Functional Impairment in Youth With ADHD: Normative Data and Norm-Referenced Cutoff Points for the Before School Functioning Questionnaire and the Parent Rating of Evening and Morning Behavior Scale, Revised

ABSTRACT

The early morning is a source of distress and disability for children with ADHD. Barkley and Cunningham1 showed that ADHD impaired early morning organization, self-care, preparation for the school day, and transportation to school. Failing to complete their morning routine puts children at risk for tardiness, forgetting to take homework to school, and family conflict. Parents of stimulant-treated ADHD children report stress while getting their child ready for school and may be late to work when their child is late to school.2 Whalen et al3 showed that children’s before-school ADHD symptoms reduced parenting effectiveness. Faraone et al4 surveyed 300 caregivers of stimulant-treated children and reported high levels of early morning functional impairments in the child. The child’s impairments reduced the emotional well-being of parents, worsened the parent’s functioning during the early morning, and increased conflict with siblings.

ADHD symptoms also impair patients in the late afternoon and evening. For children, symptom control into the evening hours is important for socialization, family interactions, and completing chores and homework.3 Many adults with ADHD have work schedules that require attention to work or driving during the evening. At home, ADHD symptoms in adults interfere with parenting and marital interactions. A survey of 201 families with ADHD children2 found that symptoms were high during dinner and evening homework time and at bedtime.

For children, the standard for assessing time-specific symptoms is the laboratory school study, which brings ADHD children together for a day in a simulated classroom during which they are assessed for behaviors relevant to ADHD.5 Likewise, workplace simulations have been developed to assess the duration of ADHD symptoms in adults with ADHD.6 Although simulation methods provide valuable data about the duration of action of medications,7 they place patients in artificial environments. A classroom that includes only ADHD children may not generalize to real classrooms. Simulated environments do not assess behaviors that are important in the naturalistic environment such as getting to school on time, driving home from work, staying seated for dinner, and settling down at bedtime. For assessing naturalistic behaviors, researchers and clinicians need reliable and valid measures to test the time-specific efficacy of treatments.

We have been evaluating the psychometric features of 2 time-sensitive measures for youth with ADHD: the Before School Functioning Questionnaire (BSFQ)8 and the Parent Rating of Evening and Morning Behaviors Scale, Revised (PREMB-R).9,10 The BSFQ comprises 20 items that assess ADHD-associated symptoms and functional impairments prior to attending school or other activities. The BFSQ investigator-rated scale has very good internal homogeneity (Cronbach α = .91), good test-retest reliability (correlation = 0.60), good concurrent validity (correlations = 0.42-0.86), and a strong treatment effect in a clinical trial of the methylphenidate transdermal system (effect size = −0.93).11 The BSFQ separated drug from placebo in trials of guanfacine extended release (parent-reported scale)12 and delayed-release and extended-release methylphenidate (DR/ER-MPH, formerly HLD200) (investigator-rated scale).13

The PREMB-R Morning (AM) subscale measures early morning functional (EMF) impairments; the Evening (PM) subscale assesses functional impairment in the late afternoon and evening. Michelson et al14 reported that atomoxetine was significantly better than placebo for reducing ADHD symptoms but was not effective for reducing early morning or late afternoon/evening functional impairments. In contrast, the early morning and late afternoon/evening scores were improved with atomoxetine in subsequent studies.15-17 The PREMB-R AM has significant internal homogeneity (α = .65) and test-retest reliability (correlations = 0.52-0.45) and good concurrent validity (correlations = 0.50-0.71).9 The PREMB-R PM also has high internal homogeneity (α = .81), good test-retest reliability (correlations = 0.74-0.83), and good concurrent validity (correlations = 0.47-0.80).10

Although prior work with the PREMB-R AM and PM and the BSFQ shows them to be reliable and valid assessments of early morning and late afternoon/evening behaviors, no normative data are available to help clinicians determine when a child’s level of impairment is clinically significant. Such norms would also be useful in clinical trials to address the degree to which treatments return ADHD patients to the normative range of functioning. To address that gap in the literature, we conducted a population survey of 1,200 families with the goal of establishing cut-points for each scale for use in clinical practice and research.

CLINICAL POINTS

• The early morning and the late afternoon/evening are times of distress and disability for youth with attention-deficit/hyperactivity disorder.
• The present work provides clinicians with normative data on validated assessment tools that will allow them to classify their patients’ risks for early morning and late afternoon/evening functional impairments.
• These norms provide a rational approach for selecting treatments to target such time-specific functional impairments.

METHODS

Research Participants

The normative online survey was conducted on a sample of 1,200 primary caregivers of children and adolescents (6-17 years old). They were recruited through a consumer research panel (N = 275,000) representative of the US population in terms of age, sex, education, employment, ethnicity, and geography. Participants completed a screening questionnaire to assess the following inclusion criteria: (1) they self-identified as a primary caregiver of a child aged 6 to 17 years; (2) their child either never had ADHD, had a history of ADHD, or currently had ADHD without treatment (medical, psychological, or holistic) during the past 3 months; and (3) their child had not been diagnosed with birth defects of the brain or spinal cord injury, traumatic brain injury, severe concussions, cerebral palsy, seizure disorders, tic disorders or Tourette disorder, bipolar disorder, or psychosis. To determine if the child had any ADHD, other psychiatric disorders, or exclusionary disorders, we asked the parent if the child had ever been diagnosed by a health professional, a method that has been used, for example, by the Centers for Disease Control and Prevention (CDC).18 We did not, however, query for specific DSM-5 or other structured diagnostic criteria. If there was more than one eligible child, respondents were instructed to select and answer the subsequent questions based on the child whose birthday was next. We recruited parents of 50 male and 50 female youth in each of the 12 age groups. We did not preferentially recruit families that had a child with ADHD.

Eligible caregivers rated their children on the BSFQ and the PREMB-R. The survey was fielded and completed in June 2016 by REPASS, Inc (Newtown Square, Pennsylvania), a market research firm. Surveys were completed voluntarily and anonymously, and respondents were blinded to the research sponsor (Ironshore Pharmaceuticals & Development, Inc; Grand Cayman, Cayman Islands). The survey was conducted in adherence to the Marketing Research Association’s Code of Marketing Research Standards (https://www.insightsassociation.org/issues-policies/mra-code-marketing-research-standards). The survey contained no option for any of the 18 standard types of protected health information (PHI) data to be collected and therefore did not require Institutional Review Board (IRB) approval.

Assessments

The BSFQ comprises 20 items that cover commonly reported areas of dysfunction in the early morning, before-school activities (eg, breakfast, hygiene, time awareness, getting to school), associated with ADHD.8 Using the BSFQ, primary caregivers rated their child’s EMF impairment between the time of awakening and before the school day or other morning activities (ie, 6:00 am and 9:00 am) on a severity scale of 0 to 3. The PREMB-R AM comprises 3 items that assess the difficulty of functioning during the early morning, and the PREMB-R PM comprises 8 items that assess the difficulty of functioning during the late afternoon and evening. Using the PREMB-R AM and PREMB-R PM, primary caregivers rated their child’s temporal functional impairment on a severity scale of 0 to 3.

Statistical Analyses

Because the rating scale data have only integer values that are right truncated by the number of items and are markedly non-normally distributed, statistical analyses relied on truncated Poisson regression with scale scores as dependent variables. To predict ADHD diagnosis from each individual item score and total scores, we used logistic regression. We used multinomial logistic regression for trinary ADHD diagnoses (no history of ADHD vs history of ADHD vs current untreated ADHD). Each model was adjusted for the number of the following psychiatric comorbidities: cigarette smoking, substance use disorder, oppositional defiant disorder, conduct disorder, depression, insomnia/sleep disorders, anxiety disorder, obsessive-compulsive disorder, learning disability, and developmental disorder.

We determined if scale scores were significantly affected by age and sex after adjusting for the presence of ADHD and other comorbidities using a Bonferroni-adjusted α level of .008. We then computed normative cutoff scores using the same percentile cut-points defined by DuPaul et al19 for the ADHD Rating Scale:

• Screening risk: 80th percentile
• Mild functional impairment: 90th percentile
• Moderate functional impairment: 93rd percentile
• Severe functional impairment: 98th percentile

Because the prevalence of ADHD in this sample (10.1%) is about twice the population expectation of 5%,20 we computed percentiles after randomly deleting half of the ADHD participants with the goal of estimating cut-points that would generalize to the overall population.

RESULTS

Of the 1,200 families surveyed, 121 index youth had been diagnosed with ADHD. Among these, 80 had current ADHD but had not been treated within the 3 months prior to participation and 41 did not currently have ADHD and were not receiving treatment. This prevalence of 10% is consistent with that reported by the CDC using similar methodology18 but larger than the population expectation of 5% from meta-analyses.20 The ADHD group was half a year older and was more likely to be male (Table 1). There were large and significant differences for most psychiatric disorders.

Adolescents had significantly lower BSFQ scores than young children; the effect of sex was not significant (Table 2). Those with a history of ADHD or other psychiatric disorders had higher BSFQ scores than other youth. Table 3 shows the same pattern of results for the PREMB-R AM and PM. Both ADHD status and number of other disorders were significant, independent predictors of BSFQ and PREMB-R scores. The BSFQ also asks respondents “How many minutes did it take your child to wake up and get out of bed?” and “How many minutes did it take your child to complete their early morning routine?” For the first question, both ADHD groups were significantly more likely to take more time getting out of bed than the non-ADHD group even after adjustment for significant effects of age, sex, and psychiatric comorbidity (P < .001). The mean ± SD times were 16.7 ± 12.2 minutes for those with a history of ADHD and 10.9 ± 10.2 minutes for others. Responses to the second question did not differ significantly by ADHD status. The mean ± SD times were 33.2 ± 18.9 minutes for those with a history of ADHD and 30.9 ± 18.3 minutes for those without such a history.

For all 3 scales, the distribution of total scores was heavily skewed. Many children had low or no impairment, and decreasing numbers of children showed higher levels of impairment. The distributions of these scores are shown in Supplementary Figure 1. The distributions of severity for each item of the BSFQ are given in Supplementary Figure 2. Across all 20 items of the BSFQ, youth with current untreated ADHD or a history of ADHD were rated as more severely ill than those without ADHD (all unadjusted P values < .001), even after adjusting for psychiatric comorbidity (all adjusted P values < .001). Supplementary Figure 3 shows a similar pattern of results for the PREMB-R. Across the 3 items of PREMB-R AM, youth with current untreated ADHD or a history of ADHD had more severe ratings than those without ADHD, even after adjusting for psychiatric comorbidity (all unadjusted P values < .001; all adjusted P values < .001, except for item 1 [P = .01]). Across the 8 items of the PREMB-R PM, youth with current untreated ADHD or a history of ADHD had more severe ratings than those without ADHD, even after adjusting for psychiatric comorbidity (all unadjusted P values < .001; all adjusted P values < .001).

Because each of the total scores was associated with age but not sex, we defined 4 cut-points based on age alone: screening risk (80th percentile), mild functional impairment (90th percentile), moderate functional impairment (93rd percentile), and severe functional impairment (98th percentile). We used 4 age bins: 6 to 8, 9 to 11, 12 to 14, and 15 to 17 years. These cut-points are given in Table 4. We also computed cut-points for a data set that was modified to have a prevalence of ADHD, which is more consistent with the population expectation (5%) from meta-analyses.19 These data are also included in Table 4. These cut-points are slightly, albeit not dramatically, lower. As a third step, we smoothed these cut-points to be consistent with our finding that all 3 scores decrease with age. Despite that highly significant effect, 4 cut-points violate that pattern due to natural variability. We smoothed these by replacing them and the cut-point for the prior age group with the mean of those 2 cut-points. This method pools data across adjacent age groups to compute a common cut-point. The modified cut-points are bolded in Table 4.

Figure 1 shows the fraction of subjects exceeding each cut-point stratified by ADHD and comorbidity status. For example, Figure 1A shows that 78% of children having both ADHD and another disorder screen positive at the 80th percentile; 53% of those having only ADHD but not another psychiatric disorder also screen positive at this cut-point. For nearly all scores and cut-points, children with ADHD and another disorder are most likely to exceed the cut-point, those with only ADHD are second most likely, those without ADHD but with another disorder are third most likely, and those with no disorder are least likely.

For the BSFQ questions asking about the amount of time to get out of bed, the 80th percentile was 15 minutes, the 90th percentile was 25 minutes, the 93rd percentile was 30 minutes, and the 98th percentile was 43 minutes. For time to complete the early morning routine, these percentiles were 45 minutes, 60 minutes, 60 minutes, and 80 minutes, respectively (the data were too sparse to separate the 90th and 93rd percentiles).

DISCUSSION

The prevalence of ADHD in this sample was similar to a prior report from the CDC18 that used a telephone interview to query about prior diagnoses of ADHD but higher than in data derived from a meta-analysis20 of studies that interviewed respondents about symptoms and impairment. We found a higher male versus female prevalence of ADHD and higher rates of psychiatric disorders in ADHD compared with non-ADHD participants—findings highly consistent with prior literature.21

The total scores from the BSFQ, PREMB-R AM, and PREMB-R PM were significantly associated with ADHD and with the presence of other psychiatric disorders. The effects of ADHD on scale scores was not accounted for by comorbid psychiatric disorders, which suggests that ADHD has an independent effect on the early morning and late afternoon/evening functioning of children and adolescents. Respondents rated youth with ADHD as showing greater severity for each item of the BSFQ, PREMB-R AM, and PREMB-R PM. For most items, about half of youth with current ADHD or a history of ADHD were rated as showing moderate or severe functional impairment.

For all scale scores, we found a significant effect for age but not for sex. The decrease in symptoms with age is consistent with other studies of ADHD,22 which show that symptoms of hyperactivity and impulsivity tend to decrease in adolescence whereas symptoms of inattention persist. The lack of sex effects raises the possibility that temporally specific measures of ADHD-associated features may be more sensitive to the disorder in females than diagnostic items. If so, that could have implications for future revisions of the diagnostic criteria for ADHD.

We presented the first norms for these 3 scores, stratifying by age. Inspection of Figure 1 motivates several recommendations. Because the PREMB-R AM has only 3 items, it cannot distinguish mild from moderate functional impairments, which suggests that the BSFQ is better for identifying EMF impairment. The 4 cut-points selected few children without ADHD and without other psychiatric disorders. This finding indicates that if these scores are applied to population samples, there would not be many false-positive identifications, especially for the more extreme cut-points. For the ADHD sample, the screening criterion (80th percentile) would select 50% to 80% of patients. The 90th percentile identifies 25% to 50% of ADHD youth as having early morning or late afternoon or evening impairments. Given that so many ADHD youth show extreme deviations from the normative level of functioning, clinicians should consider screening for such impairments, even among youth currently receiving treatment, as such youth have been documented to have EMF impairments.2,4

In clinical practice, clinicians should use the normative cut-point that fits the context of the clinical setting and the population to be assessed. For example, in a pediatrics clinic, where it is feasible to follow up many positive screens with a clinical interview of the parent, one could use a highly sensitive threshold (the 80th or 90th percentiles). When follow-up interviews are less feasible, the 93rd or 98th percentiles allow fewer false-positive findings at the price of not identifying all youth with ADHD who have impairments. When screening youth with ADHD using the BSFQ and PREMB-R PM, the mild functional impairment criterion identifies between 30% and 50% of ADHD youth as impaired (Figure 1) and should be considered as a useful threshold for optimizing early morning and late afternoon/evening functioning.

Our work is limited by using online surveys rather than in-person interviews and not collecting information about diagnostic criteria. Instead, we relied on the respondent’s report about prior psychiatric diagnoses of their child. Although not ideal, this method has been used by others18 and is unlikely to have caused false-positive associations between ADHD and time-specific functional impairments. Nevertheless, future work should attempt to replicate these findings using structured interview diagnoses of ADHD that document age at onset, impairment, and the cross-situational expression of symptoms and impairment.

In summary, functional impairments in the early morning and late afternoon/evening hours are associated with ADHD and cannot be accounted for by other disorders. We have presented norms, stratified by age, which can guide clinicians in the use of these scores for identifying those ADHD youth who may be experiencing difficulties in these time periods. Such tools are needed given the availability of treatments that can target ADHD symptoms and impairments at these extremes of the daily routine.

Submitted: June 14, 2019; accepted September 24, 2019.

Published online: December 10, 2019.

Author contributions: All authors listed in the article have seen and approved the submission of this version of the article and take full responsibility for the content.

Potential conflicts of interest: In the past year, Dr Faraone received income, potential income, travel expenses continuing education support, and/or research support from Tris; Otsuka; Arbor; Ironshore Pharmaceuticals & Development, Inc; Shire; Akili Interactive; Enzymotec; Sunovion; Supernus; and Genomind. With his institution, he has US patent US20130217707 A1 for the use of sodium-hydrogen exchange inhibitors in the treatment of attention-deficit/hyperactivity disorder (ADHD). He also receives royalties from books published by Guilford Press: Straight Talk about Your Child’s Mental Health; Oxford University Press: Schizophrenia: The Facts; and Elsevier: ADHD: Non-Pharmacologic Interventions. He is principal investigator of www.adhdinadults.com. Dr Wilens receives or has received grant support from the National Institutes of Health (NIH) National Institute on Drug Abuse (NIDA) and Lloyd Foundation. He is or has been a consultant for NIH NIDA; Ironshore Pharmaceuticals & Development, Inc; KemPharm; Otsuka/Neurovance; the National Football League Program for Substances of Abuse (ERM Associates); and Minor/Major League Baseball. He has a published book with Guilford Press: Straight Talk About Psychiatric Medications for Kids and co-edited ADHD Across the Lifespan (Cambridge Univ Press) and Comprehensive Clinical Psychiatry (Elsevier). Mr DeSousa and Dr Incledon are employees/consultants of Ironshore Pharmaceuticals & Development, Inc. Dr Sallee is an employee of Ironshore Pharmaceuticals Inc and is a member of the board of directors for P2D Bioscience. Dr Komolova is an employee of Highland Therapeutics Inc.

Funding/support: This work was funded by Ironshore Pharmaceuticals & Development, Inc.

Role of the sponsor: Ironshore Pharmaceuticals & Development, Inc, paid for the collection and analysis of data. Employees of Ironshore Pharmaceuticals & Development, Inc, and its affiliated companies were involved in the design of the study and interpretation of results and, hence, are included as authors.

Acknowledgments: The authors acknowledge Rick Nullmeier, BA, of ADVANCE Life Sciences Consulting (Hebron, Kentucky), who provided project management support; and Helen Doll, DPhil, and Hannah Lewis, PhD, of ICON (Gaithersburg, Maryland), who performed initial data analysis. These individuals were paid by the study sponsor for services provided.

Supplementary material: See accompanying pages.

REFERENCES

1.Barkley RA, Cunningham CE. The effects of methylphenidate on the mother-child interactions of hyperactive children. Arch Gen Psychiatry. 1979;36(2):201-208. PubMed CrossRef

2.Sallee FR. Early morning functioning in stimulant-treated children and adolescents with attention-deficit/hyperactivity disorder, and its impact on caregivers. J Child Adolesc Psychopharmacol. 2015;25(7):558-565. PubMed CrossRef

3.Whalen CK, Henker B, Jamner LD, et al. Toward mapping daily challenges of living with ADHD: maternal and child perspectives using electronic diaries. J Abnorm Child Psychol. 2006;34(1):115-130. PubMed CrossRef

4.Faraone SV, Schachar RJ, Barkley RA, et al. Early morning functional impairments in stimulant-treated children with attention-deficit/hyperactivity disorder versus controls: impact on the family. J Child Adolesc Psychopharmacol. 2017;27(8):715-722. PubMed CrossRef

5.Wigal SB, Wigal TL. The laboratory school protocol: its origin, use, and new applications. J Atten Disord. 2006;10(1):92-111. PubMed CrossRef

6.Fried R, Surman C, Hammerness P, et al. A controlled study of a simulated workplace laboratory for adults with attention deficit hyperactivity disorder. Psychiatry Res. 2012;200(2-3):949-956. PubMed CrossRef

7.Gomeni R, Fang LL, Bressolle-Gomeni F, et al. A general framework for assessing In vitro/In vivo as a tool for maximizing the benefit-risk ratio of a treatment using a convolution-based modeling approach. CPT Pharmacometrics Syst Pharmacol. 2019;8(2):97-106. PubMed

8.Faraone SV, Hammerness PG, Wilens TE. Reliability and validity of the Before-School Functioning Scale in children with ADHD. J Atten Disord. 2018;22(11):1040-1048. PubMed CrossRef

9.Faraone SV, Childress A, Wigal SB, et al. Reliability and validity of the Daily Parent Rating of Evening and Morning Behavior Scale, Revised. J Atten Disord. 2018;22(11):1066-1073. PubMed CrossRef

10.Faraone SV, DeSousa NJ, Sallee FR, et al. Psychometric validation of the Before School Functioning Questionnaire and Parent Rating of Evening and Morning Behavior Scale, Revised in children with attention-deficit/hyperactivity disorder. J Am Acad Child Psychiatry. 2017.56(10):S212.

11.Wilens TE, Hammerness P, Martelon M, et al. A controlled trial of the methylphenidate transdermal system on before-school functioning in children with attention-deficit/hyperactivity disorder. J Clin Psychiatry. 2010;71(5):548-556. PubMed CrossRef

12.Wilens TE, McBurnett K, Turnbow J, et al. Morning and evening effects of guanfacine extended release adjunctive to psychostimulants in pediatric ADHD. J Atten Disord. 2017;21(2):110-119. PubMed CrossRef

13.Pliszka SR, Wilens TE, Bostrom S, et al. Efficacy and safety of HLD200, delayed-release and extended-release methylphenidate, in children with attention-deficit/hyperactivity disorder. J Child Adolesc Psychopharmacol. 2017;27(6):474-482. PubMed CrossRef

14.Michelson D, Allen AJ, Busner J, et al. Once-daily atomoxetine treatment for children and adolescents with attention deficit hyperactivity disorder: a randomized, placebo-controlled study. Am J Psychiatry. 2002;159(11):1896-1901. PubMed CrossRef

15.Sutton V, Sumner C, Allen AJ, et al. Validity, Reliability, and Responsiveness of the DPREMB-R Scale for ADHD. Annual Meeting of the American Academy of Child and Adolescent Psychiatry. 2003; Miami Beach, Florida.

16.Sangal RB, Owens J, Allen AJ, et al. Effects of atomoxetine and methylphenidate on sleep in children with ADHD. Sleep. 2006;29(12):1573-1585. PubMed CrossRef

17.Kelsey DK, Sumner CR, Casat CD, et al. Once-daily atomoxetine treatment for children with attention-deficit/hyperactivity disorder, including an assessment of evening and morning behavior: a double-blind, placebo-controlled trial. Pediatrics. 2004;114(1):e1-e8. PubMed CrossRef

18.Visser SN, Danielson ML, Bitsko RH, et al. Trends in the parent-report of health care provider-diagnosed and medicated attention-deficit/hyperactivity disorder: United States, 2003-2011. J Am Acad Child Adolesc Psychiatry. 2014;53(1):34-46.e2. PubMed CrossRef

19.DuPaul GJ, Reid R, Anastopoulos AD, et al. Parent and teacher ratings of attention-deficit/hyperactivity disorder symptoms: factor structure and normative data. Psychol Assess. 2016;28(2):214-225. PubMed CrossRef

20.Polanczyk GV, Willcutt EG, Salum GA, et al. ADHD prevalence estimates across three decades: an updated systematic review and meta-regression analysis. Int J Epidemiol. 2014;43(2):434-442. PubMed CrossRef

21.Faraone SV, Asherson P, Banaschewski T, et al. Attention-deficit/hyperactivity disorder. Nat Rev Dis Primers. 2015;1(1):15020. PubMed CrossRef

22.Faraone SV, Biederman J, Mick E. The age-dependent decline of attention deficit hyperactivity disorder: a meta-analysis of follow-up studies. Psychol Med. 2006;36(2):159-165. PubMed CrossRef