Faces Scales for Anxiety and Anger: A National Study of Measurement Properties

Lisa Grossman Liu, MD, PhD^a ; David Russell, PhD^b ; Meghan Reading Turchioe, PhD, RN^c ; Annie C. Myers, MA^c ; Connie M. Baker, MS^d ; Jyotishman Pathak, PhD^c ; Ruth M. Masterson Creber, PhD, RN^c

J Clin Psychiatry 2022;83(5):21m14202

ABSTRACT

Importance: Faces scales are used worldwide to assess pain, but robust faces scales for anxiety and anger do not exist. These scales are urgently needed, because an estimated two-thirds of patients have difficulty reading written questionnaires.

Objective: To develop and evaluate measurement properties of faces scales to monitor two mental health symptoms in US adults (anxiety and anger) in accordance with the COnsensus-based Standards for health Measurement INstruments (COSMIN).

Methods: The development process included population identification, scale generation, and pretesting. The evaluation process included assessment of content validity, construct validity, criterion validity, test-retest reliability, and measurement error using 5 order-randomized, positively controlled online survey studies conducted between April and June 2020. We recruited national purposive samples of US adults representative on age, gender, and race. For each faces scale, participants assessed relevance, comprehensibility, and comprehensiveness (study 1, n = 300), strength-of-association (study 2, n = 300), convergent validity against the visual analog scale (VAS; study 3, n = 305), convergent validity against the Patient-Reported Outcomes Measurement Information System (PROMIS) questionnaires (study 4, n = 1,000), and test-retest reliability and measurement error (study 5, n = 853).

Results: The anxiety and anger faces scales showed high relevance (95%–96%), comprehensibility (93%–97%), comprehensiveness (94%–97%), and strength-of-association (74%–96%). We found very high agreement with the VAS (ρ = 0.94–0.95) and high agreement with PROMIS questionnaires (ρ = 0.74–0.79). Scales showed adequate test-retest reliability (intraclass correlation = 0.70–0.78) and measurement error (standard error of measurement = 1.14–1.22).

Conclusions: Faces scales to monitor anxiety and anger show adequate measurement properties, including content validity, construct validity, criterion validity, test-retest reliability, and measurement error. The recommended use is non-diagnostic monitoring of anxiety and anger, particularly when mental health is an ancillary but important outcome of treatment.

Continue Reading...

Members enjoy unlimited free PDF downloads as part of their subscription! Subscribe today for instant access to this article and our entire library in your preferred format. Alternatively, you can purchase the PDF of this article individually.

Subscribe Now

Already a member? Login

Buy PDF for $40

Please sign in or purchase this PDF for $40.

References (54)

Barrera TL, Norton PJ. Quality of life impairment in generalized anxiety disorder, social phobia, and panic disorder. J Anxiety Disord. 2009;23(8):1086–1090. PubMed CrossRef
DiMatteo MR, Lepper HS, Croghan TW. Depression is a risk factor for noncompliance with medical treatment: meta-analysis of the effects of anxiety and depression on patient adherence. Arch Intern Med. 2000;160(14):2101–2107. PubMed CrossRef
Nipp RD, El-Jawahri A, Moran SM, et al. The relationship between physical and psychological symptoms and health care utilization in hospitalized patients with advanced cancer. Cancer. 2017;123(23):4720–4727. PubMed CrossRef
US Food and Drug Administration. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims: Guidance for Industry. US Food and Drug Administration website. https://www.fda.gov/downloads/Drugs/Guidances/UCM193282.pdf. 2009. Accessed July 1, 2020.
Willke RJ, Burke LB, Erickson P. Measuring treatment impact: a review of patient-reported outcomes and other efficacy endpoints in approved product labels. Control Clin Trials. 2004;25(6):535–552. PubMed CrossRef
European Medicines Agency. Fourth Report on the Progress of the Interaction with Patients’ and Consumers’ Organisations. European Medicines Agency website. https://www.ema.europa.eu/en/documents/report/fourth-report-progress-interaction-patients-consumers-organisations-2010-results/analysis-degree-satisfaction-patients-consumers-involved-european-medicines-agency-act_en.pdf. 2011. Accessed July 1, 2020.
Goodman M, Finnegan R, Mohadjer L, et al. Literacy, Numeracy and Problem Solving in Technology-Rich Environments Among U.S. Adults: Results from the Program for the International Assessment of Adult Competencies 2012. Washington, DC. https://www.voced.edu.au/content/ngv%3A59624. 2012. Accessed July 1, 2020. doi:10.1787/9789264128859-en.
United States Census Bureau. 2018 Population Estimates by Age, Sex, Race and Hispanic Origin. US Census Bureau website. https://www.census.gov/newsroom/press-kits/2019/detailed-estimates.html. Published 2019. Accessed April 8, 2020.
PROMIS Cooperative Group. PROMIS instrument development and validation scientific standards version 2.0. HealthMeasures website. http://www.nihpromis.org/Documents/PROMISStandards_Vers2.0_Final.pdf?AspxAutoDetectCookieSupport=1. 2013. Accessed July 1, 2020.
Ali J, Basch E, Baumhauer J, et al. Users’ Guide to Integrating Patient-Reported Outcomes in Electronic Health Records. Baltimore, Maryland: Johns Hopkins University; 2017.
Palmer MJ, Mercieca-Bebber R, King M, et al. A systematic review and development of a classification framework for factors associated with missing patient-reported outcome data. Clin Trials. 2018;15(1):95–106. PubMed CrossRef
Mercieca-Bebber R, Palmer MJ, Brundage M, et al. Design, implementation and reporting strategies to reduce the instance and impact of missing patient-reported outcome (PRO) data: a systematic review. BMJ Open. 2016;6(6):e010938. PubMed CrossRef
Wong-Baker FACES Foundation. Wong-Baker FACES Pain Rating Scale. Wong-Baker FACES Foundation website. https://wongbakerfaces.org/. Published 2016. Accessed April 8, 2020.
Wong DL, Baker CM. Pain in children: comparison of assessment scales. Pediatr Nurs. 1988;14(1):9–17. https://www.ncbi.nlm.nih.gov/pubmed/3344163 PubMed
Chambers CT, Hardial J, Craig KD, et al. Faces scales for the measurement of postoperative pain intensity in children following minor surgery. Clin J Pain. 2005;21(3):277–285. PubMed CrossRef
West N, Oakes L, Hinds PS, et al. Measuring pain in pediatric oncology ICU patients. J Pediatr Oncol Nurs. 1994;11(2):64–68, discussion 69–70. PubMed CrossRef
Keck JF, Gerkensmeyer JE, Joyce BA, et al. Reliability and validity of the Faces and Word Descriptor Scales to measure procedural pain. J Pediatr Nurs. 1996;11(6):368–374. PubMed CrossRef
Herr KA, Garand L. Assessment and measurement of pain in older adults. Clin Geriatr Med. 2001;17(3):457–478, vi. PubMed CrossRef
Halfaker DA, Akeson ST, Hathcock DR, et al. Psychological Aspects of Pain. In: Pain Procedures in Clinical Practice. Elsevier; 2011:13–22. 10.1016/B978-1-4160-3779-8.10003-X.
Kuttner L, LePage T. Face scales for the assessment of pediatric pain: a critical review. Can J Behav Sci / Rev Can des Sci du Comport. 1989;21(2):198–209. CrossRef
McKinley S, Madronio C. Validity of the Faces Anxiety Scale for the assessment of state anxiety in intensive care patients not receiving mechanical ventilation. J Psychosom Res. 2008;64(5):503–507. PubMed CrossRef
Raccanello D, Brondino M, Pasini M. Achievement emotions in technology enhanced learning: development and validation of self-report instruments in the Italian context. Interact Des Archit. 2014;23(1):68–81.
Kontou E. Thomas SA, Lincoln NB. Psychometric properties of a revised version of the Visual Analog Mood Scales. Clin Rehabil. 2012;26(12):1133–1140. PubMed CrossRef
Turner J, Kelly B. Emotional dimensions of chronic disease. West J Med. 2000;172(2):124–128. PubMed CrossRef
Schalet BD, Pilkonis PA, Yu L, et al. Clinical validity of PROMIS depression, anxiety, and anger across diverse clinical samples. J Clin Epidemiol. 2016;73(10):119–127. PubMed CrossRef
Cella D, Lai J-S, Jensen SE, et al. PROMIS fatigue item bank had clinical validity across diverse chronic conditions. J Clin Epidemiol. 2016;73(10):128–134. PubMed CrossRef
Howland M, Tatsuoka C, Smyth KA, et al. Evaluating PROMIS() applied cognition items in a sample of older adults at risk for cognitive decline. Psychiatry Res. 2017;247(1):39–42. PubMed CrossRef
Mokkink LB, Prinsen CA, Patrick DL, et al. COSMIN Study Design checklist for patient-reported outcome measurement instruments: Version July 2019. COSMIN website. https://www.cosmin.nl/wp-content/uploads/COSMIN-study-designing-checklist_final.pdf. 2019. Accessed July 1, 2020.
Schenker Y, Stewart A, Na B, et al. Depressive symptoms and perceived doctor-patient communication in the Heart and Soul Study. J Gen Intern Med. 2009;24(5):550–556. PubMed CrossRef
Read C, Armstrong AW. Association between the mental health of patients with psoriasis and their satisfaction with physicians. JAMA Dermatol. 2020;156(7):754–762. PubMed CrossRef
Sowden GL, Huffman JC. The impact of mental illness on cardiac outcomes: a review for the cardiologist. Int J Cardiol. 2009;132(1):30–37. PubMed CrossRef
Bremner JD, Campanella C, Khan Z, et al. Brain correlates of mental stress-induced myocardial ischemia. Psychosom Med. 2018;80(6):515–525. https://www.ncbi.nlm.nih.gov/pubmed/29794945. PubMed CrossRef
Arcia A, Suero-Tejeda N, Spiegel-Gotsch N, et al. Helping Hispanic family caregivers of persons with dementia “get the picture” about health status through tailored infographics. Gerontologist. 2019;59(5):e479–e489. PubMed CrossRef
Pollak JP, Adams P, Gay G. PAM: A Photographic Affect Meter for Frequent, In Situ Measurement of Affect. Conf Hum Factors Comput Syst - Proc; 2011:725–734. 10.1145/1978942.1979047.
Bieri D, Reeve RA, Champion DG, et al. The Faces Pain Scale for the self-assessment of the severity of pain experienced by children: development, initial validation, and preliminary investigation for ratio scale properties. Pain. 1990;41(2):139–150. PubMed CrossRef
Tomlinson D, von Baeyer CL, Stinson JN, et al. A systematic review of faces scales for the self-report of pain intensity in children. Pediatrics. 2010;126(5):e1168–e1198. PubMed CrossRef
Stange M, Barry A, Smyth J, et al. Effects of smiley face scales on visual processing of satisfaction questions in web surveys. Soc Sci Comput Rev. 2018;36(6):756–766. CrossRef
Peer E, Brandimarte L, Samat S, et al. Beyond the Turk: alternative platforms for crowdsourcing behavioral research. J Exp Soc Psychol. 2017;70:153–163. CrossRef
Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med. 2004;36(8):588–594. PubMed
R Core Team. R: A Language and Environment for Statistical Computing. R-Project website. https://www.r-project.org/. 2015. Accessed July 1, 2020.
Gould D, Kelly D, Goldstone L, et al. Examining the validity of pressure ulcer risk assessment scales: developing and using illustrated patient simulations to collect the data: Information Point: Visual Analogue Scale. J Clin Nurs. 2001;10(5):706. PubMed CrossRef
Wewers ME, Lowe NK. A critical review of visual analogue scales in the measurement of clinical phenomena. Res Nurs Health. 1990;13(4):227–236. PubMed CrossRef
HealthMeasures. PROMIS (Patient-Reported Outcomes Measurement Information System). https://www.healthmeasures.net/explore-measurement-systems/promis. Published 2020. Accessed April 8, 2020.
Cella D, Riley W, Stone A, et al; PROMIS Cooperative Group. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. J Clin Epidemiol. 2010;63(11):1179–1194. PubMed CrossRef
Cella D, Yount S, Rothrock N, et al; PROMIS Cooperative Group. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007;45(suppl 1):S3–S11. PubMed CrossRef
Kim TK. Practical statistics in pain research. Korean J Pain. 2017;30(4):243–249. PubMed CrossRef
Paivio A, Csapo K. Picture superiority in free recall: imagery or dual coding? Cognit Psychol. 1973;5(2):176–206. CrossRef
HealthMeasures. Interpret Scores: PROMIS: Meaningful Change. HealthMeasures website. https://www.healthmeasures.net/score-and-interpret/interpret-scores/promis/meaningful-change. Published 2019. Accessed May 27, 2019.
Hallgren KA. Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol. 2012;8(1):23–34. PubMed CrossRef
Payne RW. Reliability theory and clinical psychology. J Clin Psychol. 1989;45(2):351–353. PubMed CrossRef
Tkacz J, Brady B. The increasing rate of childhood mental illnesses and associated healthcare costs in the United States: trends over the past decade. Value Health. 2019;22(suppl 2):S231. CrossRef
Veinot TC, Mitchell H, Ancker JS. Good intentions are not enough: how informatics interventions can worsen inequality. J Am Med Inform Assoc. 2018;25(8):1080–1088. PubMed CrossRef
Lorenc T, Petticrew M, Welch V, et al. What types of interventions generate inequalities? evidence from systematic reviews. J Epidemiol Community Health. 2013;67(2):190–193. PubMed CrossRef
Hart JT. The inverse care law. Lancet. 1971;297(7696):405–412. PubMed CrossRef