Measuring Empathy in Artificial Intelligence: Insights From Psychodermatology and Implications for General Practice

To the Editor: Delusional parasitosis (DP), also known as Ekbom syndrome, is a psychocutaneous disorder wherein patients have a fixed, false, unwavering belief that they are infected with parasites.¹ This disorder often presents with extensive pruritus as well as hesitancy to obtain psychiatric treatment.¹ Because of a lack of insight, many patients may continue to suffer.

Studies have demonstrated that artificial intelligence (AI), particularly specific chatbots programmed for cognitive-behavioral therapy, may serve as cost-effective and accessible psychiatric therapeutic options.² With the rise of AI models such as ChatGPT-4, the potential of AI in psychodermatology remains unexplored.

Methods

The goal of this study was to assess which AI modality has the most potential to help a patient with DP based on the Empathy Scale for Human-Computer Communication (ESHCC) score.³ The ESHCC is an analytical framework created to measure perceived empathy in large language models.³ Each item on the ESHCC is scored using a 7-point Likert-type scale (1 = not at all to 7 = extensively).³ In Table 1, comparisons of responses between ChatGPT-4, Google, Claude, and Barde are shown.

Results

Statistically significant differences between all groups were observed. Claude had the highest score on the ESHCC, while Google search results had the lowest. For one of the prompts, Google had a link to a Cleveland Clinic website, stating that persistent itching may be a sign of an underlying neurological disorder such as Parkinson disease or multiple sclerosis. In contrast, the AI chatbots provided more direct, reassuring answers such as “No, you are not crazy” and “I understand this is very distressing.” For a concerned patient, learning that their symptoms may be a sign of a much more serious neurological condition may be more stress-inducing. While preliminary, our results suggest that AI chatbots may provide more supportive answers than Google, reflected by higher ESHCC scores.

Discussion

Physicians may use the results of this study in their interactions with patients with DP. As in all patient-provider relationships, but particularly in conditions where patient insight is lacking, validating the patient’s experience with empathetic answers while providing accurate information is crucial. For example, physicians may use the ESHCC to grade their responses to patients, such as in the setting of responding to a patient question on an online health portal. Furthermore, studies have demonstrated that AI chatbots generate quality and empathetic responses to patient questions asked in an online forum. Taking advantage of such technology has the potential to prevent provider burnout and even improve patient outcomes.⁴ For instance, machine learning algorithms have already demonstrated excellent performance in objective, accurate measurements of itch by quantifying scratching behavior.⁵ Further directions may involve AI chatbots being used as distractions to pruritic behavior, resources for patient education, or starting points for seeking support. Limitations of this study include a limited number of prompts and differing responses to the same prompts by AI chatbots. Additionally, it is important to consider that empathy may vary among individuals and thus ESHCC scores may be user dependent.³

Article Information

Published Online: October 10, 2024. https://doi.org/10.4088/PCC.24lr03782
© 2024 Physicians Postgraduate Press, Inc.
Prim Care Companion CNS Disord 2024;26(5):24lr03782
To Cite: Ahuja K, Lio P. Measuring empathy in artificial intelligence: insights from psychodermatology and implications for general practice. Prim Care Companion CNS Disord. 2024;26(5):24lr03782.
Author Affiliations: Eastern Virginia Medical School, Norfolk, Virginia (Ahuja); Northwestern University, Department of Dermatology, Chicago, Illinois (Lio).
Corresponding Author: Kripa Ahuja, MS, Eastern Virginia Medical School, 825 Fairfax Ave, Norfolk, VA 23507 ([email protected]).
Relevant Financial Relationships: Dr Lio reports research grants/funding from AbbVie and AOBiome; is on the speaker’s bureau for AbbVie, Arcutis, Eli Lilly, Galderma, Hyphens Pharma, Incyte, La Roche Posay/L’Oreal, MyOR Diagnostics, ParentMD, Pfizer, Pierre-Fabre Dermatologie, Regeneron/ Sanofi Genzyme, and Verrica; reports consulting/ advisory boards for Alphyn, AbbVie, Almirall, Amyris, Arcutis, ASLAN, Boston Skin Science, Bristol Myers Squibb, Burt’s Bees, Castle Biosciences, Codex Labs, Concerto Biosciences, Dermavant, Eli Lilly, Galderma, Janssen, Johnson & Johnson, Kimberly-Clark, LEO Pharma, Lipidor, L’Oreal, Merck, Micreos, MyOR Diagnostics, Regeneron/ Sanofi Genzyme, Skinfix, Theraplex, UCB, Unilever, and Verrica Yobee Care; has stock options with Codex, Concerto Biosciences and Yobee Care; has a patent pending for a Theraplex product with royalties paid; and is a board member and scientific advisory committee member of the National Eczema Association. Mr Ahuja reports no relevant financial relationships.
Funding/Support: None.