Objective: Suicide is a critical global health concern. Research indicates that generative artificial intelligence (GenAI) and large language models, such as generative pretrained transformer-3 (GPT-3) and GPT-4, can evaluate suicide risk comparably to experts, yet the criteria these models use are unclear. This study explores how variations in prompts, specifically regarding past suicide attempts, gender, and age, influence the risk assessments provided by ChatGPT-3 and ChatGPT-4.
Methods: Using a controlled scenario based approach, 8 vignettes were created. Both ChatGPT-3.5 and ChatGPT 4 were used to predict the likelihood of serious suicide attempts, suicide attempts, and suicidal thoughts. A univariate 3-way analysis of variance was conducted to analyze the effects of the independent variables (previous suicide attempts, gender, and age) on the dependent variables (likelihood of serious suicide attempts, suicide attempts, and suicidal thoughts).
Results: Both ChatGPT-3.5 and ChatGPT-4 recognized the importance of previous suicide attempts in predicting severe suicide risks and suicidal thoughts. ChatGPT-4 also identified gender differences, associating men with a higher risk, while both models disregarded age as a risk factor. Interaction analysis revealed that ChatGPT-3.5 associated past attempts with a higher likelihood of suicidal thoughts in men, whereas ChatGPT-4 showed an increased risk for women.
Conclusions: The study highlights ChatGPT-3.5 and ChatGPT-4’s potential in suicide risk evaluation, emphasizing the importance of prior attempts and gender, while noting differences in their handling of interactive effects and the negligible role of age. These findings reflect the complexity of GenAI decision-making. While promising for suicide risk assessment, these models require careful application due to limitations and real-world complexities.
J Clin Psychiatry 2024;85(4):24m15365
Author affiliations are listed at the end of this article.
Continue Reading...
Members enjoy unlimited free PDF downloads as part of their subscription! Subscribe today for instant access to this article and our entire library in your preferred format. Alternatively, you can purchase the PDF of this article individually.
Elyoseph Z, Levkovich I. Beyond human expertise: the promise and limitations of ChatGPT in suicide risk assessment. Front Psychiatry. 2023;14:1213141. PubMedCrossRef
Levkovich I, Elyoseph Z. Suicide risk assessments through the eyes of ChatGPT-3.5 versus ChatGPT-4: vignette study. JMIR Ment Health. 2023;10:e51232þ. PubMedCrossRef
Stone DM, Jones CM, Mack KA. Changes in suicide rates - United States, 2018- 2019. MMWR Morb Mortal Wkly Rep. 2021;70(8):261–268. PubMed
Zalsman G. [Suicide: epidemiology, etiology, treatment and prevention]. Harefuah. 2019;158(7):468–472. PubMed
Li Y, Guo Z, Tian W, et al. An investigation of the relationships between suicidal ideation, psychache, and meaning in life using network analysis. BMC Psychiatry. 2023;23(1):257. PubMed
Gvion Y, Levi-Belz Y. Serious suicide attempts: systematic review of psychological risk factors. Front Psychiatry. 2018;9:56. PubMed
Fazel S, Runeson B. Suicide. N Engl J Med. 2020;382(3):266–274. PubMedCrossRef
Demesmaeker A, Chazard E, Hoang A, et al. Suicide mortality after a nonfatal suicide attempt: a systematic review and meta-analysis. Aust N Z J Psychiatry. 2022;56(6):603–616. PubMedCrossRef
Fehling KB, Selby EA. Suicide in DSM-5: current evidence for the proposed suicide behavior disorder and other possible improvements. Front Psychiatry. 2020;11:499980. PubMedCrossRef
Eaton DK, Kann L, Kinchen S, et al; Centers for Disease Control and Prevention (CDC). Youth risk behavior surveillance - United States, 2011. MMWR Surveill Summ. 2012;61(4):1–162. PubMed
Schrijvers DL, Bollen J, Sabbe BGC. The gender paradox in suicidal behavior and its impact on the suicidal process. J Affect Disord. 2012;138(1–2):19–26. PubMedCrossRef
Freeman A, Mergl R, Kohls E, et al. A cross-national study on gender differences in suicide intent. BMC Psychiatry. 2017;17(1):1–11.þ PubMedCrossRef
Stone DM, Mack KA, Qualters J. Notes from the field: recent changes in suicide rates, by race and ethnicity and age group - United States, 2021. MMWR Morb Mortal Wkly Rep. 2023;72(6):160–162. PubMed
Wasserman D, Rihmer Z, Rujescu D, et al. The European Psychiatric Association (EPA) guidance on suicide treatment and prevention. Eur Psychiatry. 2012;27(2):129–141. PubMedCrossRef
Bejan CA, Ripperger M, Wilimitis D, et al. Improving ascertainment of suicidal ideation and suicide attempt with natural language processing. Sci Rep. 2022; 12(1):15146. PubMedCrossRef
Vioules MJ, Moulahi B, Azé J, et al. Detection of suicide-related posts in Twitter data streams. IBM J Res Dev. 2018;62(1):7–1.
Ghanadian H, Nejadgholi I, Osman HA. ChatGPT for suicide risk assessment on social media: quantitative evaluation of model performance, potentials and limitations. Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis. Association for Computational Linguistics; 2023:172–183.
Tlili A, Shehata B, Adarkwah MA, et al. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learn Environ. 2023;15(23):1–24.
Farhat F, Sohail SS, Madsen DØ. How trustworthy is ChatGPT? The case of bibliometric analyses. Cogent Eng. 2023;10(1):2222988.
Surameery NMS, Shakor MY. Use Chat GPT to solve programming bugs. Int J Inf Technol Comput Eng. 2023;3(1):17–22.
Balas M, Ing EB. Conversational AI models for ophthalmic diagnosis: comparison of ChatGPT and the Isabel Pro differential diagnosis generator. JFO Open Ophthalmol. 2023;1:100005.
Kung TH, Cheatham M, Medenilla A, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2):e0000198. PubMedCrossRef
He N, Yan Y, Wu Z, et al. Chat GPT-4 significantly surpasses GPT-3.5 in drug information queries. J Telemed Telecare. 2023:1357633X231181922. doi:10.1177/1357633X231181922 PubMedCrossRef
Nori H, King N, McKinney SM, et al. Capabilities of GPT-4 on medical challenge problems. arXiv. 2023:13375. doi:10.48550/arXiv.2303.13375 CrossRef
Lee P, Bubeck S, Petro J. Benefits, limits, and risks of GPT-4 as an AI Chatbot for medicine. N Engl J Med. 2023;388(13):1233–1239. PubMedCrossRef
Elyoseph Z, Hadar-Shoval D, Asraf K, et al. ChatGPT outperforms humans in emotional awareness evaluations. Front Psychol. 2023;14:1199058. PubMedCrossRef
Singh OP. Artificial intelligence in the era of ChatGPT - opportunities and challenges in mental health care. Indian J Psychiatry. 2023;65(3):297–298. PubMedCrossRef
Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthc (Basel). 2023;11(6):887.
Imran N, Hashmi A, Imran A. Chat-GPT: opportunities and challenges in child mental healthcare. Pak J Med Sci. 2023;39(4):1191–1193. PubMedCrossRef
Elyoseph Z, Refoua E, Asraf K, et al. Capacity of generative AI to interpret human emotions from visual and textual data: pilot evaluation study. JMIR Ment Health. 2024;11:e54369. PubMedCrossRef
Hadar-Shoval D, Elyoseph Z, Lvovsky M. The plasticity of ChatGPT’s mentalizing abilities: personalization for personality structures. Front Psychiatry. 2023;14:1234397. PubMedCrossRef
Girard C. Age, gender, and suicide: a cross-national analysis. Am Sociol Rev. 1993;58(4):553–574.
Levi-Belz Y, Gamliel E. The effect of perceived burdensomeness and thwarted belongingness on therapists’ assessment of patients’ suicide risk. Psychother Res. 2016;26:436–445. PubMedCrossRef
Beghi M, Butera E, Cerri CG, et al. Suicidal behaviour in older age: a systematic review of risk factors associated to suicide attempts and completed suicides. Neurosci Biobehav Rev. 2021;127:193–211. PubMedCrossRef
Barrigon ML, Cegla-Schvartzman F. Sex, gender, and suicidal behavior. Curr Top Behav Neurosci. 2020;46:89–115. PubMedCrossRef
Lewandowski M, Łukowicz P, Świetlik D, et al. ChatGPT-3.5 and ChatGPT-4 dermatological knowledge level based on the specialty certificate examination in dermatology. Clin Exp Dermatol. 2024;49(7):686–691. PubMedCrossRef
de la Torre-Luque A, Pemau A, Ayad-Ahmed W, et al. Risk of suicide attempt repetition after an index attempt: a systematic review and meta-analysis. Gen Hosp Psychiatry. 2023;81:51–56. PubMed
Milton AC, Davenport TA, Iorfino F, et al. Suicidal thoughts and behaviors and their associations with transitional life events in men and women: findings from an international web-based sample. JMIR Ment Health. 2020;7(9):e18383. PubMedCrossRef
Ferrara E. Should ChatGPT be biased? Challenges and risks of bias in large language models. arXiv. 2023. doi:10.5210/fm.v28i11.13346 CrossRef
Ivey-Stephenson AZ, Crosby AE, Hoenig JM, et al. Suicidal thoughts and behaviors among adults aged ≥18 Years - United States, 2015–2019. MMWR Surveill Summ. 2022;71(1):1–19. PubMed
Stone DM, Mack KA, Qualters J. Notes from the field: recent changes in suicide rates, by race and ethnicity and age group - United States, 2021. MMWR Morb Mortal Wkly Rep. 2023;72(6):160–162. PubMed