与标准化病人互动是医学生病史采集技能训练的重要途径。最新发展的大语言模型ChatGPT具有强大的语言功能,能够以类人类的自然语言进行沟通。为深入探索和开发ChatGPT-4作为虚拟标准化病人的提示词工程,2023年10月至2024年2月,本研究经过提示词的开发、测试和迭代,收集ChatGPT-4与参与者互动的日志和参与测试医学生的评价,通过质性文本分析,评估不同版本的提示词对提升ChatGPT-4作为虚拟标准化病人效果的影响。结果显示,ChatGPT-4对叙事病史的理解更为透彻,能够掌握标准化病人的互动技巧,并对问诊进行有效评估。经过调整优化后的提示词,能够适应各种医学病例。学生认为其表现优秀,若能够有情绪互动则效果更佳。该提示词工程的建立为医学教师轻松创建虚拟标准化病人提供了新途径,也为医学生的病史采集练习创造了机会。
Acting with standardized patients is an important way to train the skills of medical history taking. The newly developed large language model ChatGPT has powerful language function and can communicate with human-like natural language. To further explore and develop the prompt engineering for ChatGPT-4 to act as virtual standardized patients (VSP), this study established, tested, and iterated the prompts from October 2023 to February 2024. The logs of ChatGPT-4′s interaction with participants and the evaluations of medical students participating in the test were collected. Qualitative text analysis was used to evaluate the influence of different versions of the prompt lexemes on improving the effectiveness of ChatGPT-4 as a VSP. The results showed that ChatGPT-4 had a better understanding of narrative history, mastered the interaction skills of SP, and effectively evaluated the consultation. After adjustment and optimization, the prompts can adapt to various medical cases. Students rated their performance as excellent, especially if there was emotional interaction. The establishment of the prompt engineering provides a new way for medical teachers to easily create VSPs, and creates a chance for medical students to practice medical history taking.
[1] Maicher K, Danforth D, Price A, et al. Developing a conversational virtual standardized patient to enable students to practice history-taking skills[J]. Simul Healthc, 2017,12(2):124-131. DOI: 10.1097/SIH.0000000000000195.
[2] Plaksin J, Nicholson J, Kundrod S, et al. The benefits and risks of being a standardized patient: a narrative review of the literature[J]. Patient, 2016,9(1):15-25. DOI: 10.1007/s40271-015-0127-y.
[3] Bosse HM, Nickel M, Huwendiek S, et al. Cost-effectiveness of peer role play and standardized patients in undergraduate communication training[J]. BMC Med Educ, 2015,15:183. DOI: 10.1186/s12909-015-0468-1.
[4] Gillette C, Stanton RB, Rockich-Winston N, et al. Cost-effectiveness of using standardized patients to assess student-pharmacist communication skills[J]. Am J Pharm Educ, 2017,81(10):6120. DOI: 10.5688/ajpe6120.
[5] Stevens A, Hernandez J, Johnsen K, et al. The use of virtual patients to teach medical students history taking and communication skills[J]. Am J Surg, 2006,191(6):806-811. DOI: 10.1016/j.amjsurg.2006.03.002.
[6] Peng S, Wang D, Liang Y, et al. AI-ChatGPT/GPT-4: an booster for the development of physical medicine and rehabilitation in the new era![J]. Ann Biomed Eng, 2024,52(3):462-466. DOI: 10.1007/s10439-023-03314-x.
[7] Lee P, Bubeck S, Petro J. Benefits, limits, and risks of GPT-4 as an ai chatbot for medicine[J]. N Engl J Med, 2023,388(13):1233-1239. DOI: 10.1056/NEJMsr2214184.
[8] GPT-4 technical report[EB/OL].[2024-03-01] . https://arxiv.org/pdf/2303.08774.
[9] Kuroiwa T, Sarcon A, Ibara T, et al. The potential of chatgpt as a self-diagnostic tool in common orthopedic diseases: exploratory study[J]. J Med Internet Res, 2023,25:e47621. DOI: 10.2196/47621.
[10] Topsakal O, Akinci TC, Celikoyar M. Evaluating patient and otolaryngologist dialogues generated by ChatGPT, are they adequate?[EB/OL].(2023-04-02)[2024-03-01] .https://www.researchsquare.com/article/rs-2719379/latest.
[11] Luke N, Taneja R, Ban K, et al. Large language models (ChatGPT) in medical education: embrace or abjure?[J]. Asia Pacific Scholar, 2023, 8(4): 50-52. DOI:10.29060/TAPS.2023-8-4/PV3007.
[12] Safranek CW, Sidamon-Eristoff AE, Gilson A, et al. The role of large language models in medical education: applications and implications[J]. JMIR Med Educ, 2023,9:e50945. DOI: 10.2196/50945.
[13] Milota MM, van Thiel G, van Delden J. Narrative medicine as a medical education tool: a systematic review[J]. Med Teach, 2019,41(7):802-810. DOI: 10.1080/0142159X.2019.1584274.
[14] Melvin RS, May W, Narayanan SS, et al. Creation of a doctor-patient dialogue corpus using standardized patients[EB/OL].[2024-03-01] . http://sail.usc.edu/publications/files/belvinlrec2004.pdf.
[15] Bickley L, Szilagyi PG. Bates′ guide to physical examination and history-taking[M]. Philadelphia: Lippincott Williams & Wilkins, 2012:65-106.
[16] Eysenbach G. The role of ChatGPT, generative language models, and artificial intelligence in medical education: a conversation with chatgpt and a call for papers[J]. JMIR Med Educ, 2023,9:e46885. DOI: 10.2196/46885.
[17] Boulet JR, van Zanten M, de Champlain A, et al. Checklist content on a standardized patient assessment: an ex post facto review[J]. Adv Health Sci Educ Theory Pract, 2008,13(1):59-69. DOI: 10.1007/s10459-006-9024-4.
[18] Bokken L, Linssen T, Scherpbier A, et al. Feedback by simulated patients in undergraduate medical education: a systematic review of the literature[J]. Med Educ, 2009,43(3):202-210. DOI: 10.1111/j.1365-2923.2008.03268.x.
[19] Makoul G. Essential elements of communication in medical encounters: the Kalamazoo consensus statement[J]. Acad Med, 2001,76(4):390-393. DOI: 10.1097/00001888-200104000-00021.
[20] Erby LA, Roter DL, Biesecker BB. Examination of standardized patient performance: accuracy and consistency of six standardized patients over time[J]. Patient Educ Couns, 2011,85(2):194-200. DOI: 10.1016/j.pec.2010.10.005.
[21] Danforth DR, Procter M, Chen R,et al. Development of virtual patient simulations for medical education[J]. J Virt Worlds Res, 2009, 2(2):1-11. DOI:10.4101/JVWR.V2I2.707.
[22] Talbot T, Sagae K, John B,et al. Designing useful virtual standardized patient encounters[EB/OL].[2024-03-01] . https://ict.usc.edu/pubs/Designing%20Useful%20Virtual%20Standardized%20Patient%20Encounters.pdf.
[23] Maicher KR, Stiff A, Scholl M, et al. Artificial intelligence in virtual standardized patients: combining natural language understanding and rule based dialogue management to improve conversational fidelity[J]. Med Teach, 2022, 45(3):279-285. DOI: 10.1080/0142159X.2022.2130216.
[24] Vishal K.ChatGPT-4 turbo, all you need to know[EB/OL].(2023-11-11)[2024-03-01] . https://medium.com/@kumar.vishal9626/chatgpt-4-turbo-all-you-need-to-know-e141e644bcf4.
[25] Hasle JL, Anderson DS, Szerlip HM. Analysis of the costs and benefits of using standardized patients to help teach physical diagnosis[J]. Acad Med, 1994,69(7):567-570. DOI: 10.1097/00001888-199407000-00013.
[26] Bearman M, Cesnik B, Liddell M. Random comparison of ′virtual patient′ models in the context of teaching clinical communication skills[J]. Med Educ, 2001,35(9):824-832. DOI: 10.1046/j.1365-2923.2001.00999.x.
