Advancements in evaluating the effectiveness of virtual standardized patients

doi:10.3760/cma.j.cn115259-20240722-00770

Abstract

Abstract: Virtual standardized patients (VSPs), as computer-based programs capable of accurately simulating clinical scenarios, have become important educational tools in medical training. However, their effectiveness remains debated, underscoring the need for a systematic review of evaluation approaches. This study summarizes current progress in VSP effectiveness assessment across two major dimensions: learning effectiveness and system evaluation. Learning effectiveness assessment encompasses criterion-referenced evaluation, expert evaluation, and additional emerging methods, reflecting learners' training outcomes through quantitative indicators, professional judgments, experience-level differences, and physiological or behavioral responses. System evaluation focuses on technical design, simulation fidelity, and learning practicality, covering key indicators such as usability, authenticity, and sense of presence, and incorporates multiple assessment tools and methodologies. Existing research reveals limitations in current VSP evaluation, including methodological constraints, limited tool diversity, and insufficient longitudinal follow-up. Future work should develop multidimensional and more objective evaluation frameworks to enhance the scientific rigor and practical value of VSPs in medical education.

Key words: Patient simulation, Virtual standardized patient, Effectiveness, Evaluations, Review

CLC Number:

R-05

Yu Jialong, Li Yue, Qin Jianxing, Wu Junfang, Xu Yong. Advancements in evaluating the effectiveness of virtual standardized patients[J]. Chinese Journal of Medical Education, 2026, 46(1): 15-20.

References

[1] 汪国栋,李月,秦建星,等.虚拟标准化病人在精神医学教学中的应用与展望[J].神经疾病与精神卫生,2025,25(1):68-72.
[2] Poulton T, Balasubramaniam C. Virtual patients: a year of change[J]. Med Teach, 2011,33(11):933-937. DOI: 10.3109/0142159X.2011.613501.
[3] Jensen R, Musaeus P, Pedersen K. Virtual patients in undergraduate psychiatry education: a systematic review and synthesis[J]. Adv Health Sci Educ Theory Pract, 2024,29(1):329-347. DOI: 10.1007/s10459-023-10247-6.
[4] Lee J, Kim H, Kim KH, et al. Effective virtual patient simulators for medical communication training: a systematic review[J]. Med Educ, 2020,54(9):786-795. DOI: 10.1111/medu.14152.
[5] 林崇德.心理学大辞典(下卷)[M].上海:上海教育出版社,2003:1396.
[6] 李翔, 陈晓云, 张慧群, 等. 基于VSP的病史采集思维训练系统在问诊技能教学中的应用研究[J].中国医学教育技术,2023,37(2):186-190,201. DOI: 10.13566/j.cnki.cmet.cn61-1317/g4.202302013.
[7] Lebdai S, Mauget M, Cousseau P, et al. Improving academic performance in medical students using immersive virtual patient simulation: a randomized controlled trial[J]. J Surg Educ, 2021,78(2):478-484. DOI: 10.1016/j.jsurg.2020.08.031.
[8] Kava BR, Andrade AD, Marcovich R, et al. Communication skills assessment using human avatars: piloting a virtual world objective structured clinical examination[J]. Urol Pract, 2017,4(1):76-84. DOI: 10.1016/j.urpr.2016.01.006.
[9] Andrade AD, Bagri A, Zaw K, et al. Avatar-mediated training in the delivery of bad news in a virtual world[J]. J Palliat Med, 2010,13(12):1415-1419. DOI: 10.1089/jpm.2010.0108.
[10] Guetterman TC, Sakakibara R, Baireddy S, et al. Medical students' experiences and outcomes using a virtualhuman simulation to improve communication skills: mixed methods study[J]. J Med Internet Res, 2019,21(11):e15459. DOI: 10.2196/15459.
[11] Maicher KR, Zimmerman L, Wilcox B, et al. Using virtual standardized patients to accurately assess information gathering skills in medical students[J]. Med Teach, 2019,41(9):1053-1059. DOI: 10.1080/0142159X.2019.1616683.
[12] Bond WF, Mischler MJ, Lynch TJ, et al. The use of virtual standardized patients for practice in high value care[J]. Simul Healthc, 2023,18(3):147-154. DOI: 10.1097/SIH.0000000000000659.
[13] Oliven A, Nave R, Gilad D, et al. Implementation of a web-based interactive virtual patient case simulation as a training and assessment tool for medical students[J]. Stud Health Technol Inform, 2011,169:233-237.
[14] Kron FW, Fetters MD, Scerbo MW, et al. Using a computer simulation for teaching communication skills: a blinded multisite mixed methods randomized controlled trial[J]. Patient Educ Couns, 2017,100(4):748-759. DOI: 10.1016/j.pec.2016.10.024.
[15] Washburn M, Parrish DE, Bordnick PS. Virtual patient simulations for brief assessment of mental health disorders in integrated care settings[J]. Soc Work Ment Health, 2020,18(2):121-148. DOI: 10.1080/15332985.2017.1336743.
[16] Gilbert A, Carnell S, Lok B, etal. Using virtual patients to support empathy training in health care education: an exploratory study[J]. Simul Healthc, 2024,19(3):151-157. DOI: 10.1097/SIH.0000000000000742.
[17] Foster A, Chaudhary N, Murphy J, et al. The use of simulation to teach suicide risk assessment to health profession trainees-rationale, methodology, and a proof of concept demonstration with a virtual patient[J]. Acad Psychiatry,2015,39(6):620-629. DOI: 10.1007/s40596-014-0185-9.
[18] Borja-Hart NL, Spivey CA, George CM. Use of virtual patient software to assess student confidence and ability in communication skills and virtual patient impression: a mixed-methods approach[J]. Curr Pharm Teach Learn, 2019,11(7):710-718. DOI: 10.1016/j.cptl.2019.03.009.
[19] Dupuy L, Micoulaud-Franchi JA, Cassoudesalle H, et al. Evaluation of a virtual agent to train medical students conducting psychiatric interviews for diagnosing major depressive disorders[J]. J Affect Disord, 2020,263:1-8. DOI: 10.1016/j.jad.2019.11.117.
[20] Bond WF, Lynch TJ, Mischler MJ, et al. Virtual standardized patient simulation: case development and pilot application to high-value care[J]. Simul Healthc, 2019,14(4):241-250. DOI: 10.1097/SIH.0000000000000373.
[21] Buijs-Spanjers KR, Harmsen A, Hegge HH, et al. The influence of a serious game's narrative on students' attitudes and learning experiences regarding delirium: an interview study[J]. BMC Med Educ, 2020,20(1):289. DOI: 10.1186/s12909-020-02210-5.
[22] Deladisma AM, Cohen M, Stevens A, et al. Do medical students respond empathetically to a virtual patient?[J]. Am J Surg, 2007,193(6):756-760. DOI: 10.1016/j.amjsurg.2007.01.021.
[23] Wilkening GL, Gannon JM, Ross C, et al. Evaluation of branched-narrative virtual patients for interprofessional education of psychiatry residents[J]. Acad Psychiatry, 2017,41(1):71-75. DOI: 10.1007/s40596-016-0531-1.
[24] Rakofsky JJ, Talbot TB, Dunlop BW. A virtual standardized patient-based assessment tool to evaluate psychiatric residents' psychopharmacology proficiency[J]. Acad Psychiatry, 2020,44(6):693-700. DOI: 10.1007/s40596-020-01286-x.
[25] Cook DA, Brydges R, Ginsburg S, et al. A contemporary approach to validity arguments: a practical guide to Kane's framework[J]. Med Educ, 2015,49(6):560-575. DOI: 10.1111/medu.12678.
[26] O'Rourke SR, Branford KR, Brooks TL, et al. The emotional and behavioral impact of delivering bad news to virtual versus real standardized patients: a pilot study[J]. Teach Learn Med, 2020,32(2):139-149. DOI: 10.1080/10401334.2019.1652180.
[27] Real FJ, DeBlasio D, Beck AF, et al. A virtual reality curriculum for pediatric residents decreases rates of influenza vaccine refusal[J]. Acad Pediatr, 2017,17(4):431-435. DOI: 10.1016/j.acap.2017.01.010.
[28] Zeng M, Cai Y, Cao J, et al. The agreement between virtual patient and unannounced standardized patient assessments in evaluating primary health care quality: multicenter, cross-sectional pilot study in 7 provinces of China[J]. J Med Internet Res, 2022,24(12):e40082. DOI: 10.2196/40082.
[29] Wagner R, Alsbrooks K, Bacon E, et al. Clinician satisfaction and preference for central venous catheter systems promoting patient safety, ease-of-use and reduced clinician error[J]. Expert Rev Med Devices, 2023,20(7):607-614. DOI: 10.1080/17434440.2023.2219001.
[30] Liesa-Orús M, Latorre-Cosculluela C, Sierra-Sánchez V, et al. Links between ease of use, perceived usefulness and attitudes towards technology in older people in university: a structural equation modelling approach[J]. Educ Inf Technol (Dordr), 2023,28(3):2419-2436. DOI: 10.1007/s10639-022-11292-1.
[31] 黄盈盈, 潘绥铭. 社会学问卷调查的边界与限度——一个对“起点”的追问与反思[J].学术研究,2010(7):49-53. DOI: 10.3969/j.issn.1000-7326.2010.07.007.
[32] 代沁伶. 胶印机输墨系统响应时间的研究[J].包装工程,2008,29(10):146-147,210. DOI: 10.3969/j.issn.1001-3563.2008.10.050.
[33] Petrie H, Power C, Cairns P, et al. Using card sorts for understanding website information architectures: technological, methodological and cultural issues[J]. Lecture Notes in Computer Science, 2011, 6949: 309-322. DOI: 10.1007/978-3-642-23768-3_26.
[34] Wentzel J,Müller F, Beerlage-de Jong N, et al. Card sorting to evaluate the robustness of the information architecture of a protocol website[J]. Int J Med Inform, 2016,86:71-81. DOI: 10.1016/j.ijmedinf.2015.12.003.
[35] 朱思悦, 桂莉. 标准化病人评价工具和评价指标的研究进展[J]. 中华医学教育杂志, 2019, 39(4):305-309. DOI:10.3760/cma.j.issn.1673-677X.2019.04.016.
[36] Foster A, Chaudhary N, Kim T, et al. Using virtual patients to teach empathy: a randomized controlled study to enhance medical students' empathic communication[J]. Simul Healthc, 2016,11(3):181-189. DOI: 10.1097/SIH.0000000000000142.
[37] Witmer B, Singer M.Measuring presence in virtual environments: a presence questionnaire[J]. Presence,1998,7(3):225-240. DOI:10.1162/105474698565686.
[38] Nash R, Harvey T. Student nurse perceptions regarding learning transfer following high-fidelity simulation[J]. Clinical Simulation in Nursing, 2017, 13(10): 471-477. DOI:10.1016/j.ecns.2017.05.010.
[39] Consorti F, Mancuso R, Nocioni M, et al. Efficacy of virtual patients in medical education: a meta-analysis of randomized studies[J]. Computers & Education, 2012, 59(3): 1001-1008. DOI: 10.1016/j.compedu.2012.04.017.