Application of ultrasound simulator in assessing residents' skill of obstetric ultrasound scanning

doi:10.3760/cma.j.cn115259-20210114-00070

Abstract

Abstract: Objective To explore the feasibility of introducing ultrasound simulator test in assessing residents' obstetric ultrasound scanning skills. Methods An experimental comparative method was used in this study. In September 2019, 30 residents in the Department of Ultrasound of the Third Xiangya Hospital of Central South University were included in the study as examinees. They completed two runs of obstetric ultrasound scan skill assessments on volunteer pregnant women and on ultrasound simulator (P1, S1, S2, P2), respectively. The scores in volunteer pregnant women tests and simulator tests were compared, and the correlation between scores recorded in volunteer pregnant women tests and simulator tests was analyzed. The difficulty and differentiation degrees of tests were observed. The inter-rater reliability and test-retest reliability were evaluated by intraclass correlation coefficient (ICC). Results Scores from volunteer pregnant women test were P1= (50.39±9.71), P2= (53.39±10.63) and the scores from simulator test were S1= (54.41±9.52), S2= (57.51±9.76). Scores in S1 were higher than that of P1, and scores in S2 were higher than that of P2 (P<0.01). A positive correlation between the two sets of scores was found (r=0.784, P<0.01). The difficulty degrees of simulator test and volunteer pregnant women test were P_S1=0.54, P_S2=0.58, P_P1=0.50, P_P2=0.53 respectively and the differentiation degrees of simulator test and volunteer pregnant women test were D_S1=0.67, D_S2=0.65, D_P1=0.70, D_P2=0.72 respectively. The inter-rater reliability of simulator test and volunteer pregnant women test was high (simulator test: ICC=0.838, P<0.01;volunteer pregnant women test: ICC=0.762, P<0.01). The test-retest reliability of simulator test and volunteer pregnant women test was high (simulator test: ICC=0.878, P<0.01;volunteer pregnant women test: ICC=0.760, P<0.01). Conclusions Although there are differences between scores recorded in simulator test and volunteer pregnant women test, the simulator test is highly correlated with volunteer pregnant women test. The difficulty degree of simulator test is medium, the differentiation degree is high, and the reliability is high. Simulator test is expected to be an alternative to the test with volunteer pregnant women in assessing residents' skill of obstetric ultrasound scanning.

Key words: Simulator, Obstetric ultrasound scanning, Skill assessment, Residents

CLC Number:

Zhao Yongfeng, Tang Xiaohong, Peng Hong, Liu Wengang, Li Linlu, Yuan Le, Zhou Ping. Application of ultrasound simulator in assessing residents' skill of obstetric ultrasound scanning[J]. Chinese Journal of Medical Education, 2021, 41(7): 601-605.

References

[1] Chalouhi GE, Bernardi V, Ville Y. Ultrasound simulators in obstetrics and gynecology: state of the art[J]. Ultrasound Obstet Gynecol, 2015,46(3):255-263. DOI: 10.1002/uog.14707.
[2] ISUOG Education Committee recommendations for basic training in obstetric and gynecological ultrasound[J]. Ultrasound Obstet Gynecol, 2014,43(1):113-116. DOI: 10.1002/uog.13208.
[3] Damewood SC, Leo M, Bailitz J, et al. Tools for measuring clinical ultrasound competency: recommendations from the ultrasound competency work group[J]. AEM Educ Train, 2020,4(Suppl 1):S106-S112. DOI: 10.1002/aet2.10368.
[4] Tolsgaard MG, Ringsted C, Dreisler E, et al. Reliable and valid assessment of ultrasound operator competence in obstetrics and gynecology[J]. Ultrasound Obstet Gynecol, 2014,43(4):437-443. DOI: 10.1002/uog.13198.
[5] Abuhamad A, Minton KK, Benson CB, et al. Obstetric and gynecologic ultrasound curriculum and competency assessment in residency training programs: consensus report[J]. J Ultrasound Med, 2018,37(1):19-50. DOI: 10.1002/jum.14519.
[6] Østergaard ML, Konge L, Kahr N, et al. Four virtual-reality simulators for diagnostic abdominal ultrasound training in radiology[J]. Diagnostics (Basel), 2019,9(2):50.DOI: 10.3390/diagnostics9020050.
[7] American Educational Research Association, American Psychological Association, National Council On Measurement in Education. Standards for educational and psychological testing[M]. 2014 Edition. Washington, DC: American Educational Research Association, 2014:11-22.
[8] Salomon LJ, Alfirevic Z, Berghella V, et al. Practice guidelines for performance of the routine mid-trimester fetal ultrasound scan[J]. Ultrasound Obstet Gynecol, 2011,37(1):116-126. DOI: 10.1002/uog.8831.
[9] 中国医师协会超声医师分会. 中国产科超声检查指南[M]. 北京: 人民卫生出版社, 2019:12-30.
[10] 朱德全. 教育测量学[M]. 北京: 中国人民大学出版社, 2016:73-86.
[11] 孙振球. 医学统计学[M]. 2版. 北京: 人民卫生出版社, 2005:756-769.
[12] Chalouhi GE, Bernardi V, Gueneuc A, et al. Evaluation of trainees' ability to perform obstetrical ultrasound using simulation: challenges and opportunities[J]. Am J Obstet Gynecol, 2016,214(4):521-525. DOI: 10.1016/j.ajog.2015.10.932.
[13] Burden C, Preshaw J, White P, et al. Validation of virtual reality simulation for obstetric ultrasonography: a prospective cross-sectional study[J]. Simul Healthc, 2012,7(5):269-273. DOI: 10.1097/SIH.0b013e3182611844.
[14] Dyre L, Nørgaard LN, Tabor A, et al. Collecting validity evidence for the assessment of mastery learning in simulation-based ultrasound training[J]. Ultraschall Med, 2016,37(4):386-392. DOI: 10.1055/s-0041-107976.
[15] Pietersen PI, Konge L, Graumann O, et al. Developing and gathering validity evidence for a simulation-based test of competencies in lung ultrasound[J]. Respiration, 2019,97(4):329-336. DOI: 10.1159/000493758.