目的 探讨采用模拟器考核评估住院医师Ⅲ级产科超声检查技能的可行性。方法 本研究采用试验对照方法。2019年9月,选取中南大学湘雅三医院超声科30名住院医师作为研究对象,在志愿者孕妇和模拟器上各进行2次Ⅲ级产科超声检查技能考核。比较志愿者孕妇考核与模拟器考核的成绩,分析志愿者孕妇考核与模拟器考核成绩的相关性,观察考核的难度与区分度,通过组内相关系数(intraclass correlation coefficient,ICC)评估评分者之间信度与重测信度。结果 考生的2次志愿者孕妇考核成绩分别为P1=(50.39±9.71)分,P2=(53.39±10.63)分,2次模拟器考核成绩分别为S1=(54.41±9.52)分,S2=(57.51±9.76)分,S1高于P1,S2高于P2,其差异均具有统计学意义(均P<0.01)。模拟器考核与志愿者孕妇考核成绩呈正相关(r=0.784,P＜0.01)。模拟器考核与志愿者孕妇考核的难度为:PS1=0.54,PS2=0.58,PP1=0.50,PP2=0.53,区分度为DS1=0.67,DS2=0.65,DP1=0.70,DP2=0.72。模拟器考核与志愿者孕妇考核均具有良好的评分者之间信度(模拟器考核ICC=0.838,P＜0.01;志愿者孕妇考核ICC=0.762,P＜0.01)和重测信度(模拟器考核ICC=0.878,P＜0.01;志愿者孕妇考核ICC=0.760,P＜0.01)。结论 模拟器考核与志愿者孕妇考核成绩虽然存在差异,但二者具有高度的相关性,且模拟器考核难度适中、区分度高、信度良好,有望作为传统志愿者孕妇考核方式的替代方案评估住院医师的Ⅲ级产科超声检查技能。
Objective To explore the feasibility of introducing ultrasound simulator test in assessing residents' obstetric ultrasound scanning skills. Methods An experimental comparative method was used in this study. In September 2019, 30 residents in the Department of Ultrasound of the Third Xiangya Hospital of Central South University were included in the study as examinees. They completed two runs of obstetric ultrasound scan skill assessments on volunteer pregnant women and on ultrasound simulator (P1, S1, S2, P2), respectively. The scores in volunteer pregnant women tests and simulator tests were compared, and the correlation between scores recorded in volunteer pregnant women tests and simulator tests was analyzed. The difficulty and differentiation degrees of tests were observed. The inter-rater reliability and test-retest reliability were evaluated by intraclass correlation coefficient (ICC). Results Scores from volunteer pregnant women test were P1= (50.39±9.71), P2= (53.39±10.63) and the scores from simulator test were S1= (54.41±9.52), S2= (57.51±9.76). Scores in S1 were higher than that of P1, and scores in S2 were higher than that of P2 (P<0.01). A positive correlation between the two sets of scores was found (r=0.784, P<0.01). The difficulty degrees of simulator test and volunteer pregnant women test were PS1=0.54, PS2=0.58, PP1=0.50, PP2=0.53 respectively and the differentiation degrees of simulator test and volunteer pregnant women test were DS1=0.67, DS2=0.65, DP1=0.70, DP2=0.72 respectively. The inter-rater reliability of simulator test and volunteer pregnant women test was high (simulator test: ICC=0.838, P<0.01;volunteer pregnant women test: ICC=0.762, P<0.01). The test-retest reliability of simulator test and volunteer pregnant women test was high (simulator test: ICC=0.878, P<0.01;volunteer pregnant women test: ICC=0.760, P<0.01). Conclusions Although there are differences between scores recorded in simulator test and volunteer pregnant women test, the simulator test is highly correlated with volunteer pregnant women test. The difficulty degree of simulator test is medium, the differentiation degree is high, and the reliability is high. Simulator test is expected to be an alternative to the test with volunteer pregnant women in assessing residents' skill of obstetric ultrasound scanning.
[1] Chalouhi GE, Bernardi V, Ville Y. Ultrasound simulators in obstetrics and gynecology: state of the art[J]. Ultrasound Obstet Gynecol, 2015,46(3):255-263. DOI: 10.1002/uog.14707.
[2] ISUOG Education Committee recommendations for basic training in obstetric and gynecological ultrasound[J]. Ultrasound Obstet Gynecol, 2014,43(1):113-116. DOI: 10.1002/uog.13208.
[3] Damewood SC, Leo M, Bailitz J, et al. Tools for measuring clinical ultrasound competency: recommendations from the ultrasound competency work group[J]. AEM Educ Train, 2020,4(Suppl 1):S106-S112. DOI: 10.1002/aet2.10368.
[4] Tolsgaard MG, Ringsted C, Dreisler E, et al. Reliable and valid assessment of ultrasound operator competence in obstetrics and gynecology[J]. Ultrasound Obstet Gynecol, 2014,43(4):437-443. DOI: 10.1002/uog.13198.
[5] Abuhamad A, Minton KK, Benson CB, et al. Obstetric and gynecologic ultrasound curriculum and competency assessment in residency training programs: consensus report[J]. J Ultrasound Med, 2018,37(1):19-50. DOI: 10.1002/jum.14519.
[6] Østergaard ML, Konge L, Kahr N, et al. Four virtual-reality simulators for diagnostic abdominal ultrasound training in radiology[J]. Diagnostics (Basel), 2019,9(2):50.DOI: 10.3390/diagnostics9020050.
[7] American Educational Research Association, American Psychological Association, National Council On Measurement in Education. Standards for educational and psychological testing[M]. 2014 Edition. Washington, DC: American Educational Research Association, 2014:11-22.
[8] Salomon LJ, Alfirevic Z, Berghella V, et al. Practice guidelines for performance of the routine mid-trimester fetal ultrasound scan[J]. Ultrasound Obstet Gynecol, 2011,37(1):116-126. DOI: 10.1002/uog.8831.
[9] 中国医师协会超声医师分会. 中国产科超声检查指南[M]. 北京: 人民卫生出版社, 2019:12-30.
[10] 朱德全. 教育测量学[M]. 北京: 中国人民大学出版社, 2016:73-86.
[11] 孙振球. 医学统计学[M]. 2版. 北京: 人民卫生出版社, 2005:756-769.
[12] Chalouhi GE, Bernardi V, Gueneuc A, et al. Evaluation of trainees' ability to perform obstetrical ultrasound using simulation: challenges and opportunities[J]. Am J Obstet Gynecol, 2016,214(4):521-525. DOI: 10.1016/j.ajog.2015.10.932.
[13] Burden C, Preshaw J, White P, et al. Validation of virtual reality simulation for obstetric ultrasonography: a prospective cross-sectional study[J]. Simul Healthc, 2012,7(5):269-273. DOI: 10.1097/SIH.0b013e3182611844.
[14] Dyre L, Nørgaard LN, Tabor A, et al. Collecting validity evidence for the assessment of mastery learning in simulation-based ultrasound training[J]. Ultraschall Med, 2016,37(4):386-392. DOI: 10.1055/s-0041-107976.
[15] Pietersen PI, Konge L, Graumann O, et al. Developing and gathering validity evidence for a simulation-based test of competencies in lung ultrasound[J]. Respiration, 2019,97(4):329-336. DOI: 10.1159/000493758.
