目的 探讨数字化结果评估系统(以下简称评估系统)和传统评分方法在后牙Ⅱ类洞预备评价的一致性,评价评估系统在医学生临床前技能训练中的应用价值。方法 选取南京医科大学2017级口腔医学专业5+3一体化38名学生为研究对象,在其后牙Ⅱ类洞预备的考核中,采用评估系统和3名考官进行评价。采用Kruskal-Wallis检验比较4组评价结果的差异,采用组内相关系数(intraclass correlation coefficients,ICC)评价评估系统和3名考官之间的一致性。结果 评估系统评分为80.05(76.33,80.90)分,考官1评分为79.50(75.75,80.00)分,考官2评分为82.00(78.00,83.00)分,考官3评分为81.00(76.00, 83.00)分。4组评价结果总体存在差异(H=12.90,P=0.005)。考官1评分低于考官2,其差异具有统计学意义(P=0.011);其余组评分比较,其差异均无统计学意义(均P＞0.05)。评估系统和3名考官评价结果的ICC值均大于0.90,一致性极好。结论 评估系统与考官评价结果具有较好的一致性,综合运用评估系统能降低传统评分方法的差异,有助于提高学生评价的准确性。
Objective To explore the consistency of digital outcome assessment system and traditional evaluation method in the evaluation of posterior tooth classⅡ cavity preparation, and to evaluate its application value in preclinical skill training of medical students. Methods Thirty-eight medical students from 5+3 integrated program enrolled in 2017 at stomatological college of Nanjing Medical University were selected to participate in the examination of class II cavities preparation . The prepared teeth were evaluated by digital outcome assessment system and three examiners. The Kruskal-Wallis test was used to compare the differences of scores from the four groups, and the intraclass correlation coefficients (ICC) was used to evaluate consistencies between digital outcome assessment system and examiners. Results Scores from digital outcome assessment system were 80.05(76.33,80.90) , examiner 1 were 79.50(75.75,80.00) , examiner 2 were 82.00(78.00,83.00) ,and examiner 3 were 81.00(76.00, 83.00) .There were significant differences in the evaluation results of the four groups (H=12.90, P=0.005). Scores in examiner 1 were lower than that of examiner 2, the difference was statistically significant (P=0.011); there were no significant differences in evaluation results among the other groups (all P＞0.05). The ICC values of digital outcome assessment system and the results from three examiners were all greater than 0.90, which showing excellent agreement. Conclusions There were strong consistencies between digital outcome assessment system and examiners in evaluation results. The comprehensive application of the digital outcome assessment system can reduce the individual differences by traditional scoring methods, and help improve the accuracy of student evaluations.
[1] 朱锋, 吴丽, 杨洁, 等. 客观结构化临床考试在口腔住院医师规范化培训阶段考核中的实施与评价[J].中华医学教育杂志,2020,40(6):467-470. DOI: 10.3760/cma.j.cn115259-20190603-00432.
[2] 方远鹏, 左艳萍. 数字化虚拟口腔教学系统在口腔修复实验教学中的应用[J].全科口腔医学杂志(电子版),2020,7(5):83,97.
[3] 邱莹茜. 数字化虚拟系统在口腔修复实验教学中的应用[J].中国继续医学教育,2019,11(34):84-86. DOI: 10.3969/j.issn.1674-9308.2019.34.033.
[4] 刘琦, 项小珍, 梁悦娥, 等. 数字化结果评估系统在中切牙全瓷冠牙体预备教学中的应用研究[J].中华老年口腔医学杂志,2019,17(3):169-173. DOI: 10.19749/j.cn.cjgd.1672-2973.2019.03.010.
[5] 姚瑶, 黎尚荣, 袁联雄. 原位模拟和引导性反馈在住院医师规范化培训小讲课督导培训中的应用[J].中华医学教育杂志,2021,41(12):1081-1085. DOI: 10.3760/cma.j.cn115259-20210406-00455.
[6] Wolgin M, Frank W, Kielbassa AM. Development of an analytical prepCheck-supported approach to evaluate tutor-based assessments of dental students' practical skills[J]. Int J Comput Dent, 2018,21(4):313-322.
[7] 左艳萍, 崔晓明, 马新扬, 等. 虚拟仿真系统在全冠牙体预备实验教学中的应用[J].口腔颌面修复学杂志,2021,22(3):202-205. DOI: 10.19748/j.cn.kqxf.1009-3761.2021.03.009.
[8] 于卫强, 吴玉琼, 苏庭舒, 等. 虚拟实时评估系统在口腔住院医师规范化培训口腔修复课程中的应用[J].中国毕业后医学教育,2020,4(6):527-530. DOI: 10.3969/j.issn.2096-4293.2020.06.012.
[9] 李雪铃, 杨凌, 吴淑仪, 等. 数字化口腔虚拟教学评估系统用于前牙全瓷冠牙体预备评分的初步评价[J].中华口腔医学研究杂志(电子版),2020,14(3):187-190. DOI: 10.3877/cma.j.issn.1674-1366.2020.03.010.
[10] 樊明文.牙体牙髓病学[M].4版.北京:人民卫生出版社, 2012:60-63.
[11] 乔舰. 组内相关系数的理论基础及建模应用[J]. 统计与信息论坛,2016,31(11):44-48. DOI:10.3969/j.issn.1007-3116.2016.11.008.
[12] 袁重阳, 王晓燕, 董艳梅, 等. 数字化虚拟仿真培训系统用于髓腔冠部预备临床前教学的效果评价[J].中华口腔医学杂志,2021,56(5):479-484. DOI: 10.3760/cma.j.cn112144-20200831-00485.
[13] 王艳红, 朱晓华, 李斯文, 等. 虚拟实验室Simodont系统用于口腔医学窝洞预备教学的效果评价[J].中华口腔医学研究杂志(电子版),2017,11(2):119-125. DOI: 10.3877/cma.j.issn.1674-1366.2017.02.010.
[14] 李家,史翠平,林开利,等. 数字化虚拟口腔教学系统在口腔修复实验教学中的应用与探索[J]. 科技创新导报,2017,14(1):175-176. DOI:10.16660/j.cnki.1674-098X.2017.01.175.
[15] 王东苗, 贡艳宏, 陆晓庆, 等. 数字化虚拟仿真牙医培训系统在口腔临床前技能训练中的应用[J].高校医学教学研究(电子版),2019,9(1):25-30. DOI: 10.3868/j.issn.2095-1582.2019.01.006.
[16] 林育华, 池政兵. 两种虚拟操作系统在口腔临床前实训中的应用评价[J].口腔材料器械杂志,2017,26(3):155-158. DOI: 10.11752/j.kqcl.2017.03.09.
[17] 曹勇, 杨学英, 何灏逾, 等. 数字化过程评估系统在牙体预备教学中的应用评价[J].中华口腔医学研究杂志(电子版),2017,11(1):53-57. DOI: 10.3877/cma.j.issn.1674-1366.2017.01.011.
[18] Tang L, Cao Y, Liu Z, et al. Improving the quality of preclinical simulation training for dental students using a new digital real-time evaluation system[J]. Eur J Dent Educ, 2021,25(1):100-107. DOI: 10.1111/eje.12580.
