目的 研究住院医师规范化培训(简称住培)结业临床实践能力考核中考官评分的同质性,探索提高考官评分同质化的方法。方法 2021年7月,使用一致性相关系数(concordance correlation coefficient,CCC)对中山大学105名住培医师客观结构化临床考试中的病史采集和体格检查、临床思维、技能操作的考官评分进行一致性分析。选取44名考官进行详细解读评分细则与评分要点等相关培训,并评估培训前后考官评分的一致性情况。结果 在63个考核项目中,3个(4.8%)考核项目的CCC>0.7,31个(49.2%)考核项目的CCC为0.3~0.7,29个(46.0%)考核项目的CCC＜0.3。临床思维的17个考核项目中,有14个考核项目的CCC≥0.3。对同一名考生病史采集的25个考核项目,考官评分一致性≥80.0%(35/44)的项目从培训前的8项(32.0%)增加到培训后的19项(76.0%)。结论 住培结业临床实践能力考核中考官评分的同质性有待提高,通过详细解读评分细则与评分要点等相关培训可以在一定程度上提高考官评分的同质性。
Objective This study aimed to conduct a preliminary investigation and explore methods to improve the homogeneity of examiners in clinical skill examination of standardized resident training. Methods In July 2021, the concordance correlation coefficient (CCC) was used to analyze the consistency of examiner scorings for history taking and physical examination, clinical thinking, and skills in the objective structured clinical examination (OSCE) of 105 residents at Sun Yat-sen University. A subgroup of 44 examiners received training on detailed interpretation of the rules and key criteria of scoring, and the consistency of examiner scorings was assessed before and after the training. Results Of the 63 examination items, 3 items (4.8%) had a CCC>0.7, 31 items (49.2%) had a CCC of 0.3-0.7, and 29 items (46.0%) had a CCC<0.3. Among the 17 items in clinical thinking station, 14 items had a CCC≥0.3. For the 25 items of history taking of the same candidate, the number of items with consistent examiner scores ≥80.0% (35/44) increased from 8 (32.0%) to 19 (76.0%) after intensive training. Conclusions The homogeneity of examiners in clinical skill examination of standardized resident training needs to be improved. The homogeneity of the examiners can be improved through implementing training, such as detailed interpretation of the rules and key criteria of scoring.
[1] 曾庆奇, 刘婧, 陈声宇, 等. 规范化培训对住院医师胜任力影响的研究[J]. 中华医学教育杂志, 2022,42(5):468-472. DOI: 10.3760/cma.j.cn115259-20210915-01163.
[2] Szasz P, Louridas M, de Montbrun S, et al. Consensus-based training and assessment model for general surgery[J]. Br J Surg, 2016,103(6):763-771. DOI: 10.1002/bjs.10103.
[3] 潘赛英, 周汝云, 陈宏翔, 等. 形成性评价在住院医师核心胜任力培养中的应用探索[J]. 中华医学教育杂志, 2023,43(6):467-470. DOI: 10.3760/cma.j.cn115259-20220906-01119.
[4] Pallari E, Khadjesari Z, Aceituno D, et al. Implementation and evaluation of quality improvement training in surgery: a systematic review[J]. Ann Surg, 2021,274(6):e489-e506. DOI: 10.1097/SLA.0000000000004751.
[5] 江玲, 张阳, 俞美英, 等. 浙江省住院医师规范化培训结业考核结果分析[J]. 浙江医学, 2021,43(20):2269-2274. DOI: 10.12056/j.issn.1006-2785.2021.43.20.2021-1676.
[6] 张丹丹, 金镇, 常青, 等. 2020年辽宁省妇产科住院医师规范化培训结业实践技能考核结果分析[J]. 中华医学教育杂志, 2021,41(11):1021-1025. DOI: 10.3760/cma.j.cn115259-20210303-00279.
[7] 陈建胜, 沈建箴. 临床技能多层次考试平台考官培训与管理模式探索[J]. 中国高等医学教育, 2018(10):38-39,91. DOI: 10.3969/j.issn.1002-1701.2018.10.019.
[8] 薛华丹, 张大明, 孙昊, 等. 北京市放射科住院医师规范化培训Ⅰ阶段考试中考官对客观结构化临床考试应用反馈[J]. 基础医学与临床, 2018,38(11):1669-1672. DOI: 10.3969/j.issn.1001-6325.2018.11.036.
[9] 邵珍珍. 浙江省住院医师规范化培训结业考核结果分析[D]. 杭州:浙江大学, 2018.
[10] Foster N, Price M, Bettger J P, et al. Objective test scores throughout orthopedic surgery residency suggest disparities in training experience[J]. J Surg Educ, 2021,78(5):1400-1405. DOI: 10.1016/j.jsurg.2021.01.003.
[11] Lin L I. A concordance correlation coefficient to evaluate reproducibility[J]. Biometrics, 1989,45(1):255-268.
[12] 代青. 绩效考核定量指标与定性指标的适用性分析[J]. 人力资源管理, 2016(5):68-69. DOI: 10.3969/j.issn.1673-8209.2016.05.057.
[13] 何文盛, 王焱, 蔡明君. 政府绩效评估结果偏差探析:基于一种三维视角[J].中国行政管理,2013(1):50-54. DOI: 10.3782/j.issn.1006-0863.2013.01.07.
