Objective This study aimed to conduct a preliminary investigation and explore methods to improve the homogeneity of examiners in clinical skill examination of standardized resident training. Methods In July 2021, the concordance correlation coefficient (CCC) was used to analyze the consistency of examiner scorings for history taking and physical examination, clinical thinking, and skills in the objective structured clinical examination (OSCE) of 105 residents at Sun Yat-sen University. A subgroup of 44 examiners received training on detailed interpretation of the rules and key criteria of scoring, and the consistency of examiner scorings was assessed before and after the training. Results Of the 63 examination items, 3 items (4.8%) had a CCC>0.7, 31 items (49.2%) had a CCC of 0.3-0.7, and 29 items (46.0%) had a CCC<0.3. Among the 17 items in clinical thinking station, 14 items had a CCC≥0.3. For the 25 items of history taking of the same candidate, the number of items with consistent examiner scores ≥80.0% (35/44) increased from 8 (32.0%) to 19 (76.0%) after intensive training. Conclusions The homogeneity of examiners in clinical skill examination of standardized resident training needs to be improved. The homogeneity of the examiners can be improved through implementing training, such as detailed interpretation of the rules and key criteria of scoring.
Huang Xitai
,
Zhang Kunsong
,
Feng Shaoting
,
Huang Chensong
,
Yin Xiaoyu
,
Lai Jiaming
,
Chen Liuhua
. The investigation and analysis of examiners′ homogeneity in clinical skill examination of standardized resident training[J]. Chinese Journal of Medical Education, 2024
, 44(2)
: 151
-154
.
DOI: 10.3760/cma.j.cn115259-20230210-00118
[1] 曾庆奇, 刘婧, 陈声宇, 等. 规范化培训对住院医师胜任力影响的研究[J]. 中华医学教育杂志, 2022,42(5):468-472. DOI: 10.3760/cma.j.cn115259-20210915-01163.
[2] Szasz P, Louridas M, de Montbrun S, et al. Consensus-based training and assessment model for general surgery[J]. Br J Surg, 2016,103(6):763-771. DOI: 10.1002/bjs.10103.
[3] 潘赛英, 周汝云, 陈宏翔, 等. 形成性评价在住院医师核心胜任力培养中的应用探索[J]. 中华医学教育杂志, 2023,43(6):467-470. DOI: 10.3760/cma.j.cn115259-20220906-01119.
[4] Pallari E, Khadjesari Z, Aceituno D, et al. Implementation and evaluation of quality improvement training in surgery: a systematic review[J]. Ann Surg, 2021,274(6):e489-e506. DOI: 10.1097/SLA.0000000000004751.
[5] 江玲, 张阳, 俞美英, 等. 浙江省住院医师规范化培训结业考核结果分析[J]. 浙江医学, 2021,43(20):2269-2274. DOI: 10.12056/j.issn.1006-2785.2021.43.20.2021-1676.
[6] 张丹丹, 金镇, 常青, 等. 2020年辽宁省妇产科住院医师规范化培训结业实践技能考核结果分析[J]. 中华医学教育杂志, 2021,41(11):1021-1025. DOI: 10.3760/cma.j.cn115259-20210303-00279.
[7] 陈建胜, 沈建箴. 临床技能多层次考试平台考官培训与管理模式探索[J]. 中国高等医学教育, 2018(10):38-39,91. DOI: 10.3969/j.issn.1002-1701.2018.10.019.
[8] 薛华丹, 张大明, 孙昊, 等. 北京市放射科住院医师规范化培训Ⅰ阶段考试中考官对客观结构化临床考试应用反馈[J]. 基础医学与临床, 2018,38(11):1669-1672. DOI: 10.3969/j.issn.1001-6325.2018.11.036.
[9] 邵珍珍. 浙江省住院医师规范化培训结业考核结果分析[D]. 杭州:浙江大学, 2018.
[10] Foster N, Price M, Bettger J P, et al. Objective test scores throughout orthopedic surgery residency suggest disparities in training experience[J]. J Surg Educ, 2021,78(5):1400-1405. DOI: 10.1016/j.jsurg.2021.01.003.
[11] Lin L I. A concordance correlation coefficient to evaluate reproducibility[J]. Biometrics, 1989,45(1):255-268.
[12] 代青. 绩效考核定量指标与定性指标的适用性分析[J]. 人力资源管理, 2016(5):68-69. DOI: 10.3969/j.issn.1673-8209.2016.05.057.
[13] 何文盛, 王焱, 蔡明君. 政府绩效评估结果偏差探析:基于一种三维视角[J].中国行政管理,2013(1):50-54. DOI: 10.3782/j.issn.1006-0863.2013.01.07.
