Chinese Journal of Medical Education

Chinese Journal of Medical Education >

2022 , Vol. 42 >Issue 8: 753 - 756

DOI: https://doi.org/10.3760/cma.j.cn115259-20220107-00026

Estimation of decision consistency in National Medical Licensing Examination medical comprehensive examination (2016-2020)

  • Feng Pan ,
  • Zhang Quanhui ,
  • Luo Huiqin
Expand
  • 1Department of General Management, National Medical Examination Center, Beijing 100097, China;
    2Department of Information and Research, National Medical Examination Center, Beijing 100097, China;
    3Second Department of Item Development, National Medical Examination Center, Beijing 100097, China

Received date: 2022-01-07

  Online published: 2022-08-01

Fold

Abstract

Decision consistency (DC) is a key index in National Medical Licensing Examination (NMLE) validation. In this study, W-DC method was used to estimate DC index of medical comprehensive examination during 2016 to 2020. The results show that the lowest value of DC index coefficient is 0.826 7 and the highest one is 0.970 9. Among all types of examinations, the biggest variation between years is present in public health assistant physician test (0.144 2). The consistency of decision-making in medical comprehensive examination is good on the whole, and the examinee's actual test score is close to the qualified score, which contribute to the low DC index coefficient of individual examination. Our result provides evidence to support the interpretation and application of NMLE scores.

Key words： Validation; Decision consistency; National Medical Licensing Examination

Cite this article

Feng Pan , Zhang Quanhui , Luo Huiqin . Estimation of decision consistency in National Medical Licensing Examination medical comprehensive examination (2016-2020)[J]. Chinese Journal of Medical Education, 2022 , 42(8) : 753 -756 . DOI: 10.3760/cma.j.cn115259-20220107-00026

References

[1] American educational research association, American psychological association, national council on measurement in education. Standards for educational and psychological testing[M]. Washington, DC:American Educational Research Association, 2014:40.
[2] 韩宁.评价考试质量的新指标:决策一致性和决策准确性[J].中国考试,2008(6):3-6.
[3] 陈平,李珍,辛涛,等.标准参照测验决策一致性指标研究的总结与展望[J].心理发展与教育,2011,27(2):210-215.
[4] 赵世明.资格认证测验的分类一致性信度估计[J].中国考试,2006(10):30-34.
[5] 杜佳萱,陈平,辛涛.基于IRT的决策一致性系数在大规模教育测量中的应用[J].北京师范大学学报(自然科学版),2015,51(6):643-648.
[6] Livingston SA, Lewis C.Estimating the consistency and accuracy of classifications based on test scores[J]. J Educ Meas, 1995, 32(2): 179-197. DOI: 10.1111/j.1745-3984.1995.tb00462.x.
[7] Wolkowitz AA. A computationally simple method for estimating decision consistency[J]. J Educ Meas, 2021, 58(3): 388-412.DOI: 10.1111/jedm.12297.
[8] Subkoviak MJ. A practitioner's guide to computation and interpretation of reliability indices for mastery tests[J]. J Educ Meas, 1988, 25(1): 47-55. DOI: 10.1111/j.1745-3984.1988.tb00290.x.
Options
Outlines

/

〈 〉