目的 对临床思维能力测评(clinical thinking ability assessment,CTA)系统在医师资格考试临床类别分阶段考试实证研究中的评分准确性进行分析,为CTA系统的后续研究提供参考。方法 采用简单随机抽样法,选取参加2018年CTA系统测试的35名考生作为研究对象,邀请三级甲等医院副高级及以上职称的8位医师作为临床专家,对35名考生的临床思维能力进行等级评价,通过CTA系统评分与临床专家等级评价的Pearson相关系数和概化理论(generalizability theory, GT)对CTA系统评分和专家等级评价的一致性进行分析。结果 CTA系统在信息收集站、病情分析站、临床诊疗站和动态决策站的评分与临床专家等级评价的Pearson相关系数分别为0.824、0.697、0.856、0.815;GT分析得出的概化系数分别为0.879、0.904、0.879、0.878,可靠性指数分别为0.674、0.863、0.869、0.797。结论 CTA系统评分与临床专家等级评价的一致性较高。
Objective To analyzes scoring accuracy of clinical thinking ability assessment in the empirical study of National Medical Licensing Examination(NMLE) phased examination, so as to provide reference for the follow-up study of CTA system. Methods In 2018,a simple random sampling method was used to select 35 candidates who participated in the CTA test as the research object. Eight clinical doctors of position of senior or higher from the third class hospital were invited as clinical experts to evaluate the clinical thinking ability of 35 examinees, then the consistency between CTA system score and clinical expert grade evaluation is analyzed by the Pearson correlation coefficient and general theory (GT). Results Pearsonal correlation coefficient of CTA system and clinical expert rating in information collection, disease analysis, clinical diagnosis and treatment and dynamic decision making were 0.824, 0.697, 0.856 and 0.815; The generalizability coefficient of GT analysis were 0.879, 0.904, 0.879, 0.878, dependent index were 0.674, 0.863, 0.869, 0.797. Conclusions The evaluation system of CTA is consistent with clinical experts evaluation.
[1] 杨春晖, 朱亚鑫, 曲波. 计算机模拟病例在美国医学教育中的应用现状与进展[J].中华医学教育杂志,2018,38(4):634-637. DOI: 10.3760/cma.j.issn.1673-677X.2018.04.035.
[2] Sample L, Laduca T, Leung C, et al. Comparing patient-management skills of referred physicians and non-referred physicians on a computer-based case-simulation examination[J]. Acad Med, 2001,76(10 Suppl):S24-26. DOI: 10.1097/00001888-200110001-00009.
[3] Feinberg RA, Swygert KA, Haist SA, et al. The impact of postgraduate training on USMLE step 3 and its computer-based case simulation component[J]. J Gen Intern Med, 2012,27(1):65-70. DOI: 10.1007/s11606-011-1835-1.
[4] 左天明, 于晓松, 孙宝志. 医学计算机模拟病例考试在毕业综合考试中的初步应用[J].中国高等医学教育,2003(5):48-49,52. DOI: 10.3969/j.issn.1002-1701.2003.05.023.
[5] 徐晓锋, 刘勇. 评分者内部一致性的研究和应用[J].心理科学,2007,30(5):1175-1178.
[6] Cronbach LJ, Gleser GC, Rajaratnam N. Theory of generalizability: a liberalization of reliability theory[J]. Br J Math Stat Psychol, 1963, 16(2):137-163. DOI: 10.1111/j.2044-8317.1963.tb00206.x
[7] Reynolds CR, Bigler ED, Brennan RL, et al. Cognitive assessment: a multidisciplinary perspective [M].Boston: Springer,1994:175-207.
[8] 陈康.高考英语科写作新题型的概化理论研究[J].中国考试,2018(3):27-31.DOI:10.19360/j.cnki.11-3303/g4.2018.03.007.
[9] 杨志明,张雷. 用多元概化理论对普通话的测试[J].心理学报,2002,34(1):50-55.
[10] 孙晓敏,张厚粲. 表现性评价中评分者信度估计方法的比较研究——从相关法、百分比法到概化理论[J].心理科学, 2005, 28( 3): 646-649.
[11] 史润泽,李永刚,康晓凤.评分者信度测量在护理研究中的应用[J]. 护理学杂志,2017,32(19):110-112,封3.DOI:10.3870/ j.issn.1001 -4152.2017.19.110.
