目的 基于项目反应理论为康复治疗专业技能培训师生互评构建模型,并验证该模型用于技能培训师生互评的实际效果。方法 提取南京医科大学附属逸夫医院康复技能培训管理系统中2015至2018级康复治疗专业学生的出科考核、平时成绩、病例报告等考试成绩作为研究资料。以教师考核学生(考学,13项)问卷和学生评价教师(评师,12项)问卷作为评测条目,各条目评分为0、1、3和5分,由此建立项目反应理论模型。通过对师生互评结果分析验证所得模型的拟合度、信度和局部独立性,并测算教师的教学水平、学生的实践能力与出科考核成绩的相关性。结果 两类问卷均在转置为二分类项目后完成建模。考学问卷的13个项目可构成单因素2参数模型,该模型拟合度(P=0.206)、信度(r＝0.853)和局部独立性良好,且模型测算的学生实践能力与出科考核成绩显著相关(r=0.442,P<0.001)。评师问卷的12个项目可构成单因素4参数模型,该模型拟合度(P=0.237)、信度(r=0.952)和局部独立性良好,且模型测算的教师教学水平与学生出科考核成绩显著相关(r=0.197,P=0.006)。结论 在二分类项目基础上,可以构建出适用于评价康复治疗专业教师教学水平、学生实践能力的项目反应理论模型。经验证,该模型评测效果良好,可考虑推广应用。
Objective Based on the item response theory(IRT), to build a model for the mutual evaluation between teachers and students in the professional skills training of rehabilitation therapy, and to verify the actual effect of the model for the mutual evaluation between teachers and students in the skills training. Methods To extract the examination results of the students majoring in rehabilitation therapy from grade 2015 to 2018 in the rehabilitation skills training management system of Sir Run Run Hospital affiliated to Nanjing Medical University, including the after-department examination, daily performance evaluation, case report performance, etc. The IRT model was established based on the evaluation items provided by the teacher assessment student (TAS, 13 items) questionnaire and the student assessment teacher (SAT, 12 items) questionnaire and each item is scored 0, 1, 3 and 5 points. The fitting, reliability and local independence of the model were verified through the result of TAS and SAT, and the correlation between the teaching capability, the students′ practical ability and the examination results is calculated. Results Both types of questionnaires were modeled after being converted into 2 category items. The 13 items of TAS can form a 1 factor 2 parameters model. The model has good fitness (P=0.206), reliability (r=0.853) and local independence. The level of students′ practical ability measured by the model is significantly related to the students′ performance in the examination (r=0.442,P<0.001). The 12 items of the SAT questionnaire can form a 1 factor 4 parameters model. The model has good fitness (P=0.237), reliability (r=0.952) and local independence. The teacher′s teaching level measured by the model is significantly related to the student′s performance in the examination (r=0.197, P=0.006). Conclusions On the basis of the 2 category items, the IRT model can be constructed to evaluate the professional-skills-teaching level of teachers and students′ practical ability. After verification, the model has good evaluation effect and can be considered for promotion and application.
[1] Reubenson A, Ng L, Gucciardi DF. The assessment of physiotherapy practice tool provides informative assessments of clinical and professional dimensions of student performance in undergraduate placements: a longitudinal validity and reliability study[J]. J Physiother, 2020, 66(2):113-119. DOI: 10.1016/j.jphys.2020.03.009.
[2] Hu J, Jones A, Zhou X, et al. Acceptance of the ′assessment of physiotherapy practice (chinese)′ as a standardised evaluation of professional competency in chinese physiotherapy students: an observational study[J]. BMC Med Educ, 2020, 20(1):108. DOI: 10.1186/s12909-020-02026-3.
[3] Debelak R. An evaluation of overall goodness-of-fit tests for the rasch model[J]. Front Psychol, 2018, 9:2710. DOI: 10.3389/fpsyg.2018.02710.
[4] Xia Y, Yang Y. Rmsea, cfi, and TLI in structural equation modeling with ordered categorical data: the story they tell depends on the estimation methods[J]. Behav Res Methods, 2019, 51(1):409-428. DOI: 10.3758/s13428-018-1055-2.
[5] Akoglu H. User′s guide to correlation coefficients[J]. Turk J Emerg Med, 2018, 18(3):91-93. DOI: 10.1016/j.tjem.2018.08.001.
[6] O′Connor A, Tai A, Brinn M, et al. The acceptability of using augmented reality as a mechanism to engage children in asthma inhaler technique training: qualitative interview study with deductive thematic analysis[J]. JMIR Pediatr Parent, 2023, 6:e40231. DOI: 10.2196/40231.
[7] Choi Y J, Asilkalkan A. R Packages for item response theory analysis: descriptions and features [J]. Measurement, 2019, 17(3): 168-75. DOI: 10.1080/15366367.2019.1586404.
[8] Dalton M, Davidson M, Keating J. The Assessment of Physiotherapy Practice (APP) is a valid measure of professional competence of physiotherapy students: a cross-sectional study with Rasch analysis[J]. J Physiother. 2011;57(4):239-246. doi:10.1016/S1836-9553(11)70054-6
[9] Gorgon EJ, Basco MD, Manuel AT. Teaching evidence based practice in physical therapy in a developing country: a national survey of philippine schools[J]. BMC Med Educ, 2013, 13:154. DOI: 10.1186/1472-6920-13-154.
[10] Chen W, Reeves TC. Twelve tips for conducting educational design research in medical education[J]. Med Teach, 2020, 42(9):980-986. DOI: 10.1080/0142159X.2019.1657231.
[11] 李博杨, 吴惠群, 卢守四, 等. 我国康复机构人才培养现状调查与分析[J]. 中国医院, 2021, 25(8):12-13. DOI: 10.19660/j.issn.1671-0592.2021.8.04.
