Exploration of a mutual evaluation model between teachers and students for professional skills training in rehabilitation therapy based on project response theory

doi:10.3760/cma.j.cn115259-20220624-00820

Abstract

Abstract: Objective Based on the item response theory(IRT), to build a model for the mutual evaluation between teachers and students in the professional skills training of rehabilitation therapy, and to verify the actual effect of the model for the mutual evaluation between teachers and students in the skills training. Methods To extract the examination results of the students majoring in rehabilitation therapy from grade 2015 to 2018 in the rehabilitation skills training management system of Sir Run Run Hospital affiliated to Nanjing Medical University, including the after-department examination, daily performance evaluation, case report performance, etc. The IRT model was established based on the evaluation items provided by the teacher assessment student (TAS, 13 items) questionnaire and the student assessment teacher (SAT, 12 items) questionnaire and each item is scored 0, 1, 3 and 5 points. The fitting, reliability and local independence of the model were verified through the result of TAS and SAT, and the correlation between the teaching capability, the students′ practical ability and the examination results is calculated. Results Both types of questionnaires were modeled after being converted into 2 category items. The 13 items of TAS can form a 1 factor 2 parameters model. The model has good fitness (P=0.206), reliability (r=0.853) and local independence. The level of students′ practical ability measured by the model is significantly related to the students′ performance in the examination (r=0.442,P<0.001). The 12 items of the SAT questionnaire can form a 1 factor 4 parameters model. The model has good fitness (P=0.237), reliability (r=0.952) and local independence. The teacher′s teaching level measured by the model is significantly related to the student′s performance in the examination (r=0.197, P=0.006). Conclusions On the basis of the 2 category items, the IRT model can be constructed to evaluate the professional-skills-teaching level of teachers and students′ practical ability. After verification, the model has good evaluation effect and can be considered for promotion and application.

Key words: Rehabilitation, Practical training, Item response theory, Teacher assessment, Student assessment, Build a model

CLC Number:

Yu Wei, Chen Qun, Geng Baofeng, Bian Jiajia, Jiang Zhongli, Shi Yawei, Lin Feng. Exploration of a mutual evaluation model between teachers and students for professional skills training in rehabilitation therapy based on project response theory[J]. Chinese Journal of Medical Education, 2023, 43(4): 316-320.

References

[1] Reubenson A, Ng L, Gucciardi DF. The assessment of physiotherapy practice tool provides informative assessments of clinical and professional dimensions of student performance in undergraduate placements: a longitudinal validity and reliability study[J]. J Physiother, 2020, 66(2):113-119. DOI: 10.1016/j.jphys.2020.03.009.
[2] Hu J, Jones A, Zhou X, et al. Acceptance of the ′assessment of physiotherapy practice (chinese)′ as a standardised evaluation of professional competency in chinese physiotherapy students: an observational study[J]. BMC Med Educ, 2020, 20(1):108. DOI: 10.1186/s12909-020-02026-3.
[3] Debelak R. An evaluation of overall goodness-of-fit tests for the rasch model[J]. Front Psychol, 2018, 9:2710. DOI: 10.3389/fpsyg.2018.02710.
[4] Xia Y, Yang Y. Rmsea, cfi, and TLI in structural equation modeling with ordered categorical data: the story they tell depends on the estimation methods[J]. Behav Res Methods, 2019, 51(1):409-428. DOI: 10.3758/s13428-018-1055-2.
[5] Akoglu H. User′s guide to correlation coefficients[J]. Turk J Emerg Med, 2018, 18(3):91-93. DOI: 10.1016/j.tjem.2018.08.001.
[6] O′Connor A, Tai A, Brinn M, et al. The acceptability of using augmented reality as a mechanism to engage children in asthma inhaler technique training: qualitative interview study with deductive thematic analysis[J]. JMIR Pediatr Parent, 2023, 6:e40231. DOI: 10.2196/40231.
[7] Choi Y J, Asilkalkan A. R Packages for item response theory analysis: descriptions and features [J]. Measurement, 2019, 17(3): 168-75. DOI: 10.1080/15366367.2019.1586404.
[8] Dalton M, Davidson M, Keating J. The Assessment of Physiotherapy Practice (APP) is a valid measure of professional competence of physiotherapy students: a cross-sectional study with Rasch analysis[J]. J Physiother. 2011;57(4):239-246. doi:10.1016/S1836-9553(11)70054-6
[9] Gorgon EJ, Basco MD, Manuel AT. Teaching evidence based practice in physical therapy in a developing country: a national survey of philippine schools[J]. BMC Med Educ, 2013, 13:154. DOI: 10.1186/1472-6920-13-154.
[10] Chen W, Reeves TC. Twelve tips for conducting educational design research in medical education[J]. Med Teach, 2020, 42(9):980-986. DOI: 10.1080/0142159X.2019.1657231.
[11] 李博杨, 吴惠群, 卢守四, 等. 我国康复机构人才培养现状调查与分析[J]. 中国医院, 2021, 25(8):12-13. DOI: 10.19660/j.issn.1671-0592.2021.8.04.

[1]	Duan Zhouying, Chen Wenhua, Sheng Yilan. Exploration on the applied talent training model of ″2+1+1″ in rehabilitation therapeutics [J]. Chinese Journal of Medical Education, 2023, 43(1): 5-9.
[2]	Zhang Quanhui, He Ju, Ren Jie, Zhang Ying, Lu Yan. A comparative study of equating methods applied in standardized competence test for clinical medicine undergraduates [J]. Chinese Journal of Medical Education, 2022, 42(7): 577-580.
[3]	Jing Xuebing, Wang Shuo, Han Qingkun, Fang Xiaojie, Wei Yulian, Qiao Juan, Cui Hanfei. Research progress and expectations on the application of multi-disciplinary team model in clinical nursing practice teaching [J]. Chinese Journal of Medical Education, 2022, 42(7): 651-655.
[4]	Niu Yuhong, Feng Caili, Feng Hao. Investigation and analysis on professional sense and employment of students majoring in rehabilitation in Shaanxi Province [J]. Chinese Journal of Medical Education, 2022, 42(6): 485-489.
[5]	Cao Zhenyu, Wang Lei, Tu Yue, Lin Feng. Exploring the key points and difficulties of therapeutic exercise course based on knowledge graph and item response theory [J]. Chinese Journal of Medical Education, 2022, 42(12): 1083-1088.
[6]	Jiang Ting, Xiang Wei, Shi Feng, Gong Yu, Guan Weichang, Liao Weijing. Survey on the interest in and influencing factors of cognitive rehabilitation for interns from rehabilitation therapeutics [J]. Chinese Journal of Medical Education, 2022, 42(12): 1123-1127.
[7]	Lu Weiwei, Zhang Jian, Chen Jun, Liu Guanghua, Huang Yiming, Yu Qing. Current status and influencing factors of satisfaction with clinical teaching among interns in rehabilitation therapy-related majors [J]. Chinese Journal of Medical Education, 2022, 42(11): 1024-1027.
[8]	Ma Yubao, Guo Qingcheng, Zhang Lihua, Ma Xinxin, He Yu, Liu Tiejun. A contrastive study on cultivating models for undergraduates in rehabilitation therapeutics in China and Japan [J]. Chinese Journal of Medical Education, 2022, 42(10): 951-955.
[9]	Jiang Xiaocui, Liu Zhen, Lu Fei, Zhao Qin, Su Qinglun. Application of problem-based learning combined with case three-dimensional teaching method in skill training for students majoring in rehabilitation therapy [J]. Chinese Journal of Medical Education, 2022, 42(1): 42-45.
[10]	Meng Lijiao, Ge Yanlei, Wei Qingchuan, Peng Panxing, Pang Xiaofan, Yang Yonghong, Yang Lin. Research on the curriculum construction standard of undergraduate pediatic rehabilitation [J]. Chinese Journal of Medical Education, 2022, 42(1): 60-64.
[11]	Zhao Xiaoyi, Shu Xuan, Chen Zhengwei, Chen Lixia. Application of interactive case teaching in after-class learning of rehabilitation medicine for eight-year program medical students [J]. Chinese Journal of Medical Education, 2021, 41(9): 814-817.
[12]	Liu Ying, Shi Mingnan, Zhao Wowa, Chen Lixia. Analysis of current situation of work and training of hemophilia rehabilitation practitioners [J]. Chinese Journal of Medical Education, 2021, 41(4): 297-300.
[13]	Zhou Chaosheng, Tang Weizhen, Xu Jie, Xu Gang. Application of metacognition and SOAP assessment in clinical practice of rehabilitation therapy students [J]. Chinese Journal of Medical Education, 2020, 40(3): 225-228.
[14]	Pan Yanxia, Gao Songling, Wang Shizhong, Zeng Yufang, Lin Cheng, Xu Guiqing, Pei Jun. Countermeasure of talent demand and professional construction of sports rehabilitation speciality [J]. Chinese Journal of Medical Education, 2019, 39(9): 647-651.
[15]	Lyu Ying, Wang Fengqing, Wang Qing, Xu Ming, Duan Liping. Current situation and training pathway exploration for high-level staff in rehabilitation therapy [J]. Chinese Journal of Medical Education, 2019, 39(3): 207-210.