Exploring the key points and difficulties of therapeutic exercise course based on knowledge graph and item response theory

doi:10.3760/cma.j.cn115259-20220530-00704

Abstract

Abstract: Objective Applying item response theory (IRT) and knowledge graph (KG) to explore the quantitative indicators for measuring the key points and difficults of medical professional courses, with a view to providing reference and example for promoting refined teaching management. Methods In January 2022, a knowledge point difficulty questionnaire was developed based on the Therapeutic Exercise course syllabus, and a knowledge point difficulty self-assessment survey was administered to 119 students in the 2019 class of rehabilitation therapy at Nanjing University of Chinese Medicine who had completed the course final examination. The students′ examination results and their self-assessment results of 61 course-specific knowledge points were extracted, and the Mokken scale analysis and Roche model construction were conducted based on IRT to assess the students′ learning ability and its correlation with the examination results, and to estimate the difficulty values of the knowledge points; the correlations of the knowledge points were quantified and measured with the help of KG technology, and the core structure of KG was extracted and analysed. Results Forty-two knowledge points were selected for the construction of the Roche model by the Mokken scale analysis, and the resulting model had good fit (P=0.065 for the fit test), reliability (Cronbach coefficient of 0.968) and validity (students′ test scores were weakly and significantly correlated with the learning ability values estimated by the model: P=0.014, r=0.23). The model fit was good for each knowledge difficulty (Bonferroni corrected p-values were >0.05). the core structure of the KG covered 61 knowledge points and visual analysis showed it to be presented as a hierarchical dominant structure with pivot nodes. Conclusions The combination of IRT model and KG technology can quantitatively measure students′ learning ability and the key points and difficults of their knowledge, providing a refined management tool for teaching medical professional courses.

Key words: Therapeutic exercise, Item response theory (IRT), Knowledge graph (KG), Teaching management

CLC Number:

Cao Zhenyu, Wang Lei, Tu Yue, Lin Feng. Exploring the key points and difficulties of therapeutic exercise course based on knowledge graph and item response theory[J]. Chinese Journal of Medical Education, 2022, 42(12): 1083-1088.

References

[1] Franz A, Oberst S, Peters H, et al. How do medical students learn conceptual knowledge? High-, moderate- and low-utility learning techniques and perceived learning difficulties[J]. BMC Med Educ, 2022,22(1):1-8. DOI: 10.1186/s12909-022-03283-0.
[2] Boone WJ, Staver JR, Yale MS. Rasch analysis in the human sciences[M]. Dordrecht: Springer Netherlands,2014:35-45.
[3] Neumann KL, Kopcha TJ. The use of schema theory in learning, design, and technology[J]. Tech Trends, 2018, 62(5): 429-431. DOI: 10.1007/s11528-018-0319-0.
[4] Koponen IT, Nousiainen M. Concept networks in learning: finding key concepts in learners′ representations of the interlinked structure of scientific knowledge[J]. J Complex Netw, 2014, 2(2): 187-202. DOI: 10.1093/comnet/cnu003.
[5] 陆泉, 谢祎玉, 陈静,等. 临床医学课程知识主题图谱构建研究[J].图书情报工作, 2019, 63(9): 101-108. DOI: 10.13266/j.issn.0252-3116.2019.09.011.
[6] 许嘉, 韦婷婷, 于戈, 等. 题目难度评估方法研究综述[J].计算机科学与探索,2022,16(4):734-759.
[7] Arifin WN, Yusoff MSB. Item Response Theory for Medical Educationists[J]. Edu Med J, 2017; 9(3):69-81. DOI: 10.21315/eimj2017.9.3.8.
[8] Brodin U, Fors U, Laksov KB. The application of item response theory on a teaching strategy profile questionnaire[J]. BMC Med Educ, 2010,10:14. DOI: 10.1186/1472-6920-10-14.
[9] Koopman L, Zijlstra B, van der Ark LA. A two-step, test-guided Mokken scale analysis, for nonclustered and clustered data[J]. Qual Life Res, 2022,31(1):25-36. DOI: 10.1007/s11136-021-02840-2.
[10] 袁淑莉, 何壮. 非参数项目反应理论模型——Mokken模型[J]. 贵阳学院学报(自然科学版), 2020, 15(4): 101-106. DOI: 10.16856/j.cnki.52-1142/n.2020.04.024.
[11] Stochl J, Jones PB, Croudace TJ. Mokken scale analysis of mental health and well-being questionnaire item responses: a non-parametric IRT method in empirical research for applied health researchers[J]. BMC Med Res Methodol, 2012,12:74. DOI: 10.1186/1471-2288-12-74.
[12] Straat JH, van der Ark LA, Sijtsma K. Using conditional association to identify locally independent item sets[J]. Methodology, 2016, 12(4): 117-123. DOI: 10.1027/1614-2241/a000115.
[13] Wouter DN,Andrej M,Vladimir B. 蜘蛛:社会网络分析技术[M].林枫,译.2版. 北京:世界图书出版公司,2014:108-110.
[14] 林枫, 江钟立. 基于《国际功能、残疾和健康分类(ICF)》的康复信息平台设计与实践初探[J]. 中国康复医学杂志, 2019, 34(2): 125-132. DOI:10.3969/j.issn.1001-1242.2019.02.002.
[15] R Core Team. R: a language and environment for statistical computing[EB/OL].(2022-3-10)[2022-3-26]. https://www.R-project.org.
[16] Sijtsma K, van der Ark LA. A tutorial on how to do a Mokken scale analysis on your test and questionnaire data[J]. Br J Math Stat Psychol, 2017,70(1):137-158. DOI: 10.1111/bmsp.12078.
[17] Rusch T, Mair P, Hatzinger R. Psychometrics with R: a review of CRAN packages for Item response theory[M]. Vienna: Wirtschafts University, 2013:3-27.
[18] PatiI I. Visualizations with statistical details: the ggstatsplot approach[J]. JOSS, 2021, 6(61): 3167. DOI: 10.21105/joss.03167.
[19] Williams, DR. Beyond Lasso: a survey of nonconvex regularization in Gaussian graphical models[J/OL].PsyArXiv, 2020 (2020-11-09).http://psyarxiv.com/ad57p. DOI:10.31234/osf.io/ad57p.[网络预发表].
[20] Csardi G, Nepusz T. The igraph software package for complex network research[J]. Int J Complex Syst, 2006,1695(5): 1-9.
[21] Hubert M, Vandervieren E. An adjusted boxplot for skewed distributions[J]. Comput Stat Data Anal, 2008,52(12): 5186-5201. DOI: 10.1016/j.csda.2007.11.008.
[22] Aliyu I, Kana AFD, Aliyu S. Development of knowledge graph for university courses management[J]. Int J Educ Manage Eng, 2020, 10(2): 1-10. DOI: 10.5815/ijeme.2020.02.01.