目的 应用项目反应理论(item response theory,IRT)和知识图谱(knowledge graph,KG)探析衡量医学专业课程重点与难点的量化指标,以期为促进精细化教学管理提供参考和借鉴。方法 2022年1月,基于运动疗法学课程大纲编制知识点难度问卷,对完成课程期末考试的南京中医药大学2019级康复治疗学专业119名学生进行知识点难度自评调查。提取学生考试成绩及其对61个课程专有知识点的自评结果,基于IRT进行摩肯量表分析和罗氏模型构建,评估学生的学习能力及其与考试成绩的相关性,并估算知识点的难度值;借助KG技术量化测评知识点的相关关系,并提取和分析KG核心结构。结果 摩肯量表分析筛选出42个知识点用于构建罗氏模型,所得模型具有良好的拟合度(拟合度检验P=0.065)、信度(克隆巴赫系数为0.968)和效度(学生的考试成绩与模型估算的学习能力值呈显著弱相关:P=0.014,r=0.23)。各知识点难度的模型拟合度良好(Bonferroni校正P值均>0.05)。KG的核心结构覆盖61个知识点,可视化分析结果显示其呈现为含有枢纽节点的层次支配结构。结论 IRT模型和KG技术相结合,可以量化测评学生的学习能力和知识的重点和难点,为医学专业课程的教学提供精细化管理工具。
Objective Applying item response theory (IRT) and knowledge graph (KG) to explore the quantitative indicators for measuring the key points and difficults of medical professional courses, with a view to providing reference and example for promoting refined teaching management. Methods In January 2022, a knowledge point difficulty questionnaire was developed based on the Therapeutic Exercise course syllabus, and a knowledge point difficulty self-assessment survey was administered to 119 students in the 2019 class of rehabilitation therapy at Nanjing University of Chinese Medicine who had completed the course final examination. The students′ examination results and their self-assessment results of 61 course-specific knowledge points were extracted, and the Mokken scale analysis and Roche model construction were conducted based on IRT to assess the students′ learning ability and its correlation with the examination results, and to estimate the difficulty values of the knowledge points; the correlations of the knowledge points were quantified and measured with the help of KG technology, and the core structure of KG was extracted and analysed. Results Forty-two knowledge points were selected for the construction of the Roche model by the Mokken scale analysis, and the resulting model had good fit (P=0.065 for the fit test), reliability (Cronbach coefficient of 0.968) and validity (students′ test scores were weakly and significantly correlated with the learning ability values estimated by the model: P=0.014, r=0.23). The model fit was good for each knowledge difficulty (Bonferroni corrected p-values were >0.05). the core structure of the KG covered 61 knowledge points and visual analysis showed it to be presented as a hierarchical dominant structure with pivot nodes. Conclusions The combination of IRT model and KG technology can quantitatively measure students′ learning ability and the key points and difficults of their knowledge, providing a refined management tool for teaching medical professional courses.
[1] Franz A, Oberst S, Peters H, et al. How do medical students learn conceptual knowledge? High-, moderate- and low-utility learning techniques and perceived learning difficulties[J]. BMC Med Educ, 2022,22(1):1-8. DOI: 10.1186/s12909-022-03283-0.
[2] Boone WJ, Staver JR, Yale MS. Rasch analysis in the human sciences[M]. Dordrecht: Springer Netherlands,2014:35-45.
[3] Neumann KL, Kopcha TJ. The use of schema theory in learning, design, and technology[J]. Tech Trends, 2018, 62(5): 429-431. DOI: 10.1007/s11528-018-0319-0.
[4] Koponen IT, Nousiainen M. Concept networks in learning: finding key concepts in learners′ representations of the interlinked structure of scientific knowledge[J]. J Complex Netw, 2014, 2(2): 187-202. DOI: 10.1093/comnet/cnu003.
[5] 陆泉, 谢祎玉, 陈静,等. 临床医学课程知识主题图谱构建研究[J].图书情报工作, 2019, 63(9): 101-108. DOI: 10.13266/j.issn.0252-3116.2019.09.011.
[6] 许嘉, 韦婷婷, 于戈, 等. 题目难度评估方法研究综述[J].计算机科学与探索,2022,16(4):734-759.
[7] Arifin WN, Yusoff MSB. Item Response Theory for Medical Educationists[J]. Edu Med J, 2017; 9(3):69-81. DOI: 10.21315/eimj2017.9.3.8.
[8] Brodin U, Fors U, Laksov KB. The application of item response theory on a teaching strategy profile questionnaire[J]. BMC Med Educ, 2010,10:14. DOI: 10.1186/1472-6920-10-14.
[9] Koopman L, Zijlstra B, van der Ark LA. A two-step, test-guided Mokken scale analysis, for nonclustered and clustered data[J]. Qual Life Res, 2022,31(1):25-36. DOI: 10.1007/s11136-021-02840-2.
[10] 袁淑莉, 何壮. 非参数项目反应理论模型——Mokken模型[J]. 贵阳学院学报(自然科学版), 2020, 15(4): 101-106. DOI: 10.16856/j.cnki.52-1142/n.2020.04.024.
[11] Stochl J, Jones PB, Croudace TJ. Mokken scale analysis of mental health and well-being questionnaire item responses: a non-parametric IRT method in empirical research for applied health researchers[J]. BMC Med Res Methodol, 2012,12:74. DOI: 10.1186/1471-2288-12-74.
[12] Straat JH, van der Ark LA, Sijtsma K. Using conditional association to identify locally independent item sets[J]. Methodology, 2016, 12(4): 117-123. DOI: 10.1027/1614-2241/a000115.
[13] Wouter DN,Andrej M,Vladimir B. 蜘蛛:社会网络分析技术[M].林枫,译.2版. 北京:世界图书出版公司,2014:108-110.
[14] 林枫, 江钟立. 基于《国际功能、残疾和健康分类(ICF)》的康复信息平台设计与实践初探[J]. 中国康复医学杂志, 2019, 34(2): 125-132. DOI:10.3969/j.issn.1001-1242.2019.02.002.
[15] R Core Team. R: a language and environment for statistical computing[EB/OL].(2022-3-10)[2022-3-26]. https://www.R-project.org.
[16] Sijtsma K, van der Ark LA. A tutorial on how to do a Mokken scale analysis on your test and questionnaire data[J]. Br J Math Stat Psychol, 2017,70(1):137-158. DOI: 10.1111/bmsp.12078.
[17] Rusch T, Mair P, Hatzinger R. Psychometrics with R: a review of CRAN packages for Item response theory[M]. Vienna: Wirtschafts University, 2013:3-27.
[18] PatiI I. Visualizations with statistical details: the ggstatsplot approach[J]. JOSS, 2021, 6(61): 3167. DOI: 10.21105/joss.03167.
[19] Williams, DR. Beyond Lasso: a survey of nonconvex regularization in Gaussian graphical models[J/OL].PsyArXiv, 2020 (2020-11-09).http://psyarxiv.com/ad57p. DOI:10.31234/osf.io/ad57p.[网络预发表].
[20] Csardi G, Nepusz T. The igraph software package for complex network research[J]. Int J Complex Syst, 2006,1695(5): 1-9.
[21] Hubert M, Vandervieren E. An adjusted boxplot for skewed distributions[J]. Comput Stat Data Anal, 2008,52(12): 5186-5201. DOI: 10.1016/j.csda.2007.11.008.
[22] Aliyu I, Kana AFD, Aliyu S. Development of knowledge graph for university courses management[J]. Int J Educ Manage Eng, 2020, 10(2): 1-10. DOI: 10.5815/ijeme.2020.02.01.
