Construction and empirical research on a BOPPPS and large language models (LLMs) integrated teaching quality evaluation system

doi:10.3760/cma.j.cn115259-20250815-00959

Abstract

Abstract: Objective To construct a BOPPPS-Large Language Models (LLMs) integrated teaching quality evaluation system, verify its scientific rigor and applicability, and inform the establishment of similar digital teaching quality evaluation systems in the field of traditional Chinese medicine (TCM). Methods With the BOPPPS teaching model as the structural anchor and LLMs as the quantitative support, an evaluation system covering five core dimensions of core cognition, teaching process, teaching effect, non-cognitive ability, and long-term migration was constructed. A cluster randomized controlled trial was conducted from September 2023 to June 2024, and 120 undergraduate students of the 2021 cohort majoring in TCM who took the Traditional Chinese Internal Medicine course at Guangxi University of Chinese Medicine enrolled in the study. They were equally divided into the experimental group and the control group by random number table method, 60 students in the experimental group adopted the BOPPPS-LLMs integrated evaluation system, and 60 students in the control group continued with the traditional experience-based evaluation system. Independent samples t-test was used for inter-group comparison, Pearson correlation analysis was adopted to analyze the correlation between indicators, and Cronbach′s α coefficient was applied to test the internal consistency reliability of the system. Results The overall Cronbach′s α coefficient of the evaluation system was 0.86, indicating good internal consistency reliability. All indicators of the five core dimensions in the experimental group were significantly better than those in the control group (all P<0.001), for example, the confusion degree of case/text interpretation in the core cognition dimension was (18.5±3.8) points, which was significantly lower than (32.1±4.5) points in the control group; the average score of the final comprehensive assessment of the teaching effect dimension in the experimental group was (89.5±4.2) points, which was significantly higher than (77.2±5.1) points in the control group. In addition, there was a strong negative correlation between core cognitive confusion score and teaching effect (|r|≥0.78, P < 0.001), which confirmed the good construct validity of the system. Conclusions The BOPPPS-LLMs integrated teaching quality evaluation system is scientific, reliable and highly practical. It can accurately and comprehensively evaluate the teaching quality of TCM courses, and provide an empirical basis for the implementation of digital teaching quality evaluation systems in TCM higher education.

Key words: Artificial intelligence, Educational model, Educational evaluation, Traditional Chinese medicine

CLC Number:

Ou Jian, Hu Yueqiang, Huang Hongna, Luo Jun, Wang Chunling, Liang Jiyao. Construction and empirical research on a BOPPPS and large language models (LLMs) integrated teaching quality evaluation system[J]. Chinese Journal of Medical Education, 2026, 46(8): 615-621, DOI: 10.3760/cma.j.cn115259-20250815-00959.

References

[1] 燕姣云,安俊丽,孙国红.课堂教学质量评价指标体系重构[J]. 中国大学教学,2023(12):74-78,91. DOI:10.3969/j.issn.1005-0450.2023.12.013.
[2] 胡钦太,伍文燕,冯广,等.人工智能时代高等教育教学评价的关键技术与实践[J]. 开放教育研究,2021,27(5):15-23.DOI:10.13966/j.cnki.kfjyyj.2021.05.002.
[3] 王晓璐, 杨云轩, 谢阳斌. 创造人机对话式学习新形态——大语言模型的教育应用现状与展望[J]. 中小学信息技术教育,2024(5):15-17. DOI:10.3969/j.issn.1671-7384.2024.05.006.
[4] 李悦. 梳理与深化:BOPPPS教学模式研究现状透视与分析[J]. 广东职业技术教育与研究,2023(7):48-53. DOI:10.3969/j.issn.1672-2744.2023.07.016.
[5] 雍彦礼, 李苏迪, 何鑫宇. 中医内科学教学目标之中医临床思维能力培养的研究[J]. 中国中医药现代远程教育,2023,21(4):198-200. DOI:10.3969/j.issn.1672-2779.2023.04.070.
[6] 赵红,彭周华,朱景伟,等.基于BOPPPS模型的混合式教学研究与实践[J]. 航海教育研究,2022,39(2):41-45.
[7] 郝梦.面向大语言模型的中医辨证思维测评数据集构建研究[D]. 北京:中国中医科学院,2025.
[8] 陈元电,陈静,徐胜.人工智能技术在教学质量评价中的研究与实践[J]. 中国现代教育装备,2024(9):20-22,26.DOI:10.13492/j.cnki.cmee.2024.09.018.
[9] 杨海燕, 赵海平, 程绍民, 等. 中医药教育培养模式现状及当前中医药课程体系下的课程融合性教学实践[J]. 中国中医药现代远程教育,2023,21(13):149-151. DOI:10.3969/j.issn.1672-2779.2023.13.054.
[10] 罗江华,王琳,刘璐.人工智能赋能课堂反馈的伦理困境及风险化解[J]. 现代远程教育研究,2022,34(2):29-36.
[11] 魏亚丽,张亮.从“基于经验”到“数据驱动”:大数据时代的教学新样态[J]. 当代教育科学,2022(2):50-56. DOI:10.3969/j.issn.1672-2221.2022.02.008.
[12] 周辉.数智赋能中医药高校教育教学的现实桎梏与突破路径[J]. 湖南中医药大学学报,2024,44(6):1095-1099. DOI:10.3969/j.issn.1674-070X.2024.06.024.

[1]	Liu Kan, Zheng Shuyuan, Wang Zhenyu, Liu Zidong, Yang Hai, Tian Tian. Application of generative large language models in improving the quality of basic medical science examination items [J]. Chinese Journal of Medical Education, 2026, 46(8): 628-635.
[2]	Li Xinhui, Wang Niqing, Jin Jingjing. Ethical governance and teaching optimization of generative artificial intelligence in the ″Anatomy and Physiology″ course [J]. Chinese Journal of Medical Education, 2026, 46(7): 502-508.
[3]	Cheng Wenjie, Lyu Feng, Min Su. The application and prospects of embodied artificial intelligence in medical education [J]. Chinese Journal of Medical Education, 2026, 46(7): 509-514.
[4]	Liu Siyan, Zhou Yaqian, Chu Jiaqi, Jiang Yutong, Sun Lingzhi. Construction of a knowledge graph for the textbook History of Chinese Medicine based on large language models and Neo4j [J]. Chinese Journal of Medical Education, 2026, 46(7): 515-520.
[5]	Wang Chenghe, Gu Qian, Chen Lu, Li Yinglin, He Hongchao, Huang fang, Xu Danfeng. Design and implementation of an intelligent agent-assisted cystoscopy training curriculum for surgical residents [J]. Chinese Journal of Medical Education, 2026, 46(7): 534-538.
[6]	Cao Wei, Hu Bin, Jiang Zhehan, Nie Jiashuai, Jin Huimin. Research on examiner allocation for the practical skills examination of traditional Chinese medicine category physician qualification examination [J]. Chinese Journal of Medical Education, 2026, 46(7): 549-553.
[7]	Xu Fangyong, Zhang Dandan, Zhang Yingying, Huang Yuenuo, Pan Susu, Zhang Jizhou. Educational work preferences among TCM residency training faculty [J]. Chinese Journal of Medical Education, 2026, 46(5): 351-357.
[8]	Zhang Kunsong, Zhang Junlong, Feng Shaoting, Chen Wei, Kuang Ming, Xiao Haipeng. Opportunities, challenges and integration paths of medical humanities education in the AI era [J]. Chinese Journal of Medical Education, 2026, 46(4): 241-246.
[9]	Chen Lihua, Liu Shuo, Xin Siyan, Wang Yu, Wu Hongbin. A study on the influencing factors of the use of generative artificial intelligence by clinical medicine students [J]. Chinese Journal of Medical Education, 2026, 46(3): 199-205.
[10]	Qian Sishi, Zhao Hanqi, Kang Rui, Li Yong, Hu Yimei. Study on the construction of clinical ability evaluation index system for traditional Chinese medicine students of ″5+3″ integrated program during standardized residency training [J]. Chinese Journal of Medical Education, 2026, 46(2): 141-145.
[11]	Wen Deliang, Wei Xiaotong. Application progress and future trends of artificial intelligence in medical education [J]. Chinese Journal of Medical Education, 2026, 46(1): 1-7.
[12]	Wei Xiaofan, Wu Jianghua, Chi Xiaochun, Zhang Weiguang, Peng Yihong. Exploring the path, challenges, and prospects of artificial intelligence in empowering human morphology curriculum construction [J]. Chinese Journal of Medical Education, 2026, 46(1): 8-14.
[13]	Tan Jing, Kong Xiangguo, Chen Zijie, Su Dan, Wu Yufeng. Current status survey and improvement strategies of traditional Chinese medical cultural literacy for non-medical major students in traditional Chinese medicine universities [J]. Chinese Journal of Medical Education, 2025, 45(9): 646-650.
[14]	Zhu Jing, Wang Xiaofei, Zhou Weibing, Liu Hengyu. Visual analysis of research on generative artificial intelligence in medical education based on scientific citation databases [J]. Chinese Journal of Medical Education, 2025, 45(9): 661-665.
[15]	Wei Fangze, Liu Qian. Bibliometric study on the current application status of artificial intelligence in surgical education [J]. Chinese Journal of Medical Education, 2025, 45(9): 666-670.