Abstract: Addressing the insufficient support for medical educators′ self-directed learning and personalized tutoring, as well as the predominant use of didactic instruction in faculty development activities for medical educators, this study aims to explore the potentials of an AI agent applied in faculty development for medical educators. The researchers developed the “Huaxi Xiaozhu” AI Faculty Development Assistant for Medical Education (hereinafter referred to as “Huaxi Xiaozhu”), which was applied in faculty development in West China Medical Center of Sichuan University from May to August 2024. The output quality of “Huaxi Xiaozhu” was evaluated, and feedback from medical educators was collected. Data were analyzed and visualized using methods such as analysis of variance and radar charts. The results showed that “Huaxi Xiaozhu” achieved a higher output quality score (24.92±2.56) for 13 medical education-related prompts compared to mainstream general large language models in China, such as Wenxin Yiyan (21.84±3.08) (P<0.05). The radar charts indicated that “Huaxi Xiaozhu” performed well across six dimensions: accuracy, completeness, understandability, clarity of opinion, domain adaptability, and creativity & inspiration. Among the 92 medical educators who participated in the satisfaction survey, 90 (97.8%) were satisfied or very satisfied with the integration of “Huaxi Xiaozhu” into workshops of faculty development programs. However, in the follow-up survey on the use of “Huaxi Xiaozhu”, only 4 out of 16 medical educators continued to use it for review and consolidation within 40 days after a faculty development program. The findings suggest that the high output quality of the AI agent and the high satisfaction of medical educators make it a promising pathway for generative AI-empowered faculty development. However, the actual use of the AI agent by medical educators shortly after programs was not optimistic. These findings highlight the necessity of enhancing the development of AI agents for medical education, and employing them into the design of faculty development programs to mutually promote the development of teaching abilities and enhancement of digital literacy of medical educators.

Key words: Education, Medical, Faculty development, Generative artificial intelligence, AI agent, Digital literacy of teachers