Abstract: Objective To analyze the application effect of a competency-oriented modified three-dimensional Milestones evaluation system in tracking the development of clinical core competencies among pediatric residency trainees. Methods A prospective cohort study was conducted from October 2021 to October 2024. Based on the ACGME Milestones framework, a modified ″Pediatric Resident Competency Assessment Scale″ tailored to pediatric clinical practice was developed. Seventy-eight trainees who underwent standardized residency training at the Pediatric Specialty Base of Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine from 2020 to 2022 were consecutively enrolled. During the second and third years of training, while completing the annual assessment (teacher evaluation), trainees also performed self-evaluations using the milestones assessment form (trainee self-evaluation), establishing a three-dimensional evaluation framework integrating teacher evaluation, trainee self-evaluation, and system tracking to assess the clinical core competencies of the trainees. Paired t-tests and independent samples t-tests were used for within-group longitudinal comparisons and between-group comparisons, respectively. Pearson linear correlation analysis was employed to assess correlations between variables. Results Both teacher evaluation scores and trainee self-evaluation scores in the third year were significantly higher than those in the second year [(433.60 ± 25.59) vs. (393.08 ± 48.37), (63.11 ± 13.63) vs. (54.39 ± 12.16), all P < 0.001]. There were no statistically significant differences in teacher evaluation scores or trainee self-evaluation scores between independent residency trainees and clinical medicine master's degree residents in either the second or third year [Year 2: (393.36 ± 49.68) vs. (387.37 ± 53.97), (58.85 ± 12.17) vs. (59.96 ± 12.55); Year 3: (431.77 ± 28.39) vs. (437.26 ± 23.20), (62.79 ± 14.64) vs. (62.41 ± 13.69); all P > 0.05]. The correlation between total teacher evaluation scores and total trainee self-evaluation scores was negative in the second year (r=-0.26, P =0.003) but positive in the third year (r = 0.24, P = 0.039). Conclusions The modified three-dimensional milestones evaluation system can effectively track the developmental trajectory of clinical core competencies in pediatric residency trainees. The core competencies of trainees increase in a stepwise manner with additional years of training, and competencies among trainees with different backgrounds tend to converge in the mid-to-late stages of training. The mutual verification and integration of relatively objective teacher evaluations and subjective trainee self-evaluations contribute to a comprehensive and systematic assessment of residents' clinical core competencies.

Key words: Pediatrics, Competence, Milestones evaluation system, Standardized residency training, Residency trainees