Objective To explore the role of formative evaluation based on Python Programming in improving the clinical reasoning by residents in neurosurgery department.Methods Totally 116 residents trained in neurosurgery department of Sir Run Run Shaw Hospital Affiliated to Zhejiang University were randomly divided into test group (n=49) and control group (n=67). The residents in the control group were trained with the conventional pathway and while the residents in the Python group received formative assessment once a week in addition to the conventional training. Python programming was used in Python group to quickly make examination questions. At the end of the training, the students will join a final exam and their evaluation of teaching will be collected.Results The scores of theory and skill in the experimental group were significantly higher than those in the control group [(89.94±5.43) vs. (84.82±5.73), (80.18±7.23) vs. (73.67±7.48), P<0.01]. In the interview, 61.2% (30/49) of the experimental group students said that the formative evaluation based on python programming was very helpful to improve their capacity of clinical reasoning.Conclusions The formative assessment based on Python programming can improve the clinical reasoning by residents. This method can be implemented quickly and more frequently as a supplement for the formative assessment of mini-CEX and DOPS.