目的 了解问卷调查在我国医学教育研究领域的应用现状,并对问卷质量进行评价和分析。方法 以2018年12月至2019年11月,《中华医学教育杂志》和《中华医学教育探索杂志》连续出版的12期期刊为样本,筛选基于问卷调查的原创性研究,对问卷调查的设计、开发、实施等相关资料进行收集和分析。结果 212篇原创研究将问卷调查作为研究方法之一,68篇可纳入分析(含73个问卷调查表),其中,分别有18个和20个问卷调查表报道了效度和信度资料,33个问卷调查报道了问卷回收率。对58个采用李克特选项的问卷调查表进行5个方面的质量分析,17.2%(10/58)的问卷调查表未犯任何设计错误。结论 问卷调查在我国医学教育研究领域应用广泛,但问卷调查的质量仍有待提高。问卷调查表的题干及选项设置容易出现错误,加强对问卷调查的设计、开发、实施、信度、效度资料等研究信息的披露,有助于准确传递学术信息。
Objective Surveys are widely used in medical education research, yet little is known about the quality of the instruments employed. This study evaluated the quality of published survey instruments in the field of medical education in China.Methods The project covers original research articles that employed at least one survey which is published in two high-impact Chinese journals in the field of medical education from December 2018 to November 2019. The descriptive data on the design, development and implementation of surveys, as well as the quality of survey items were collected and assessed independently.Results Totally 212 articles employed surveys as research method, 68 articles included 73 surveys met inclusion criteria. Response rate, validity and reliability evidence were reported in 33, 18 and 20 surveys, respectively. Item-quality assessment revealed that 17.2% (10/58) articles did not violate established best practices in the design of Likert-type rating items.Conclusions Although surveys were commonly used in medical education research in China, their quality need to be improved. The setting of questions and options in the questionnaire was prone to errors. Aimed to improve the quality of survey and facilitate the delivery of academic information, detail information regarding the design, development and implementation of surveys should be reported.
[1] Phillips AW, Artino AR Jr.Lies, damned lies, and surveys[J]. J Grad Med Educ, 2017,9(6):677-679. DOI: 10.4300/JGME-D-17-00698.1.
[2] Artino AR Jr, Phillips AW, Utrankar A, et al.“The questions shape the answers”: assessing the quality of published survey instruments in health professions education research[J]. Acad Med, 2018,93(3):456-463. DOI: 10.1097/ACM.0000000000002002.
[3] Artino AR Jr, La Rochelle JS, Dezee KJ, et al.Developing questionnaires for educational research: AMEE Guide No.87[J]. Medical Teacher, 2014, 36(6):463-474. DOI: 10.3109/0142159X.2014.889814.
[4] Gehlbach H, Artino AR Jr.The survey checklist (manifesto)[J]. Acad Med, 2018,93(3):360-366. DOI: 10.1097/ACM.0000000000002083.
[5] Artino AR Jr, Gehlbach H.AM last page.Avoiding four visual-design pitfalls in survey development[J]. Acad Med, 2012,87(10):1452. DOI: 10.1097/ACM.0b013e31826a06b2.
[6] Artino AR Jr, Gehlbach H, Durning SJ.AM last page: avoiding five common pitfalls of survey design[J]. Acad Med, 2011,86(10):1327. DOI: 10.1097/ACM.0b013e31822f77cc.
[7] Cook DA, Beckman TJ.Current concepts in validity and reliability for psychometric instruments: theory and application[J]. Am J Med, 2006,119(2):166.e7-166.e16. DOI: 10.1016/j.amjmed.2005.10.036.
[8] Presser S, Couper MP, Lessler JT, et al.Methods for testing and evaluating survey questions [J]. Public Opinion Quarterly, 2004,68(1):109-130. DOI: 10.1093/poq/nfh008.
[9] 曾庆奇, 刘婧, 高杰, 等.某住院医师规范化培训基地不同身份住院医师的培训师资均质化分析[J]. 中华医学教育杂志,2019,39(1):64-68. DOI: 10.3760/cma.j.issn.1673-677X.2019.01.016.
[10] 梁春蕾,斯洪桥,王燕, 等.四川省住院医师规范化培训学员人文素质教育的效果研究[J]. 中华医学教育探索杂志,2019,18(10):1070-1075. DOI:10.3760/cma.j.issn.2095-1485.2019.10.023.
[11] 潘登, 余良仁.健康中国背景下基层就业医学生职业生涯发展现状研究[J]. 中华医学教育杂志,2019,39(3):236-240. DOI: 10.3760/cma.j.issn.1673-677X.2019.03.019.
[12] 谭斯品, 肖子辉, 涂自智, 等.基于案例和问题的交互式病理生理学教学效果评价[J]. 中华医学教育探索杂志,2019,18(9):884-888. DOI: 10.3760/cma.j.issn.2095-1485.2019.09.005.
[13] 郑志楠, 李文玉, 李福军, 等.麻醉医生工作-家庭冲突对工作满意度及离职倾向的影响[J]. 中华医学教育探索杂志,2019,18(2):198-203. DOI: 10.3760/cma.j.issn.2095-1485.2019.02.021.
[14] 雷丽萍, 王樵砚, 李学谨, 等.北京市住院医师与专业学位硕士研究生培养效果评价研究[J]. 中华医学教育探索杂志,2019,18(1):5-12. DOI: 10.3760/cma.j.issn.2095-1485.2019.01.002.
[15] Sullivan GM, Artino AR Jr.Analyzing and interpreting data from likert-type scales[J]. J Grad Med Educ, 2013,5(4):541-542. DOI: 10.4300/JGME-5-4-18.
[16] Peeters MJ .Measuring rater judgment within learning assessments—part 1: why the number of categories matters in a rating scale[J]. Currents in Pharmacy Teaching and Learning, 2015, 7(5):656-661. DOI:10.1016/j.cptl.2015.06.015
[17] Peeters MJ.Measuring rater judgments within learning assessments—part 2: a mixed approach to creating rubrics [J]. Currents in Pharmacy Teaching and Learning, 2015,7(5):662-668. DOI:10.1016/j.cptl.2015.06.022.
[18] Artino AR Jr, Durning SJ, Sklar DP.Guidelines for reporting survey-based research submitted to academic medicine[J]. Acad Med, 2018,93(3):337-340. DOI: 10.1097/ACM.0000000000002094.
[19] Colbert CY, French JC, Arroliga AC, et al.Best practice versus actual practice: an audit of survey pretesting practices reported in a sample of medical education journals[J]. Med Educ Online, 2019,24(1):1673596. DOI: 10.1080/10872981.2019.1673596.
[20] Cook JV, Dickinson HO, Eccles MP.Response rates in postal surveys of healthcare professionals between 1996 and 2005: an observational study[J]. BMC Health Serv Res, 2009,9:160. DOI: 10.1186/1472-6963-9-160.
