The development of the Best Evidence Medical Education (BEME) Collaboration is introduced in the context of other systematic review initiatives, specifically the Cochrane and Campbell collaborations. The commentary addresses two goals: to describe the current status of BEME and to situate BEME in the broader context of the medical education community's need to be accountable, to conduct research to understand educational processes and results, and the key role that medical educational research must play within the quality-improvement agenda. Lessons drawn from the evidence-based practice movement of the last ten years and the current experience with BEME suggest that, although BEME will inform some educational policies and practices, its initial success may be limited because of the paucity of studies that meet current standards for evidence and the great difficulty in conducting methodologically rigorous studies in the complex social interaction called education. Nonetheless, the need exists for medical education research to continue to address key issues in medical education using experimental designs, while at the same time anticipating the need for more situation-specific data to permit educators to monitor and benchmark their existing programs within a quality-improvement and accountability framework. The authors conclude that the very nature of being professional in today's social and fiscal context demands that medical educators provide evidence of effectiveness and efficiency of their programs while at the same time BEME and medical education research continue to grow and mature.