The Peninsula College of Medicine and Dentistry (PCMD) was among the five new medical schools approved in the United Kingdom in 2000. PCMD required a new curriculum and a new outcomes-oriented assessment system based on sound educational theory. The resulting system was designed to use multiple sampling and show increasing authenticity as students progress through the curriculum. A “frequent look and rapid remediation” structure was incorporated to allow faculty to evaluate student competence throughout the year and take immediate action when warranted. Using a recently published framework, the authors retrospectively evaluate the assessment system and share many of the decisions they and their colleagues had to make. They discuss how to support stakeholders in understanding and contributing to the development of an assessment system and how to meet some of the challenges they encountered. They consider indicators of success in terms of the performance of the assessment system as a whole and the ways the system contributes to educational research. They suggest that future research should focus on building greater flexibility into the system so that assessment decisions can be individualized to particular students.