Sport is an international obsession for many people. Ensuring that athletes can compete in as safe of an environment as possible is vital not only in terms of the players' individual health and the teams' success in competition but also serves as a point of public health impact and as an example to young athletes. Sports injury surveillance and player health research is an essential component of injury prevention, reduction, and ultimately, player wellness. The National Football League (NFL) has established a robust injury research structure across all facets of athlete health (1,2). Consistent with established ethical principles and good practice for medical research, the NFL and NFL Players Association (NFLPA) jointly developed a Player Scientific and Medical Research Protocol Approval Process (MRAP) through which NFL player electronic injury and illness data may be used for the purposes of observational and interventional research. MRAP was agreed to by the NFL and NFLPA as a side letter agreement to the 2011 NFL-NFLPA Collective Bargaining Agreement (CBA), dated January 26, 2015, and has been updated over time. This process provides oversight of the access, utilization, and management of NFL health data to: 1) ensure that the potential research is feasible, valuable, and methodologically sound; (2) protect the confidentiality interests of NFL players and assure that appropriate personal information is not released to the public; and 3) avoid the duplication of research efforts between the NFL's medical committees, with the overall goal of ensuring that the goals of the intended research are consistent with the interests of the patient (player) and institution (NFL).
The establishment of this process acknowledges the sensitivity of the workplace injury and illness data shared by NFL athletes, as well as the value of these data toward driving actionable evidence for injury prevention and player care. While these are supremely fit, talented young men, they are at risk for sustaining significant injuries and developing medical conditions throughout their time playing football — both at amateur and professional levels — based on the inherent risks associated with American football. Collaborative development of MRAP by the NFL and NFLPA ensures that both the League and the player representatives agree to the principles of conducting research on these “player-patients” with the goal of understanding and improving health and safety in a sound and ethical fashion within a collectively bargained employment environment.
The steps within the application process and various levels of approval required to conduct research using NFL player workplace injury and illness data are defined in the MRAP protocol. The level of approval is based on whether the research is: 1) active (interventional) or observational (noninterventional); 2) the intended use of the data; and 3) the sensitivity of the requested data. Sequential approvals by each party in the process are required; denial at any point by either party results in the full request being denied. Additionally, MRAP establishes a process whereby a research product (e.g., manuscript, presentation) is submitted for approval prior to submission for public presentation or editorial review and subsequent dissemination. However, it is stated explicitly in the MRAP protocol that “the NFL and NFLPA may offer nonbinding comments but shall not have editorial control or input” into any scientific publication or professional presentation. This caveat applies to all types of research disseminated via the MRAP governance structure.
Player-related research approved through MRAP is based on data that are collected directly on active players by NFL team medical staff or NFL team medical/scientific consultants, rather than data compiled and promoted by external public or private sources. Data pertaining to prospective players evaluated at the National Invitational Scouting Combine who do not sign an NFL contract are not eligible for use in MRAP research. This approach ensures that the medical research data obtained from NFL players and its subsequent analyses and interpretation are as consistent and accurate as possible, particularly with regard to completeness, injury classification, and diagnosis. Studies describing the incidence of, and prognostic implications following specific injuries among NFL players have recently been published (3–23); these studies are most frequently based on convenience samples obtained from publicly reported sources, such as the lay press or dedicated web logs (blogs), which may be inaccurate and incomplete, and are likely to reflect more severe injuries or the experiences of more popular players. Use of a comprehensive, curated electronic medical record-based injury system linked with official game statistics, player demographics, and injury mechanisms provides the most reliable and robust data. Such data are necessary to capture accurate injury statistics, which are essential for impact assessment of these injuries.
Every NFL club is responsible for distributing this MRAP protocol to its medical team. Club research requests for approval may only be submitted by the club's head team orthopedist, head team medical physician, or head team athletic trainer (AT). Requests to undertake research also may come from an NFL club or a member or consultant of the NFL's Medical Committees (24) (e.g., health and safety, musculoskeletal, head, neck and spine, general medical). Failure to obtain the requisite approvals prior to initiating a research project or disseminating NFL player injury data will result in removal of the responsible member from their respective committee and/or sanctions to the club at the discretion of the NFL Commissioner.
The purposes of this article are to outline the MRAP governance structure (which explains the details of the medical research processes in the NFL) and highlight the importance of a structured, multilevel research approval process in a professional sport league whose injury and illness data are governed by a collectively bargained agreement.
MRAP Protocol Components and Resources
There are two components to the MRAP governance structure:
- 1) The MRAP Protocol details the process for seven distinct pathways of approval based on the (a) person requesting data, (b) intent of research (i.e., publication/external presentation vs. occupational/internal club, league or committee use), and (c) sensitivity/identifiability of data requested (Table 1).
- 2) Initial request form through which applications are initiated, designated by MRAP as Appendix A (Supplemental Digital Content 1, http://links.lww.com/CSMR/A47). This form requires disclosure of the study objectives, the data being requested, identification of the intended audience, and intended use of the data.
Based on the information submitted in the initial request form, the research proposal request is assigned to one of the seven pathways outlined within the MRAP protocol, which specifies the sequential reviews and approvals required for the request. All requests are made via a dedicated web site and reviewed by an independent third-party research company (IQVIA, Durham, NC) specializing in health information technology, analytics, and clinical research. The initial request initiates a feasibility review to assess whether: 1) data are available for the requested research; and 2) the research methodology is scientifically valid. Each approval party then reviews the proposal with consideration toward the strength of the scientific aims, availability and sensitivity of the data being requested, relative importance of the research objectives, and whether there is overlapping research underway by one of the NFL's medical committees and advisors.
The MRAP protocol contains seven pathways of approval for research activity, each with specific, sequential requirements. These are detailed below, and the language is taken nearly verbatim from the NFL-NFLPA CBA side letter agreement.
I. Active/Interventional Player Research
Club-related or third-party request to conduct “active” or interventional research on an NFL player or players (e.g., conducting diagnostic tests for the purpose of gathering data rather than as clinically or medically indicated; circulating questionnaires or interviewing players directly; or administering any form of medication and/or blood testing for the purpose of research rather than as clinically or medically indicated).
Once an interventional research project obtains the approval of the relevant NFL medical committee, that committee designates one of its members to serve as the “sponsor” of the research project. The project sponsor is responsible for monitoring the progress of the research and periodically reporting back to the larger committee. The project sponsor serves as the liaison between the researcher and the NFL for any logistical issues. Data or analysis may be shared per NFL and NFLPA determination provided: 1) institutional review board (IRB) approval is attained and current; 2) player privacy concerns are addressed with prior notice to and approval as set forth above by the NFL, NFLPA, and relevant medical committee(s); and 3) notification and dissemination of findings and reports to aforementioned parties is provided 60 d prior to submission for publication and 30 d prior to publication (final article upon acceptance). Player participation in any research project must be and remain entirely voluntary. Players retain the right to withdraw their consent at any time and no longer participate in the research project.
Prior to publication or presentation of results, the researcher must send a draft of the report or manuscript regarding the study to the NFL (NFL chief medical officer and NFL management council legal) and the NFLPA to ensure that any player privacy concerns are appropriately addressed. This must occur 60 d prior to submission and 30 d prior to publication (final article upon acceptance) and applies to any abstracts or presentation given or submitted based on the research. As mentioned previously, the NFL and NFLPA may only offer nonbinding comments and has neither editorial control nor input. If an NFL club physician engages in active or interventional player research without obtaining the requisite approvals described above, discipline may be imposed against the club with which that physician is affiliated. Other remedies also may be available to individual players and/or the NFLPA pursuant to the CBA.
II NFL Club Physician EMR Data Request: Internal/Club Use Only
Each NFL club may access the data related to its own players; clubs may not obtain identified data related to players in other clubs. Beginning in the 2019 season, individual NFL clubs have access to an injury dashboard showing injury incidence overall and by setting trends over time, and return to play metrics for players from their own club, as well as comparison with select aggregate results across the League. For many injuries and health-related conditions, NFL clubs can compare their injury/health condition data with league averages, which are provided to the club by IQVIA.
Clubs may request analysis of their own player data by submitting the request form to IQVIA, which will comment on the feasibility and resources needed for the project. The requesting Club also must submit a proposal through the NFL MRAP web site. IQVIA's feasibility assessment, when applicable, and the request form will then be forwarded to NFL management council legal, the NFL chief medical officer and the NFL executive vice president (EVP) of Health and Safety Initiatives for approval. The project will pass through the approvals required as set forth within Table 2 via the NFL MRAP web site (https://legacy.nflfoundation.org/mrap/login). Please note that one cannot log in or view anything on the site unless they are provided with login information by the NFL. Projects will be approved and prioritized according to the complexity of the analysis and availability of resources. Data produced under this section may not be disclosed to anyone who is not affiliated with the club. In other words, a club medical team member may not use data produced under this research pathway in any publication or presentation to any audience outside of the club.
III NFL Medical Committee Member: Internal Committee Use Only/De-Identified Data
NFL medical committees shall be given access to deidentified player data in connection with their respective missions. Data produced via this research pathway may not be disclosed to anyone who is not affiliated with the relevant committee. In other words, an NFL medical committee member may not use data produced under this section in any publication or presentation to any audience outside of the committee, the NFL, or the NFLPA, absent appropriate approval. The NFL will give notice to the NFLPA legal of such a request and provide a copy of such data prior to sending the data to the requesting committee. In addition, a committee member or chairperson may request access to research results to consider for broader distribution to relevant NFL medical advisors by contacting the NFL chief medical officer and IQVIA. Should the committee member/chairperson conclude that the broader committee would benefit from distribution of the data, the protocol reflected in this research pathway shall apply.
NFL medical committee members may request deidentified player analyses or data by submitting the request form to IQVIA, which will assess the feasibility and resources needed to perform the recommended project. The requesting NFL medical committee member also must submit a proposal through the NFL MRAP web site. IQVIA's assessment and the request form will then be forwarded to NFL management council legal, the NFL chief medical officer and the NFL executive vice president of Health and Safety Initiatives for approval. The project will pass through the approvals required as set forth within Table 3, via the NFL MRAP web site. Data produced under this section may not be disclosed to anyone who is not affiliated with the committee.
IV NFL Medical Committee Member: Internal Committee Use Only — Identified or Identifiable Data
NFL medical committees shall be given access to player data in connection with their respective missions. Identified or identifiable data may include the names of players or could be identifiable by virtue of the data set or information included (e.g., date of game, type of injury, position). These data may be requested by an individual NFL medical committee member for review by the larger committee or to identify potential issues.
NFL medical committee members may request analysis of identified or identifiable player data by submitting a request to IQVIA, which will assess the feasibility and resources needed to perform the recommended project. The requesting NFL medical committee member also must submit a proposal through the NFL MRAP web site. IQVIA's assessment and the request form will then be forwarded to NFL management council legal, the NFL chief medical officer and the NFL executive vice president of Health and Safety Initiatives for approval. NFLPA legal also must consent to the production of these data. The project will pass through the approvals required as set forth within Table 3, via the NFL MRAP web site. Data produced under this section may not be disclosed to anyone who is not affiliated with the committee. In other words, an NFL medical committee member may not use data produced under this section in any publication or presentation to any audience outside of the committee.
V NFL Club Physician/NFL Medical Committee Member: EMR Data Research — Publication/Public Disclosure Intended — De-identified, Identifiable, and Identified Data
NFL club physicians (head orthopedic or head internal medicine physicians only), head club ATs, or members of NFL medical committees may request either their club-specific or league-wide data with the intent to publish the results.
To obtain such data, the Head Club Physician/AT or NFL medical committee member must submit the request form through the NFL MRAP web site, which will direct the request to the NFL chief medical officer who will, upon approval, direct the request to the appropriate NFL medical committee chairperson. The committee will review all proposals to determine whether or not the proposal raises an issue of substantial relevance to the general NFL population and is worthy of the dedication of resources required to gather the data requested. If approved by the committee, IQVIA conducts analyses which ensure that the request is feasible and may request additional detail should it be required to evaluate the request. Upon approval of the relevant NFL medical committee, the request will be directed to the remaining parties for approval (Table 3).
The final draft of the article or report must be presented to IQVIA and the relevant medical committee chairperson 60 d prior to submission and 30 d prior to publication (final article upon acceptance), and this requirement applies to any abstracts or presentation given or submitted based on the study. No submissions or presentations may occur until approvals are provided. The manuscript must be finalized within 24 months from the initial request date and IRB review must remain up-to-date for the full course of the project; updated approvals must be provided to IQVIA. International Committee of Medical Journal Editors guidelines for review and authorship must be followed (25).
VI NFL Club Physician/NFL Medical Committee Member: EMR Data Research — Publication/Public Disclosure Intended — Case Study
NFL club physicians (head orthopedic and head internal medicine physicians only), club head ATs, or NFL medical committee members may request either their club-specific or league-wide identified player data, in furtherance of a specifically defined research project, with the intent to publish the results.
To obtain such data, the head club physician/AT or committee member must submit the request through the NFL MRAP web site, which will direct the request through the required approvals, as set forth in Table 3. The submission to the NFL and NFLPA must include a proposed IRB-approved individual consent form for players whose data will be the subject of the research to execute. Upon approval from those entities, the proposal will be submitted to the IRB. Such submission to the IRB must include the IRB-approved proposed individual player consent form. Following approval of the IRB, the approved consent form must be used to obtain the consent of individual players whose identifiable data will be the subject of the research.
Upon approval of the IRB and the individual player(s) whose identified data will be the subject of the research, the proposal shall be sent to the chairperson of the NFL Health and Safety Committee for designation to the appropriate NFL medical committee chairperson. The NFL medical committee chairperson will assign the project to a committee member for review and presentation to the committee. The committee will review all proposals to determine whether or not the proposal raises an issue of substantial relevance to the general NFL population and is worthy of the dedication of resources required to gather the data requested. If approved, the designated committee member shall be responsible for monitoring the project's progress and providing periodic updates to the committee.
Data or analyses may be shared after approval from NFL and the NFLPA. The final draft of the article or report by which the results of the study will be publicly released must be presented to and approved by the designated committee member and NFL chief medical officer. This must occur 60 d prior to submission and 30 d prior to publication (final article upon acceptance) and applies to any abstracts or presentation given or submitted based on the study. As in all other MRAP-related research endeavors, the NFL and NFLPA can offer nonbinding comments but do not have editorial control or input.
VII Research by Third Parties without NFL Affiliation
Pursuant to the terms of the collectively bargained NFL Data Management Policy, NFL player medical data will not ordinarily be available for third-party research, absent agreement by the NFL management council and NFLPA. Should the parties agree that a proposed research project presents a clear opportunity to enhance NFL players' health and safety and has not been duplicated by previous or ongoing NFL research, the parties may consent to the disclosure of NFL player data. In such an instance, the research project must, at a minimum, follow the steps outlined in Pathway III. Data or analysis may be shared after approval from NFL and NFLPA.
The MRAP governance structure was devised to maintain high standards of research and appropriate use of sensitive data by structuring mutually agreed oversight of the use of NFL player workplace injury and illness data within clinical research. The goals of this process are to ensure that proposed research provides value to the NFL and its players, is feasible and methodologically sound, respects patient (player) confidentiality, and avoids the duplication of research efforts among the NFL's medical committees. Similar to typical American university or hospital-based IRBs and research governance systems in Europe and the rest of the world, MRAP serves to primarily protect the rights and welfare of human research subjects (the NFL players) while secondarily protecting the interests of the institution (the NFL). In general terms, IRBs and Research Ethics Committees (RECs) are charged with providing an independent evaluation that proposed research is ethically acceptable, devoid of investigator bias, and is compliant with regulations designed to protect human subjects. Such research governance bodies are given the authority to approve, require modifications to, or disapprove research activities undertaken in their institution or geographical setting, irrespective of the relative merit or purpose of the research. To accomplish this duty, IRBs and RECs often function through a committee format comprised of clinician-scientists, nonscientists, and often at least one layperson representing the perspective of the research subjects. Some research activities (e.g., patient surveys, existing data) are often designated as exempt from medical research governance approval processes because they pose “minimal risk” to human subjects. IRBs and RECs do not determine the relative value the research provides to the institution or to society and may not have detailed knowledge of the occupational and demographic risk factors of the patient population to be studied. The NFL and NFLPA MRAP governance structure adds additional layers of scientific oversight, all in the context of respecting the parameters of conducting scientific research in the context of a collectively bargained workforce.
The seven distinct pathways of MRAP ensure that approvals specific to the use of the data and the risk the research poses to the players are attained. MRAP also assesses the merit and need for proposed research activities so as to avoid committing resources to studies whose objectives have limited clinical relevance to the NFL player population, have already been completed, or that cannot be performed in a robust, quality manner with the proposed methodology or available data.
Because NFL players are members of a union, and MRAP has been collectively bargained, all NFL player research evaluated through this process must be approved by the players' bargaining representative, the NFLPA. The structure of MRAP enhances and augments the required evaluation performed by an independent IRB by requiring additional layers of oversight by both NFL and NFLPA medical advisors, legal representatives, and senior level administrators as well as experts in injury data analysis. In this way, all involved stakeholders — the players, NFL, and NFLPA — have their interests protected. This unique model can serve as a template for other nonmedical entities or organizations that may wish to perform clinical research that involves human subjects.
An additional benefit of MRAP is the improved reliability of research data that is used either internally by each NFL club or made publicly available through peer-reviewed publications, academic conferences, or media releases. Multiple research studies (3–23) have been published in the medical literature that use Internet-based injury reports, press releases, and team web sites to obtain injury data in an effort to describe injury incidence or treatment outcomes. This practice can be problematic since there is no confirmation that the publicly reported data are accurate or complete. In some cases, the publicly collected injuries may closely match the data within the NFL's electronic medical record. However, the true incidence is unlikely to match perfectly and, in many cases, may differ meaningfully. For example, if the injury in question is of a lower grade that does not result in playing time lost, then this less severe injury may not be detected by these public data sources. As a result, lower-grade injuries will likely be underreported in publicly available data sources, leading to calculation of an erroneously low injury incidence and/or a subsequently inaccurate report of specific injury burden. Second, these public data sources may not report injuries that occur in players whose contracts are terminated and enter into Injury Settlements postinjury or who are on the practice squad. Finally, failure to account for each injury, especially less severe injuries that are not media-reported, also may lead to inaccurate conclusions regarding treatment outcomes when generalizing results to a specific injury. Since all NFL medical research is first approved through MRAP, the appropriate data set for the research question is able to be identified. Use of a comprehensive, curated EMR-based injury system linked with official game statistics, player demographics, and injury mechanisms provides the most reliable and valid data, and accessing these data through formal channels allows external researcher access for consultation on methodological use of the data, operational definitions of variables, understanding of limitations, and ultimately, appropriate interpretations.
MRAP was uniquely developed through the mutual efforts of the NFL and NFLPA to enhance oversight of the access and utilization of NFL player electronic workplace injury and illness data for the purposes of regulating clinical research. By governing across all proposed research to ensure each project is unique, methodologically sound, and properly addresses confidentiality and privacy concerns, the protected interests of the patients (player) and institution (NFL) remain paramount to the goals of the intended research. MRAP has proved successful in achieving its aims and could be adapted for other sports and areas of similar sports medicine research interest.
Dr. Mack and Ms. Zeidler are employees of IQVIA. Dr. Matava is an employee of the Washington University School of Medicine and is chair of the NFL Player Health and Safety Department’s Research and Innovation Committee. Dr. Sills is chief medical officer and an employee of the National Football League. Dr. Solomon is senior medical advisor (paid consultant) to the National Football League, Player Health and Safety Department. All authors have received expense reimbursements for professional and/or scientific activities related to National Football League Player Health and Safety Department initiatives.
There were no funding sources for the writing of this article.
1. Dreyer N, Mack C, Anderson R, et al. Using a real-world evidence platform for player health and safety in the NFL: lessons for electronic-health-record driven research. Sports Health
. 2019; 11:440–5.
2. National Football League. NFL Medical Committees Overview. Play Smart Play Safe
web site. 2018. [cited 2019 July 17]. Available from: https://www.playsmartplaysafe.com/resource/nfl-medical-committees-overview/
3. Carey J, Huffman G, Parekh S, et al. Outcomes of anterior cruciate ligament injuries to running backs and wide receivers in the National Football League. Am. J. Sports Med
. 2006; 34:1911–7.
4. Cinque M, Hannon C, Bohl D, et al. Return to sport and performance after anterior cruciate ligament reconstruction in National Football League linemen. Orthop. J. Sports Med
. 2017; 5:2325967117711681.
5. Dai JB, Li AY, Haider SF, et al. Effects of game characteristics and player positions on concussion incidence and severity in professional football. Orthop. J. Sports Med
. 2018; 6:2325967118815448.
6. Dodson CC, Secrest ES, Bhat SB, et al. Anterior cruciate ligament injuries in National Football League athletes from 2010 to 2013: a descriptive epidemiology study. Orthop. J. Sports Med
. 2016; 4:2325967116631949.
7. Erickson B, Harris J, Heninger J, et al. Performance and return-to-sport after ACL reconstruction in NFL quarterbacks. Orthopedics
. 2014; 37:e728–34.
8. Hsu W. Performance-based outcomes following lumbar discectomy in professional athletes in the National Football League. Spine
. 2010; 35:1247–51.
9. Hsu W. Outcomes following nonoperative and operative treatment for cervical disc herniations in National Football League athletes. Spine
. 2011; 36:800–5.
10. Jack RA, Evans DC, Echo A, et al. Performance and return to sport after sports hernia surgery in NFL players. Orthop. J. Sp. Med
. 2017; 5:2325967117699590.
11. Jack RA 2nd, Sochacki KR, Navarro SM, et al. Performance and return to sport after clavicle open reduction and internal fixation in National Football League players. Orthop. J. Sp. Med
. 2017; 5:2325967117720677.
12. Jack RA 2nd, Sochacki KR, Gardner SS, et al. Performance and return to sport after achilles tendon repair in National Football League players. Foot Ankle Int
. 2017; 38:1092–9.
13. Jack RA 2nd, Sochacki KR, Navarro SM, et al. Performance and return to sport after nonoperative treatment of clavicle fractures in National Football League players. Orthopedics
. 2017; 40:e836–43.
14. Krill MK, Borchers JR, Hoffman JT, et al. Effect of position, time in the season, and playing surface on Achilles tendon ruptures in NFL games: a 2009-10 to 2016-17 review. Phys. Sportsmed
. 2017; 45:259–64.
15. Krill M, Hoffman J, Yang J, et al. Previous foot injuries associated with a greater likelihood of Achilles tendon ruptures in professional American football players. Phys. Sportsmed
. 2018; 46:342–8.
16. Mai H, Schneider J, Mitchell S, et al. The difference in clinical outcomes between ACDF and posterior foraminotomy in professional athletes. Spine
. 2016; 242–3.
17. Mai H, Chun D, Schneider A, et al. Performance-based outcomes after anterior cruciate ligament reconstruction in professional athletes differ between sports. Am. J. Sports Med
. 2017; 45:2226–32.
18. Mai MT, Burgmeier RJ, Mitchell SM, et al. Does the level of cervical disc herniation surgery affect performance-based outcomes in National Football League athletes? Spine (Phila Pa 1976)
. 2016; 41:1785–9.
19. Mai H, Alvarez A, Freshman R, et al. The NFL Orthopaedic Surgery Outcomes Database (NO-SOD): the effect of common orthopaedic procedures on football careers. Am. J. Sports Med
. 2016; 44:2255–62.
20. Lynch T, Kosanovic R, Gibbs D, et al. Performance-based outcomes after operative management of athletic pubalgia/core muscle injury in national football league players. Orthop. J. Sports Med
. 2017; 5:2325967117S00243.
21. Myer GD, Smith D, Barber Foss KD, et al. Rates of concussion are lower in National Football League games played at higher altitudes. J. Ortho. Sports Phys. Th
. 2014; 38:164–72.
22. Nathanson JT, Connolly JG, Yuk F, et al. Concussion incidence in professional football: position-specific analysis with use of a novel metric. Orthop. J. Sports Med
. 2016; 4:2325967115622621.
23. Navarro SM, Sokunbi OF, Haeberle HS, et al. Short-term outcomes following concussion in the NFL: a study of player longevity, performance, and financial loss. Orthop. J. Sports Med
. 2017; 5:2325967117740847.
24. National Football League [Internet]. Play Smart Play Safe. 2019. NFL Medical Committees Overview. [cited 2019 Aug 23]. Available from: https://www.playsmartplaysafe.com/resource/nfl-medical-committees-overview/
25. International Committee of Medical Journal Editors [Internet]. Philadelphia (PA): American College of Physicians. 2017. Roles & Responsibilities: Defining the Role of Authors and Contributors. [cited 2017 Feb 08]. Available from: http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html