Design and analysis of high-risk medical device clinical trials for diabetes monitoring and treatment: a review

Abstract

Class III medical devices are those devices that support or sustain human life or prevent impairment of human health but, when used, pose high risks to the patients. There were 15 class III medical devices that were approved by the US Food and Drug Administration from September 1, 2005 to December 31, 2018 for glucose monitoring and diabetes treatment. In this article, we reviewed the study design and statistical analysis in clinical trials that were conducted to provide evidences for safety and effectiveness of these medical devices. Two exemplars of clinical trials for “Eversense continuous glucose monitoring system” and “t:slim X2 Insulin Pump with Basal-IQ Technology” are discussed in details.

Introduction

Diabetes mellitus (DM) is a metabolic disorder characterized by a chronic state of hyperglycemia, caused by absolute or relative insulin deficiency and its metabolic consequences. The prevalence of diabetes is steadily on the rise, and the number of adults with diabetes has quadrupled in the past 25 years, from 108 million in 1980 to 422 million in 2014,^[1] and will almost double by 2040.^[2] Current guidelines for the management of hyperglycemia recommend the use of intravenous insulin in the intensive care unit and subcutaneous basal or basal-bolus insulin regimens in general medicine and surgery settings.^[3,4] Improving glycemic control, while minimizing the hypoglycemia and hyperglycemia rate, is majorly important in the hospital as both hyperglycemia and hypoglycemia pose significant risks for patients with DM, inducing poor clinical outcomes and mortality.^[5,6] During recent decades, the treatment of diabetes has changed profoundly, and the ability to control blood glucose levels has improved dramatically. The improvements are particularly evident through the medical devices used to monitor glucose and to administer insulin.

To be legally marketed in the United States, many medical devices must be reviewed by the US Food and Drug Administration (FDA), with FDA's Center for Devices and Radiological Health (CDRH) being primarily responsible for medical device review. The pre-market evaluation process, which is based on FDA's regulations, varies according to the complexity of the device and the degree of risk to patients posed by the device. Devices are categorized into 3 regulatory classes by the increasing risk to patients: classes I, II, and III, and the stringency of the approval process is proportional to the device risk.^[7,8] All FDA regulated medical devices are subject to “general controls,” and the minimum regulations include 5 elements: establishment registration, device listing, good manufacturing practices, labeling, and pre-market notification. Class I devices are low-risk devices for which general controls “are sufficient to provide reasonable assurance of the safety and effectiveness of the device.” Class II devices are moderate-risk devices and may include new devices for which information or special controls are available to reduce or mitigate risk. Special controls may include special labeling requirements, mandatory performance standards, and postmarket surveillance. Most class II devices require pre-market notification via the 510(k) process, in which the equivalence of safety and effectiveness to a device already on the market (a predicate device) is demonstrated, although some class II devices may be exempted from this process.^[9] Class III includes devices which are life-supporting or life-sustaining and devices which present a high or potentially unreasonable risk of illness or injury to a patient. The assurance of the safety and effectiveness for these devices is demonstrated by a thorough pre-market approval (PMA) application or a 510(k) notification.

The class III devices enter the market through 2 different paths. Some class III devices may be cleared via the 510(k) process if they show substantial equivalence to pre-amendment class III devices (i.e., introduced to the US market before May 28, 1976), for which the regulation calling for the PMA application has not been published in Code of Federal Regulations Title 21.^[10] The other path involves conducting clinical studies and submitting a PMA application, with a requirement of sufficient valid scientific evidence to assure that the device is safe and effective. The clinical study design and data necessary to support the approval of a specific class III device depends on the intended use that was proposed for the new device. After a device receives PMA approval, the FDA makes its approval order, labeling guidelines, and a summary of safety and effectiveness data (SSED) publicly available. The SSED summarizes the Device Description, Preclinical Evidence, and Clinical Evidence that serves as the basis of the decision to approve or deny the PMA.^[11] In this paper, the study evidence in the approved PMAs of high-risk medical devices for diabetes monitoring and treatment from 2005 to 2018 were systematically reviewed. The goals were to (1) select SSEDs of PMAs for approved diabetes monitoring or treatment medical devices; (2) provide exemplars for 2 representative diabetes diagnosis or treatment devices; (3) summarize the clinical study design and analysis in the approved PMAs of high-risk medical devices for diabetes monitoring and treatment.

Review method: selection of summary of safety and effectiveness data

The SSED for the PMAs of high-risk (class III) FDA-approved medical devices were downloaded from the public database of PMA at http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMA/pma.cfm. We considered all PMAs published within the “Decision Date” from September 1, 2005 to December 31, 2018 but specified the “Supplementary Type” as “Original Only.” We did not restrict the search on devices in any medical field. Therefore, the SSEDs from all the 18 “Advisory Committee” panels were included (Advisory Committees are the experts outside the FDA that are organized to panels to provide independent advice on regulatory issues related to human and veterinary drugs, vaccines and other biological products, and medical devices; please see Fig. 1 for the names of 18 panels for the CDRH). All the SSEDs that met our search criteria were downloaded, and the product names, approval data, PMA numbers, indications for use, and product categories were summarized in a database. This search was conducted in August 2019.

Figure 1: Numbers of SSEDs of FDA-approved high-risk medical devices in 18 advisory committee panels (SSEDs published by the CDRH between September 1, 2005 and December 31, 2018 were counted). All 15 medical devices for diabetes diagnosis or treatment were classified to clinical chemistry panel. CDRH = Center for Devices and Radiological Health, FDA = US Food and Drug Administration, SSED = summaries of safety and effectiveness data.

A total of 414 SSEDs have been published during the 14-year interval. During the year of 2017, the highest number of high-risk medical devices were approved (n = 46), and the lowest number (excluding the year of 2005 as we only considered 2005 partially) occurred in the year of 2009 (n = 15). By categories, 414 SSEDs belonged to 18 categories according to the FDA advisory committee classification (see Fig. 1). Most of the devices (n = 145, 35.0%) were classified as cardiovascular devices. Of other categories, 12.8% were classified into microbiology panel, 8.7% belonged to the orthopedic panel, 7.5% belonged to general and plastic surgery devices, 6.0% belonged to the ophthalmic panel, 3.9% belonged to the clinical chemistry panel, 3.9% belonged to the pathology panel, and 3.6% belonged to the radiology panel.

We selected the medical devices for diabetes diagnosis or treatment by first searching the keywords “diabetes,” “glucose,” and “insulin” in our database. A total of 17 SSEDs had these keywords in their indications for use. We excluded “COBRA PzFTM NanoCoated Coronary Stent System” and “Resolute MicroTrac and Resolute Integrity Zotarolimus-Eluting Coronary Stent Systems” because both devices were used for improving coronary luminal diameter in patients. As a result, a total of 15 medical devices for diabetes diagnosis or treatment were selected. We then repeated this procedure by searching through the downloaded SSED files using the same keywords. We excluded the SSEDs that contained the keywords in their study designs but were not related to diabetes diagnosis or treatment. Then, the same 15 devices were selected. The product names, approval data, and PMA numbers of the 15 selected medical devices were summarized in Table 1.

Table 1: Product names, approval dates, and PMA numbers of selected 15 medical devices for diabetes diagnosis or treatment: 8 CGM devices, 4 insulin pumps, and 3 artificial pancreas device systems.

The selected 15 devices were all classified as clinical chemistry devices. Of these 15 devices, 2/3 of the devices were approved after 2015. These devices for diabetes diagnosis or treatment can be mainly classified into 3 categories: 8 diagnostic devices were all continuous glucose monitor (CGM) devices, 4 devices were insulin pumps for continuous subcutaneous insulin infusion, and the remaining 3 devices were artificial pancreas device systems (APDSs) that linked a CGM to an insulin pump to automatically adjust insulin infusion based on pre-specified thresholds.^[12]

Exemplars of clinical trials for evaluating high-risk medical devices trials for diabetes monitoring and treatment

Eversense continuous glucose monitoring system

The Eversence CGM System (P160048^[13]) is a CGM manufactured by Senseonics, Incorporated (Germantown, MD). This system was approved by the FDA on June 21, 2018 and is “indicated for continually monitoring glucose levels in adults (age 18 and older) with diabetes for up to 90 days. The system is intended to provide real-time glucose readings, provide glucose trend information, and provide alerts for the detection and prediction of episodes of low blood glucose (hypoglycemia) and high blood glucose (hyperglycemia).” This CGM is a prescription device, only functioning as an adjunctive device not a replacement for standard home glycose monitoring devices. The system consists of 4 parts: the Eversense Sensor, Eversense Smart Transmitter, Eversense Mobile Medical Application, and Insertion Tools.

The sponsor designed and performed 2 principal clinical studies, PRECISE II and PRECISION, to demonstrate the safety and effectiveness of the device. The PRECISE II study was a multicenter, non-randomized, single-arm, blinded, prospective study. 90 adult subjects with DM were enrolled at 8 sites in the United States for both clinic visits and home use of the system. 82 of the enrolled subjects completed the day 90 visit with accuracy data collection, participating in a total of 7 visits over a period of approximately 5 months. The Eversence System's accuracy was evaluated on days 1, 30, 60, and 90 during clinic visits by comparing sensor glucose values and plasma glucose values measured on a bedside glucose analyzer every 5 to 15 min for a period of approximately 4.5 to 12.5 h. The PRECISION study design was similar to the PRECISE II trial except for additional accuracy assessments on days 7 and 14 for sensory accuracy; patients underwent sleep assessments for accuracy and system performance during sleep; no blinding of the patients to the glucose values and alerts during the study. In the PRECISION study, 35 enrolled subjects completed the 90-day accuracy evaluation. The sponsor presented data from a retrospective processing of the raw sensor data from the PRECISE II and PRECISION studies using a new algorithm, “SW 602” which was improved for a better early sensor life and hypoglycemic range over the sensor life.

The clinical endpoint of both studies was the performance of the system compared to the laboratory reference venous plasma sample measurements while assessing the system-reference matched pairs obtained in the in-clinic sessions. The primary safety endpoint was the incidence of device-related or sensor insertion/removal procedure-related serious adverse events (SAEs) through 90 days postinsertion or sensor removal and follow-up. A total of 1.1% and 8.6% of subjects in the PRECISE II and PRECISION studies, respectively, experienced a SAE and were resolved. No unanticipated adverse events and no infections occurred. Analysis of dexamethasone exposure during the PRECISION study showed that in the first 8 days and through the visits over 90 days, a subset of 8 subjects did not have dual-energy x-ray absorptiometry (DXA) levels greater than 50 pg/mL. At 90 days, the removed sensors retained approximately 80% to 90% of their original DXA content; thus, over the course of 90 days, approximately 0.18 to 0.35 mg of DXA was released into the body from a single sensor.

The effectiveness of the Eversense CGM System was analyzed with “SW-602” on the observed accuracy of sensors in 90 evaluable patients in the PRECISE II study and 35 patients in the PRECISION study. These patients contributed 15,753 and 15,170 CGM-comparator matched glucose data pairs, respectively. The primary effectiveness endpoint for both studies was the performance evaluation of the Eversense CGM System compared to the blood glucose values measured by laboratory glucose analyzer during days 1, 7, 14, 30, 60, and 90 of the in-clinic sessions. The CGM system and comparator agreed between 71.4% to 100.0% and 60.0% to 100.0% within a given range of paired comparator measurements in the PRECISE II and PRECISION study, respectively. The mean absolute relative difference was 8.5% and 9.6%, and the median absolute relative difference was 9.6% and 7.1% for the overall (40–400 mg/dL) CM glucose range for the PRECISE II and PRECISION study, respectively. The system's precision was determined through analysis of the data from 2 sensors at the same time on 15 subjects (10,371 between-sensor matched pairs) in the PRECISE II trial and on 27 subjects (37,307 between-sensor matched pairs) in the PRECISION trial. For all the pairs, the mean difference between sensor 1 and sensor 2 was −1.1 mg/dL with a standard deviation (SD) of 18.2 mg/dL, percent absolute relative difference (PARD) of 9.0% and percent coefficient of variation (%CV) of 6.4% in the PRECISE II study. For all the pairs, the mean difference was −0.8 mg/dL, SD of 10.5 mg/dL, PARD of 9.9%, and %CV of 7.0% in the PRECISION study.

The analysis of the data supports acceptable accuracy across the claimed measuring range (40–400 mg/dL), precision, 90-day wear period claims for the sensor, and effective alerts for detection and prediction episodes of hypoglycemia and hyperglycemia. The analysis of the data from the clinical studies demonstrates the safety, with associated risks, and effectiveness of the Eversense CGM System in the study population, which is representative of the intended use population.

t:slim X2 Insulin Pump with Basal-IQ Technology

The t:slim X2 Insulin Pump with Basal-IQ Technology (P180008^[14]) is an automated insulin dosing, threshold suspending, non-adjunctive invasive glucose sensor that was approved by the FDA on June 21, 2018. This system is manufactured by Tandem Diabetes Care, Inc (San Diego, CA) and is made up of 2 parts: the t:slim X2 Insulin Pump, which contains the Basal-IQ Technology, and a CGM. This system is intended for the use of “subcutaneous delivery of insulin, at set and variable rates, for the management of DM in persons requiring insulin. The t:slim X2 Insulin Pump can be used solely for continuous insulin delivery and as part of the t:slim X2 Insulin Pump with Basal-IQ Technology System. When the system is used with the Dexcom G5 Mobile CGM or a compatible integrated continuous glucose monitoring (iCGM), the Basal-IQ Technology can be used to suspend insulin delivery based on CGM sensor readings.” Furthermore, the t:slim X2 Insulin Pump with Basal-IQ Technology is indicated for use in individuals 6 years of age and higher; for single patient use, requiring a prescription; and for use with NovoLog or Humalog U-100 insulin. The Basal-IQ Technology is an insulin pump software with a predictive low glucose suspend (PLGS) algorithm that assesses glucose information provided by the CGM ever 5 min, detects glucose levels below 70 mg/dL or predicts if glucose levels will decrease below 80 mg/dL, and temporarily suspends insulin delivery in response.

The sponsors designed and performed a pivotal clinical study to demonstrate that the Basal-IQ Technology of the already approved t:slim X2 Insulin Pump with Dexcom G5 CGM works safely and as intended. This pivotal study was a multicenter, randomized, crossover design, at-home evaluation of 107 subjects with type 1 diabetes. Subjects were enrolled at 4 study sites in the United States, and they were either insulin pump users, multiple daily injection of insulin (MDI) users, CGM naïve users (pump or MDI users), or experienced CGM users (also pump or MDI users). Of the 107 enrolled subjects, 102 subjects participated in the study with 1 3-week period of t:slim X2 Insulin Pump with Basal-IQ (Basal-IQ enabled) use and another 3-week period of the control, sensor augmented pump (SAP) use. Before the study, the study participants underwent a run-in phase where they either completed CGM training for 10 to 14 days and/or SAP training for 14 to 28 days, according to the subject's current use of a CGM or Tandem pump with a Dexcom CGM. Before the initiation of the crossover study, a 10-day pilot period was conducted where 10 subjects wore the Dexcom G5 CGM and t:slim X2 Insulin Pump with Basal-IQ to evaluate the safety metrics and usability. After determining patient eligibility and obtaining consent, subjects were randomly assigned to either Group A or Group B. For the first 3 weeks of the crossover study (period 1), Group A used PLGS (Dexcom G5 CGM with the t:slim X2 Insulin Pump with Basal-IQ), while Group B used SAP (Dexcom G5 CGM with t:slim X2 Insulin Pump without Basal-IQ). For period 2, the study treatments crossed over between the groups, with Group A using SAP and Group B using PLGS. At the end of both study periods, these parameters were assessed: adverse events, new medical conditions, and medications, hemoglobin A1c (HbA1c) level, and a Basal-IQ usability survey following the PLGS treatment. The sponsor conducted a reanalysis of the clinical data from previous Dexcom G5 CGM studies (P120005^[15]) through a retrospective application of the Basal-IQ algorithm performance to evaluate the accuracy of the Basal-IQ Technology.

This clinical study was designed to show the effectiveness of the t:slim X2 Insulin Pump with Basal-IQ Technology by comparing (1) the Dexcom G5 CGM with the t:slim X2 Insulin Pump with Basal-IQ (PGLS) to (2) the Dexcom G5 CGM with the t:slim X2 Insulin Pump without Basal-IQ (SAP). When comparing the percent of CGM glucose sensor values below 70 mg/dL for 102 subjects, the median was 2.6% (1.4%, 4.0%) and mean was 2.1% ± 2.8% for the PLGS arm, while the median was 3.2% (1.9%, 6.1%) and the mean was 4.5% ± 3.9% for the SAP arm. The mean sensor glucose was also found to be 159 ± 25 mg/dL for the PGLS arm, which was similar to the SAP mean of 150 ± 27 mg/dL. The similarity was also found among the percentage of sensor values 70 to 180 mg/dL at baseline as it was 65% ± 15% and 63% ± 15% for the PGLS and SAP arms, respectively. These comparative performance data demonstrated “the Basal-IQ feature of the already approved t:slim X2 Insulin Pump” with the Dexcom G5 CGM (P140015^[16]) functions safely and as intended. Analysis to determine the Basal-IQ algorithm accuracy showed that 83.91% of the suspends from the analysis of 109 subjects (6–86 years) were True Suspends (Basal-IQ commanded the suspend and blood glucose became <80 within 30 min), and there were 46.4%, 60.5%, and 68.9% of True Resumes (Basal-IQ commands resume and blood glucose > previous blood glucose or current blood glucose ≥70 and in 30-min blood glucose >80) after 0, +5, and +10 min, respectively. These results presented that the Basal-IQ feature, as intended, suspends and resumes insulin delivery according to low and high blood glucose levels, respectively.

Safety outcomes were collected from the 102 subjects to demonstrate the safeness of the device. No unanticipated adverse device effects or SAEs were reported related to study procedure and device. The results for the safety outcomes were found to be comparable between the PLGS and SAP arms of the study.

Overview on design and analysis of high-risk medical device clinical trials for diabetes monitoring and treatment

High-risk diabetes monitoring medical devices

For the CGM devices, 8 PMAs included 9 pivotal clinical studies (P160048 described 2 pivotal clinical studies, the PRECISE II study and PRECISION study), and all 9 pivotal clinical studies are non-randomized, single-arm, multicenter, prospective studies. The subjects were enrolled at 3 to 8 centers for 6 pivotal clinical studies, while the number of centers was not mentioned in the other 3 studies. A mean (±SD), median (minimum, maximum) of 71.9 (±20.5), 72 (35, 91) patients were enrolled in these 9 pivotal clinical studies. Eight studies enrolled patients with type I or type II DM, and the pivotal clinical study of P050020,^[17] FreeStyle Navigator Continuous Glucose Monitoring System only enrolled type I DM patients. The proportion of type I DM patients were mentioned in 7 out of 8 studies with a mean (±SD), median (minimum, maximum) of 78.9% (±9.5%), 80% (67.8%, 95.8%). The pivotal clinical studies generally include an in-clinic and home use portion, with the main analysis of the clinical study focusing on the in-clinic portion of the study. Across different pivotal clinical studies, the length that the subjects wore the electrochemical sensors of the glucose monitoring system varies from a 5-day study and 1-week follow-up visit (P050020,^[17] FreeStyle Navigator Continuous Glucose Monitoring System) to 10 days following a 12-h warm up period (P160030,^[18] FreeStyle Libre Flash Glucose Monitoring System, Alameda, CA) or up to 90 days of use (P160048,^[13] Eversense Continuous Glucose Monitoring System). The study length depended on the full claimed use life following sensor application for each device. In-clinic testing consisted of frequent sample testing (FST) of blood samples that were obtained every 5 to 15 min, and the FST lasted for a period of approximately 4 to 12 h and generally occurred at the beginning, middle, and end days of the CGM use. Each subject was fitted with 2 to 3 sensors in all 9 pivotal clinical studies except for some patients in P120005^[19] (Dexcom G4 PLATINUM Continuous Glucose Monitoring System, San Diego, CA) and P160048 pivotal clinical studies who only received 1 sensor. Only 3 clinical studies from the P160030 and P160048 submissions clearly defined the first and secondary sensor, and the analysis of effectiveness was based on all data points collected from the primary sensors worn by the study subjects, while other studies analyzed data from all inserted sensors. During sensor clinic visits, qualifying subjects underwent hyperglycemia and hypoglycemia challenges in 2 pivotal clinical studies (P150029,^[20] iPro2 Continuous Glucose Monitoring System and P160048).

The primary objective of the pivotal clinical studies for the CGM was to evaluate the system performance to laboratory reference measurements on venous blood samples for all study subjects. The effectiveness measurements were based on the comparison between the measurements from the proposed glucose monitoring systems and the blood glucose values measured by a clinical laboratory reference method, generally from the Yellow Springs Instrument (YSI) during in-clinic sessions. These studies generally do not include clearly specified primary endpoints, and the pivotal clinical study data were presented using multiple analyses. The total mean (±SD), median (minimum, maximum) of 15,759 (±12,231), 15,001 (2846, 45,051) sensor-YSI matched pairs were collected in these 9 pivotal clinical studies for the effectiveness analysis. One key effective analysis to evaluate the agreement of the new device to the blood glucose value was assessed by calculating the percentage of new CGM readings within 15%, 20%, 30%, 40%, and greater than 40% of the YSI values for values greater than 80 mg/dL and within absolute 15, 20, 30, 40 mg/dL, and greater than 40 mg/dL for values less than or equal to 80 mg/dL. The mean (±SD), median (minimum, maximum) percentage of total readings within 15 mg/dL or 15%, 20 mg/dL or 20% are 78.1% (±7.4%), 80.9% (68.9%, 86.8%) and 82.8% (±13.3%), 83.8% (50.2%, 83.8%), respectively. Of note, the pivotal clinical studies from 2 early PMAs (P050012,^[21] DexCom STS Continuous Glucose Monitoring System and P050020, San Diego, CA) did not provide 15%/15% results in the SSED.

Additional effectiveness analyses from these CGM pivotal clinical studies include (1) mean and median absolute relative differences from YSI; (2) number and percentage of reference glucose measurements collected while the CGM read “low” (generally <40 mg/dL), or “high” (>400 or >500 mg/dL depending on whether the CGM system does not display glucose readings above 400 or 500 mg/dL); (3) the percentage of concurring glucose values for each reference glucose range with each CGM glucose range; (4) alert performance that may include hypoglycemia alert rate, hyperglycemia alert rate, hypoglycemia detection rate, and hyperglycemia detection rate. The hypoglycemia detection rate alerted when the CGM reads were below the low threshold and the user's blood glucose was actually below that low threshold within ±15 or 30 min of the alert, and the hypoglycemia alert rate alerted when the CGM reads were below the low threshold and the user's blood glucose was actually below that low threshold within 15 or 30 min following the alert; (5) precision analysis, evaluated by comparing the results from 2 separate sensors worn on the same subject at the same time; (6) sensor stability by stratifying the comparison results between the reference glucose values and their paired sensor points at different FST days. The pivotal clinical studies for early PMA submissions (P050012 and P050020) also included the Clarke Error Grid Analysis in the SSED, which was developed in 1987 and broke down a scatterplot of a reference glucose meter and an evaluated glucose meter into 5 different regions.^[13] The pivotal clinical studies from recent PMA submissions (P160030 and P160048) generally included the glucose trend accuracy analysis to show the percentage of time that a specific trend arrow displayed by the CGM was observed, as it corresponds to the true direction and rate of change in glucose levels as measured using the reference method. The analysis of device safety was generally based on all the subjects enrolled to evaluate the adverse events that occurred during the course of the study, the mean (±SD), median (minimum, maximum) number of adverse events per subject is 0.29 (±0.17), 0.24 (0.06, 0.59) for these 9 pivotal clinical studies. Detailed information about the adverse events for the 8 selected CGM devices was summarized in Table 2.

Table 2: Adverse event summary table from the PMA SSED of 8 selected continuous glucose monitor devices (9 pivotal clinical studies).

High-risk diabetes treatment medical devices

The selected 7 diabetes treatment devices (either insulin pumps or APDSs) are all insulin delivery systems, generally paired with a specific CGM sensor and transmitter to detect trends and tracking patterns in persons with diabetes. The major pivotal clinical studies for these treatment devices are also to evaluate the performance of the paired CGM to laboratory reference measurements which are similar to those mentioned in the High-risk diabetes monitoring medical devices section. For 2 out of the 4 insulin pumps (P130007,^[22] Animas Vibe System and P140015,^[16] t:slim G4 Insulin Pump with Dexcom G4 Platinum CGM), the sponsor did not provide any additional clinical study data and simply referred to previous pivotal clinical studies of the paired CGM devices. The other 2 insulin pumps (P150019, Paradigm REAL-Time Revel System and P180008, t:slim X2 Insulin Pump with Basal-IQ Technology) conducted a reanalysis to characterize the performance of the system and determine the accuracy since different algorithms were used to convert signals from the sensor into glucose values for the proposed insulin pump and previously approved CGM. Two APDSs (P120010,^[23] MiniMed 530G System and P160017,^[24] MiniMed 670G System) performed new pivotal accuracy studies, while the other artificial pancreas device (P150001,^[25] MiniMed 630G System with SmartGuard) referred to the clinical study described in the SSED of the previously approved similar device. All 5 clinical accuracy studies enrolled patients with type I or type II DM (P150001 referred the same study as that of P120010 and P180008 did not have a full performance evaluation study) and the mean (±SD), median (minimum, maximum) of 103.4 (±41.3), 90 (72, 176) patients were enrolled in these 5 pivotal clinical studies. The total mean (±SD), median (minimum, maximum) of 13,190 (±7678), 14,847 (2922, 23,754) sensor-YSI matched pairs were collected in these 5 pivotal clinical studies for the effectiveness analysis and the mean (±SD), median (minimum, maximum) percentage of total readings within 15 mg/dL or 15%, 20 mg/dL or 20% are 66.7% (±6.8%), 70.4% (55.0%, 71.0%), and 78.3% (±5.9%), 81.3% (68.0%, 82.0%), respectively.

Three APDSs (P120010, P150001, and P160017) and 1 insulin pump (P180008) included a threshold suspend feature in the intended use, which can be programmed to temporarily suspend delivery of insulin when the sensor glucose value falls below a pre-defined threshold. Therefore, the sponsors for these 4 devices also conducted additional clinical studies to evaluate the safety and efficacy of the threshold feature of the devices. All 4 clinical studies only enrolled subjects with type I diabetes, and the study design varied across these studies. The additional clinical studies of the P120010 and P180008 submissions applied randomized, simple 2-period, 2-treatment cross-over studies to assess the efficacy of the threshold suspend tool. The 2-treatment options are threshold suspend tool turned “ON” or “OFF.” Fifty and 103 patients were enrolled in these 2 clinical studies. The effectiveness results for P120010 only demonstrated that the pump successfully suspended insulin delivery when the sensor value fell below the threshold (70 mg/dL), and there was not a clinically significant difference in the nadir glucose between the 2-treatment groups. The effectiveness results for P180008 provides the information on the percentage of sensor glucose values below the threshold (70 mg/dL), and the study also compared the time spent in low sensor glucose value ranges, time spent in high sensor glucose value ranges, the mean sensor glucose value, and the percentage of sensor values 70 to 180 mg/dL at baseline and for each arm of the study. The P150001 SSED included a multicenter, in-home, randomized parallel adaptive study designed to compare HbA1c and CGM-based nighttime low sensor glucose events in a treatment arm (threshold suspend) to a control arm (disenabled threshold suspend). A total of 247 subjects were randomized, 121 to the Threshold Suspend arm, and 126 to the Control arm. The sponsor pre-specified a primary safety endpoint regarding the change of HbA1c level from randomization to the end of the study, and a primary effectiveness endpoint regarding the reduction of the area under the curve for nighttime low sensor glucose events (≤65 mg/dL) during the 3-month study phase between the Threshold Suspend and the Control Group. P160017's additional clinical study was a multicenter, single-arm, observational, at-home, and hotel clinical study with no controls. The sponsor enrolled 126 subjects (ages 14–75 years) at 10 investigational centers. There were no statistically powered endpoints in this Auto Mode study, which was a descriptive study to evaluate the safe use of the proposed device. The effectiveness results provide an overall summary of the run-in phase and study phase (home and hotel) for all subjects in the study, and the data presented includes information about subjects’ glucose levels, insulin delivered, and weight during run-in vs study phases.

Discussion and conclusion

In this manuscript, we selected 15 PMAs, reviewed the SSEDs of approved high-risk diabetes monitoring or treatment medical devices from 2005 to 2018, and discussed 2 representative devices. We also summarized the pivotal clinical study design and analysis in these 15 SSEDs. The primary objective of the pivotal clinical studies is generally intended to determine the clinical accuracy of the device with respect to a glucose reference method, either targeting the diabetes monitoring devices or the paired CGM for the treatment devices. For the diabetes treatment devices, the sponsor justified that the clinical accuracy testing for the paired CGM plus the pre-clinical and human factors/usability testing completed for the PMA may support the operation of this device as a system. There is generally no pre-specified primary effectiveness endpoint for this clinical accuracy study, and the agency may make decisions based on multiple analyses and the integrity of the effectiveness results. For some approved diabetes treatment devices, in addition to the proposed clinical accuracy testing, the sponsor also conducted additional clinical studies intended to support specific features proposed in the intended use, and the study design and data analysis for these studies are not consistent across different devices. The clinical study proposed should be adequate to support the proposed intended use, and the sponsor is encouraged to obtain FDA feedback regarding clinical study design and analysis before an intended pre-market submission through the pre-submission program.^[26]

Acknowledgments

HZ would thank the Robert H. Lurie Comprehensive Cancer Center of Northwestern University in Chicago, IL for the use of the Quantitative Data Sciences Core. The Lurie Cancer Center is supported in part by an NCI Cancer Center Support Grant #P30CA060553.

Author contributions

None.

Financial support

None.

Conflicts of interest

The authors declare no conflicts of interest.