Patient response prediction with logistic regression in gastrointestinal endoscopy under midazolam-alfentanil sedation performed as well as response surface model

Abstract

Background: 

Researchers have used logistic regression (LR) and non-linear response surface models (RSMs) to predict patient responses to sedation. The reduced Greco and hierarchy RSMs have proven to be more appropriate than other RSMs in gastrointestinal endoscopies using midazolam and alfentanil. In this study, we evaluate the performance of a simpler model, LR, and compared it with that of RSM.

Methods: 

Thirty-three patients who received esophagogastroduodenoscopy (EGD) and colonoscopy sedation with midazolam and alfentanil were enrolled in the study. LR was performed for the EGD group and validated using the colonoscopy group. The two RSMs were performed using the same process, and performances and receiver operating characteristic (ROC) curves of the models were evaluated.

Results: 

The native EGD LR model had an ROC curve area of 0.94. For external validation, the ROC curves were 0.92, 0.94, and 0.94 for the reduced Greco, hierarchy, and LR models, respectively. Pairwise comparison between models was not significant.

Conclusion: 

The LR model performed as well as RSM in generalizing the predicted sedative effect of midazolam and alfentanil during gastrointestinal endoscopies. LR may be used for generalization across patients experiencing procedures with similar stimulus intensities.

1. Introduction

Pharmacodynamic is a primary concern for clinicians. Data sets are gathered to fit a mathematical model, which is then used to make future predictions. Pharmacodynamics studies are traditionally performed with isobolograms^1,2 or concentration effect curves.³ The major drawback of these methods is the lack of ability to cover the entire spectrum of the specific pharmacodynamic endpoints (for example 0%–100% chance of loss of response). The need for this is especially important in anesthetic practice, where drugs are administered rapidly, have rapid onset and offset that occur within minutes and often have very serious side effects if dosing is not carefully handled. Large drug concentration fluctuations in the body are inevitable in this particular dynamic phase of anesthesia. A model therefore has to be flexible enough to be useful under such conditions. Constructing such a model would greatly help anesthetists maintain a suitable anesthetic depth, which in turn could reduce procedure time and post-procedural pain.⁴

In the past decade, pharmacodynamic studies in anesthesia have involved a versatile group of models called response surface models (RSMs).⁵ A RSM is a surface that accommodates the entire set of isobolograms and concentration effect curves. It usually deals with two drugs simultaneously,^6–8 but a three drug model can also be derived using complex mathematics.^9–11 Many studies have validated the clinical application of RSMs.^9,12,13 Our group has reported midazolam-alfentanil RSMs¹⁴ that can reasonably predict a patient's response during gastrointestinal endoscopy sedation.

Logistic regression (LR) is an important tool that offers both simplicity and accuracy simultaneously. LR can take different forms to calculate the probability of a binary outcome.¹⁵ Drug analyses including pharmacokinetics and pharmacodynamics are often performed with LR.^16–19 We hypothesize that a simpler model such as LR can predict patient responses as well as the RSMs during gastrointestinal endoscopy sedation.

2. Methods

Patient management: After approval from the Institutional Review Board (IRB 2014–12–001BC) at the Taipei Veterans General Hospital, we enrolled patients younger than 65 years with ASA (American Society of Anesthesiologists) physical statuses of I to III. The requirement for informed consent was waived by the IRB. All methods were performed in accordance with the IRB's ethical and clinical regulations. All patients received esophagogastroduodenoscopy (EGD) and colonoscopy as a single-stage procedure and were sedated with midazolam and alfentanil. They were excluded if they had a history of verbal communication impairments or a history of sedative, opioid, or chronic alcohol use. The initial dosage of Midazolam and Alfentanil was 0.03–0.04 mg kg⁻¹ and 6–9 ng kg⁻¹ respectively. Subsequent doses were given according to the preference of the anesthesiologist in charge. The detailed anesthetic management and dosing procedure was described in a previous study.¹⁴ The Observer Assessment of Alertness/Score (OAA/S)²⁰ was used to measure the state of sedation by clinical observation on a 1–5 scale (Table 1). The defined loss of response (LOR) to instrumentation is an OAA/S < 2 during EGD and colonoscopy.

Table 1: Observer's assessment of alertness/sedation scale.

Response surface models: Effect-site concentrations (Ce) were calculated with the pharmacokinetic simulation software TIVA trainer (Version 9.1, Build 5, Euro SIVA). Matlab (R2013a, The MathWorks, Inc., Natick, MA, USA) was used for pharmacodynamic analysis and modeling of the RSMs. The derived Ces were divided into colonoscopy group and EGD group, along with their corresponding patient responses. The reduced Greco model⁸ and the hierarchy²¹ RSMs are more accurate than the Minto or the full Greco RSMs at predicting patients’ responses during gastrointestinal endoscopy sedation using midazolam and alfentanil.¹⁴ The final form of each model is shown below (Eq. (1), reduced Greco model; Eq. (2), hierarchy model)

where E is the drug effect according to OAA/S scoring. E has a value between 0 and 1, which also corresponds to the pre-defined LOR probability. This produces binary data from observations for analysis (LOR = 1, and no LOR = 0). Ce_a and Ce_m are the alfentanil and midazolam Ce values, respectively, in ng mL⁻¹. Ce_50a and Ce_50m are the Ce values at which 50% of patients experience the independent maximal clinical effect for alfentanil and midazolam, respectively. The parameter α represents the degree of interaction between alfentanil and midazolam, and γ is the steepness of the concentration-effect relationship. Parameter estimation was performed with the EGD group (training group) for the two response surface models. The colonoscopy group acted as the validation group. These two models' parameters are listed in Table 2.

Table 2: Patient demographic and original model parameters.

Logistic regression: SAS (V9.2; SAS Institute Inc., Cary, NC, USA.) was used for data and statistical analysis. The model was constructed using EGD data set and tested against the colonoscopy data set as training group to validate the predictive performance. Mean values, standard deviations, and 95% confidence intervals were calculated as metric variables. We used nonparametric independent t-tests to analyze metric variables between the training and testing sets. Categorical variables were assessed for a significant association by either chi-square statistics or Student's t-test. Variable selection was also applied using the forward selection algorithm which was performed with a pre-assigned p-value equal to 0.05 for controlling the stepping retention. This automatically selects variables for inclusion or exclusion by calculating their respective contributions to the model. At each step, each variable that is not in the model is tested for inclusion in the model. Therefore, the algorithm begins by including the variable that is most significant in the initial analysis. The coefficients of these variables (β) were evaluated by logistic regression. An interaction term between midazolam and alfentanil were included. Cross-validation using the leave-one-out method was used to test the performance of the model. In summary, applying the logistic equation to these results allowed us to estimate the probability of LOR. The discriminating power of each prediction model was measured by receiver operating characteristic (ROC) curves.

Model external validation: Additional external validation of the reduced Greco RSM, the hierarchy RSM and the LR model was performed by applying the EGD model to the colonoscopy concentrations. The discriminating power were also measured by ROC curve analysis.

3. Results

Patient and pharmacokinetic profiles: Thirty-three patients were included for model construction. All patients exhibited ASA physical statuses of I or II, and the mean age was 49.3 ± 9.2 years. The calculated effective site concentrations ranged from 1 to 76 ng mL⁻¹ for alfentanil and 5–80 ng mL⁻¹ for midazolam during the course of the examination. The high variation in concentration is quite common in the induction or emergence phase during anesthesia. There were 68 and 75 concentration sets (alfentanil and midazolam for a given observed response) for EGD and colonoscopy respectively. Only one patient experienced pulse oximetry saturation lower than 90% briefly, which was managed with the chin-lift maneuver.

Logistic regression EGD model: The final model was built with the EGD concentration sets. The coefficient estimate were 0.23 and 0.04 (odds ratio of 1.26 and 1.10) for Ce_m and Ce_a, respectively (Table 3), indicating that for every unit increase in Ce, midazolam was more likely to increase the probability of LOR. The Wald test chi-square values were 13.22 and 5.71 for Ce_m and Ce_a, respectively, both of which were significant at p < 0.05 with one degree of freedom. These results were in accordance with our previous results, which also delineated a more influential role of midazolam on the probability of LOR.¹⁴ ROC curve analysis of the native EGD model (Fig. 1B) and cross-validation (Fig. 1A) both indicated ROC curve areas of 0.94 (Table 3).

Table 3: Results of logistic regression for the EGD group.

Fig. 1.: Logistic regression (LR) model for esophagogastroduodenoscopy. Logistic regression was performed with the leave-one-out cross-validation technique (A) and the entire data pool (B). The results of the two were similar. The 95% receiver operating characteristic (ROC) curve area confidence intervals were 0.90–0.99 (A) and 0.94–0.95 (B).

Response surface modeling: In our previous study,¹⁴ the reduced Greco model and hierarchy model (originally the fixed C₅₀ hierarchy model) were shown to be superior to other RSMs during EGD and colonoscopy in terms of accurate predictions and avoiding overparameterization. The reported predictive accuracies were 82% and 84% during EGD for the reduced Greco and hierarchy models, respectively. An isobolographic plot depicted synergism between midazolam and alfentanil during the EGD procedure (Fig. 2B,C).

Fig. 2.: Contour plot for logistic regression, reduced Greco, and hierarchy models for external validation. External validation was performed with all three models using the esophagogastroduodenoscopy (EGD) construct to fit the colonoscopy group. Non-responsive was defined as OAA/S < 2 and was plotted with red circles. Blue circles (awake) were patient responses with an OAA/S ≥ 2. The contour plot identified the type of interaction with the isoboles (black, bold lines). The 20%, 40%, 60%, and 80% isoboles are shown. The logistic regression model (A) showed the classical appearance of an additive interaction. The reduced Greco (B) and hierarchy (C) models were synergistic, as depicted by the isoboles bowing toward the origin. The interaction was consistent across the surface. Ce = effect-site concentration.

Comparison between models: The reduced Greco and hierarchy models were 64% accurate based on the previous definition for predictive accuracy,^7,14 while the LR model exhibited 82.7% accuracy using the same standard. The ROC curve areas were 0.94, 0.92, and 0.94 for the LR, reduced Greco, and hierarchy models, respectively (Fig. 3). Pairwise comparison of the three models did not show any significant differences in their abilities to predict the probability of LOR (Table 4).

Fig. 3.: Receiver operating characteristic (ROC) analysis of external validation. External validation was performed with all three models using the esophagogastroduodenoscopy (EGD) construct to fit the colonoscopy group. The areas under the curves were similar, and pairwise comparisons were not significant.

Table 4: ROC curve results.

4. Discussion

We compared LR with the reduced Greco and hierarchy RSMs during gastrointestinal endoscopy sedation. Generally, all three models were accurate in the EGD group. The EGD LR model was as precise as the reduced Greco and hierarchy RSMs, the latter two showing the highest accuracy among five RSMs in our previous published study.¹⁴ The performances of the three models were similar when external validation was conducted with the colonoscopy group.

LR is a simple tool that was developed in 1958 and gained popularity rapidly because of its simple design and wide range of applications.²² The type of drug interaction was not directly given by the LR model, but isobolographic approach (Fig. 2) showed an additive interaction between midazolam and alfentanil. We attempted to include an interaction term between the drugs but it did not yield a better fit. Ce_m and Ce_a were the only significant variables. Other demographic variables were included in the forward stepwise selection process but were discarded in the results. This was anticipated because the contributions of these variables were already considered in the initial pharmacokinetic simulation.

We previously reported that the reduced Greco and hierarchy models achieved better fits than the Minto¹¹ or the full Greco models²³ during midazolam/alfentanil-based gastrointestinal endoscopies.¹⁴ In this study, the two RSMs tended to overestimate the rate of LOR during external validation when compared with the LR model (Fig. 2). This was most likely due to different pain intensities between EGD and colonoscopy. The average pain intensity was approximately 9.4% higher in colonoscopy than in EGD.^6,14 This ratio was extrapolated from the pharmacodynamic intensity by comparing light verbal stimulus (OAA/S = 4) in EGD and colonoscopy using the C₅₀ hierarchy RSMs. This could partially explain why a higher proportion of patients with RSM-predicted LOR actually exhibited non-LOR during colonoscopy. This does not imply failure of the RSMs because by adjusting the model parameters, they also achieved high accuracy in the colonoscopy group. The LR model however did not require coefficient readjustments to fit the data well in both the EGD and colonoscopy groups. One explanation for this might lie in the degree of interaction between midazolam and alfentanil. Weak synergism was observed in the EGD and colonoscopy groups. It is possible that the two RSMs did not inherently capture the additivity well and that an additive model might be more suitable than an intuitive synergistic model.

Drug interactions are highly dependent on the intensity of the stimulus in investigations. Interactions between midazolam and opioids are not consistently synergistic. A light stimulus may produce additivity between midazolam and an opioid.^14,24 In contrast, earlier studies reported synergism during induction of anesthesia using midazolam and alfentanil or fentanyl.^11,25–27 Opioids are not considered true anesthetics because they do not produce reliable hypnosis even at high doses.^28,29 The LR isobologram implied that an OAA/S < 2 can be reached with sufficient alfentanil (Fig. 2A). According to the isobologram, the probability of LOR is expected to be 80% and 50% at Ce_a = values of 124.1 and 106.5 ng mL⁻¹, respectively, when alfentanil is given alone. True clinical hypnosis is unlikely to occur at this range. Egan et al. reported a Ce_50a of 375.9 ng mL⁻¹ by monitoring the spectral edge in electroencephalography.³⁰ The reduced Greco and hierarchy models deal with this problem by modifying parameter definitions, which can be seen in the non-approaching isoboles toward the alfentanil axes (Fig. 2B,C).

Several limitations are present in the study. First, our sample size was limited. Sample size requirements differ for LR and RSMs. As few as 20 patients are required to build an RSM.³¹ For LR, van der Ploeg et al. reported that a stable area-under-curve was reached when 20 to 50 events were available for each variable,³² which is higher than the commonly used 10 events per variable.³³ Lu et al. performed naïve pooled analysis and mixed-effects analysis on midazolam using an LR that took the form of the Hill model.¹⁵ They concluded that accurate estimation of C₅₀ from sparse data was possible (even with one observation per patient), although denser data should be used to accurately estimate other parameters. We believe that our sample size reached the minimal requirements for model development. Second, Ce values outside our clustered data sets (high midazolam/low alfentanil or low midazolam/high alfentanil) should be interpreted with caution because the results were extrapolated in these regions. The absent concentrations are not routinely used in clinical sedation. All models were only approximations and they unavoidably had inherent limitations.³⁴ It is possible that the degree of drug interaction may differ if extreme concentrations were available. Third, the choice to transform categorical data into dichotomous data inevitably results in information loss. We simplified the original OAA/S score into binary outcomes, which would reduce the discriminating power. We did not investigate other OAA/S cut-off points for categorization and we do not know how this would affect the final models.

In conclusion, the classical LR model performed well and can be applied to EGD and colonoscopy sedation or to procedures that might share similar stimulus intensities, such as endoscopic submucosal dissection or endoscopic mucosal resection. The models can be applied to computer simulation for customized optimal dosing regimens. Hardware implementation can offer visual aids for anesthesiologists to titrate drug dosages. The RSMs failed to achieve generalization in a similar manner and its construction is far more sophisticated and less commercialized. LR offers better accessibility and may therefore be a simpler alternative to RSMs for predicting patient responses during gastrointestinal sedation using midazolam and alfentanil.

Acknowledgments

This work was supported by the Taiwan Ministry of Science and Technology [grant number MOST-104–2314-B-075–008, MOST 103–2314-B-075–030] and Taipei Veterans General Hospital [grant number V100C-172 and V107C-101].