- Journal List
- HHS Author Manuscripts
- PMC6145132
As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsem*nt of, or agreement with, the contents by NLM or the National Institutes of Health.
Learn more: PMC Disclaimer | PMC Copyright Notice
N Engl J Med. Author manuscript; available in PMC 2018 Sep 19.
Published in final edited form as:
N Engl J Med. 2017 Feb 9; 376(6): 526–535.
Published online 2017 Jan 25. doi:10.1056/NEJMoa1516333
PMCID: PMC6145132
NIHMSID: NIHMS968172
PMID: 28121489
Frederic S. Resnic, MD, MSc,a,b Arjun Majithia, MD,a,b Danica Marinac-Dabic, PhD,c Susan Robbins,a Henry Ssemaganda, MD, MSc,a Kathleen Hewitt,d Angelo Ponirakis, PhD,d Nilsa Loyo-Berrios, PhD,c Issam Moussa, MD,e Joseph Drozda, MD,f Sharon-Lise Normand, PhD,g and Michael E. Matheny, MD, MSc, MPHh
Author information Copyright and License information PMC Disclaimer
The publisher's final edited version of this article is available free at N Engl J Med
Associated Data
- Supplementary Materials
Abstract
Background
Assuring the safety of medical devices is challenged by reliance on voluntary reporting of adverse events. We evaluated a strategy of prospective, active surveillance of a national clinical registry to monitor the safety of a vascular closure device suspected as potentially associated with increased adverse events.
Methods
We used an integrated clinical-data-surveillance system to conduct a prospective, propensity-matched analysis of the safety of the Mynx vascular closure device, as compared with alternative approved vascular closure devices, using the National Cardiovascular Data Repository CathPCI Registry. Outcomes were vascular complications including the composite of access-site bleeding, access-site hematoma, retroperitoneal bleeding, any vascular complication requiring intervention or blood transfusion.
Results
Between 1/1/2011 and 9/30/2013, 73,124 patients receiving Mynx devices used following femoral percutaneous coronary intervention (PCI) procedures were analyzed. When compared with alternative vascular closure devices, the Mynx device was associated with an increased risk of vascular complications (absolute risk [AR]: 1.21% vs. 0.76%; relative risk [RR]: 1.59; 95% CI: 1.42-1.78), access-site bleeding (AR: 0.38% vs. 0.28%; RR: 1.34; 95% CI: 1.10-1.62) and transfusion (AR: 1.82% vs. 1.48%; RR: 1.23; 95% CI: 1.13-1.34). The initial alerts occurred within the first 12 months of monitoring. Relative risks were greater for three prespecified high-risk subsets (diabetics, elderly patients and women). These safety signals were confirmed in an independent sample of 48,992 cases between 4/1/14 and 9/30/15.
Conclusions
A strategy of prospective, active surveillance of a clinical registry was capable of rapidly identifying potential safety signals following use of an implantable medical device.
Introduction
Medical devices have revolutionized healthcare, but assurance of their post-market safety relies on voluntary reporting of adverse events, resulting in incomplete ascertainment and an inability to evaluate the safety of medical devices in an active, prospective fashion1–12. Early identification of real-world differences in clinical performance may provide opportunities to support iterative device design improvements, and complement traditional pre-market evidence to refine patient selection and optimize provider training. Active surveillance of medical device safety, through the continuous monitoring of large clinical data sources, has been proposed to address these limitations13–15 and has been identified as a priority by the Food and Drug Administration (FDA)16.
The current study was designed to evaluate a strategy of active surveillance of a national cardiovascular registry in the assessment of the post-market safety of an implantable medical device. To test this strategy, we considered implantable devices for which previous reports had identified potential safety concerns. Prior studies had suggested important differences in the rates of post-procedural complications after the use of one vascular closure device (Mynx vascular closure device, Cardinal Health, Dublin OH) as compared to alternative, approved vascular closure devices when implanted after percutaneous coronary intervention (PCI)13,17. We therefore sought to use registry data on this device to test our prospective, active data-surveillance strategy.
Methods
Study Design
We have previously validated an active clinical-data-surveillance system, denoted DELTA (Data Extraction and Longitudinal Trend Analysis), capable of prospectively monitoring clinical registries and other detailed clinical data sources for safety signals13,15,18–22. DELTA is a collection of integrated software components linking open-source database management and statistical analysis tools, and is designed to simultaneously support multiple risk-adjusted prospective safety-surveillance analyses of complex clinical datasets13,15 (See Supplementary Appendix for additional information regarding DELTA).
The CathPCI DELTA study was designed to demonstrate the feasibility of using the DELTA system for prospective, active surveillance of medical device safety by monitoring clinical outcomes with the identified vascular closure device following PCI procedures. The National Cardiovascular Data Repository (NCDR) CathPCI Registry was used as the data source. The Mynx vascular closure device was identified as the device of interest based on evidence from prior national registry and regional studies that suggested an increased rate of access-site bleeding, injury, or need for repeat intervention as compared to alternative vascular closure devices13,17. Use of the Mynx device has been increasing since 2012 (Supplementary Appendix Figure S1), giving our analyses practical clinical importance.
A study oversight committee was established and a written protocol was approved prior to review of any study data. The protocol (available at NEJM.org) included pre-specified endpoints, analytic methods, sensitivity analyses and plans for two interim data reviews. The study protocol was reviewed and approved as per the institutional review board policies of the NCDR and the Lahey Hospital and Medical Center. The study was funded by the FDA and the William M. Wood Foundation.
Device Exposures and Adverse Outcomes
All patients who received a vascular closure device following PCI performed between 1/1/2011 and 9/30/2013 were identified from the CathPCI Registry. We excluded those patients in whom an intra-aortic balloon pump or ventricular assist device was used, cases in which non-femoral arterial access was utilized, cases in which more than one vascular closure device was deployed, and cases for which the vascular closure device used did not have an implantable component. Vascular closure devices using identical mechanisms of action and identical implanted materials were grouped together into “device families” (see Supplementary Appendix Table S1) for the study in order to increase analytic power.
The primary safety outcome was overall vascular complications, a composite endpoint that included access-site bleeding requiring treatment, access-site hematoma requiring treatment, retroperitoneal bleeding, or any vascular complication requiring intervention. Secondary safety endpoints included access-site bleeding requiring treatment alone and post-procedural blood transfusion. All endpoints and covariates were defined according to the CathPCI version 4 data element definitions (see Supplementary Appendix). All outcomes were assessed through time of hospital discharge, as limited by the CathPCI Registry dataset.
Propensity Score Matching
For each analysis, a propensity-matched concurrent control population was developed based on a non-parsimonious propensity model. The model included previously identified risk factors as well as covariates selected by the authors based on information from prior publications (see the Supplementary Appendix for details)13. Variables were excluded if they were found to be co-linear with covariates already included in the propensity model. Missing data were imputed using univariate rules, by assuming the absence of a clinical condition for dichotomous variables, and by using the median observed value for continuous variables. The final propensity model is shown in Supplementary Appendix Table S2.
Data Surveillance Process
Fully de-identified case-level CathPCI data were provided to DELTA based on scheduled quarterly data updates, and the cumulative safety analysis was automatically regenerated within DELTA. Propensity-score matched control patients were selected in a 1:1 ratio, within 6 months of the date of the case device implant, using a fixed propensity probability caliper width of 0.0110,15,23 through a greedy matching algorithm. The relative imbalance between the exposed and unexposed groups was assessed using the absolute standardized difference in covariate means and proportions, with values less than 10% considered adequately balanced24.
The primary measure of interest was the relative risk for development of an adverse safety outcome, defined as the ratio of observed event rates in the Mynx-treated patients to the observed event rates in the matched population of patients treated with other vascular closure devices. DELTA alerts were triggered if the differences between the independent proportions for the Mynx device and the alternative vascular closure devices were statistically significant, as measured by the Wilson method25. Confidence intervals (CI) were established using a 95% CI corrected for multiple comparisons through use of the O’Brien-Fleming alpha-spending method adapted for use in prospective surveillance26 (see the Supplementary Appendix).
Pre-Specified Sensitivity Analyses and Analyses of High-Risk Subsets
Protocol pre-specified high-risk patient subsets included patients 70 years or older, diabetics, and female patients. Sensitivity analyses included alternative event-rate estimation using logistic regression risk adjustment in place of propensity matching.
Interim Review and Additional Analyses
An early alert (after nine months of monitoring) led the study steering committee to recommend that an additional analysis be performed to address potential bias due to center-level effects. It was hypothesized that higher adverse event rates might be expected at those centers where the Mynx device was used infrequently, and where the device may have been reserved for patients with higher risk. We therefore performed a propensity-matched analysis including only those cases performed at centers that frequently used the Mynx device compared with patients from centers that used, predominantly, an alternative vascular closure device.
After review of the final study results, the FDA requested an additional analysis using the most recent data collected in CathPCI in order to assess persistence of the DELTA safety signals. A protocol amendment was finalized and CathPCI data for the period 10/1/13 through 9/30/15 were made available to DELTA. Propensity-score models were re-calibrated using cases performed from 10/1/13 through 3/31/14. An independent propensity-matched analysis was performed for cases performed between 4/1/14 and 9/30/15.
The study steering committee also recommended several post-hoc sensitivity analyses to assess the robustness of the primary findings. These included performance of a falsification hypothesis analysis in which the original matched patient cohorts were evaluated for the development of post-procedural contrast-induced nephropathy, as defined by a maximal increase of ≥25% or ≥0.5 mg/dl in pre-PCI serum creatinine27. For this outcome, no difference in risk between different vascular closure devices was expected. Additional sensitivity analyses included alternative approaches to handling missing data, as well as an analysis using center-level matching, rather than global registry matching, in an effort to address any confounding due to center-level effects. Analyses similar to that performed for the Mynx device were performed for three other commonly used vascular closure devices. Finally, event rates with vascular closure devices were compared with manual or mechanical access-site compression for hemostasis. Additional details regarding analytic methods and assumptions are provided in the Supplementary Appendix.
Results
Study Population
Between 1/1/2011 and 9/30/2013, 73,164 patients received the Mynx vascular closure device and met the inclusion criteria for the analysis from among 1,822,575 PCI cases submitted to CathPCI (Supplementary Appendix Figure S2). Missing data did not differ between the Mynx-treated patients and those receiving alternative vascular closure devices and represented 0.04% of all risk factor data (Supplementary Appendix Table S3). As shown in Table 1, propensity matching resulted in 73,124 (99.9%) Mynx cases being successfully matched with alternative vascular-closure-device cases. Matching resulted in an adequate distribution of risk factors between cohorts, with all post-match standardized differences less than the threshold for imbalance of 0.10.
Table 1
Distribution of Clinical Covariates before and after propensity matching, in patients treated with the Mynx VCD or alternative VCD.
| Covariate | Mynx VCD (n=73,164) | Total Study Population | Std. Diff | Mynx VCD (n=73,124) | After Propensity Match | Std. Diff | Unmatched Exposures | Std. Diff |
|---|---|---|---|---|---|---|---|---|
| Alternate VCD (n=603,437) | Alternate VCD (n=73,124) | Mynx VCD (n=40) | ||||||
| Age (yrs) | 65.3 ± 11.9 | 65.1 ± 12.1 | 0.0166 | 65.3 ± 11.9 | 65.3 ± 12.0 | 0.0007 | 64.4 ± 11.9 | 0.075 |
| Female Gender | 34.3% | 30.5% | 0.0814 | 34.3% | 34.2% | 0.0022 | 30.0% | 0.092 |
| Body Mass Index (kg/m2) | ||||||||
| <21 | 3.66% | 3.69% | 0.0018 | 3.66% | 3.70% | 0.0022 | 7.50% | 0.168 |
| ≥25 and <30 | 35.4% | 37.0% | 0.0333 | 35.4% | 35.4% | 0.0007 | 37.5% | 0.043 |
| ≥30 | 45.22% | 43.19% | 0.0410 | 45.23% | 45.25% | 0.0005 | 40.00% | 0.106 |
| Diabetes | 39.8% | 35.9% | 0.0802 | 39.8% | 40.2% | 0.0080 | 45.0% | 0.105 |
| Chronic Lung Disease | 16.5% | 13.6% | 0.0814 | 16.5% | 16.7% | 0.0026 | 35.0% | 0.433 |
| Hypertension | 84.8% | 81.1% | 0.0992 | 84.8% | 84.8% | 0.0014 | 80.0% | 0.128 |
| Baseline Creatinine (mg/dL) | 1.20 ± 1.08 | 1.17 ± 0.98 | 0.0341 | 1.20 ± 1.08 | 1.20 ± 1.10 | 0.0004 | 1.39 ± 1.30 | 0.177 |
| Peripheral Arterial Disease | 12.7% | 9.8% | 0.0918 | 12.6% | 12.6% | 0.0007 | 25.0% | 0.320 |
| Emergent Procedure Status | 13.0% | 18.7% | 0.1575 | 13.0% | 12.9% | 0.0036 | 17.5% | 0.126 |
| NSTEMI on Presentation | 18.9% | 20.3% | 0.0362 | 18.9% | 18.9% | 0.0001 | 27.5% | 0.205 |
| Bivalirudin exposure | 67.9% | 64.4% | 0.0723 | 67.9% | 68.7% | 0.0188 | 67.5% | 0.008 |
| Left Main Coronary Artery PCI | 2.09% | 2.23% | 0.0092 | 2.09% | 2.16% | 0.0049 | 2.50% | 0.027 |
| Number of vessels treated during index PCI | 1.42 ± 0.71 | 1.44 ± 0.73 | 0.0236 | 1.42 ± 0.71 | 1.42 ± 0.70 | 0.0042 | 1.78 ± 0.97 | 0.498 |
| Fluoroscopy time (min) | 12.5 ± 9.7 | 14.1 ± 11.2 | 0.1419 | 12.5 ± 9.2 | 12.6 ± 8.9 | 0.0091 | 122.6 ± 72.2 | 11.733 |
| Total number of PCI during admission | 1.05 ± 0.22 | 1.05 ± 0.22 | 0.0020 | 1.05 ± 0.22 | 1.05 ± 0.22 | 0.0033 | 1.03 ± 0.16 | 0.102 |
Open in a separate window
Abbreviations: Standard Deviation (Std Dev), Absolute Percentile Standardized Difference (Std Diff), Percutaneous coronary intervention (PCI); Number (Num); Non-ST elevation myocardial infarction (NSTEMI); Coronary artery disease (CAD)
Mynx Vascular Closure Device Safety Alerts
A DELTA alert was triggered after nine months of monitoring for the primary outcome of any vascular complication, and was sustained throughout the duration of the surveillance study (Figure 1). At the end of the study, the absolute risk of a vascular complication was 1.21% with use of the Mynx versus 0.76% with use of an alternate vascular closure device, with a relative risk of 1.59 (95% CI: 1.42-1.78, p<0.001). Each component endpoint of the primary safety outcome was significantly more common in the Mynx-treated patients, including the risk of significant access-site bleeding, vascular complication requiring intervention, significant access-site hematoma, and development of post-procedural retroperitoneal bleeding. Alerts for the secondary endpoints of significant access-site bleeding and post-procedure blood transfusion were triggered after 30 and 15 months, respectively, and were sustained through the remainder of the surveillance period (see Table 2).
Figure 1
Safety analysis of “any vascular complication” following use of the Mynx vascular closure device versus propensity-matched alternative vascular closure devices among patients treated with percutaneous coronary intervention between 1/1/11 and 9/30/13.
Propensity-matched analysis of cumulative incidence of any vascular complication following use of the Mynx vascular closure device. Circles indicate observed event rates in Mynx-treated patients. Red colored circles indicate higher than expected rates (safety alerts). Solid blue squares indicate observed event rates in the matched alternative vascular closure device group. Blue vertical lines indicate 95% CI, with dashed lines indicating adjustment for multiple comparisons. The solid black line indicates cumulative sample size (with values shown on the right vertical axis).
Table 2
Summary of Outcomes and Final Alert Status for Propensity Matched Analysis of Mynx VCD for patients undergoing PCI between 1/1/11 and 9/30/13
| Cohort Analyzed | Mynx VCD | Alternative VCD | Relative Risk | (95% CI) | p-value | Absolute Risk Difference (%) | Time to Alert (Months) | ||
|---|---|---|---|---|---|---|---|---|---|
| Overall Study Population | |||||||||
| Patients | 73,124 | 73,124 | |||||||
| Vascular complications | 883 | 1.21% | 555 | 0.76% | 1.59 | (1.42-1.78) | <0.001 | 0.45% | 9 |
| Access-Site Bleeding | 277 | 0.38% | 207 | 0.28% | 1.34 | (1.10-1.62) | 0.001 | 0.10% | 30 |
| Blood Transfusion | 1,328 | 1.82% | 1,080 | 1.48% | 1.23 | (1.13-1.34) | <0.001 | 0.34% | 15 |
| High Risk Patient Subsets: | |||||||||
| 70 Years or Greater | |||||||||
| Patients | 27,293 | 27,293 | |||||||
| Vascular complications | 447 | 1.64% | 231 | 0.85% | 1.94 | (1.63-2.29) | <0.001 | 0.79% | 9 |
| Access-Site Bleeding | 142 | 0.52% | 70 | 0.26% | 2.03 | (1.49-2.76) | <0.001 | 0.26% | 15 |
| Blood Transfusion | 754 | 2.76% | 621 | 2.28% | 1.21 | (1.08-1.36) | <0.001 | 0.49% | 27 |
| Diabetes | |||||||||
| Patients | 29,097 | 29,097 | |||||||
| Vascular complications | 307 | 1.06% | 178 | 0.61% | 1.72 | (1.42-2.10) | <0.001 | 0.44% | 15 |
| Access-Site Bleeding | 101 | 0.35% | 63 | 0.22% | 1.60 | (1.14-2.25) | 0.003 | 0.13% | 24 |
| Blood Transfusion | 634 | 2.18% | 539 | 1.85% | 1.18 | (1.04-1.33) | 0.005 | 0.33% | 24 |
| Female | |||||||||
| Patients | 25,065 | 25,065 | |||||||
| Vascular complications | 544 | 2.17% | 297 | 1.18% | 1.83 | (1.58-2.13) | <0.001 | 0.99% | 9 |
| Access-Site Bleeding | 182 | 0.73% | 100 | 0.40% | 1.82 | (1.40-2.36) | <0.001 | 0.33% | 15 |
| Blood Transfusion | 795 | 3.17% | 593 | 2.37% | 1.34 | (1.20-1.50) | <0.001 | 0.81% | 15 |
Open in a separate window
Pre-Specified Sensitivity Analyses and Analyses of High-Risk Subsets
Protocol-specified propensity-matched analyses were performed for high-risk patient subsets including those age 70 years or greater, females, and diabetics, as shown in Table 2. More than 99% of identified cases were successfully matched within each subset, yielding more than 25,000 Mynx-treated patients within each high-risk subset. Safety alerts for vascular complications were triggered within 15 months of monitoring for all subsets and were sustained for the duration of the study. The relative risks for the development of any vascular complication ranged from 1.72 (95% CI: 1.42-2.10) in diabetic patients to 1.94 (95% CI: 1.63-2.29) in patients 70 years or older.
As a pre-specified sensitivity analysis, a logistic model was used to adjust for baseline risk, in place of propensity matching, and confirmed safety alerts for all outcomes within three months of the propensity-matching analysis, thereby supporting the primary results (Supplementary Appendix Table S4).
Post-Hoc Analysis at Mynx High-Usage Centers
A total of 28,141 cases using the Mynx device were identified from those hospitals which used Mynx most frequently among available vascular closure devices, with greater than 99% of these cases matched with the AngioSeal vascular closure device from centers in which this device was preferentially used. A safety alert for vascular complications was triggered after the first twelve months of monitoring, with a relative risk of 1.43 (95% CI 1.18-1.73). Safety alerts were triggered for the secondary endpoints of access-site bleeding and transfusion within 21 months of surveillance (Table 3), and all safety alerts persisted throughout the monitored period.
Table 3
Summary of Outcomes and Final Alert Status for Propensity Matched Analysis for post-hoc High Mynx Utilizing centers, Signal Persistence Analysis and Falsification Hypothesis analysis.
| Cohort Analyzed | Mynx VCD | Alternative VCD | Relative Risk | (95% CI) | p-value | Absolute Risk Difference (%) | Time to Alert (Months) | ||
|---|---|---|---|---|---|---|---|---|---|
| Hi Mynx Utilizing Centers | |||||||||
| Patients | 28,107 | 28,107 | |||||||
| Vascular complications | 297 | 1.06% | 208 | 0.74% | 1.43 | (1.18-1.73) | <0.001 | 0.32% | 12 |
| Access-Site Bleeding | 101 | 0.36% | 63 | 0.22% | 1.60 | (1.14-2.25) | 0.003 | 0.14% | 15 |
| Blood Transfusion | 444 | 1.58% | 364 | 1.30% | 1.22 | (1.05-1.41) | 0.005 | 0.28% | 21 |
| Signal Persistence Analysis | |||||||||
| Patients | 48,992 | 48,992 | |||||||
| Vascular complications | 704 | 1.44% | 472 | 0.96% | 1.49 | (1.32-1.68) | <0.001 | 0.47% | 6 |
| Access-Site Bleeding | 355 | 0.72% | 248 | 0.51% | 1.43 | (1.21-1.69) | <0.001 | 0.22% | 12 |
| Blood Transfusion | 725 | 1.48% | 614 | 1.25% | 1.18 | (1.06-1.32) | <0.001 | 0.23% | 12 |
| Falsification Analysis | |||||||||
| Patients | 73,124 | 73,124 | |||||||
| Contrast Induced Nephropathy | 2,507 | 3.43% | 2,384 | 3.26% | 1.05 | (0.99-1.11) | 0.070 | 0.17% | n/a |
Open in a separate window
Post-hoc Signal Persistence Analysis
At the request of FDA, 49,037 additional PCI procedures in which the Mynx device was used between 4/1/14 and 9/30/15 were analyzed in order to assess the persistence of the potential safety signal identified in the primary analysis. Of these cases, 48,992 (99.9%) were successfully matched to patients receiving alternative vascular closure devices, and DELTA alerts were triggered within twelve months for all endpoints (see Table 3 and Supplementary Appendix Table S5).
Additional Post-hoc Sensitivity Analyses
Repeating the primary analysis using multivariate normal imputation28 in place of simple imputation rules for missing data did not lead to any significant change in alert timing or estimated relative risks. Separately, in order to assess potential confounding due to between-center differences, we repeated the primary analysis with case matching performed at the center level. Using the same propensity methods, 35,192 (48.1%) of all eligible Mynx exposures were successfully matched, resulting in a relative risk of vascular complications of 1.59 (CI: 1.40-1.78). No significant differences in estimated relative risk for any outcome or patient subset was found using center-level matching.
A falsification hypothesis analysis was performed using the outcome of contrast-induced nephropathy, which was not anticipated to differ between patients treated with the Mynx device and patients treated with alternative vascular closure devices. The relative risk for the development of contrast-induced nephropathy after receiving a Mynx device was 1.05 (CI: 0.99 – 1.11, p=0.07) indicating a non-significant trend toward a small increase in risk of developing contrast-induced nephropathy for the Mynx-treated patients as compared to those patients receiving alternative vascular closure device (Table 3 and Supplementary Appendix Figure S3). Analyses of three other commonly used vascular closure devices demonstrated no increase in the risk of vascular complications (Supplementary Appendix Table S6), while manual or mechanical compression was associated with a higher risk of vascular complications than the Mynx device or any vascular closure device (Supplementary Appendix Table S7).
Discussion
In our study, we evaluated the feasibility of applying active surveillance to assess the safety of a commonly used cardiovascular device through prospective monitoring of a national clinical registry. Using the CathPCI Registry, we monitored the comparative safety of a vascular closure device suspected of having increased safety risks over a three-year study period using a pre-specified, active surveillance plan. Our analysis demonstrated statistically significantly increased risks of vascular complications, access-site bleeding, and transfusion requirement following PCI with the use of the Mynx vascular closure device as compared with alternative vascular closure devices, though the absolute risk differences were small. Alerts were triggered early, persisted throughout the duration of surveillance and were present in all patient subsets (see Figure 2). In addition, the primary results were confirmed in an independent, more contemporary, cohort of patients. Analyses of other vascular closure devices did not demonstrate a similar safety signal.
Figure 2
Forest plot of relative risk in all analyzed groups.
An analysis restricted to those centers that preferentially utilized Mynx among available vascular closure devices demonstrated an attenuated, though persistent, safety signal as compared with the primary analysis. These findings may suggest that a portion of the difference in complications is attributable to center level experience in using the Mynx device, a finding that has previously been demonstrated with other vascular closure devices29.
Potential safety signals generated through active surveillance should be interpreted with caution, as all such analyses are observational by design. Despite robust risk adjustment through propensity matching, one cannot exclude residual confounding as being responsible for some portion of the observed difference in adverse events associated with Mynx device as compared with alternative vascular closure devices. The limited number of covariates selected for inclusion in the propensity model could contribute to such residual confounding, though the covariate selection strategy employed guarded against over-fitting of the models. In addition, important clinical variables such as anatomical puncture site location may have contributed to confounding the observed relationships, but were not available in the CathPCI Registry. While the duration of follow up following the PCI procedure was limited by the CathPCI Registry to in-hospital events, the adverse events of interest would generally be expected to occur during the index hospitalization.
We attempted to assess for residual confounding through several post-hoc sensitivity analyses. Though the dataset was large, there was a small amount of missing data (<0.04%) that were imputed using both simple imputation and multivariate normal imputation. Global registry case matching, as used here, may bias results in that patients treated in different centers may have different unmeasured clinical characteristics. However, an analysis using center-specific matching resulted in no significant changes to the estimated risk differences for Mynx device as compared with alternative vascular closure devices.
Finally, we performed a falsification hypothesis analysis in which we monitored the registry for the development of contrast-induced nephropathy following PCI; an adverse outcome for which an association with exposure to a particular vascular closure device was not anticipated. We found a trend toward developing contrast-induced nephropathy in Mynx recipients that did not meet statistical significance, indicating the possibility of a small amount of residual risk imbalance between Mynx recipients and patients treated with alternative vascular closure devices.
In conclusion, a strategy of prospective, active surveillance of a representative clinical registry was capable of rapidly identifying potential safety signals following use of an implantable medical device.
Supplementary Material
Appendix/supplement
Click here to view.(351K, pdf)
Acknowledgments
Funding
This research was primarily supported through research grants from the U.S. Food and Drug Administration (HHSF Contract 223200830058C) and the William M. Wood Foundation. In addition, Drs. Resnic and. Normand’s efforts were funded, in part, by Contract HHSF223201110172C (MDEpiNet Methodology Center) and grant 1U01 FD004493-01 (MDEpiNet Medical Counter Measures Study), both from the U.S. Food and Drug Administration. Dr. Matheny’s efforts were funded, in part, by VA HSR&D CDA 08-020, VA HSR&D IIR 11-292 and 1U01 FD004493-01.
References
1. Hauser RG, Kallinen LM, Almquist AK, Gornick CC, Katsiyiannis WT. Early failure of a small-diameter high-voltage implantable cardioverter-defibrillator lead. Heart Rhythm. 2007;4:892–6. [PubMed] [Google Scholar]
2. Steele GD, Fehring TK, Odum SM, Dennos AC, Nadaud MC. Early failure of articular surface replacement XL total hip arthroplasty. J Arthroplasty. 2011;26:14–8. [PubMed] [Google Scholar]
3. Maisel WH. Semper fidelis–consumer protection for patients with implanted medical devices. N Engl J Med. 2008;358:985–7. [PubMed] [Google Scholar]
4. Shah JS, Maisel WH. Recalls and safety alerts affecting automated external defibrillators. JAMA. 2006;296:655–60. [PubMed] [Google Scholar]
5. Maisel WH. Unanswered questions–drug-eluting stents and the risk of late thrombosis. N Engl J Med. 2007;356:981–4. [PubMed] [Google Scholar]
6. Rosen CJ. The rosiglitazone story–lessons from an FDA Advisory Committee meeting. N Engl J Med. 2007;357:844–6. [PubMed] [Google Scholar]
7. Mehran R, Leon MB, Feigal DA, et al. Post-market approval surveillance: a call for a more integrated and comprehensive approach. Circulation. 2004;109:3073–7. [PubMed] [Google Scholar]
8. Kessler DA. Introducing MEDWatch. A new approach to reporting medication and device adverse effects and product problems. Gen Hosp Psychiatry. 1994;16:96–101. discussion 2. [PubMed] [Google Scholar]
9. O’Shea JC, Kramer JM, Califf RM, Peterson ED. Part I: Identifying holes in the safety net. Am Heart J. 2004;147:977–84. [PubMed] [Google Scholar]
10. Vidi VD, Matheny ME, Resnic FS. Post-marketing device safety surveillance. Contemp Clin Trials. 2011;32:307–8. [PubMed] [Google Scholar]
11. Normand SL, Hatfield L, Drozda J, Resnic FS. Postmarket surveillance for medical devices: America’s new strategy. BMJ. 2012;345:e6848. [PubMed] [Google Scholar]
12. Resnic FS, Normand SL. Postmarketing surveillance of medical devices–filling in the gaps. N Engl J Med. 2012;366:875–7. [PubMed] [Google Scholar]
13. Kumar A, Matheny ME, Ho KK, et al. The data extraction and longitudinal trend analysis network study of distributed automated postmarket cardiovascular device safety surveillance. Circ Cardiovasc Qual Outcomes. 2015;8:38–46. [PMC free article] [PubMed] [Google Scholar]
14. Krucoff MW, Sedrakyan A, Normand SL. Bridging Unmet Medical Device Ecosystem Needs With Strategically Coordinated Registries Networks. JAMA. 2015;314:1691–2. [PubMed] [Google Scholar]
15. Resnic FS, Gross TP, Marinac-Dabic D, et al. Automated surveillance to detect postprocedure safety signals of approved cardiovascular devices. JAMA: the journal of the American Medical Association. 2010;304:2019–27. [PMC free article] [PubMed] [Google Scholar]
16. Strengthening our national system for medical device postmarket surveillance. 2012 (Accessed February 20, 2016, at http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/XDRHReports/UCM301924.pdf)
17. Tavris DR, Wang Y, Jacobs S, et al. Bleeding and vascular complications at the femoral access site following percutaneous coronary intervention (PCI): an evaluation of hemostasis strategies. J Invasive Cardiol. 2012;24:328–34. [PubMed] [Google Scholar]
18. Matheny ME, Ohno-Machado L, Resnic FS. Monitoring device safety in interventional cardiology. J Am Med Inform Assoc. 2006;13:180–7. [PMC free article] [PubMed] [Google Scholar]
19. Matheny ME, Arora N, Ohno-Machado L, Resnic FS. Rare adverse event monitoring of medical devices with the use of an automated surveillance tool. AMIA Annu Symp Proc. 2007:518–22. [PMC free article] [PubMed] [Google Scholar]
20. Matheny ME, Morrow DA, Ohno-Machado L, Cannon CP, Sabatine MS, Resnic FS. Validation of an automated safety surveillance system with prospective, randomized trial data. Med Decis Making. 2009;29:247–56. [PMC free article] [PubMed] [Google Scholar]
21. Matheny ME, Normand SL, Gross TP, et al. Evaluation of an automated safety surveillance system using risk adjusted sequential probability ratio testing. BMC Med Inform Decis Mak. 2011;11:75. [PMC free article] [PubMed] [Google Scholar]
22. Hauser RG, Mugglin AS, Friedman PA, et al. Early detection of an underperforming implantable cardiovascular device using an automated safety surveillance tool. Circ Cardiovasc Qual Outcomes. 2012;5:189–96. [PubMed] [Google Scholar]
23. Austin PC. Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharm Stat. 2011;10:150–61. [PMC free article] [PubMed] [Google Scholar]
24. Austin PC. The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies. Med Decis Making. 2009;29:661–77. [PubMed] [Google Scholar]
25. Newcombe RG. Interval estimation for the difference between independent proportions: comparison of eleven methods. Stat Med. 1998;17:873–90. [PubMed] [Google Scholar]
26. O’Brien P, Fleming T. A multiple testing procedure for clinical trials. Biometrics. 1979;35:549–56. [PubMed] [Google Scholar]
27. Mehran R, Aymong ED, Nikolsky E, et al. A simple risk score for prediction of contrast-induced nephropathy after percutaneous coronary intervention: development and initial validation. J Am Coll Cardiol. 2004;44:1393–9. [PubMed] [Google Scholar]
28. Bernaards CA, Belin TR, Schafer JL. Robustness of a multivariate normal approximation for imputation of incomplete binary data. Stat Med. 2007;26:1368–82. [PubMed] [Google Scholar]
29. Resnic FS, Wang TY, Arora N, et al. Quantifying the learning curve in the use of a novel vascular closure device: an analysis of the NCDR (National Cardiovascular Data Registry) CathPCI registry. JACC Cardiovascular interventions. 2012;5:82–9. [PubMed] [Google Scholar]
