Article Text
PDF
Abstract
Background Patients with diabetes mellitus (DM) are at substantial risk of developing peripheral artery disease (PAD). We recently developed a clinical/proteomic panel to predict obstructive PAD. In this study, we compare the accuracy of this panel for the diagnosis of PAD in patients with and without DM.
Methods and results The HART PAD panel consists of one clinical variable (history of hypertension) and concentrations of six biomarkers (midkine, kidney injury molecule-1, interleukin-23, follicle-stimulating hormone, angiopoietin-1 and eotaxin-1). In a prospective cohort of 354 patients undergoing peripheral and/or coronary angiography, performance of this diagnostic panel to detect ≥50% stenosis in at least one peripheral vessel was assessed in patients with (n=94) and without DM (n=260). The model had an area under the receiver operating characteristic curve (AUC) of 0.85 for obstructive PAD. At optimal cut-off, the model had 84% sensitivity, 75% specificity, positive predictive value (PPV) of 84% and negative predictive value (NPV) of 75% for detection of PAD among patients with DM, similar as in those without DM. In those with DM, partitioning the model into five levels resulted in a PPV of 95% and NPV of 100% in the highest and lowest levels, respectively. Abnormal scores were associated with a shorter time to revascularisation during 4.3 years of follow-up.
Conclusion A clinical/biomarker model can predict with high accuracy the presence of PAD among patients with DM.
Trial registration number NCT00842868.
- peripheral vascular disease
- risk factors
- claudication
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Key questions
What is already known about this subject?
The ankle-brachial index (ABI) is most commonly used to diagnose lower extremity peripheral artery disease (PAD); however, its diagnostic accuracy is limited in patients with stiff, calcified arteries which is common among patients with diabetes mellitus (DM).
We recently developed a clinical/proteomic panel (HART PAD) using machine learning, capable of diagnosing obstructive PAD with high accuracy; however, the utility of this score in patients with DM is uncertain.
What does this study add?
The HART PAD panel predicted with high accuracy the presence of PAD among patients with DM.
Furthermore, the HART PAD panel was predictive of revascularisation among patients with DM.
How might this impact on clinical practice?
The HART PAD panel offers an attractive alternative to ABI for diagnosing PAD among patients with DM.
The panel could act as a gatekeeper to imaging or invasive testing, thereby reducing costs, and exposures to intravenous contrast and/or ionising radiation by avoiding expensive imaging modalities when unwarranted.
Furthermore, the panel could be used for prognostic purposes to guide more intensification of medical therapies.
Introduction
Diabetes mellitus (DM) is a global health problem; it is estimated, by 2030, approximately 366 million people worldwide will suffer from the disease.1 Patients with DM are at substantial risk for developing both microvascular and macrovascular complications.2 One notable macrovascular complication of DM is peripheral artery disease (PAD) which is prevalent in approximately 20%–30% of patients.3 4 PAD is associated with a considerable increase in the risk of fatal and non-fatal cardiovascular and cerebrovascular events,5 and event rates are higher among patients with DM.6
Symptoms of PAD are variable, especially in patients with DM who may suffer from concomitant peripheral neuropathy, thus it is often undiagnosed until its advanced stages. As a result, patients with DM and PAD often receive suboptimal management that may prevent progression of disease.4 The ankle-brachial index (ABI) is the most common non-invasive diagnostic modality used to detect the presence of lower extremity PAD; however, its accuracy is reduced in patients with stiff, calcified arteries. Approximately 60% of patients with DM have calcified lower extremity peripheral arteries, and expectantly, ABI has correlated poorly with angiographic PAD in this population.7 Imaging modalities are also used to diagnose PAD but imaging is expensive, has variable availability and requires intravenous contrast and/or ionising radiation. For these reasons, we recently developed a clinical/proteomic panel (HART PAD) using machine learning, capable of diagnosing obstructive PAD with high accuracy.8 In this study, we compare the accuracy of this panel for the diagnosis of obstructive PAD in patients with and without DM a population at high risk for PAD that is particularly challenging to evaluate and manage.
Methods
Study population
The Catheter Sampled Blood Archive in Cardiovascular Diseases study was a prospective, single-centre, observational cohort study that was undertaken at the Massachusetts General Hospital in Boston, Massachusetts, between 2008 and 2011. The investigators enrolled 1251 subjects undergoing coronary and peripheral angiography with or without intervention over the study period.9
For the purpose of this study, we included 354 patients who underwent peripheral angiography only (n=140), peripheral and coronary angiography but without significant coronary artery disease (CAD) (n=11) and those who underwent coronary angiography alone without significant CAD and no history of PAD (n=203). The latter group were incorporated to increase cohort size and were assumed to have an absence of PAD, based on their medical history. The indications for peripheral angiography included claudication (n=96), carotid artery stenosis with/without stroke (n=11), hypertension (n=21) and other PAD without claudication (n=25). The peripheral angiograms included: lower extremity (n=129), renal arteries (n=59) and carotid/subclavian (n=18). All study procedures were approved by the Partners HealthCare Institutional Review Board and carried out in accordance with the Declaration of Helsinki.
Data acquisition
After obtaining informed consent and at the time of the procedure, clinical and historical variables and reason for referral for angiography were recorded. Results of the peripheral +/−coronary angiography (based on visual estimation at the time of the procedure) were recorded. For the purposes of this analysis, ≥50% luminal obstruction in at least one peripheral vessel was considered obstructive PAD.
HART PAD model
The derivation of the HART PAD panel has been previously described.8 Briefly, using the same 354 patients included in this study, we used machine learning, a subset of artificial intelligence, to identify predictors of significant PAD. We examined 109 biomarkers and more than 50 clinical variables. Candidate panels of proteins and clinical features were generated via least angle regression. In this method, factors were included in the model one at a time, with their coefficients determined by their correlation with the outcome. This was repeated until all factors were included in the model, and the step at which the performance plateaued resulted in our initial panel of interest. With this panel of interest, predictive analyses were run on the training set using least absolute shrinkage and selection operator with logistic regression, predicting the outcome of obstructive PAD using only the variables in the panel of interest. If the contribution of the least performing variable was not statistically significant, then that variable was removed and the analysis rerun. This process was repeated until the predictive contribution of all variables in the model was statistically significant. This model development process was done via Monte Carlo cross-validation, using 400 iterations with an 80:20 (training:test) split. The final panel was used to create a final model with the entire sample, and this model was then evaluated to predict obstructive PAD.
After the machine learning model building process, the final panel consisted of one clinical variable (systemic hypertension) and six biomarkers: follicle-stimulating hormone, angiopoietin-1, kidney injury molecule-1 (KIM-1), midkine, interleukin-23 and eotaxin-1.
Biomarker testing
Using a centrally placed vascular access sheath, 15 mL of blood was taken immediately prior to angiography. Blood samples were then stored in a 4°C refrigerator until centrifuging was undertaken. After a single freeze-thaw cycle, 200 µL of plasma was used for biomarker analysis on a Luminex 100/200 xMAP technology platform. The biomarkers were obtained from a commercially available kit, Myriad RBM MAP; this technology uses multiplexed, microsphere-based assays in a single reaction vessel whereby an assay-specific capture antibody on each microsphere binds to a protein of interest. Comparable to a flow cytometer, as each individual microsphere passes through a series of excitation beams, it is analysed for size, encoded fluorescence signature and the amount of fluorescence generated is proportionate to the protein concentration.
Follow-up
A review of patients’ medical records from time of enrolment to the end of follow-up was undertaken. For identification of clinical endpoints, in addition to a review of medical records, telephone follow-up was performed with patients and/or their managing physicians. Study investigators adjudicating angiographic severity of PAD or events during follow-up were blinded to results of all biomarker testing.
Statistical analysis
Baseline characteristics between those with and without DM were compared; dichotomous variables were compared using two-sided Fisher’s exact test, while continuous variables were compared using two-sided two-sample t-test. The biomarkers compared were tested with the Wilcoxon rank-sum test, as their concentrations were not normally distributed. A complete case analysis was performed; two patients were missing at least one of the concentration read-outs for the six proteins in the final panel, so they were excluded, leaving 352 samples available for analysis. For any biomarker result that was unmeasurable, we used a standard approach of imputing concentrations 50% below the limit of detection.
The HART PAD panel was originally developed to predict obstructive PAD using Monte Carlo cross-validation, and the final model was evaluated using in-sample validation. In this study, we used the final model trained to diagnose PAD in all patients, and evaluated it on all patients with DM. We generated a receiver operating characteristic curve and determined an optimal diagnostic cut-off using the optimal Youden’s index. With this cut-off we calculated sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). The predictive score generated by the diagnostic model was rescaled to a range of 0–10 to facilitate interpretation. The score was then partitioned into five different risk levels, corresponding to multiple levels of PAD risk. Time to revascularisation as a function of elevated PAD score was calculated from 7 days after index angiography over a mean of 4.3 years’ follow-up period and displayed as a Kaplan-Meier survival curve. Lastly, after excluding patients with non-lower extremity PAD (eg, carotid and renal artery disease), we then assessed time to revascularisation or amputation in patients with lower extremity PAD as a function of the continuous HART PAD score (adjusting for age) over a mean of 4.3 years’ follow-up period and displayed this as a Kaplan-Meier survival curve.
All statistics were performed by using R software, V.3.4 (R Foundation for Statistical Computing, Vienna, Austria). P values are two sided, with a value <0.05 considered significant.
Results
Baseline characteristics
Of the 354 patients included in this study, 132 had obstructive PAD (online supplementary table 1). Baseline characteristics of study subjects, dichotomised as a function of DM, are detailed in table 1. Patients with DM were older, more likely to be male and had a higher prevalence of hypertension, CAD, prior myocardial infarction and chronic kidney disease. Notably, of the biomarkers measured, those with DM had higher concentrations of KIM-1 and midkine.
Supplemental material
Baseline characteristics of patients with and without diabetes mellitus
HART PAD diagnostic performance
With the data represented in the 0–10 scale, the optimal cut-off for the panel was determined to be 5.607 using the optimal Youden’s index, which identifies the optimal balance of sensitivity and specificity. In receiver operating characteristic testing, the model generated an in-sample AUC of 0.848 for patients with DM (figure 1); slightly higher than the performance for patients without DM (AUC 0.83).
Receiver operating characteristic curve for the HART PAD score to predict obstructive peripheral arterial disease in patients with diabetes mellitus. The score had a very robust area under the receiver operating characteristic curve (AUC).
At its optimal cut-off to diagnose PAD, for patients with DM, we observed a sensitivity of 84%, specificity of 75%, NPV of 75% and PPV of 84%.
HART PAD five-level score
Partitioning the score into five categories yielded a PPV of 95% and NPV of 100% in the highest and lowest scores, respectively, for patients with DM (table 2). We found a higher prevalence of obstructive PAD in those with higher scores and lower prevalence among those with lower scores (figure 2). When the score was divided into five categories of predicted risk, an increasing score correlated with an increasing degree of mean PAD stenosis in both patients with and without DM (figure 3).
Distribution of score among patients with diabetes (positive) and without diabetes (negative) in a histogram.
Correlation between peripheral artery disease (PAD) score and mean degree of arterial stenosis in patients with and without diabetes mellitus (DM).
Predictive performance as a five-level score
Performance in various vascular territories
The score had a similar in-sample performance in each individual vascular territory using the optimal cut-off among patients with DM. In the diagnosis of obstructive lower extremity artery disease, the score had a sensitivity of 83%, specificity of 62%, PPV of 70% and NPV of 78%. For diagnosing obstructive carotid and renal artery stenosis, the score appeared most useful for its NPV. The score had a sensitivity of 80%, specificity of 58%, PPV of 10% and NPV of 98% for obstructive carotid arterial disease. While for the renal arteries, the corresponding sensitivity was 88%, specificity 59%, PPV 17% and NPV 98%.
Predicting the need for revascularisation
In adjusted Cox proportional hazards models, from 7 days after index angiography to the end of follow-up, those patients with DM who had a dichotomously elevated score had higher risk for revascularisation, compared with patients with DM who had a lower PAD score (HR: 2.88; 95% CI 1.16 to 7.19, p=0.02); those with higher scores also had shorter time to first revascularisation event (figure 4). After excluding patients with obstructive PAD in non-lower extremity territories (figure 5), the score predicted time to revascularisation or amputation as a continuous score among patients with DM and lower extremity PAD (HR: 1.3; 95% CI 1.02 to 1.66, p=0.04).
Kaplan-Meier survival curves depicting time to revascularisation as a function of peripheral artery disease (PAD) score. Patients in the positive group had a score greater than or equal to the optimal cut-off for the score, which was determined to be 5.607 using the optimal Youden's index (with the model’s output rescaled to the range of 0–10). Patients in the negative group had a score below 5.607. DM, diabetes mellitus.
Kaplan-Meier survival curves depicting time to revascularisation as a function of continuous HART PAD score in patients with lower extremity peripheral artery disease (PAD). Patients in the positive group had a score greater than or equal to the optimal cut-off for the score, which was determined to be 5.607 using the optimal Youden’s index (with the model’s output rescaled to the range of 0–10). Patients in the negative group had a score below 5.607. DM, diabetes mellitus.
Discussion
Though prevalent in patients with DM, PAD is a challenge to identify and manage; tools typically used for diagnosis of PAD are often less accurate in those with DM. Accordingly, we wished to verify the performance of a biomarker-leveraged scoring system derived using machine learning recently found to predict angiographically significant PAD.8 We demonstrate excellent performance of the model in patients with DM. When analysed as a five-level score, the highest and lowest scores yielded a PPV of 95% and NPV of 100%, respectively, for diagnosing obstructive PAD among patients with DM.
As discussed, the biomarkers used in this model all have credible biological links to atherosclerotic PAD.10–19 We believe the HART PAD panel could be useful in those with DM. As 60% of patients with DM have calcified lower extremity peripheral arteries, ABI, the most commonly used non-invasive diagnostic tool, has correlated poorly with angiographic PAD in this population.7 The HART PAD panel offers an attractive alternative to ABI with high NPV and PPV in patients with DM. Given a range of score values that provides both strong PPV and NPV, the panel could potentially avoid the need for imaging or invasive testing, thereby reducing costs, and exposures to intravenous contrast and/or ionising radiation. Of note, the model performed similarly well in other vascular regions (renal and carotid arteries), in particular for its NPV. Undoubtedly, the panel should serve as an adjunct to a thorough history and physical examination.
Furthermore, we demonstrate that the HART PAD panel has prognostic utility, with a shorter time to revascularisation in those with an elevated score. Consequently, the scoring system may also be used to predict patients at risk for vascular complications, which could then be used to guide therapeutic intervention. Aspirin and statins are already standard therapies in PAD, but other medications such as novel oral anticoagulants (NOAC) and proprotein convertase subtilisin/kexin type 9 (PCSK9) inhibitors have recently demonstrated promise in this population.20 21 Cost (PCSK9 inhibitors) and side effect profile (NOACs) may limit their utility and, thus, appropriate risk stratification will be important to guide therapy.
Lastly, the HART PAD panel may play a role in clinical trials to enrich for PAD-related events or to identify patients at risk for adverse effects of drug therapies. For example, although the sodium-glucose co-transporter-2 inhibitor canagliflozin significantly reduced the risk of cardiovascular events by 14%, it doubled the risk of amputation in patients with type 2 DM. Our panel could be useful in predicting underlying PAD or the need for revascularisation in patients who are being considered for these agents.22
Despite being novel, our study has limitations. Notably, the number of patients from which we derived our findings was relatively small and included patients who only underwent coronary angiography as negative controls; though such patients were low risk for PAD, unexpected disease might have been present. As such, the results of our study should serve as preliminary evidence that requires confirmation in larger and ad hoc cohorts. The study participants were predominantly Caucasian which limits to the external validity of our panel to African-American patients. Patients with critical limb ischaemia were not included in our study and the utility of the panel in this cohort is uncertain. The diagnostic model was not compared with other non-invasive modalities such as ABI or ultrasonography; a comparison which will necessitate investigation in future studies. Our results need further validation and should not be extrapolated to the general population; the patients in our study were referred with clinical suspicion for significant PAD. Lastly, using the clinical/biomarker panel alone will not be sufficient to differentiate the exact territory of PAD and thus clinical correlation with history and physical examination will be an important component of assessment.
Conclusion
We describe a clinical/biomarker panel with high accuracy for predicting the presence of PAD in patients with DM.
References
Footnotes
Contributors All authors played a role in the drafting and editing of the manuscript.
Funding NI is supported by the Dennis and Marilyn Barry Fellowship in Cardiology Research. JLJ is supported in part by the Hutter Family Professorship in Cardiology. HKG is supported in part by the Ruth and James Clark Fund for Cardiac Research Innovation. This work was supported by a grant from Prevencio.
Competing interests JLJ has received grant support from Abbott, Cleveland Heart Labs, Singulex and Prevencio; has received consulting income from Roche Diagnostics, Critical Diagnostics and Novartis; and has participated in clinical endpoint committees/data or safety monitoring boards for Novartis, Amgen, GE, Janssen, Pfizer and Boehringer Ingelheim. CM is a consultant to Prevencio. HKG has received grant support from Roche and Portola; consulting income from Roche Diagnostics, American Regent, Amgen, Boston Heart Diagnostics and Critical Diagnostics; research payments for clinical endpoint committees for EchoSense. JMG has received consulting income from Siemens, Applied Clinical Intelligence, Bayer and Merck, Boehringer Ingelheim and AbbVie. NI has received speaker fees from Novartis. RRJvK has received grant support from Novartis. RR and GB are employees of Prevencio.
Patient consent for publication Obtained.
Ethics approval Partners HealthCare Institutional Review Board (Study ID No: 2008P001076).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.