Article Text
PDF
Abstract
Background This paper will focus on outcome reporting within percutaneous coronary intervention (PCI) trials. A core outcome set (COS) is a standardised set of outcomes that are recommended to be reported in every clinical trial. Using a COS can help to ensure that all relevant outcomes are consistently reported across clinical trials. In 2018, the European Society of Cardiology outlined the only COS published for PCI trials.
Methods We searched the literature for all randomised controlled trials published between 2014 and 2022. PCI trials included were late-phase trials and must investigate coronary intervention. The primary outcome was the proportion of trials that reported all of the COS-defined outcomes within their publication as either a primary, secondary or safety endpoint. The secondary outcomes included; the number of primary outcomes reported per study, the proportion of studies which use patient and public involvement (PPI) during trial design, outcome variability and outcome consistency.
Results 9580 trials were screened and 115 studies met inclusion/exclusion criteria. Our study demonstrated that 55% (34/62) of PCI trials used a COS when it was available, compared with 40% (21/53) before the availability of a PCI COS set, p=0.121. Fewer primary outcomes were reported after the implementation of the COS, 2 compared with 2.3, p=0.014. There was no difference in the use of PPI between either group. There was a higher level of variability in outcomes reported before the availability of the COS, while the consistency of outcome reporting remained similar.
Conclusion The use of a COS in PCI trials is low. This study provides evidence that there still is a lack of awareness of the COS among those who design clinical trials. We also presented the inconsistency and heterogenicity in reporting clinical trial outcomes. Finally, there was a clear lack of PPI utilisation in PCI trials.
- Acute Coronary Syndrome
- Coronary Vessels
- Myocardial Infarction
- Outcome Assessment, Health Care
- Biostatistics
Data availability statement
Data sharing not applicable as no data sets generated and/or analysed for this study.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- Acute Coronary Syndrome
- Coronary Vessels
- Myocardial Infarction
- Outcome Assessment, Health Care
- Biostatistics
WHAT IS ALREADY KNOWN ON THIS TOPIC
The implementation of a core outcome set (COS) in percutaneous coronary intervention (PCI) trials holds significant potential to enhance the consistency and quality of reported outcomes, ultimately benefiting daily practice in cardiology.
WHAT THIS STUDY ADDS
The findings underscore a current lack of awareness surrounding the COS among clinical trial designers, highlighting an opportunity for increased education and dissemination of COS guidelines within the cardiology research community.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
As we strive for methodological standardisation in PCI trials, widespread adoption of a COS has the potential to elevate the quality and relevance of evidence generated, fostering advancements in clinical decision-making and patient care in the field of cardiology.
Introduction
Coronary artery disease (CAD) is one of the leading causes of death worldwide.1 It imposes a substantial economic burden on healthcare systems, providers, and patients worldwide.2
This paper will focus on outcome reporting within percutaneous coronary intervention (PCI) trials. PCI can be defined as the use of a non-surgical procedure that aims to relieve any blockage or narrowing within a coronary artery to supply blood to ischaemic myocardial tissue.3 4 PCI can include a multitude of different techniques to achieve this goal such as balloon angioplasty, the deployment of stents or more advanced techniques such as rotablation and thrombectomy.3 PCI trials are frequently designed with composite endpoints, which may risk conveying inaccurate treatment effects and potentially leading to confusion among patients and clinicians.5 6
A core outcome set (COS) is a standardised set of outcomes that are recommended to be reported in every clinical trial.7 Using a COS can help to ensure that all relevant outcomes are consistently reported across clinical trials.4 8 This will make it easier to compare findings across studies, for example, in systematic reviews.9 In addition, the use of a COS will augment trial design by encouraging the use of both important and relevant outcomes for patients.7 The use of COS in reporting can ultimately lead to a more meaningful and applicable evidence base for both clinicians and decision-makers.8 A COS aims to help advance treatments and interventions for patients, thus improving the quality of care.10 The development of a COS additionally integrates the use of patient and public involvement (PPI) within its design stages.7 PPI is important as it demonstrates that the trial is being directed in a way that can answer questions deemed important to patients, and not just researchers.11
In 2018, the European Society of Cardiology outlined a COS in the cardiology audit and registration data standards (CARDS) for PCI outcomes and this was published by Flynn et al and Daeter et al. This is currently the only COS published for PCI trials available. The authors have 11 specific PCI-related outcomes.12 These can be seen in online supplemental appendix 1.
Supplemental material
The purpose of this study was to determine the effectiveness of the PCI COS by comparing the outcomes reported within PCI trials before and after the COS was published in 2018. In addition, we investigated the use of PPI within outcome selection and reporting.
Methods
Search strategy
We searched PubMed, Embase, Scopus, Google Scholar and Web of Science for all randomised controlled trials (RCTs) published between 2014 and 2022.
The search strategy from Embase is presented in online supplemental appendix 2. We sought papers published in the English language. The search strategy was conducted in accordance with the Peer Review Electronic Search Strategies checklist.13
Following the initial search, the trials were then categorised into two groups. This was based on the year of publication. The first group was named pre-COS (January 2014 to June 2018) and the second group was named post-COS (July 2018 to December 2022).
Inclusion and exclusion criteria
All RCTs published between January 2014 and December 2022 involving PCI were included in this study. This constitutes both 4 years before and after the COS became available, for comparison purposes. PCI trials included were late-phase trials (phase 3 or 4) and trials must have investigated coronary intervention. Trials including patients admitted with suspected acute coronary syndrome for primary PCI and studies examining elective and non-elective PCI were included. Trials that investigated antiplatelet therapy or interventional diagnostics, such as fractional flow reserve, intravascular ultrasound or optical coherence tomography strategies with PCI were included to make the inclusion criteria as broad as possible. The trial must be published and only trials in English with a full manuscript available online were included. RCTs using any type of design and any method of randomisation were eligible for inclusion. There was no restriction based on sample size. Trials must have had clearly reported primary and secondary outcomes.
Trials not including PCI were excluded. RCTs investigating structural percutaneous intervention, non-coronary vascular intervention, non-interventional coronary trials and surgical revascularisation were excluded. Electrophysiology procedure interventional trials were excluded. Systematic reviews and meta-analyses were excluded. Trials investigating arterial access sites as the primary outcome were excluded. Furthermore, trials that were based on a post-hoc analysis or follow-on data of already published data were excluded, due to the likely repetition of the same data points.
Identification of relevant studies
Titles and abstracts were screened for keywords, followed by a full-text review to assess the inclusion/exclusion criteria.
Next, the use of a COS in the selected studies was analysed. This included the types of outcomes reported, the methods used to measure outcomes and the statistical analysis used to evaluate the effects of the intervention on the outcome. This was followed by the subanalysis of the use of PPI. As mentioned above, this will include any keywords or phrases mentioned in the trial or trial protocol (online supplemental appendix 3).
Data extraction and management
The data extracted from each trial was independently extracted from each RCT using a data-extraction form (online supplemental appendix 4).
Outcomes
The primary outcome was the proportion of trials that reported all of the COS-defined outcomes within their publication as either a primary, secondary or safety endpoint. These comprise of; reperfusion strategy, events during the in-hospital stay, mortality, procedure indication, use of any diagnostic or functional testing and degrees of recovery (DOR). This is further explained in table 1.
PCI COS-defined outcomes
The secondary outcomes were:
The number of primary outcomes reported per study.
The proportion of studies that report using PPI during trial design.
The number of studies that report ‘degrees of recovery’. This variable is one of the PCI-COS which is completely patient-focused and rarely reported.
Outcome variability (outlined in online supplemental appendix 5).
Outcome consistency, measured as the proportion of trials that report primary/secondary outcomes that match the COS within each domain of (1) efficacy, (2) safety and (3) quality of life (QOL).
Analysis
For categorical data, a χ2 test was used to compare the proportion of trials reporting the COS-defined outcomes, DOR and utilisation of PPI, in the pre-COS and post-COS groups. Independent sample t-tests were used to compare the mean number of outcomes reported by the pre-COS and post-COS groups. Results were reported as percentages (proportion of trials reporting a COS-defined outcome), alongside corresponding p values.
The consistency of outcome measurement was assessed by calculating the proportion of trials that report a primary or secondary outcome that matches the COS for each clinical domain (efficacy, safety and QOL). This was reported as a percentage (proportion of trials within that domain reporting a COS). In addition to reporting the proportion of trials consistent with the COS, the number of trials reporting unique outcomes was analysed. This provided further information on the extent of outcome heterogeneity and the potential impact of using a COS on reducing outcome variability.14
Outcome variability was calculated by using propensity scoring. This was calculated by using the ‘Inverse probability of treatment weighting with survival analysis’ technique as described by Rosenbaum and Rubin, 1983; Austin, 2011;15 16 2016. A full detailed description of our technique used is found in online supplemental appendix 5. This was reported as a weighted HR, effect size and p value.
Analysis was performed using IBM SPSS Statistics V.28 (IBM Corp, Armonk, New York, USA). The level of statistical significance was set at α= 0.05.
Results
The Preferred Reporting Items for Systematic review and Meta-Analysis flow diagram showing the search results with reasons for exclusion is presented in figure 1. Our final number of papers included was 115.
Flow diagram of study identification. RCT, randomised controlled trial.
There were 53 trials in the pre-COS group and 62 in the post-COS group. Table 2 shows the breakdown of trials pre and post availability of the COS. The proportion of trials that reported COS-defined variables increased from 40% (21/53) pre-COS availability to 55% (34/62) post-COS availability. This did not reach statistical significance, p=0.121.
Trials reporting entire COS pre and post availability of COS
There was a significant difference in the mean number of primary outcomes being reported in the pre-COS versus post-COS group. The post-COS group reported fewer outcomes compared with the pre-COS group with a mean of 2.0 and 2.3, respectively, p=0.014.
The proportion of trials that recorded the use of PPI in trial design was higher in the pre-COS group. This was recorded at 26% (14/53) and 16% (10/62), respectively. This did not reach statistical significance, p=0.25 (table 3).
Trials reporting the usage of PPI
The proportion of trials that used outcome sets consistent with DOR in trial design was similar in both groups. This was recorded at 13% (7/53) for the pre-COS group and 15% (9/62) for the post-COS group with p=0.9 (table 4).
Trials using degrees of recovery for outcomes
A total of 54 studies were included in the variability analysis. There was a statistically significant difference between the pre-COS and post-COS groups with a weighted HR of 2.0, p=0.044, df=54. The effect size calculated as log HR was 4.2 (95% CI 0.33 to 7.07). This indicates a strong level of variability between the results pre and post-implementation of the COS.17
Outcome consistency can be seen reported in table 5. This shows that there is similar outcome reporting across each domain before and after implementation of the COS and there was no statistically significant difference between the two groups.
Outcome consistency across COS reported outcomes for efficacy, safety and QOL
Discussion
Our study demonstrated that only 55% of PCI trials used the COS after it was introduced. This is compared with 37% before the availability of a PCI COS set. There was a higher proportion of trials reporting COS-compliant outcomes after COS implementation however, this did not reach statistical significance (p=0.121). This identification of low-COS use, in a cohort of RCTs was not an unexpected finding. Matvienko-Sikar et al estimate that 50% of all trials published in major medical journals implemented a COS between 2019 and 2020.18 Our results were similar to previous evidence published.18 19 Prior explanations for this poor usage are thought to be low levels of awareness about the COS existence and preferences for researchers to choose their own outcomes.18 20 Our results would support similar findings.
The Core Outcome Measures in Effectiveness Trials guidelines would suggest that the measurement and reporting of a COS are expected as a minimum standard in clinical trials.7 Current trial reporting guidelines do not require a rationale or justification for choosing an outcome.18 This study highlights the need for awareness within the cardiovascular research community surrounding the availability of the COS, which intends to improve outcome reporting in clinical trials. The reason for the recommendation of the COS implementation is to develop a more standardised selection of outcomes, minimising research waste and improving patient outcomes.19 It also adds a level of clarity for researchers, clinicians and patients, in the interpretation of clinical trial results.19
Notably, there was a significantly lower level of variability in outcomes reported after the availability of the COS, while the consistency of outcome reporting remained similar. Outcome heterogeneity can make it difficult to determine the true treatment effect as well as cause hesitancy over the external validity of the results.21 Our results suggest that this heterogeneity has significantly improved. However, a variable that is not accounted for in our variability equation is time, which could potentially add an element of time-leading bias here.22
Both groups displayed similar implementation of DOR. This result was numerically low in both groups at 13% pre-COS and 15% post-COS. The DOR outcome is the most subjective outcome reported with the assessment of the patient’s angina burden by class, QOL and physical baseline status. However, it is arguably one of the outcomes that is most important to patients and clinicians as these symptoms are often key factors in decisions to undergo further revascularisation.23 Recent evidence from the ISCHAEMIA research group has shown that one of the major drivers towards revascularisation should be symptom burden in stable CAD.24 Our results suggest that this outcome is generally poorly reported within the literature and is not a major influence on the decision for revascularisation in clinical trials.
The lack of utilisation of PPI in PCI trials was notable within our results with only 21% (24/115) of RCTs implementing its use in design. This is similar to previously reported literature.11 The patient representatives’ involvement is crucial, however many barriers exist, for example, the availability of financial reimbursement or training for implementation.11 Our results show the importance of continuing to highlight PPI as a tool to be used in trial design.
Future recommendations would include; the promotion of COS reporting in journals, ongoing education regarding COS availability and promotion within the scientific community.
Strengths and limitations
The strengths of this study include the large number of publications screened and analysed with broad inclusion/exclusion criteria. This is the first study to investigate the implementation of a COS within CAD and PCI.
Our limitations include only researching PCI and CAD-related publications, which could be critiqued as a narrow field of study. Another limitation of this study was that it did not investigate grey literature and focused solely on published, peer-reviewed RCTs. This did not include any systematic reviews. We only were able to retrieve information from what the authors reported in the publication and did not seek further clarification if this was not clear. However, we do feel that this gave good scope for obtaining generalisability within our findings.
Another limitation to note is that we considered trials as implementing the COS if the design included all of the prespecified outcomes only. This gave no scope to include trials that included a component of the COS. This could be a potential source of selection bias and would be a point for future research.
Conclusion
This study provides evidence that there still is a lack of awareness of COS’ among those who design clinical trials. We also presented the inconsistency and heterogenicity in reporting clinical trial outcomes. This can impact the reader’s interpretation and understanding while leading to research waste. Finally, the findings of this study reveal a clear lack of PPI utilisation in PCI trials.
Highlighting the importance of COS utilisation should now be a priority among the research community. To help address this issue, trial developers should ensure the involvement of specialists with a broad knowledge of COS in clinical trials. This move to COS outcomes will ultimately lead to improved standardisation of outcomes and minimising research waste. This will all aim to lead to an improvement in patient standards of care.
Data availability statement
Data sharing not applicable as no data sets generated and/or analysed for this study.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
Acknowledgments
University College Cork, work undertaken as part of MSc in Clinical Trials Thesis.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
X @aaroduncan
Presented at The abstract of this paper was presented at the Irish Cardiology Society AGM 2023; ‘18 The outcomes we choose in cardiology: a literature review and analysis of core outcome sets in PCI’.
Contributors AD: guarantor and main author, background research, statistics, write-up. FS: thesis supervisor and editorial work.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.