Aim To discuss ethical issues related to a complex study (PROFID) involving the development of a new, partly artificial intelligence-based, prediction model to enable personalised decision-making about the implantation of an implantable cardioverter defibrillator (ICD) in postmyocardial infarction patients, and a parallel non-inferiority and superiority trial to test decision-making informed by that model.
Method The position expressed in this paper is based on an analysis of the PROFID trials using concepts from high-profile publications in the ethical literature.
Results We identify ethical issues related to the testing of the model in the treatment setting, and to both the superiority and the non-inferiority trial. We underline the need for ethical-empirical studies about these issues, also among patients, as a parallel to the actual trials. The number of ethics committees involved is an organisational, but also an ethical challenge.
Conclusion The PROFID trials, and probably other studies of similar scale and complexity, raise questions that deserve dedicated parallel ethics and social science research, but do not constitute a generic obstacle. A harmonisation procedure, comparable to the Voluntary Harmonization Procedure (VHP) for medication trials, could be needed for this type of trials.
- heart arrest
- outcome assessment
- health care
- research design
- myocardial infarction
Data availability statement
There are no data in this work.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
The PROFID project (implementation of personalised risk prediction and prevention of sudden cardiac death after myocardial infarction) seeks to solve the problem that current guidelines on the implantation of implantable cardioverter defibrillators (ICDs) are insufficiently personalised, leading to both overtreatment and undertreatment with serious personal and societal consequences.1 PROFID aims to develop a personalised decision support tool and test it in a double multicentre and multinational trial in postmyocardial infarction patients (post-MI): both a non-inferiority trial in patients who, according to current guidelines, should receive an ICD because they have a left ventricular ejection fraction (EF) below 35% (but of whom more than 90% are overtreated because annually, only 1%–4% of patients with an ICD receive a life-saving shock), and a superiority trial in the group that is currently considered low risk (EF >35%), but is undertreated, because most deaths in post-MI patients occur in this category.
The PROFID project has an innovative and quite complex set-up that may be seen more often in future research involving both the use of big data and large-scale clinical trials. First, a prediction tool will be developed based on a very large datasets from nine European countries, the USA, and Israel, using standard statistics, machine learning, and/or other forms of artificial intelligence (AI); to be able to validate the prediction tool externally before using it in the trials, 90% of the data from each data set will be used to develop the model, and 10% will be kept apart as a separate cohort for validation by applying leave-one-dataset out cross validation across all test sets. Subsequently, the tool will be tested in a combination of a non-inferiority trial and a superiority trial, both multicentre, involving a total of 3920 patients to be enrolled in more than 150 clinical sites.
Such complex studies are partly uncharted ethical territory, especially regarding the non-inferiority trial, informed consent, and the number of ethics approvals needed. That is the reason why an ethics work package runs parallel to the study, during the entire study period. As a part of that work package, this paper aims to explore the specific questions it raises for the ethical assessment by the many institutional review boards that will have to assess the studies. We will address ethical considerations around personalisation of treatment and big-data based prognostic tools elsewhere.
Ethical questions about the PROFID-trials
PROFID contains two parallel randomised controlled trials (RCTs). The non-inferiority trial (PROFID-Reduced) will include post-MI patients who have an EF <35% and an annual SCD risk of 2.5% or lower according to the personalised prediction model. According to the current, but insufficient guidelines, these patients should receive an ICD. In the non-inferiority trial, these patients will be randomised 1:1 to receive or not receive an ICD. The null hypothesis of inferiority of optimal medical therapy relative to ICD treatment can be claimed, if the upper boundary of 95% CI for the HR falls below 1.336 for all time points.
The superiority trial (PROFID-Preserved) will include post-MI patients who have an EF >35% and an annual SCD risk of 3% or higher according to the personalised prediction model. Following the current guidelines, these patients would not have received an ICD. These patients will be randomised 1:1 to receive or not receive an ICD. If the z-value from the one-sided log-rank test (for the difference in survival between the two groups) is larger than 1.96, the null hypothesis is rejected in favour of the superiority of ICD treatment relative to optimal medical therapy. The target relative risk is 0.75, that is, a reduction of the 30-month event rate (all cause death) from 24.5% under the reference treatment to 18.375% under the index treatment. The patients with a risk between 2.5% and 3% according to the prediction model will be treated according to the current guidelines.
The first ethical question is whether it is justifiable to test the validated prognostic model including consequences for treatment, and not parallel to current treatment (as is often done). Whether this is justifiable depends on the harm done by current treatment. Important negative effects (mainly endocarditis and perforation) have been found in many studies. To cite but a few, Kirkfeldt et al, in a study of a large Danish database, studied all complications of ICDs such as systemic infections, lead-related complications, and perforation, and found that 9.5% of ICD users suffered at least one complication.2 Van Barreveld et al, in the DO-IT study, found major complications in 7%–8% of patients.3
In the PROFID non-inferiority trial, the personalised approach intends to save the lives and the quality of life that are now lost due to these adverse effects. In the superiority trial, the standard approach clearly costs lives, and the personalised approach is expected to save more lives than it costs. This leads to the conclusion that, in the case of PROFID, testing the prediction model against current treatment, as proposed in both RCTs, is ethically justifiable.
Recently, there has been discussion about the ethical underpinnings of non-inferiority trials generally, and more specifically when death is the primary outcome. The fundamental ethical motivation for non-inferiority trials is to show that it is possible to maintain efficacy while increasing other benefits. The main ethical pitfall is the non-inferiority margin: non-inferiority trials will never show complete and certain equivalence and thus are forced to accept a margin of clinically acceptable loss of efficacy—the question then is what loss of efficacy would be acceptable. This is, at least in part, an empirical question which will be addressed in the interview study accompanying the trials.
In a controversial Lancet paper, Garattini and Bertele’ stated that strict non-inferiority trials are always unethical, because they would involve no benefit for patients.6 As was pointed out in the discussion that followed,7 they disregard the trade-off character of non-inferiority studies: they trade a potential, and minimum loss of efficacy for other benefits such as fewer adverse effects (including death, see above), better compliance, and lower costs.
However, even if these authors are wrong on a general level, non-inferiority trials with mortality as an endpoint are sometimes regarded as particularly difficult to justify ethically.8 The European Medicines Agency states that it is ‘very difficult to justify a noninferiority margin of any size in a study where the treatment under consideration is used for the prevention of death or irreversible morbidity and there is no second chance for treatment. Discussion of the number of extra deaths that are acceptable is ethically very difficult’.9
We do think that the PROFID-Reduced trial is ethically justifiable, because the risk of dying as a consequence of participating in the trial has been reduced to a minimum. To explain this technically: the null hypothesis of inferiority will be rejected if the upper boundary of 95% CI for the HR falls below 1.336. This non-inferiority margin was taken marginally larger than the inverse of the historic HR 0.75 (for proportions) of ICD versus no ICD implantation .
However, because this is such an ethically difficult area, a parallel ethical analysis is part of the PROFID project with the double aim of strengthening its ethical justification and of learning for other non-inferiority trials with mortality as an endpoint. Since patient views are crucial for the justification of the non-inferiority margin that is chosen, a study among patients will be part of a dedicated social science work package of the PROFID project.
In any trial, patients need to be informed properly to be able to decide whether they want to participate or not. The consequences of both options (participation or non-participation) should be clearly described.
Against this background, two questions may be raised about informed consent in the PROFID trials. First, what will participants need or want to know, to be able to decide about participation, about the prediction tool that was used to assess their risk and thus to decide about their inclusion in one of the two trials? More specifically, will they need or want to know that machine learning or AI was used for this? It is conceivable that this knowledge will be relevant for their decision to participate (‘why was I asked, on what grounds?’), so patient information should be as clear as possible about this. Part of the explanation may be that the AI part of the tool may remain a black box to some extent, but that it has been validated externally.
Second, what do participants need or want to know about the exact purpose of the trials? The PROFID superiority trial is relatively straightforward, but how is this for the non-inferiority trial? In a study by Doshi et al, patients and methodologists assessed information leaflets of 50 non-inferiority trials studying antibiotics.10 They found that the study purpose was explained in only 11 of these according to patients, and in just one according to methodologists. Moreover, none of the trials had an explicit clinical justification for the non-inferiority margin that was chosen—let alone that patients were informed about it. In a commentary, Menikoff argued that in the case of non-inferiority trials, the study purpose may be both very complicated to explain and less relevant for participants, and could lead to unwarranted nonconsent, even though it was unclear whether this actually occurred. He concludes that what patients need above all is a clear explanation of risks.11 This may seem plausible, but will be investigated in the abovementioned studies among patients. Awaiting the results of these studies, we think that purpose, non-inferiority margin, and risks should be clearly explained, even though this may not be simple.
The number of approvals needed
In most countries, international multicentre trials need approvals from the Research Ethics Committees (REC) of all institutions involved. Some countries, however, delegate the assessment of multicentre trials to one committee. For PROFID, the number of submissions is estimated to be a 100, or even more RECs. This clearly has a positive side, because ethical frameworks may differ locally or nationally, and such differences must be respected. On the other hand, the number of assessments needed and also the differences in speed may jeopardise the timely execution of the trials in all sites. An Australian case study from 2009, however, could show no difference between a centralised national approval system and a non-centralised system regarding time to overall approval.12 More fundamentally, separate assessments may lead to different demands for modification, or even approvals by some RECs and rejections by others.13 This, in turn, could and sometimes does invalidate the comparability of outcomes in different centres, and it may make results more difficult to apply.
So, a procedure is needed that respects local and national value differences, without unnecessarily endangering the trials. For medication trials, as a consequence of obvious commercial interests that are not at stake here, the Voluntary Harmonization Procedure (VHP) has been put in place to solve these issues. It allows for a coordinated and therefore less time-consuming assessment in the participating states.14 We think something similar might also be needed for publicly funded international non-medication studies. In the meantime, the only thing large-scale projects such as PROFID can do is informing ethics committees as completely as possible not only about the technical, but also about the ethical aspects of the studies.
Studies such as PROFID, in which a Big Data based personalised prediction model is tested in both a non-inferiority and a superiority trial with overall mortality as a primary endpoint, raise several difficult, but answerable ethical questions. The most important of these are to the testing of the prognostic model with treatment outcomes, to the non-inferiority margin, and to informed consent. These ethical issues will be studied parallel to the project, without constituting a generic obstacle to this type of study. In the future, a procedure similar to the VHP could be a solution, also for non-drug studies and for publicly funded studies.
Data availability statement
There are no data in this work.
Contributors DW had the idea for this viewpoint, performed the initial literature search and wrote the manuscript. He is the guarantor of the article. MB, HT and ND contributed to the literature search and to the writing of the article. GL, AK, ASS, GS and GH contributed to the writing of the article.
Funding This study was funded by European Commission Horizon 2020 (847999).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.