Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Reporting guideline for priority setting of health research (REPRISE)

BMC Medical Research Methodology volume 19, Article number: 243 (2019) Cite this article

Abstract

Background

Research priority setting with stakeholders can help direct the limited resources for health research toward priority areas of need. Ensuring transparency of the priority setting process can strengthen legitimacy and credibility for influencing the research agenda. This study aims to develop a reporting guideline for priority setting of health research.

Methods

We searched electronic databases and relevant websites for sources (frameworks, guidelines, or models for conducting, appraising, reporting or evaluating health research priority setting, and reviews (including systematic reviews)), and primary studies of research priority setting to July 2019. We inductively developed a list of reporting items and piloted the preliminary guideline with a diverse range of 30 priority setting studies from the records retrieved.

Results

From 21,556 records, we included 26 sources for the candidate REPRISE framework and 455 primary research studies. The REporting guideline for PRIority SEtting of health research (REPRISE) has 31 reporting items that cover 10 domains: context and scope, governance and team, framework for priority setting, stakeholders/participants, identification and collection of priorities, prioritization of research topics, output, evaluation and feedback, translation and implementation, and funding and conflict of interest. Each reporting item includes a descriptor and examples.

Conclusions

The REPRISE guideline can facilitate comprehensive reporting of studies of research priority setting. Improved transparency in research priority setting may strengthen the acceptability and implementation of the research priorities identified, so that efforts and funding are invested in generating evidence that is of importance to all stakeholders.

Trial registration

Not applicable.

Peer Review reports

Background

Historically, the health research agenda has been largely investigator-driven with limited input from other stakeholders including patients, caregivers and the community [1, 2]. Given the evident mismatch between the research interests of patients and researchers, investment into health research may be misdirected to areas of low priority or fail to address important needs of relevant stakeholders [1, 3,4,5,6,7,8]. For example, an analysis of 14 research priority setting partnerships involving patients with different medical conditions found that pharmacological interventions were prioritized only in 18% of the total priorities but 58% of the clinical trials in those fields evaluated pharmacological interventions [4].

Globally, there have been calls for research priority setting with stakeholders to be done at all levels of health systems, jurisdictions, and health areas [9,10,11,12]. There is no consensus on the definition of research priority setting but most definitions refer to a range of activities that involve identifying, prioritizing, and achieving consensus on the research areas or questions of importance to stakeholders [13, 14]. The past two decades have seen increasing efforts to develop better ways to engage all relevant stakeholders, particularly patients, in setting priorities for research across different health disciplines and populations [15,16,17]. Involving stakeholders in an explicit manner in research priority setting can help to: 1) ensure that funding decisions and research meet critical evidence gaps to inform decision making; 2) facilitate shared responsibility and accountability in implementing the research agenda; 3) improve the relevance and legitimacy of research; and 4) ultimately achieve better health outcomes [12, 17].

A diverse range of methods are used to prioritize research given the different healthcare contexts, populations, environments and resources available in which the priority setting is undertaken [14]. The process of research priority setting can be complex, political and value-laden. It can also be challenging to identify, address and integrate the different perspectives and values held by diverse stakeholders. While there is no consensus on what constitutes “successful” research priority setting, it has been advocated that processes must be fair, legitimate, informed by credible evidence, involve a broad spectrum of stakeholders, and be transparent [12, 13, 18,19,20].

However, reviews of published research priority setting exercises have consistently demonstrated a lack of transparency because of suboptimal reporting [17, 21,22,23,24,25,26]. A systematic review of research priority setting in childhood chronic disease, in which most studies were conducted in the UK, US, and Australia, found that methods for collecting and prioritizing research topics were reported in only 50 (60%) of the 84 studies included [23]. Another review of research priority setting exercises in Zambia reported that details about the process and the stakeholders involved were omitted in the majority of studies [22], and similarly, a review of studies in the Islamic Republic of Iran revealed that 22 (61%) of the 36 priority setting studies did not report methods and only listed the research priorities [21]. Inadequate description of the stakeholders and the methods makes it difficult to assess the validity of research priorities identified, and limits the ability to aggregate, analyze or compare research priorities that have been established [27].

Frameworks and guidelines are available for conducting and evaluating research priority setting, which mainly focus on criteria related to the process rather than the outcomes and impact of priority setting. There are no published guidelines for reporting priority setting for health research [5, 13, 18, 27,28,29]. Ensuring the transparency of the process for research priority setting can strengthen legitimacy and credibility to support implementation and maximise impact. A reporting checklist for research priority setting may facilitate more consistent and comprehensive reporting and enable researchers and end-users to better understand the processes taken in developing research priorities. The aim of this paper is to introduce the reporting guideline for priority setting of health research (REPRISE), describe its development and provide a rationale for the items included.

Methods

REPRISE development

We used the Enhancing the Quality and Transparency of Health Research (EQUATOR) toolkit [30], for developing the REPRISE Guideline and reported our approach based on the “Guidance for developers of health research reporting guidelines” where possible [31]. We have also registered REPRISE with the EQUATOR Network.

Purpose and context

The purpose of REPRISE is to facilitate comprehensive and transparent reporting of health research priority setting exercises, in which there is direct involvement of stakeholders setting research priorities. The REPRISE guideline is flexible so that it may be used for a range of approaches. The scope of REPRISE does not cover approaches without direct involvement of stakeholders such as documentary analysis (e.g. evidence mapping), and econometrics methods (e.g. value of information). REPRISE is not intended for use to appraise the quality of priority setting studies, establish or evaluate criteria for research priorities (e.g. evidence gaps, prevalence of disease, economic considerations), and does not recommend a preferred approach.

Identify the need for a guideline

Systematic reviews have consistently shown the reporting of the process of research priority setting with stakeholders is highly variable and limited with many details omitted [21, 23, 25, 26, 32]. The need for a reporting guideline has also been identified through our workshops and forums at national and international meetings (e.g. Cochrane Colloquia and Symposia [12, 33,34,35]; James Lind Alliance [5]), and through our collective experiences of conducting, publishing, reviewing, and using research priority setting studies.

We conducted a comprehensive search for frameworks, guidelines or models for conducting, appraising, reporting and evaluating health research priority setting, reviews (including systematic reviews) of research priority setting studies, and primary research priority setting studies. We searched electronic databases including MEDLINE, Embase, CINAHL, PsycINFO from inception to 23rd July 2019 using sensitive search strategies provided in Additional file 1. We used Medical Subject Heading (MeSH) terms and text words for research priorities and combined this with terms related to reporting, conduct and evaluation. We also searched Google Scholar, relevant organizational websites (e.g. WHO, EQUATOR, Cochrane, James Lind Alliance and PCORI), and reference lists of articles. The search results are shown in Additional file 2. From the 21,556 records retrieved, we identified 13 frameworks or guidelines for conducting or evaluating research priority guidelines [5, 13, 18,19,20, 28, 36,37,38,39,40,41,42] (none designed for reporting research priority setting), and 13 reviews of research priority setting [15,16,17, 21,22,23,24,25,26,27, 32, 43, 44], of which four were systematic reviews [23,24,25,26]. (Additional file 3) We also identified 455 primary research priority setting studies.

Generating reporting items for the candidate checklist

We extracted items related to the process of priority-setting from the frameworks and systematic reviews included (the sources are listed in Additional file 2). We translated these into reporting items for the candidate REPRISE reporting guidelines by grouping similar items and rephrasing the statements as a reporting item. We inductively developed the initial list of reporting items. This was reviewed by two other investigators (AB, AM) to ensure all relevant items were included in the list. The reporting items were compiled into 10 domains: context and scope, governance and team, framework for priority setting, stakeholders/participants, identification and collection of priorities, prioritization of research topics, output, evaluation and feedback, translation and implementation, and funding and conflict of interest. (Additional file 4) We imported all sources (frameworks and systematic reviews) into HyperRESEARCH software for coding textual data, and AT conducted line-by-line coding of each source to the initial items. We generated a report of each reporting item and the corresponding coded text (content). We developed descriptors and examples based on the content of the sources and input from all the investigators. The sources that contributed to each reporting item, and examples of the original extracted items are shown in Additional file 5.

Pilot testing the checklist

The preliminary REPRISE guideline was presented at the Australasian Cochrane Symposium, in which participants used the guideline to assess the reporting of a research priority exercise, and provided feedback on the guideline [34]. We subsequently used the REPRISE guideline in two systematic reviews of research priority setting studies in childhood chronic conditions [23] and organ transplantation [26]. We also applied the preliminary guideline to report a research priority setting exercise in health communication and participation [45].

In the final stage, we piloted the guideline with research priority setting studies. We used a purposive sampling strategy to select 30 priority setting exercises from the total of 455 studies retrieved from the search to ensure a diverse range of health topics, regions, stakeholders involved, framework or methods used, and type of output. Using a standardized data extraction template with the reporting items from the REPRISE guideline, the investigators (two per study) independently assessed if the study reported on each item (yes/no), added comments, and suggested new reporting items that were not yet captured. The results are provided in Additional file 6. After completion, the investigators discussed the relevance and applicability of the items, clarity of the items, comprehensiveness of the descriptor and examples, and any new reporting items proposed. These were integrated into the final reporting guideline, which was reviewed and approved by all investigators.

REPRISE framework: content and rationale

The REPRISE guideline includes ten domains and a total of 31 reporting items with a descriptor and examples provided. (Table 1) The principles, rationale, and explanation for the domains are detailed below, which are based on the synthesis of sources listed in Additional file 3, priority setting studies, and discussion among the investigators.

Table 1 Reporting guideline for health research priority setting with stakeholders (REPRISE)
Full size table

Context and scope (items 1–7)

Establishing the context and scope is recommended as these “underpin the process of research priority setting,” [13] including the selection of relevant stakeholders and methods used. In terms of geographical scope, priority setting may be done at an institutional, local, national, or international level [13, 19, 21, 39,40,41]; recognizing that each will have its own “sense of mandate, capacity, culture, and resources.” [39] Research priorities may address a specific condition, disease or risk factor (e.g. cancer, mental health), population (e.g. elderly, adolescents), health system, research design, or interventions (e.g. vaccination) [5, 13, 19, 21, 24, 27, 37, 38, 40, 41, 44]. These can be decided upon based on the evidence [5, 38] and initial deliberations with stakeholders [38].

The intended beneficiaries [13] may include patients, caregivers, or the general community who could benefit from the priority setting exercise, and the target audience are those who have the potential to implement or fund the research priorities identified [13, 17, 39,40,41]. The focus, content and type of research to be considered can determine the scope of the priority setting exercise. In terms of the broad research areas, these generally span public health, health services, clinical research and basic science [24, 37, 41, 44]. The types of research questions that may be included can range from etiology, diagnosis, prognosis, treatment, to behavioural; and social science, economic evaluation and implementation [23, 24, 27, 39]. It is not necessary for the type of research questions to be determined a priori.

Providing an estimated time frame that the priorities are expected to be valid or relevant may be relevant. This is because research priorities may evolve due to the development of new technology or interventions, emerging evidence, or changes to the health system or socio-political contexts [13, 17, 19, 37,38,39, 41, 42]. If there are plans to update the priority setting or to monitor the priorities for the need to update, these could be described. There has been suggestion of 3–5 year cycles of prioritization if the priority setting exercise is to be repeated [37, 38].

Governance and team (items 8–10)

It has been argued that priority setting requires “credible” [39] leadership to support acceptability and uptake. This may require leaders who are trusted by stakeholders and who have the necessary expertise, knowledge, decision-making skills, and ability and deliver the project. The leadership and management team is usually responsible for overseeing, developing and implementing the process for priority setting [5, 13, 19]. The leadership group may take the form of, for example, an Executive Committee, Advisory Group, Technical Expert Group [13]. Members of the leadership team would generally be expected to contribute broad and relevant collective insights, harness their networks for engagement and partnership; and include a diversity of members to offer legitimacy to wider stakeholder networks (e.g. patients, caregivers, researchers, policy makers, clinicians, representatives from other non-government or government organizations) [5, 21], and those with technical expertise [5]. The membership and selection of stakeholders may need to take into consideration the need for equity [12, 36]. Also, it has been suggested that the involvement of individuals or organizations with experience in priority setting and relevant research skills can ensure a “high quality process.” [5, 13, 40] With regard to facilitators, neutrality and facilitations skills may be important to elicit input from diverse and mixed stakeholders [5, 36].

Framework for priority setting (item 11)

Some priority setting studies use or adapt frameworks to guide the process [17, 22, 23, 25, 44]. Common frameworks include the James Lind Alliance [5], Council on Health Research for Development Essential National Health Research (COHRED/ENHR) [19], Essential National Health Research (EHNR), Child Health and Nutrition Research Initiative (CHMRI) [41], and the Dialogue Model [36]. A summary of these frameworks is provided in Table 2. Some priority setting exercises may develop and use a different approach, and not necessarily follow an existing or established framework.

Table 2 Summary of frameworks for conducting health research priority setting
Full size table

Stakeholders or participants (items 12–16)

Stakeholder involvement in priority setting can vary across the priority setting exercises. In some cases, they are involved in all key stages of the process and in others, they are consulted in specific steps and existing data or documents are used instead of consultation. Relevant stakeholders whose “values and interests should be respected in setting health research priorities” [41] can include patients, caregivers, clinicians, policy makers, representatives from non-governmental organizations [5, 18, 21, 39, 44]; and diverse groups, for example based on demographic or clinical characteristics, may need to be included in research priority setting [5, 18, 20, 36, 37, 39, 40, 42]. It has been emphasized that patients/caregivers (and if relevant the public) need to be directly involved in the priority setting process [5, 18, 38,39,40, 42], as they have direct experience of the health condition or context and often have different priorities to researchers and clinicians. There is also recognition of the need to involve individuals from vulnerable or marginalized groups, particularly in equity-focused research priority setting exercises [5, 12, 21, 36, 39].

Multiple strategies may be used to engage stakeholders in the priority setting process, and this is namely through partnership with relevant stakeholder organizations [5, 39]. The number and characteristics of the participants involved enables assessment of the degree of inclusivity, diversity and equity [5, 13, 39] in priority setting processes. The characteristics to specify may include role and expertise, discipline, organizational affiliations, demographics (e.g. age, sex, socio-economics status, ethnicity), and clinical factors [13, 21,22,23, 36, 42, 44]. Support for patients/caregivers involved in priority setting may include reimbursement for travel, arranging care for dependents, and time [5]. This may indicate to readers the degree to which the team was able to ensure inclusivity across the different groups. Of note, there is recognition that attention must be given to power dynamics, otherwise the engagement of disadvantaged and marginalized groups may lead to “presence without voice and voice without influence.” [2] Therefore, it may be relevant to acknowledge and discuss how hierarchies and “asymmetries between stakeholders” [36] are addressed to maximize constructive and balanced interaction. For example, some groups, such as patients, may require additional time, training, resources, or other strategies to be able to engage; to have the opportunity to contribute meaningfully [36].

Identification and collection of research priorities (items 17–22)

Different methods and approaches are available for collecting and selecting initial research priorities from stakeholders and developing the first list of priorities. This can be one or a combination of methods including interviews, focus groups, workshops, and surveys; and consensus methods (e.g. Delphi survey, nominal group technique); and these may be conducted through various modes such as face-to-face or online [5, 13, 23, 25,26,27,28, 37,38,39]. Documents such as systematic reviews, technical data, and other relevant reports may be used to identify the initial list of priorities [5, 13, 18, 21, 24, 39, 40]. In some priority setting exercises, the initial list of research priorities is derived from literature or existing data rather than consultation or engagement of stakeholders [47,48,49].

If a wide range of different initial research priorities are submitted or identified, it can be challenging to manage and synthesise to capture the diversity of views in a concise manner, whilst also retaining the context and nuances of the submissions. They may need to be organized, usually by collating and categorizing them into themes, topics or other relevant taxonomy [5, 22, 36, 38, 39, 41]; and by removing those that are “out-of-scope,” [5] or duplicative [39]. They may then be translated into “indicative, researchable questions” [5] and edited for clarity [21, 27, 35, 36, 38]. Some priority setting exercises conduct cross checking of the priorities against the evidence (i.e. systematic reviews [5]) and evidence mapping [48, 50,51,52].

The number of research priorities identified at each stage vary widely [24]. Generally, 10 to 20 questions/topics are included in the final set of priorities [5, 23,24,25,26, 38, 40].

Prioritization of research topics/questions (items 23–24)

Prioritization techniques can include scoring, ranking, voting, and ordering, and these are usually embedded in similar methods and modes used for collecting priorities as outlined in Section E. Some frameworks and priority setting exercises use explicit criteria to prioritize questions [13, 21, 22, 38, 39]. Examples include condition-related criteria (burden of disease, variation in care and outcome, evidence gaps), and research-related criteria (resources required, likelihood of success and impact) [37, 42]. The CHNRI method proposes criteria including: answerability, attractiveness (likely to be published in high-impact journals), novelty, potential for translation, effectiveness (likely to identify better interventions), affordability, deliverability, sustainability, public opinion (acceptability to the general public), equity (leads to interventions that will be accessible to marginalized or vulnerable populations), and cost and feasibility [27, 41]. Using specific criteria can facilitate a deliberative and rational process, particularly when there is limited information [21]. It may be relevant to report the processes for selecting, defining and changing the criteria. Of note, the use of criteria can add complexity to the process, and strategies may be needed to avoid inadvertent exclusion of other stakeholder values that influence prioritization. Whilst assigning scores based on such criteria may be rational, there are concerns that it may give a false sense of objectivity. The method for excluding priorities at this stage i.e. based on a quantified threshold or other criteria should be provided. Any processes to appeal or challenge the results may be specified.

Output (item 25)

The output should be “clear and of value to the research community.” [5] The final priorities generated can range from having a specific structure i.e. the Population, Intervention, Comparator, Outcome (PICO) format [5, 37], to broader outputs such as topics or themes/areas [38]. It is possible that components of PICO are not specified in original submission of priorities, or that it cannot be applied to some types of research questions [5]. In some circumstances, attempting to produce very technical research questions can potentially place non-researchers, who may include community members, patients, caregivers, at a disadvantage, as they may feel unable to articulate or consider the specific technical components. Also, consideration may need to be given to ensure that the contextual data and values around the questions are not missed. Some priority setting exercises seek to identify broader themes or areas and translate these into research questions after prioritization.

Evaluation and feedback (items 26–27)

While there is no “gold standard” [37] approach for evaluating the process of research priority setting, process evaluation can provide information about the acceptability, “reliability and usefulness” [37] of the process and results [5, 13, 37]. Stakeholder satisfaction with the process in terms of being able to engage and express opinions, and whether the priorities are considered meaningful and valid may be evaluated [13, 18]. Participants and stakeholders could have an opportunity to review and provide feedback on the prioritized questions [22, 36, 39]; and having “revision or appeal” [18] mechanisms available to identify and address disagreements in a constructive manner [18] have been suggested.

Implementation (items 28–29)

Strategies to implement the research priorities could involve informing and garnering support from government, policy makers, and funding agencies to allocate funding and resources toward the priorities identified [13, 19, 22, 39], and working with researchers to develop proposals [5, 19, 39]. Assessing the impact of research priority setting is challenging but needs to be considered [5, 22]. This may include the impact on decision-making, allocation of funding and resources, and research output [18, 19, 38].

Funding and conflict of interest (items 30–31)

There are different sources of funding that can affect a priority setting process. The funding and resources used to conduct the priority setting exercise and support the stakeholders (directly or indirectly) and whether the priority setting exercise is connected to a funding source to support the identified research priorities. Reporting the sources of funding and support is usually required. The resources required for research priority setting will depend on the size, scope, timeline, methods used, and personnel required [5, 19], and providing information about the budget may be useful for others who are planning on conducting research priority setting. It is recommended that any relevant disclosures be stated for transparency, to allow assessment of potential political or commercial influences or undue bias [5, 13, 24]. For example, pharmaceutical companies may have close ties with patient organisations and clinicians, and the potential influence this may have on the priority setting process would need to be addressed explicitly. This may be declared at an individual level, or at a process or contextual level, for example, providing a narrative of any political issues, conflict or controversies that may affect the process, output or implementation of the priority setting exercise [20].

Discussion

The REPRISE Guideline is intended to facilitate transparent and comprehensive reporting of research priority setting studies that involve stakeholders. The guideline has 31 reporting items that cover 10 domains: context and scope, governance and team, framework for priority setting, stakeholders/participants, identification and collection of research priorities, prioritization of research topics, output, evaluation and feedback, translation and implementation, and funding and conflict of interest. The REPRISE guideline is flexible without being unduly prescriptive because different approaches for health research priority setting are necessary to ensure they are contextually appropriate, respect the underpinning values and criteria, and are feasible based upon resources available. By piloting the guideline with a broad selection of research priority setting studies, we have demonstrated the feasibility, acceptability and relevance of the REPRISE reporting guidelines. We emphasise that REPRISE is not designed for making judgements about the quality of the conduct in research priority setting studies.

The REPRISE guideline may be used as a roadmap for reporting research priority setting studies, or to assess reporting of research priority setting studies as has been done in systematic reviews [25, 26]. REPRISE is focussed on the reporting of process or conduct and does not address in detail the values and criteria for establishing priorities, though these may be described in reporting the process of prioritizing research topics/questions. We did not conduct a Delphi survey, which has been used in other reporting guidelines to prioritize and achieve consensus on what reporting items should be included [30, 31]. Instead, we sought to be comprehensive, included all reporting items, and did not eliminate any items based on judgement about relevance or importance. We believe this increases the practical utility of the REPRISE checklist considering the diverse range of methods and approaches that are used for research priority setting exercises unlike other reporting guidelines which are based on study designs in one particular domain.

We acknowledge that there may be other potentially relevant items that could warrant further discussion, consideration, and evidence to support their inclusion in subsequent revisions of this framework. The items addressing diversity and hierarchies amongst group members and the networks they represent, the criteria and degree of formality in decision making processes, and the medium of communication for sharing information and making decisions are all factors affecting good group decision making [53].. Additional factors, not addressed by the priority setting literature, are the size of a group making decisions, the time available for them to explore their knowledge to make choices or solve problems and the facilitation skills for managing constructive conflict. We seek further feedback from researchers, end-users and other stakeholders, to inform future efforts to refine and revise the guideline as needed.

Conclusions

The REPRISE guideline has the potential to improve transparency in reporting research priority setting studies. Improved explicitness in how research priority setting studies are conducted could strengthen legitimacy, confidence, and acceptability of the findings, and thereby support the implementation and impact of these efforts.

Availability of data and materials

Not applicable. All guidelines and publications used for this article are available in the public domain.

Abbreviations

CHNRI:

Child Health and Nutrition Research Initiative

CINAHL:

Cumulative Index for Nursing and Allied Health Literature

COHRED:

Council on Health Research for Development

ENHR:

Essential National Health Research

EQUATOR:

Enhancing the Quality and Transparency of Health Research

JLA:

James Lind Alliance

PCORI:

Patient-centered outcomes research institute

REPRISE:

Reporting guideline for priority setting of health research

WHO:

World Health Organization

References

  1. Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, Al-Shahi Salman R, Chan AW, Glasziou P. Biomedical research: increasing value, reducing waste. Lancet. 2014;383(9912):101–4.

    Article  PubMed  Google Scholar 

  2. Pratt B. Towards inclusive priority-setting for global health research projects: recommendations for sharing power with communities. Health Policy Plan. 2019. https://doi.org/10.1093/heapol/czz1041 (ahead of print).

  3. Tallon D, Chard J, Dieppe P. Relation between agendas of the research community and the research consumer. Lancet. 2000;355(9220):2037–40.

    Article  CAS  PubMed  Google Scholar 

  4. Crowe S, Fenton M, Hall M, Cowan K, Chalmers I. Patients', clinicians' and the research communities' priorities for treatment research: there is an important mismatch. Res Involv Engagem. 2015;1:2.

    Article  PubMed  PubMed Central  Google Scholar 

  5. JLA: The James Lind Alliance Guidebook Version 8 November 2018. In. Southampton, UK: National Institute for Health Research Evaluation, Trials and Studies Coordinating Centre University of Southampton; 2018.

  6. Corner J, Wright D, Hopkinson J, Gunaratnam Y, McDonald JW, Foster C. The research priorities of patients attending UK cancer treatment centres: findings from a modified nominal group study. Br J Cancer. 2007;96(6):875–81.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Jun M, Manns B, Laupacis A, Manns L, Rehal B, Crowe S, Hemmelgarn BR. Assessing the extent to which current clinical research is consistent with patient priorities: a scoping review using a case study in patients on or nearing dialysis. Can J Kidney Health Dis. 2015;2:35.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Nasser M, Clarke M, Chalmers I, Brurberg KG, Nykvist H, Lund H, Glasziou P. What are funders doing to minimise waste in research? Lancet. 2017;389(10073):1006–7.

    Article  PubMed  Google Scholar 

  9. Ferri M, Davoli M. D'Amico R.\: involving patients in setting the research agenda in drug addiction. BMJ. 2013;347:f4513.

    Article  PubMed  Google Scholar 

  10. Liberati A. Need to realign patient-oriented and commercial and academic research. Lancet. 2011;378(9805):1777–8.

    Article  PubMed  Google Scholar 

  11. Richards T, Montori VM, Godlee F, Lapsley P, Paul D. Let the patient revolution begin. BMJ. 2013;346:f2614.

    Article  PubMed  Google Scholar 

  12. Bhaumik S, Rana S, Karimkhani C, Welch V, Armstrong R, Pottie K, Dellavalle R, Dhakal P, Oliver S, Francis DK, et al. Ethics and equity in research priority-setting: stakeholder engagement and the needs of disadvantaged groups. Indian J Med Ethics. 2015;12(2):110–3.

    PubMed  Google Scholar 

  13. Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Sys. 2010;8:36.

    Article  Google Scholar 

  14. Bryant J, Sanson-Fisher R, Walsh J, Stewart J. Health research priority setting in selected high income countries: a narrative review of methods used and recommendations for future practice. Cost Eff Resour Alloc. 2014;12:23.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Oliver S, Gray J. A bibliography of research reports about patients', clinicians' and researchers' priorities for new research. In. James Lind Alliance: London; 2006.

    Google Scholar 

  16. Stewart RJ, Caird J, Oliver K, Oliver S. Patients' and clinicians' research priorities. Health Expect. 2011;14(4):439–48.

    Article  PubMed  Google Scholar 

  17. Tomlinson M, Chopra M, Hoosain N, Rudan I. A review of selected research priority setting processes at national level in low and middle income countries: towards fair and legitimate priority setting. Health Res Policy Syst. 2011;9:19.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Sibbald SL, Singer PA, Upshur R, Martin DK. Priority setting: what constitutes success? A conceptual framework for successful priority setting. BMC Health Serv Res. 2009;9:43.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Montorzi G, De Haan S, Ijsselmuiden C. Priority setting for research for health. A management process for countries. In. Durban: Council on Health Research for Development; 2010.

    Google Scholar 

  20. Nasser M, Welch V, Tugwell P, Ueffing E, Doyle J, Waters E. Ensuring relevance for Cochrane reviews: evaluating processes and methods for prioritizing topics for Cochrane reviews. J Clin Epidemiol. 2012;66(5):474–82.

    Article  PubMed  Google Scholar 

  21. Badakhshan A, Arab M, Rashidian A, Gholipour M, Mohebbi E, Zendehdel K. Systematic review of priority setting studies in health research in the Islamic Republic of Iran. East Mediterr Health J. 2018;24(8):753–69.

    Article  PubMed  Google Scholar 

  22. Chanda-Kapata P, Ngosa W, Hamainza B, Kapiriri L. Health research priority setting in Zambia: a stock taking of approaches conducted from 1998 to 2015. Health Res Policy Sys. 2016;14(1):72.

    Article  Google Scholar 

  23. Odgers HL, Tong A, Lopez-Vargas P, Davidson A, Jaffe A, McKenzie A, Pinkerton R, Wake M, Richmond P, Crowe S, et al. Research priority setting in childhood chronic disease: a systematic review. Arch Dis Child. 2018;103(10):942–51.

    Article  PubMed  Google Scholar 

  24. Rylance J, Pai M, Lienhardt C, Garner P. Priorities for tuberculosis research: a systematic review. Lancet Infect Dis. 2010;10(12):886–92.

    Article  PubMed  Google Scholar 

  25. Tong A, Chando S, Crowe S, Manns B, Winkelmayer WC, Hemmelgarn B, Craig JC. Research priority setting in kidney disease: a systematic review. Am J Kidney Dis. 2015;65(5):674–83.

    Article  PubMed  Google Scholar 

  26. Tong A, Sautenet B, Chapman JR, Harper C, MacDonald P, Shackel N, Crowe S, Hanson CS, Hill S, Synnot A, et al. Research priority setting in organ transplantation: a systematic review. Transpl Int. 2017;30(4):327–43.

    Article  PubMed  Google Scholar 

  27. Terry RF, Charles E, Purdy B, Sanford A. An analysis of research priority-setting at the World Health Organization - how mapping to a standard template allows for comparison between research priority-setting approaches. Health Res Policy Syst. 2018;16(1):116.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. PCORI: Workshop on methods for setting research priorities. Executive summary available at https://www.pcori.org/sites/default/files/Workshop-on-Methods-for-Setting-Research-Priorities1.pdf (accessed 24th July 2019). In. Baltimore: Patient-Centered Outcomes Research Institute Methodology Committee; 2012.

  29. Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, Chant A. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect. 2019:1–17.

  30. EQUATOR: Toolkit. How to develop and reporting guidelines. Available at http://www.equator-network.org/toolkits/developing-a-reporting-guideline/ (Accessed 1 February 2019). In. Oxford, UK: Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network; 2018.

  31. Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2):e10000217.

    Article  Google Scholar 

  32. Reveiz L, Elias V, Terry RF, Alger J, Becerra-Posada F. Comparison of national health research priority-setting methods and characteristics in Latin America and the Caribbean, 2002-2012. Rev Panam Salud Publica. 2013;34(1):1–13.

    PubMed  Google Scholar 

  33. Cochrane Colloquium Seoul, Korea. (2016) Improving transparency and accountability okf priority setting exercised: time for a reporting guideline? Available at http://2016.colloquium.cochrane.org/workshops/improving-transparency-and-accountability-priority-setting-exercises-time-reporting (accessed 2nd July 2019). In.

  34. Synnot A, Tong A, Craig JC, Hills S: Practical approaches to undertaking research priority setting in health, Workshop program and materials available at https://www.latrobe.edu.au/__data/assets/pdf_file/0009/687339/ACC_Nov_15_prioritysettingworkshop_program_vFINAL_251115.pdf (Accessed 24th July 2019. In. Australian Cochrane Symposium. 25–26 November. Melbourne, Australia; 2015.

  35. Nasser M, Baumik S, Tong A, Welsh E, Synnot A, Crowe S, Morley R, Pardo Pardo J: Practical approaches to conducting an evidence-informed research priority setting exercise. Cochrane Colloquium 2015 Vienna, Australia available at https://abstracts.cochrane.org/2015-vienna/practical-approaches-conducting-evidence-informed-research-priority-setting-exercise (Accessed 1 February 2019). In. London, UK; 2015.

  36. Abma TA, Broerse JEW. Patient participation as dialogue: setting researchagendas. Health Expect. 2010;13:160–73.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Dubois RW, Graff JS. Setting priorities for comparative effectiveness research: from assessing public health benefits to being open with the public. Health Aff. 2011;30(12):2235–42.

    Article  Google Scholar 

  38. Lomas J, Fulop N, Gagnon D, Allen P. On being a good listener: setting priorities for applied health services research. Milbank Q. 2003;81(3):363–88.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Okello D, Chongtrakul P. A manual for research priority setting using the ENHR strategy. Durban: The Council on Health Research fo Development (COHRED); 2000.

    Google Scholar 

  40. Ranson MK, Benett SC. Priority setting and health policy and systems research. Health Res Policy Syst. 2009;7:27.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Rudan I, Gibson JL, Ameratunga S, El Arifeen S, Bhutta ZA, Black M, Black RE, Brown KH, Campbell H, Carneiro I, et al. Setting priorities in global child health research investments: guidelines for implementation of CHNRI method. Croat Med J. 2008;49(6):720–33.

    Article  PubMed  PubMed Central  Google Scholar 

  42. VanLare JM, Conway PH, Sox HC. Five next steps for a new national program for comparative-effectiveness research. N Engl J Med. 2010;362(11):970–3.

    Article  CAS  PubMed  Google Scholar 

  43. McGregor S, Henderson KJ, Kaldor JM. How are health research priorities set in low and middle income countries? A systematic review of published reports. PLoS One. 2014;9(10):e108787.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  44. Swingler GH, Irlam JH, Macharia WM, Tietche F, Meremikwu MM. A systematic review of existing national priorities for child health research in sub-Saharan Africa. Health Res Policy Sys. 2005;3:7.

    Article  Google Scholar 

  45. Synnot A, Bragge P, Lowe D, Nunn JS, O'Sullivan M, Horvat L, Tong A, Kay D, Ghersi D, McDonald S, et al. Research priorities in health communication and participation: international survey of consumers and other stakeholders. BMJ Open. 2018;8(5):e019481.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Rudan I. Global health research priorities: mobilizing the developing world. Public Health. 2012;126:237–40.

    Article  CAS  PubMed  Google Scholar 

  47. Dear RF, Barratt AL, Evans A, Simes J, Newsom J, Kent D, Crossing S, Holliday C, Segelov E, Hruby G, et al. Identifying and prioritising gaps in colorectal cancer trials research in Australia. Med J Aust. 2012;197(9):507–11.

    Article  PubMed  Google Scholar 

  48. Gierisch JM, Myers ER, Schmit KM, Crowley MJ, McCrory DC, Chatterjee R, Coeytaux RR, Kendrick A, Sanders GD. Prioritization of research addressing management strategies for ductal carcinoma in situ. Ann Intern Med. 2014;160(7):484–91.

    Article  PubMed  Google Scholar 

  49. Arora NK, Mohapatra A, Gopalan HS, Wazny K, Thavaraj V, Rasaily R, Das MK, Maheshwari M, Bahl R, Qazi SA, et al. Setting research priorities for maternal, newborn, child health and nutrition in India by engaging experts from 256 indigenous institutions contributing over 4000 research ideas: a CHNRI exercise by ICMR and INCLEN. J Glob Health. 2017;7(1):011003.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Lindson N, Richards-Doran D, Heath L, Hartmann-Boyce J. Setting research priorities in tobacco control: a stakeholder engagement project. Addiction. 2017;112(12):2257–71.

    Article  PubMed  PubMed Central  Google Scholar 

  51. O'Neill B, Aversa V, Rouleau K, Lazare K, Sullivan F, Persaud N. Identifying top 10 primary care research priorities from international stakeholders using a modified Delphi method. PLoS One. 2018;13(10):e0206096.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  52. Zimmerman EB, Cook SK, Haley AD, Woolf SH, Price SK, Berman D, DeLeire T, Etz R, Khalsa J, Knutson K, et al. A patient and provider research agenda on diabetes and hypertension management. Am J Prev Med. 2017;53(1):123–9.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Oliver S, Hollingworth K, Briner B, Swann C, Hinds K, Roche C. Effective and efficient committee work: a systematic overview of multidisciplinary literatures. Evidence Base. 2018;2:1–28.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

AT is supported by a National Health and Medical Research Council Fellowship (APP1106716). The funding organization had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Author information

Authors and Affiliations

  1. Sydney School of Public Health, University of Sydney, Sydney, NSW, Australia

    Allison Tong, Nicole Scholes-Robertson, Talia Gutman & Amanda Baumgart

  2. Centre for Kidney Research, The Children’s Hospital at Westmead, Westmead NSW, Sydney, 2145, Australia

    Allison Tong, Andrea Matus, Nicole Scholes-Robertson, Talia Gutman & Amanda Baumgart

  3. Centre for Health Communication and Participation, School of Psychology and Public Health, La Trobe University, Melbourne, Victoria, Australia

    Anneliese Synnot & Sophie Hill

  4. Cochrane Australia, School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia

    Anneliese Synnot

  5. Crowe Associates Ltd, Oxon, UK

    Sally Crowe

  6. Institute of Education, University College London, London, UK

    Sandy Oliver

  7. Faculty of Humanities, University of Johannesburg, Johannesburg, South Africa

    Sandy Oliver

  8. Katherine Cowan Consulting Ltd, East Sussex, UK

    Katherine Cowan

  9. Peninsula Dental School, University of Plymouth, Plymouth, UK

    Mona Nasser

  10. The George Institute for Global Health, New Delhi, India

    Soumyadeep Bhaumik

  11. College of Medicine and Public Health, Flinders University, Adelaide, South Australia, Australia

    Jonathan C. Craig

Authors
  1. Allison Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anneliese Synnot
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sally Crowe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sophie Hill
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andrea Matus
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Nicole Scholes-Robertson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sandy Oliver
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Katherine Cowan
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Mona Nasser
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Soumyadeep Bhaumik
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Talia Gutman
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Amanda Baumgart
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Jonathan C. Craig
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AT contributed to the conception and design, acquisition of the data, analysis and interpretation of the data, and drafted the manuscript. AS contributed to the conception and design, acquisition of the data, analysis and interpretation of the data, and drafted the manuscript. SC contributed to the conception and design, acquisition of the data, analysis and interpretation of the data. SH contributed to the conception and design and interpretation of the data. AM acquisition of the data, analysis and interpretation of the data. NSR contributed to the acquisition of the data, analysis and interpretation of the data. SO contributed to design, acquisition of the data, analysis and interpretation of the data. KC contributed to the interpretation of the data. MN contributed to the design, analysis and interpretation of the data. SB contributed to the interpretation of the data. TG contributed to the interpretation of the data. AB contributed to the interpretation of the data. JCC contributed to the conception and design, interpretation of the data. All authors revised the article critically for important intellectual content. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Allison Tong.

Ethics declarations

Ethics approval and consent to participate

Not required.

Consent for publication

Not applicable.

Competing interests

Allison Tong is a member of the Editorial Board of BMC Medical Research Methodology. The authors declare that they have no other competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Search strategies.

Additional file 2.

Search results.

Additional file 3.

Research priority setting: frameworks and reviews.

Additional file 4.

Preliminary REPRISE guideline.

Additional file 5.

Sources contributing to the reporting items.

Additional file 6.

Results of the pilot test.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tong, A., Synnot, A., Crowe, S. et al. Reporting guideline for priority setting of health research (REPRISE). BMC Med Res Methodol 19, 243 (2019). https://doi.org/10.1186/s12874-019-0889-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12874-019-0889-3

Keywords

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us