Skip to main content
Download PDF

Registration of systematic reviews in PROSPERO: 30,000 records and counting

Systematic Reviews volume 7, Article number: 32 (2018) Cite this article

Abstract

Background

The International Prospective Register of Systematic Reviews (PROSPERO) was launched in February 2011 to increase transparency of systematic reviews (SRs). There have been few investigations of the content and use of the database. We aimed to investigate the number of PROSPERO registrations from inception to 2017, and website usage in the last year. We also aimed to explore the epidemiological characteristics of and completeness of primary outcome pre-specification in a sample of PROSPERO records from 2017.

Methods

The PROSPERO database managers provided us with data on the annual and cumulative number of SR registrations up to October 10, 2017, and the number of visits to the PROSPERO website over the year preceding October 10, 2017. One author collected data on the focus of the SR (e.g. therapeutic, diagnostic), health area addressed, funding source and completeness of outcome pre-specification in a random sample of 150 records of SRs registered in PROSPERO between April 1, 2017 and September 30, 2017.

Results

As of October 10, 2017, there were 26,535 SRs registered in PROSPERO; guided by current monthly submission rates, we anticipate this figure will reach over 30,000 by the end of 2017. There has been a 10-fold increase in registrations, from 63 SRs per month in 2012 to 800 per month in 2017. In the year preceding October 10, 2017, the PROSPERO website received more than 1.75 million page views. In the random sample of 150 registered SRs, the majority were focused on a therapeutic question (78/150 [52%]), while only a few focused on a diagnostic/prognostic question (11/150 [7%]). The 150 registered SRs addressed 18 different health areas. Any information about the primary outcome other than the domain (e.g. timing, effect measures) was not pre-specified in 44/150 records (29%).

Conclusions

Registration of SRs in PROSPERO increased rapidly between 2011 and 2017, thus benefiting users of health evidence who want to know about ongoing SRs. Further work is needed to explore how closely published SRs adhere to the planned methods, whether greater pre-specification of outcomes prevents selective inclusion and reporting of study results, and whether registered SRs address necessary questions.

Peer Review reports

Background

Prior to 2011, users of health care evidence would be hard-pressed to find information about ongoing systematic reviews (SRs). Only a few organisations, including Cochrane and the Joanna Briggs Institute, disseminated protocols for SRs that were underway. Yet these organisations produce a minority of all published SRs [1, 2], so most would only become known at the time the SR was completed and published. This is despite the many benefits of registering SRs prior to their conduct. For example, prospective SR registration stimulates authors to anticipate methodological challenges that may arise, helps minimise potential for reporting bias by encouraging registrants to publish their SR and report all pre-specified outcomes and serves to reduce waste from unintended duplication of SRs by different teams of authors [3, 4]. Prospective SR registration is one of several processes that can facilitate optimal transparency, reproducibility and usability of SRs [5].

Calls for more extensive prospective registration of SRs started to gain traction around the late 2000s. An international group of epidemiologists, clinicians, statisticians and editors recommended in the 2009 Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement that systematic reviewers provide registration information, including a registration number, for their SR [6, 7]. That same year, Tricco et al. advocated the establishment of a database to register SRs at inception, after identifying in an international survey of 348 authors that 199 SRs they had conducted had not been published [8]. And concerns about potential for reporting bias in unregistered SRs were raised after a 2010 study identified discrepancies between the pre-specified and reported outcomes of more than a fifth of Cochrane SRs (e.g. some outcomes were omitted, others were downgraded from primary to secondary) [9].

The world’s first international prospective register of systematic reviews (PROSPERO) was launched in February 2011 to mitigate these problems with SR transparency [10, 11]. PROSPERO is produced by the Centre for Reviews and Dissemination at the University of York (UK), and funded by the UK National Institute for Health Research (NIHR). The PROSPERO register accepts any SR with a health-related outcome, regardless of whether the focus is on a diagnostic, prognostic, genetic association or intervention question. Since November 2013, new protocols for Cochrane intervention and diagnostic test accuracy SRs have been added to PROSPERO. While SRs including any type of study design (e.g. randomised trials, cohort studies, qualitative studies) are eligible for inclusion, PROSPERO currently does not accept scoping reviews or literature reviews. For a SR on a methodological topic to be included, at least one patient-oriented or clinically relevant outcome needs to be included (therefore, methodological SRs looking only at, say, the reporting of particular methods in studies would not be included). The register requires systematic reviewers to provide information on 22 mandatory and 18 optional items, which were selected following an international consultation exercise [12]. Items include administrative information (e.g. SR title, anticipated or actual start date), SR methods (e.g. eligibility criteria, methods for collecting, handling and analysing data) and other general information (e.g. reference or URL for an accompanying SR protocol). PROSPERO also provides users with information on the status of a SR, with options including “ongoing”, “completed but not published” and “completed and published”. Once the SR is complete, authors can update the PROSPERO record to provide the full citation for the final report or publication of the SR, including the URL where available.

Along with facilitating SR transparency, the PROSPERO database is a valuable source of data for meta-research (i.e. research on research). For example, Booth et al. evaluated the number of registrations and number of visitors to the website between February 2011 and February 2012 [13]. Tricco et al. investigated discrepant outcome reporting between the PROSPERO record and publication of 98 SRs published before November 2013 [14]. Borah et al. used the start dates recorded in PROSPERO to estimate the time required to complete 195 registered SRs that were published before July 2014 [15]. And Sideri et al. evaluated how frequently SRs of orthodontic research published between 2012 and 2016 were registered a priori in PROSPERO [16]. To our knowledge, there has been no investigation of the number of registrations and usage of the PROSPERO website since its first year of operation. Also, epidemiological characteristics (e.g. health areas addressed, funding source) of SR registrations have not been examined. Further, there has been no evaluation of how completely outcomes are pre-specified in PROSPERO records. Complete pre-specification of SR outcomes is necessary to protect against bias due to selective inclusion and reporting of results, where the selection of data to include from studies, and subsequent reporting of results, is influenced by the nature of the findings [17,18,19].

We aimed to investigate the number of PROSPERO registrations from inception to 2017, and website usage in the last 12 months. We also aimed to explore the epidemiological characteristics of and completeness of primary outcome pre-specification in a 2017 sample of PROSPERO records.

Methods

Evaluation of the number of PROSPERO registrations and website usage

We obtained the following aggregate data upon request from the PROSPERO database managers:

  • Total number of registrations by October 10, 2017;

  • Annual registration numbers between February 1, 2011 and October 10, 2017;

  • Country of the corresponding author for all registrations up to October 10, 2017;

  • Number of website visits and page views of the PROSPERO website (https://www.crd.york.ac.uk/prospero/), and countries of users accessing the website, within the last 12 months (October 10, 2016 to October 10, 2017) and in the 12 months prior (October 10, 2015 to October 9, 2016). A website visit is counted any time a visitor reaches the PROSPERO website from somewhere outside the website domain. A page view is counted when a page on the PROSPERO website is loaded by a browser.

Evaluation of epidemiological characteristics of and completeness of outcome pre-specification in PROSPERO records

We collected data on the focus of the SR (e.g. therapeutic, diagnostic), health area addressed, country of corresponding author, funding and completeness of outcome pre-specification in a random sample of 150 SRs registered in PROSPERO within the last 6 months. One author (MJP) downloaded the URLs of all records registered between April 1, 2017 and September 30, 2017 (n = 6070) from the PROSPERO database. The same author drew a random sample using the random number generator in Microsoft Excel, and retrieved the full PROSPERO record for all 150 SRs selected.

One author (MJP) collected data on epidemiologic characteristics from each PROSPERO record using a standardised data collection form, adopting the same terms used in a previous study evaluating the epidemiological characteristics of published SRs [1]. The focus of the SR was sought from the “Review question” field of the PROSPERO record, and classified as therapeutic (i.e. effects of a treatment/preventive intervention), epidemiologic (i.e. prevalence/incidence, or association between exposure and outcome), diagnostic (i.e. diagnostic test accuracy), prognostic (i.e. prognostic factors, biomarkers or clinical prediction rules) or other (e.g. qualitative analysis, measurement properties of instruments). Health area addressed was sought from the “Condition or domain being studied” field of the PROSPERO record, and classified using the International Classification of Diseases, Tenth Revision (ICD-10, http://apps.who.int/classifications/icd10/browse/2016/en). Country of corresponding author was sought from the “Country” and “Contact details for further information” fields of the PROSPERO record. Funding was sought from the “Funding sources/sponsors” field of the PROSPERO record, and classified as non-profit (e.g. government, university/hospital/research institute, charitable foundation), for-profit (e.g. pharmaceutical company) or no funding (as stated by the authors).

One author (MJP) classified completeness of outcome pre-specification for the primary outcome of the SR using the five-element framework developed by Saldanha et al. [20]. This framework recommends that authors of SRs pre-specify the:

  1. 1.

    Domain or outcome title (e.g. pain);

  2. 2.

    Specific measurement or technique/instrument used to make the measurement (e.g. 10-point visual analogue scale);

  3. 3.

    Specific metric or format of the outcome data from each participant that will be used for analysis (e.g. change in pain from baseline);

  4. 4.

    Method of aggregation or how data from each group will be summarised (e.g. mean change in pain from baseline);

  5. 5.

    Time points that will be used for analysis (e.g. at or closest to 6 weeks post-randomisation).

Information on outcome pre-specification was sought from the “Primary Outcome” field of the PROSPERO record and, if filled in, the “Timing and effect measures” field (the latter is an optional field on the PROSPERO registration form). Information was also sought from the “Data extraction (selection and coding)” and “Strategy for data synthesis” sections of the record. If the authors pre-specified multiple primary outcomes, we analysed only the outcome listed first in the “Primary Outcome” field of the PROSPERO record.

We performed all analyses using Stata version 14 software [21]. Data for all variables were summarised as frequency and percentage. We explored whether completeness of outcome pre-specification was associated with whether the focus of the SR was therapeutic or not. Associations were quantified as risk ratios, with 95% confidence intervals (CIs), using a log-binomial regression model.

Results

Registration statistics

As of October 10, 2017, there were 26,535 SRs registered in PROSPERO; guided by current monthly submission rates, we anticipate this figure will reach over 30,000 by the end of 2017. There has been a 10-fold increase in registrations, from 63 SRs per month in 2012 (the first complete calendar year of registrations) to 800 per month in 2017 (Fig. 1).

Fig. 1
figure 1

Annual and cumulative total number of PROSPERO registrations, 2011–2017. *Projected figures for year-end 2017 based on current monthly submission rates

Full size image

PROSPERO includes registrations from all over the world, with more than 100 countries contributing. The highest number of registrations originate from England, who were responsible for 4828/26,535 (18%) records registered from inception to October 10, 2017 (Table 1).

Table 1 Number of records registered in PROSPERO by the top 10 contributing countries, from inception to October 10, 2017
Full size table

Website usage

In the year preceding October 10, 2017, the PROSPERO website received more than 1.75 million page views. Visits to the website almost doubled in the last 12 months, with 525,750 visits occurring between October 2016 and October 2017 compared to 327,439 visits from October 2015 to October 2016. Use of PROSPERO is global, with the greatest number of website visits in the past year originating from the UK (117,878 visits).

Epidemiological characteristics of SRs registered in PROSPERO

In the random sample of 150 SRs registered between April 1, 2017 and September 30, 2017, the majority were focused on a therapeutic question (78/150 [52%]; Table 2). Almost a third of registrations addressed an epidemiologic question (47/150 [31%]), while only a few focused on a diagnostic/prognostic question (11/150 [7%]). The 150 SRs addressed a wide range of health areas; 18 ICD-10 chapters were recorded across the records. The top three most common ICD-10 chapters were diseases of the circulatory system (16/150 [11%]), mental and behavioural disorders (16/150 [11%]) and diseases of the musculoskeletal system and connective tissue (15/150 [10%]). Three ICD-10 chapters—diseases of the eye and adnexa, diseases of the ear and mastoid process and external causes of morbidity and mortality—were not covered by a single SR. Corresponding authors came from 28 countries, with England (27/150 [18%]), Australia (21/150 [14%]) and China (17/150 [11%]) the most common. Almost half of the SRs were funded by a non-profit source (66/150 [44%]), while the remainder were declared as being conducted without any funding. None of the SRs were funded by a for-profit source. Also, all of the registrations were for non-Cochrane SRs.

Table 2 Epidemiological characteristics of a random sample of 150 SRs registered in PROSPERO between April 1, 2017 and September 30, 2017
Full size table

Completeness of primary outcome pre-specification in PROSPERO records

Pre-specification of primary outcomes was incomplete in most of the 150 SRs registered between April 1, 2017 and September 30, 2017 (Table 3). The primary outcome was completely pre-specified (i.e. all five components were stated) in the PROSPERO record in 9/150 (6%) cases. In 44/150 (29%) cases, the domain was the only information about the primary outcome that was pre-specified. Approximately 40% of the PROSPERO records included information about the specific measurement, metric or method of aggregation for the primary outcome. Fewer PROSPERO records (25/150 [17%]) included information on the time points of interest for the primary outcome of the SR. Completeness of pre-specification was similar between therapeutic SRs and non-therapeutic SRs for all outcome components, except for time point, which was pre-specified more often in therapeutic SRs (20/78 [26%] therapeutic SRs versus 5/72 [7%] non-therapeutic SRs; risk ratio 3.69, 95% CI 1.46 to 9.32).

Table 3 Completeness of primary outcome pre-specification in a random sample of 150 SRs registered in PROSPERO between April 1, 2017 and September 30, 2017
Full size table

Discussion

Registration of SRs in PROSPERO has increased rapidly, from 285 records in the year of inception (i.e. 2011) to 30,000 by the end of 2017. Half a million visits to the website occurred in the last 12 months from all over the world, most commonly from the UK. In a random sample of 150 SRs registered in PROSPERO between April 1, 2017 and September 30, 2017, the majority were focused on a therapeutic or epidemiologic question, and are being conducted without a dedicated funding source. A wide range of health areas were addressed across the 150 SRs, the most common being diseases of the circulatory system, mental and behavioural disorders and diseases of the musculoskeletal system and connective tissue. Pre-specification of primary outcomes was incomplete in most of the 150 records, with almost a third specifying the domain only.

Strengths and limitations

A strength of our study is that the PROSPERO database managers provided us with routinely collected data on the number of PROSPERO registrations and website usage, which removed the potential for errors due to manual data collection. Further, we were able to explore trends in these variables over time, unlike a previous evaluation which was limited to the first year of activity [13]. However, there are also some limitations. We analysed other variables (epidemiological characteristics and completeness of outcome pre-specification) in a random sample of SRs registered in PROSPERO between April and September 2017 only. Therefore, our frequency statistics may not generalise to SRs registered earlier. It is possible that some errors exist in our data on epidemiological characteristics and completeness of outcome pre-specification, given that data collection and classification were performed by one author only. However, we expect the number of errors to be low given the extensive experience that the data collector (MJP) has from recording this type of information in previous studies [1, 14, 17, 22].

Comparison with other studies

Compared with a previous analysis of the characteristics of PROSPERO registrations [13], the geographical scope of SRs registered has changed considerably over time. Registrations in the first year (2011–2012) came from 33 different countries, whereas that figure has risen to over 100 countries now. England remains the country responsible for most registrations, and Italy has moved into the list of top ten contributors, overtaking Denmark. The number of registrations submitted by authors based in Australia and China has also increased; in the first year of operation, these countries were in fourth and eighth position on the list of top contributing countries [13], and have moved into second and fourth position, respectively.

Explanations and implications

The exponential increase in SR registrations from 2011 to 2017 is a surprising, albeit welcome, development. Unlike for clinical trialists, who rapidly embraced trial registration only after the International Committee of Medical Journal Editors (ICMJE) announced that they would no longer publish trials that were not registered at inception [23], SR registration is not yet required by most journals (nor is it an ICMJE journal requirement). Therefore, the motivation of systematic reviewers to register their SR is likely due to other factors. It is possible that authors are paying attention to the increasing number of publications promoting the advantages of pre-registration of scientific studies and open science practices [24,25,26]. The uptake of registration could also be driven in part by journals that endorse the PRISMA Statement, which encourages SR registration in item 5 [6]. In addition, awareness of the benefits of SR registration likely increased following the dissemination of the PRISMA for Protocols Statement in January 2015 [27, 28], which mentions PROSPERO specifically in item 2. Further, many authors of SRs may be trialists as well, so an acceptance of the need for trial registration may have translated to an acceptance of SR registration.

The epidemiological characteristics of the registered SRs we examined share some similarities, and some differences, with a recent sample of completed SRs [1]. In both samples, most of the SRs focused on a therapeutic question (55% of 300 SRs indexed in MEDLINE® in February 2014 versus 52% of 150 SRs registered in PROSPERO in 2017), and few focused on a diagnostic/prognostic question (11 versus 7%, respectively). This possibly reflects the fact that methods for therapeutic SRs are more established than methods for other types of SRs [29, 30]. Diseases of the circulatory system was one of the most common health areas addressed in both samples; a positive sign given that conditions such as ischaemic heart disease and stroke are leading causes of mortality globally [31, 32]. Neoplasms and certain infectious and parasitic diseases—other leading causes of mortality [32]—were the most common areas addressed in the SRs indexed in MEDLINE® in February 2014 (in 16 and 14% of 300 SRs, respectively), yet both diseases were addressed in fewer (7%) of the 150 SRs registered in PROSPERO in 2017. We encourage systematic reviewers planning SRs to align their topic with global burden of disease data to ensure they are targeting clinically important questions [33].

None of the 150 registered SRs that we studied in detail were funded by a for-profit source. This is concerning given that other research has revealed a proliferation of meta-analyses are being commissioned by industry, most of which are not registered or published [34]. Clearly, more work is needed to encourage for-profit companies to embrace SR transparency practices. In the future, it would be worthwhile to explore (e.g. via surveys and interviews) the reasons for non-registration of SRs by investigators working in for-profit and non-profit organisations.

The incomplete pre-specification of outcomes that we observed in PROSPERO records is consistent with that observed in previous studies evaluating outcomes in Cochrane SR protocols [17, 18] and published SRs [17, 20]. This is likely because full pre-specification of outcomes is not yet a mandatory requirement for SR registration. We believe this should be reconsidered. There is emerging evidence that multiple results are often available for the same outcome in clinical trials (e.g. pain is measured using three scales, each at two time points), and that failure to completely pre-specify SR outcomes can lead to challenges with selecting and interpreting results from the included trials [17, 35, 36]. For example, Mayo-Wilson et al. found that across 21 trials of gabapentin for neuropathic pain, the number of unique within-trial results for pain intensity that systematic reviewers could select from ranged from 1 to 68 (because of within-trial multiplicity of outcome measurements, metrics, methods of aggregation and time points). For this reason, the total number of possible meta-analyses for pain intensity that systematic reviewers could (in theory) calculate was more than 34 trillion [36]. To deal with multiplicity challenges such as these, and prevent cherry-picking of the most favourable results when multiple are available within studies, systematic reviewers should make greater use of the “Timing and effect measures” field in PROSPERO, to completely pre-specify the outcomes of interest to the SR. In addition, such information in PROSPERO records should be accurately reflected in public versions of SR protocols and vice versa, and any amendments should be documented in completed SR articles.

There are several avenues of further work relating to PROSPERO. The field of knowledge synthesis is evolving to include other types of reviews, such as scoping reviews [37], living SRs [38] and rapid reviews [39]. Given the rapidly changing environment and methods, PROSPERO may need to adapt the criteria for inclusion in the database, as well as items on the registration form, to reflect the current state of the field. Also, it would be worthwhile to conduct a large-scale investigation of publication rates of registered SRs, the prevalence of and reasons for discontinued SRs and discrepancies in the planned and reported SR methods. Finally, that 30,000 SRs are underway suggest that the mass production of SRs previously discussed [40, 41] shows no sign of waning. It remains to be seen whether all of these SRs address questions that are relevant to end-users (e.g. patients, health care providers and policy makers), are well conducted, and are free of financial conflicts of interest. The PROSPERO management team have advised us that to minimise the potential for redundancy of SRs, forthcoming changes to the user interface will require those registering SRs to consider whether similar SRs already exist, and whether a new SR is necessary (L. Stewart, personal communication).

Conclusion

Registration of SRs in PROSPERO increased rapidly between 2011 and 2017, thus benefiting users of health evidence who want to know about ongoing SRs. Further work is needed to explore how closely published SRs adhere to the planned methods, whether greater pre-specification of outcomes prevents selective inclusion and reporting of study results, and whether registered SRs are addressing necessary questions.

Abbreviations

CI:

Confidence interval

ICD-10:

International Classification of Diseases, Tenth Revision

NIHR:

National Institute for Health Research

PRISMA:

Preferred Reporting Items for Systematic reviews and Meta-Analyses

SR:

Systematic review

References

  1. Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13(5):e1002028.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4:e78.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Stewart L, Moher D, Shekelle P. Why prospective registration of systematic reviews makes sense. Syst Rev. 2012;1:7.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Moher D, Booth A, Stewart L. How to reduce unnecessary duplication: use PROSPERO. BJOG. 2014;121(7):784–6.

    Article  CAS  PubMed  Google Scholar 

  5. Page MJ, Altman DG, Shamseer L, McKenzie JE, Ahmadzai N, Wolfe D, et al. Reproducible research practices are underused in systematic reviews of biomedical interventions. J Clin Epidemiol. 2018;94:8–18.

    Article  PubMed  Google Scholar 

  6. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6(7):e1000100.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Tricco AC, Pham B, Brehaut J, Tetroe J, Cappelli M, Hopewell S, et al. An international survey indicated that unpublished systematic reviews exist. J Clin Epidemiol. 2009;62(6):617–23. e5

    Article  PubMed  Google Scholar 

  9. Kirkham JJ, Altman DG, Williamson PR. Bias due to changes in specified outcomes during the systematic review process. PLoS One. 2010;5(3):e9810.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, et al. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev. 2012;1:2.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L. An international registry of systematic-review protocols. Lancet. 2011;377(9760):108–9.

    Article  PubMed  Google Scholar 

  12. Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L. Establishing a minimum dataset for prospective registration of systematic reviews: an international consultation. PLoS One. 2011;6(11):e27319.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, et al. PROSPERO at one year: an evaluation of its utility. Syst Rev. 2013;2:4.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Tricco AC, Cogo E, Page MJ, Polisena J, Booth A, Dwan K, et al. A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study. J Clin Epidemiol. 2016;79:46–54.

    Article  PubMed  Google Scholar 

  15. Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7(2):e012545.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Sideri S, Papageorgiou SN, Eliades T. Are orthodontic systematic reviews registered a priori in PROSPERO? J Orthod. 2017;44(4):249-55.

  17. Page MJ, McKenzie JE, Chau M, Green SE, Forbes A. Methods to select results to include in meta-analyses deserve more consideration in systematic reviews. J Clin Epidemiol. 2015;68(11):1282–91.

    Article  PubMed  Google Scholar 

  18. Liu Z, Saldanha IJ, Margolis D, Dumville JC, Cullum NA. Outcomes in Cochrane systematic reviews related to wound care: an investigation into prespecification. Wound Repair Regen. 2017;25(2):292–308.

    Article  PubMed  Google Scholar 

  19. Page MJ, McKenzie JE, Kirkham J, Dwan K, Kramer S, Green S, et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev. 2014;10:MR000035.

    Google Scholar 

  20. Saldanha IJ, Dickersin K, Wang X, Li T. Outcomes in Cochrane systematic reviews addressing four common eye conditions: an evaluation of completeness and comparability. PLoS One. 2014;9(10):e109400.

    Article  PubMed  PubMed Central  Google Scholar 

  21. StataCorp. Stata statistical software: release 14. College Station: StataCorp LP; 2015.

    Google Scholar 

  22. Page MJ, Forbes A, Chau M, Green SE, McKenzie JE. Investigation of bias in meta-analyses due to selective inclusion of trial effect estimates: empirical study. BMJ Open. 2016;6(4):e011863.

    Article  PubMed  PubMed Central  Google Scholar 

  23. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004;351(12):1250–1.

    Article  PubMed  Google Scholar 

  24. Nosek BA, Alter G, Banks GC, Borsboom D, Bowman SD, Breckler SJ, et al. Scientific standards. Promoting an open research culture. Science. 2015;348(6242):1422–5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. McKiernan EC, Bourne PE, Brown CT, Buck S, Kenall A, Lin J, et al. How open science helps researchers succeed. elife. 2016;5:e16800.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Kidwell MC, Lazarevic LB, Baranski E, Hardwicke TE, Piechowski S, Falkenberg LS, et al. Badges to acknowledge open practices: a simple, low-cost, effective method for increasing transparency. PLoS Biol. 2016;14(5):e1002456.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647.

    Article  Google Scholar 

  29. Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration. 2011. Available from http://training.cochrane.org/handbook.

  30. Tricco AC, Zarin W, Ghassemi M, Nincic V, Lillie E, Page MJ, et al. Same family, different species: methodological conduct and quality varies according to purpose for five types of knowledge synthesis. J Clin Epidemiol. 2017.

  31. Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the global burden of disease study 2010. Lancet. 2012;380(9859):2095–128.

    Article  PubMed  Google Scholar 

  32. GBD 2016 Causes of Death Collaborators. Global, regional, and national age-sex specific mortality for 264 causes of death, 1980-2016: a systematic analysis for the global burden of disease study 2016. Lancet. 2017;390(10100):1151–210.

    Article  Google Scholar 

  33. Yoong SL, Hall A, Williams CM, Skelton E, Oldmeadow C, Wiggers J, et al. Alignment of systematic reviews published in the Cochrane database of systematic reviews and the database of abstracts and reviews of effectiveness with global burden-of-disease data: a bibliographic analysis. J Epidemiol Community Health. 2015;69(7):708–14.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Schuit E, Ioannidis JP. Network meta-analyses performed by contracting companies and commissioned by industry. Systematic Reviews. 2016;5(1):198.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Mayo-Wilson E, Fusco N, Li T, Hong H, Canner JK, Dickersin K. Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis. J Clin Epidemiol. 2017;86:39–50.

    Article  PubMed  Google Scholar 

  36. Mayo-Wilson E, Li T, Fusco N, Bertizzolo L, Canner JK, Cowley T, et al. Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy. J Clin Epidemiol. 2017. J Clin Epidemiol. 2017;91:95–110.

  37. Tricco AC, Lillie E, Zarin W, O’Brien K, Colquhoun H, Kastner M, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16:15.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Elliott JH, Synnot A, Turner T, Simmonds M, Akl EA, McDonald S, et al. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol. 2017;91:23–30.

    Article  PubMed  Google Scholar 

  39. Abou-Setta AM, Jeyaraman MM, Attia A, Al-Inany HG, Ferri M, Ansari MT, et al. Methods for developing evidence reviews in short periods of time: a scoping review. PLoS One. 2016;11(12):e0165903.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. The Milbank Quarterly. 2016;94(3):485–514.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Page MJ, Moher D. Mass production of systematic reviews and meta-analyses: an exercise in mega-silliness? The Milbank Quarterly. 2016;94(3):515–9.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

We thank Prof. Lesley Stewart and Dr. Julie Jones-Diette (Centre for Reviews and Dissemination, University of York) for providing us with registration and website usage data from PROSPERO.

Funding

There was no direct funding for this study. MJP is supported by an Australian National Health and Medical Research Council (NHMRC) Early Career Fellowship (1088535). ACT is funded by a Tier 2 Canada Research Chair in Knowledge Synthesis. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Availability of data and materials

The data underlying this study and statistical analysis code are available on the Open Science Framework: https://osf.io/v94ym/.

Author information

Authors and Affiliations

  1. School of Public Health and Preventive Medicine, Monash University, 553 St Kilda Road, Melbourne, VIC, 3004, Australia

    Matthew J. Page

  2. Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, K1H 8L6, Canada

    Larissa Shamseer

  3. School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1H 8M5, Canada

    Larissa Shamseer

  4. Knowledge Translation Program, Li Ka Shing Knowledge Institute, St Michael’s Hospital, Toronto, ON, M5B 1W8, Canada

    Andrea C. Tricco

  5. Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, M5T 3M7, Canada

    Andrea C. Tricco

Authors
  1. Matthew J. Page
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Larissa Shamseer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrea C. Tricco
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors declare to meet the ICMJE conditions for authorship. MJP and ACT conceived the study design. MJP collected data and undertook the statistical analyses. MJP wrote the first draft of the article, with contributions from ACT. LS and ACT contributed to revisions of the article. All authors approved the final version of the submitted article.

Corresponding author

Correspondence to Matthew J. Page.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

We have read the journal’s policy and the authors of this manuscript have the following competing interests: MJP, LS, and ACT are Associate Editors for Systematic Reviews, but had no involvement in the peer review process or decision for publication.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Page, M.J., Shamseer, L. & Tricco, A.C. Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst Rev 7, 32 (2018). https://doi.org/10.1186/s13643-018-0699-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-018-0699-4

Keywords

Systematic Reviews

ISSN: 2046-4053