Cochrane Centralised Search Service showed high sensitivity identifying randomised controlled trials: A retrospective analysis.
Noel-Storr AH., Dooley G., Wisniewski S., Glanville J., Thomas J., Cox S., Featherstone R., Foxlee R.
BACKGROUND: The Cochrane Central Register of Controlled Trials (CENTRAL) is compiled from a number of sources, including PubMed and Embase. Since 2017, we have increased the number of sources feeding into CENTRAL and improved the efficiency of our processes through the use of APIs, machine learning and crowdsourcing. OBJECTIVES: Our objectives were twofold: (1) Assess the effectiveness of Cochrane's centralised search and screening processes to correctly identify references to published reports which are eligible for inclusion in Cochrane systematic reviews of randomised controlled trials (RCTs). (2) Identify opportunities to improve the performance of Cochrane's centralised search and screening processes to identify references to eligible trials. METHODS: We identified all references to RCTs (either published journal articles or trial registration records) with a publication or registration date between 1st January 2017 and 31st December 2018 that had been included in a Cochrane intervention review. We then viewed an audit trail for each included reference to determine if it had been identified by our centralised search process and subsequently added to CENTRAL. RESULTS: We identified 650 references to included studies with a publication year of 2017 or 2018. Of those, 634 (97.5%) had been captured by Cochrane's Centralised Search Service (CSS). Sixteen references had been missed by the CSS: six had PubMed-not-MEDLINE status, four were missed by the centralised Embase search, three had been misclassified by Cochrane Crowd, one was from a journal not indexed in MEDLINE or Embase, one had only been added to Embase in 2019, and one reference had been rejected by the automated RCT machine learning classifier. Of the sixteen missed references, eight were the main or only publication to the trial in the review in which it had been included. CONCLUSIONS: This analysis has shown that Cochrane's centralised search and screening processes are highly sensitive. It has also helped us to understand better why some references to eligible RCTs have been missed. The CSS is playing a critical role in helping to populate CENTRAL and is moving us towards making CENTRAL a comprehensive repository of RCTs.