Develop and Register Your Protocol
In order to be conducted in a manner that is logical, transparent, reproducible, thorough and subject to a minimum level of bias, systematic reviews need to follow a well-defined, comprehensive, and explicit methodology, or "protocol.'' This protocol must be developed and agreed upon in advance of any attempt to accumulate, manage, or evaluate search results.
The protocol is used as a roadmap throughout the research process and ensures research integrity, accountability, and that the research team remains focused on the original research question.
Elements of the Protocol
- Research question, rationale and objectives
- Ample relevant literature to warrant a systematic review/meta-analysis
- Inclusion and exclusion criteria
- Information sources to be searched
- Proposed key search terms and search strategy
- Methods and tools for screening, data extraction, quality assessment, and bias analysis
- Data management process and tools
- Team members and contact details
- Timeline
Protocol Development: Checklists*
There are several checklists available to guide the development of a protocol.
Protocol Development: The Importance of Registering
It is highly recommended that a protocol is registered once it has been finalized. The development of registries evolved due to the significant increase in systematic reviews being conducted. Protocol registries perform several important functions.
- Reduce duplication of systematic reviews
- Enhance transparency
- Demonstrate credibility
Registries
Protocols can be registered with one or more registries. Listed below are two well-known registeries.
*Additional guidance for protocol development is provided in the 'Further Guidance' section of this guide.
Identifying a Topic & Developing a Research Question
Formulating a research question for a systematic review may be a lengthy process that requires rethinking and revisions. Follow these steps to begin your process.
Background Searching
- Conduct exploratory searches to scope the literature on the topic and determine how your review will differ from others
- Write down keywords and concepts
- Identify key studies related to your topic
- Find systematic reviews on related topics by searching in registries such as:
Form the Question
- Research questions for systematic reviews should be narrow and specific
- If relevant, choose a question framework such as PICO or SPIDER to structure your question
Focus & Revise
- During the earliest stages of the search process, your research question will likely change in scope or terminology
- Use Thesaurus, Subject Terms, or Medical Subject Headings in databases to identify alternative keywords and concepts
Choosing Databases
Systematic reviews require searching in a variety of information sources including but not limited to subscription databases, open access databases, and grey literature. Consider the various types of information you will need to fully answer your research question, and what types of resources you will need to search for this information.
- DePaul Library A-Z DatabasesFilter by subject, source type, and vendor/provider.
- DePaul Library Research GuidesFor a guided choice of appropriate databases available at DePaul, use one or more of the library's subject research guides .
- Research Databases - Illinois State LibraryResources marked as 'In-library Access Only' represent subscription databases that are available only to users at the Illinois State Library. All other resources are either available freely via the Web or through remote access by current Illinois state government employees.
- Data.govHome of U.S. government open data.
- National Archives Online DatabasesSome databases are freely accessible online, while others are freely accessible only at a National Archives facility.
- Open Government ResourcesGuide to government resources from Fordham University Libraries.
Grey Literature
Grey Literature Definition
"Grey literature stands for manifold document types produced on all levels of government, academics, business and industry in print and electronic formats that are protected by intellectual property rights, of sufficient quality to be collected and preserve by libraries and institutional repositories, but not controlled by commercial publishers; i.e., where publishing is not the primary activity of the producing body."
- 2010 International Conference on Grey Literature, Prague
Loosely defined, grey literature consists of any information not published in a commercial, peer-reviewed journal. Some types of grey literature used in systematic reviews include, but are not limited to:
- Abstracts
- Conference Proceedings
- Dissertations & Theses
- Pre-Prints
- Clinical Trials
- Clinical or Practice Guidelines
- Working Papers
- White Papers
- Reports
- Standards & Patents
- Policy & Legislation
Why Search Grey Literature?
Searching for grey literature can help reduce the effects of publication bias that exist in peer-reviewed literature. Peer-reviewed journals often favor publishing studies with significant and positive results, so that studies with conflicting or negative results remain unpublished. It is important to survey as much literature as possible for a systematic review, including studies with negative or low-impact results.
Many library databases include filters for different types of grey literature, and there are various online and open-access sources to find grey literature. Grey literature searches require multiple searches across several platforms to survey what is available.
Find links to selected sources for grey literature below.
Further Reading
Grey Literature Sources
- Dissertations & Theses Global (ProQuest) This link opens in a new window
ProQuest Dissertations & Theses Global (PQDT Global) database is the world's most comprehensive collection of multidisciplinary dissertations and theses, featuring 6 million records from more than 70 countries.
Dates Covered: 1637-present. - Open Access Theses and Dissertations This link opens in a new window
Electronic theses & dissertations openly available through institutional repositories worldwide.
Access note: Freely available to the public. - PapersFirst This link opens in a new window
Indexes unpublished papers presented at conferences in many academic fields; with abstracts.
Dates Covered: 1993-present - ProceedingsFirst This link opens in a new window
Indexes documents produced by congresses, symposia and conferences in many academic fields; with abstracts.
Dates Covered: 1993-present. - Sociological Abstracts (ProQuest) This link opens in a new window
Abstracts and indexes the international literature in sociology and related disciplines in the social and behavioral sciences. The database provides abstracts of journal articles and citations to book reviews drawn from thousands of serials publications, and also provides abstracts of books, book chapters, dissertations, and conference papers. Includes Social Services Abstracts, and forms part of the Sociology Collection.
Dates Covered: 1952-present.
- Google ScholarIncludes results for conference proceedings, books, and abstracts. Be aware that Google Scholar is not an exhaustive search and may exclude key pieces of evidence.
- Cochrane Library This link opens in a new window
Find high quality, independent evidence-based health care information (protocols, methods, trials, evaluations and assessments) from six different Cochrane Library databases, including The Cochrane Database of Systematic Reviews (CDSR), a comprehensive collection of peer-reviewed systematic reviews produced by the Review Group of The Cochrane Collaboration.
- IEEE Xplore This link opens in a new window
The IEEE/IET Electronic Library (IEL), on the IEEE Xplore platform, provides access to journals, transactions, magazines, conference proceedings, and IEEE Standards in computing, software, engineering, communications, biomedical engineering, information science, and other related areas.
- SciFinder-n This link opens in a new window
Chemical Abstracts database allows searching for journal articles, patents, conference proceedings, dissertations, books, reviews, technical reports and substances by topic, author, company, substance name, reaction, or chemical structure.
Dates Covered: 1907-present.
Access note: Not available to walk-in users.
- ClinicalTrials.govClinicalTrials.gov hosts a large collection of clinical studies from around the world. You can search for studies using keywords such as a condition or disease, drug, or medical device.
- Grey Literature Report - The New York Academy of MedicineA list of websites and organization that publish grey literature reports related to public health research.
- medRxiv (pronounced "med-archive")Free preprint server for unpublished manuscripts in the health sciences. Founded by Cold Spring Harbor Laboratory, Yale University, and BMJ.
- NIH RePORTERThe Research Portfolio Online Reporting Tools (RePORT) website provides access to reports, data, and analyses of NIH research activities, including information on NIH expenditures and the results of NIH-supported research.
- World Health Organization - International Clinical Trials Registry Platform (ICTRP)Databases of trial registration data sets from providers around the world.
- Congress.gov This link opens in a new window
Congress.gov is the official website for U.S. federal legislative information. Search through Committee Hearings, Committee Reports, Legislation, and Members of Congress. Bills are searchable from 1973-present, the Congressional Record from 1995-present, and Members from 1929-present. Data are usually updated the morning after a session adjourns.
- Congressional (ProQuest) This link opens in a new window
Provides the tools for searching the Congressional Record, finding members of Congress, tracking proposed legislation by keyword and sponsor, and searching Congressional hearings, the Federal Register, the U.S. Serial Set Digital Collection, Congressional committee schedules, Committee Prints and Miscellaneous Publications, and Congressional Research Service (CRS) Reports. Dates covered: 2004 – present.
- Govinfo This link opens in a new window
The U.S. Government Printing Office disseminates official information from all three branches of the Federal Government. Search the entire catalog of government publications. Formerly called FDsys.
Access note: Freely available to the public. - Policy File Index This link opens in a new window
Indexes, abstracts, and links directly to current research on public policy issues, with international coverage from many organizations around the world, as well as in-depth coverage of U.S. domestic policy organizations. It forms part of the Politics Collection.
Dates covered: 1990-present. - ProQuest Legislative Insight This link opens in a new window
Legislative histories, including fully searchable PDFs of full-text publications. Each history includes the full text of the public law itself, all versions of related bills, law-specific Congressional Record excerpts, committee hearings, reports, and prints. Also included are presidential signing statements, CRS reports, and miscellaneous congressional publications that provide background material to aid in the understanding of issues related to the making of the law.
- Think Tank Search - Harvard Kennedy School of GovernmentA Google Custom Search of more than 1200 think tanks and research centers.
- IEEE Xplore This link opens in a new window
The IEEE/IET Electronic Library (IEL), on the IEEE Xplore platform, provides access to journals, transactions, magazines, conference proceedings, and IEEE Standards in computing, software, engineering, communications, biomedical engineering, information science, and other related areas.
- Standard Reference Data (National Institute of Standards & Technology) This link opens in a new window
Provides access to a wide range of atomic and small molecule spectral databases, including those accessible via the “NIST Chemistry WebBook.”
Access note: Freely available to the public. - SciFinder-n This link opens in a new window
Chemical Abstracts database allows searching for journal articles, patents, conference proceedings, dissertations, books, reviews, technical reports and substances by topic, author, company, substance name, reaction, or chemical structure.
Dates Covered: 1907-present.
Access note: Not available to walk-in users.
Defining the Process
