Systematic Reviews

Collects and summarizes research on a topic.

Collects and summarizes research on a topic. Emphasis is on finding ALL available information, but there may be a focus on high quality evidence (Lefebvre et al, 2025).

A single person or a team may do the work.

Generally, at least two people are involved in a systematic review (Lefebvre et al, 2025). Most guidelines recommend at least 2 subject experts for the screening process. Other team members can include an information specialist (librarian), statistician, and administrative assistants. Some teams recruit additional subject experts for publications needing translation.

The authors may chose to include a methods section, indicating how they found the included research, or they may omit this information (Hempel, 2019, p. 119-120).

Provides detailed information about the search strategy, including the sources searched (databases, search engines, registers), date(s) on which they were searched, search strategies used, number of results found, and any additional methods used such as hand-searching, mining reference lists, citation tracking, etc... (Lefebvre et al, 2025, Page et al, 2021). Information provided should be sufficient to allow reproduction of the exact search.

The authors may or may not use inclusion and exclusion criteria.

Inclusion and exclusion criteria are clearly stated, and acceptable levels of evidence are usually defined before the search is undertaken (Lefebvre et al, 2025, Page et al, 2021). The emphasis is on what's appropriate for the subject matter - for instance, topics involving an intervention would likely focus on randomized controlled trials, while a discussion of prognostic factors would be more likely to accept cohort studies (OCEBM Levels of Evidence Working Group, 2011). Time period is usually all time, unless there is a logical start time, such as when a particular intervention was invented/discovered (Lefebvre et al, 2025).

Reviewers usually simply begin work on the review - they do not publish a protocol.

Before the search begins, the reviewers decide upon a protocol for the review, which is often published ahead of time (Page et al, 2021). The reviewers adhere strictly to this protocol to prevent bias (Thomas et al, 2023). In some cases, small departures from the protocol may occur if justifiable (Thomas et al, 2023) - for instance, if the reviewers suddenly gain or lose access to a database. 

As of June 2025, there are no official guidelines for writing literature or narrative reviews (Equator Network, n.d.).

Although various sources, such as book and articles, give advice on writing a literature review (Hempel, 2019; Dawidowicz, 2010), authors may or may not consult these sources before working on their review. When it comes time to publish, the author(s) would likely consult guidelines for authors available on the website of the journal where they plan to submit the article.

The reviewers likely select and follow at least one of the guidelines for performing a systematic review.

Some of the available advice on writing a literature review gives suggestions on how to make this process easier, but there are no requirements (Hempel, 2019).

Every search result from each of the databases is collected and stored. Duplicates are evaluated, counted, and removed. Each of the remaining citations is screened by two subject experts. A list of reasons for exclusion is drawn up, with each excluded article being tallied/recorded as it is removed. (Lefebvre et al, 2025).

Sources which are not readily available may or may not be sought. Some advice on writing a literature review actually supports limiting to readily available sources, while acknowledging that relevant information may be lost (Hempel, 2019, p. 56).

Every citation which survives the initial screening process is sought (i.e. they try to find the entire article). Often a budget has been set aside for purchasing articles which are not readily available. Final screening decisions are made after the subject experts have reviewed the entire article.

Some advice on writing a literature review recommends using secondary sources, if the topic is well-researched, for efficiency (Hempel, 2019, p. 50). However, scholars are taught to cite primary sources whenever possible, to avoid misinterpretation errors (American Psychological Association, 2022), and most reviewers follow this advice.

The systematic review itself includes only primary sources, except in some cases meta-analyses may be included. 

The reviewers may use previously published systematic and literature reviews to identify primary research studies which were not found by their own search - they then locate, analyze, and cite the primary sources for their own systematic review (Lefebvre et al, 2025). The reviewers may also cite secondary sources in the Introduction or Discussion sections of their review, as background or for comparison.

The reviewer may or may not evaluate the quality of the sources they are including. If the reviewer notices a flaw or bias in a source, they may choose not to include it, or they may discuss it but point out the shortcomings. Various sources provide varying information about parameters to evaluate (Hempel, 2019, Chapters 4 and 7; Dawidowicz, 2010, Chapters 6 and 9).

The reviewers use rigorous methods to evaluate the quality of the studies (Page et al, 2021).

Note that finding methodological flaws or high risk of bias does not result in automatic exclusion from the review. In some cases, the effect of these flaws or bias on the study results may be thoughtfully explored and discussed (Higgins et al, 2019).

The sources are indicated by in-text citations and a reference list. For the in-text citations, author-date style, endnotes, or footnotes may be used according to the chosen citation style.

Note: the absense of in-text citations does not necessarily disqualify a document from being a literature review - for instance, this is sometimes seen in older reviews. In a more recent document, absense of some kind of in-text citations should prompt suspicion - is this a non-scholarly source, or a predatory journal? However, this is still occasionally seen in literature reviews which either emphasize conciseness or a practical "how-to" focus.

In addition to in-text citations and a reference list, most systematic reviews contain tables presenting information about the included studies (Li et al, 2019). Common tables include:

• Study characteristics

• Study results

• Study appraisals and/or risk of bias analyses

The reviewer has likely picked a topic which they know has information available. If this turns out to be a mistake, they could adjust the topic.

Once the protocol has been established, the researchers do not make adjustments. A surprisingly large number of systematic reviews end with the statement that there wasn't any information of sufficient quality to be reviewed, and no conclusions can be drawn. 

The reviewer may be collecting additional sources as they write, or not. They may write the article within a short amount of time, e.g. a month, but if they take longer, they may or may not be watching for new articles, and so may or may not make updates just before submitting for publication.

Usually a significant amount of time passes between the day of database searching and the first draft of the article - the average is about a year. At that point, the subject experts ask the librarian to re-run the search and identify any new articles, at which time a mini-review (deduplication, screening, appraisal, and analysis) take place and the new information is incorporated into the article before publication (Lefebvre et al, 2025).

There are no particular rules to whether a literature review is updated or stand-alone. This may be up to the reviewer, however, certain journals may encourage authors to write updates.

Systematic reviews may be done once or on a regular basis. Some systematic reviews are scheduled to be updated every 5 years. The reviewers generally keep all documentation between updates to streamline the updating process. The search strategies are generally reviewed before each update in case of new information, for instance if a new word/phrase has been coined, or a new treatment discovered.

Collects and summarizes articles on a topic. Emphasis is on finding ALL available information.

Collects and analyzes articles on a topic. Emphasis is on recent articles of high relevance to the topic.

Generally, at least two people are involved in a systematic review (Lefebvre et al, 2025).

A single person does the entire project. They are guided by their advisor, and can seek assistance from library staff, but all the work is their own.

Provides detailed information about the search strategy, including the sources searched (databases, search engines, registers), date(s) on which they were searched, search strategies used, number of results found, and any additional methods used such as hand-searching, mining reference lists, citation tracking, etc...

As with a systematic review, provides detailed information about the search strategy, but within specific templates available on the DCP8806 course pages in Brightspace.

Inclusion and exclusion criteria are clearly stated, and acceptable levels of evidence are usually defined before the search is undertaken (Lefebvre et al, 2025, Page et al, 2021). The emphasis is on what's appropriate for the subject matter - for instance, topics involving an intervention would likely focus on randomized controlled trials, while a discussion of prognostic factors would be more likely to accept cohort studies (OCEBM Levels of Evidence Working Group, 2011). Time period is usually all time, unless there is a logical start time, such as when a particular intervention was invented/discovered (Lefebvre et al, 2025).

Some inclusion/exclusion criteria are defined by the project, such as peer reviewed, recent articles. Other inclusion/exclusion criteria are defined by the student (approved by the advisor). Unlike a standard systematic review, these may be adjusted at the time of searching or screening to obtain the correct number of articles as defined by the advisor (if approved by the advisor).

Before the search begins, the reviewers decide upon a protocol for the review, which is often published ahead of time (Page et al, 2021). The reviewers adhere strictly to this protocol to prevent bias (Thomas et al, 2023). In some cases, small departures from the protocol may occur if justifiable (Thomas et al, 2023) - for instance, if the reviewers suddenly gain or lose access to a database.

In Unit 7, the student writes the Methodology section, which is the equivalent of the protocol. In most cases, for practical purposes, the student has already completed at least the search stage of their project, and adjustments may be made if approved by the advisor, for example, the student may limit to 5 years rather than 10 if there is a lot of research available.

The reviewers likely select and follow one of the guidelines for performing a systematic review.

The ASP project uses the PRISMA guidance, with some modification.

Every search result from each of the databases is collected and stored. Duplicates are evaluated, counted, and removed. Each of the remaining citations is screened by two subject experts. A list of reasons for exclusion is drawn up, with each excluded article being tallied/recorded as it is removed (Lefebvre et al, 2025).

Every search result from each of the databases is collected and stored. Duplicates are evaluated, counted, and removed. The student then screens the results, hopefully recording their decisions as they go along, using the PRISMA flow chart as a guide.

Every citation which survives the initial screening process is sought (i.e. they try to find the entire article). Often a budget has been set aside for purchasing articles which are not readily available. Final screening decisions are made after the subject experts have reviewed the entire article.

The student may choose to focus on articles which are readily available, or may supplement by using the YU library's document delivery request form. 

The systematic review itself includes only primary sources, except in some cases meta-analyses may be included. 

The reviewers may use previously published systematic and literature reviews to identify primary research studies which were not found by their own search - they then locate, analyze, and cite the primary sources for the systematic review (Lefebvre et al, 2025). The reviewers may also cite secondary sources in the Introduction or Discussion sections of their review, as background or for comparison.

The systematic review itself includes only primary sources. Meta-analyses are not included in the articles to be analyzed. In some cases, the advisor may suggest including only qualitative or only quantitative studies.

As with a standard systematic review, other systematic reviews and meta-analyses may be used to identify interesting studies, or be cited in the introduction or discussion sections, as appropriate.

The reviewers use rigorous methods to evaluate the quality of the studies (Page et al, 2021).

Note that studies which are found to have methodological flaws or high risk of bias may not be excluded from the review, but rather, the effect of these flaws or bias on the study results may be thoughtfully explored and discussed (Higgins et al, 2019).

The reviews use critical appraisal tools identified by the advisor to evaluate the quality of the studies.

In addition to in-text citations and a reference list, most systematic reviews contain tables presenting information about the included studies (Li et al, 2019). Common tables include:

• Study characteristics

• Study results

• Study appraisals and/or risk of bias analyses

All documents are formatted according to APA 7th edition citation style, including in-text citations, references, and all other formatting.

The ASP templates indicate which information should be included in table form.