Systematic Literature Reviews

Overview

Systematic literature reviews (SLRs) are a structured, reproducible method for identifying and synthesizing existing research to answer a focused research question. This page will guide you through the concept and process step by step.

What Is a Systematic Literature Review?

An SLR is a scholarly synthesis of evidence on a clearly defined topic, using explicit, pre-specified methods to identify, select, critically appraise, and summarise relevant studies. Unlike ad hoc reading of literature, every decision is documented so the review can be replicated.

SLR vs. Traditional Literature Review

Feature Traditional Literature Review Systematic Literature Review
Research question Broad or flexible Narrow and pre-specified
Search strategy Informal, author-led Documented, reproducible
Study selection Subjective Governed by explicit criteria
Quality appraisal Often absent Mandatory
Reporting Variable Follows standards (e.g., PRISMA) ​
Replicability Low High

When Should You Use an SLR?

The SLR Process: Step by Step

Step 1: Define Your Research Question / Select Framework

Read more

A well-defined research question is the foundation of a systematic literature review. Every subsequent decision (which databases to search, what terms to use, which studies to include) flows directly from it. A question that is too broad produces an unmanageable volume of results; one that is too narrow may yield almost nothing. Structured question frameworks give you a reliable method for making your question precise and searchable before you open a single database.

Review Define Your Research Question & Select Framework for more information on this topic.

Step 2: Write a Protocol

Read more

A protocol is a written plan that specifies, in advance, exactly how you intend to conduct your systematic literature review. It is not complete until all 11/12 sub-sections below have been worked through and recorded. Writing the protocol is not a bureaucratic hurdle; it is the mechanism that makes your review transparent, reproducible, and defensible to examiners, supervisors, and future readers.

Review Writing a Protocol for more information on why it matters and what it contains.

Step 3: Conduct Your Search

Read more

This step translates the search strategy you documented in your protocol into actual database queries, records the results systematically, and prepares a clean, deduplicated set of references for screening. Precision and documentation at this stage are critical: every decision you make must be recorded so that your search can be reported transparently in your final thesis.

Review Conducting Your Search for more information on this topic.

Before you search, use our Search Quality Self-Assessment Checklist (adapted from vom Brocke et al., 2015) to verify your search strategy meets the standards expected in a systematic review.

Step 4: Screen Results

Read more

 Screening is the process of applying your pre-specified inclusion and exclusion criteria to the deduplicated set of references produced earlier, in order to identify the studies that will form the basis of your review. It proceeds in two sequential phases: first by title and abstract, then by full text. Each phase reduces the total set further; only studies that pass both phases are included in your final review.

Review Screening the Results for more information on this topic.

Step 5: Appraise Study Quality

Read more

Quality appraisal is the systematic assessment of the methodological rigor of each study included after screening. It answers the question: how much confidence can we place in the findings of this study? Appraisal does not judge whether a study is interesting or relevant (screening already established relevance); it judges whether the study was conducted in a way that makes its findings trustworthy.

Review Appraise Study Quality for more information on this topic.

Step 6: Extract Data

Read more

Data extraction is the process of systematically pulling the information you need from each included study and recording it in a standardised form. It bridges the gap between your screened, appraised set of studies and the synthesis you will conduct later. Consistent, thorough extraction is what makes synthesis possible: if you extract different information from different papers, you cannot meaningfully compare or combine them.

Review Extract Data for more information on this topic.

Step 7: Synthesize and Report

Read more

Synthesis is where the work of the review becomes an argument. Having identified, screened, appraised, and extracted data from your included studies, you now interpret what they collectively say in response to your research question. Reporting then translates that interpretation into a structured written account that meets the standards of academic transparency required for a thesis.

Review Synthesize and Report for more information on this topic.

Before Submission: Evaluating Your Own SLR Process

A Note on Process Models

Different disciplines and authors present SLR processes with slightly different stage names and orders. The seven-step structure presented here synthesises best practices from management research (Tranfield et al., 2003), software engineering (Kitchenham & Charters, 2007), and information systems (vom Brocke et al., 2015; Bandara et al., 2015). The core sequence (question → protocol → search → screen → appraise → extract → synthesise) is common across all models; differences are primarily in emphasis rather than substance.

SLR Process by (Kitchenham and Charters 2007).png
"SLR Process of Kitchenham and Charters 2007" by Hasan Koç is licensed under CC BY-NC-SA 4.0, based on Kitchenham & Charters, 2007.

SLR Process by (Bandara et al 2015).png
"SLR Process of Bandara et al 2015" by Hasan Koç is licensed under CC BY-NC-SA 4.0, based on Bandara et al., 2015.

Define Your Research Question & Select Framework

Overview

A well-defined research question is the foundation of a systematic literature review. Every subsequent decision (which databases to search, what terms to use, which studies to include) flows directly from it. A question that is too broad produces an unmanageable volume of results; one that is too narrow may yield almost nothing. Structured question frameworks give you a reliable method for making your question precise and searchable before you open a single database.

Why Are You Conducting This Review?

Before formulating your question, be clear about the purpose of your review. A systematic literature review is not a default method; it is the right choice for specific research goals. Common justifications in business and management research include:

Your research question should follow directly from one of these purposes. If you cannot identify which of these your review serves, discuss the scope with your supervisor before proceeding.

Why Structure Your Question?

Formulating your question using a framework forces you to identify the exact components of your topic and translate them into search terms. This step is worth doing carefully in writing, not just in your head, because:

Choosing a Framework

Different frameworks suit different types of research questions. The table below helps you select the right one.

Framework Best suited to Research types
PICO Questions about the effect of something Quantitative, experimental
SPIDER Questions about experiences, perceptions, or behaviors Qualitative, mixed-methods
PCC Questions about what exists in a topic area (scoping) Exploratory, mapping reviews

As a business or management student, you will most often use SPIDER or PCC, since many management questions ask "how" or "what" rather than "does X cause Y."

The PICO Framework

Stands for: Population · Intervention · Comparison · Outcome

Originally developed for clinical research, PICO is useful when your question tests whether a specific practice, policy, or programme produces a measurable result. It is less common in pure management research but relevant if your thesis touches on organisational interventions, training effectiveness, or behavioral economics.

Element Question to ask yourself Example
Population Who or what is the focus? SMEs in the EU retail sector
Intervention What practice or factor is being examined? Agile project management adoption
Comparison What is it being compared to? Traditional waterfall project management
Outcome What result are you measuring? Employee productivity and project delivery speed

Resulting question: In EU retail SMEs (P), does adopting agile project management (I) compared to traditional methods (C) improve employee productivity and delivery speed (O)?

The SPIDER Framework

Stands for: Sample · Phenomenon of Interest · Design · Evaluation · Research type

SPIDER was developed specifically for qualitative and mixed-methods research, where the concept of an "intervention" does not apply. It is well-suited to management questions about how people experience organisational phenomena such as leadership styles, workplace culture, or sustainability reporting.

Element Question to ask yourself Example
Sample Who is the population being studied? Mid-level managers in multinational corporations
Phenomenon of Interest What experience, behavior, or issue is the focus? Remote work and perceived organisational belonging
Influencing factors What contextual factors are relevant? Post-pandemic hybrid work policies
Design What study designs will you include? Interviews, surveys, case studies
Evaluation What outcome or concept is being assessed? Employee engagement, retention intention
Research type Qualitative, quantitative, or mixed? Qualitative or mixed-methods

Resulting question: How do mid-level managers in multinationals (S) experience organisational belonging in remote/hybrid work environments (P), as shaped by post-pandemic policies (I), across qualitative and mixed-methods studies (D/R)?

The PCC Framework

Stands for: Population · Concept · Context

PCC is used for scoping reviews, a type of SLR designed to map the existing literature on a topic rather than answer a narrow effectiveness question. It is appropriate when your research question is exploratory: you want to know what has been written about a subject, identify key themes, or find gaps before proposing a more targeted study.

Element Question to ask yourself Example
Population Who or what is the subject of study? Family-owned businesses
Concept What is the core idea or issue? Succession planning practices
Context In what setting, geography, or timeframe? European markets, 2010–2025

Resulting question: What does the literature report about succession planning practices (C) in family-owned businesses (P) in European markets between 2010 and 2025 (C)?

Practical Tips

From Question to Search: A Preview

The components of your framework map directly onto the concepts in your Boolean search string. Each element becomes a concept block, and synonyms for each element become OR-connected terms within that block. The blocks are then connected with AND.

Using the PICO example above:

Framework element Concept block
Population: EU retail SMEs "SME*" OR "small firm*" OR "small business*" AND (Europe* OR "European Union")
Intervention: agile project management "agile" OR "scrum" OR "kanban" OR "agile project management"
Outcome: productivity, delivery speed "employee productivity" OR "project delivery" OR "delivery speed"

This mapping is covered in full in Conducting Your Search. The point here is that a well-structured question makes the search string nearly self-evident; a vague question makes it nearly impossible.

Self-Assessment Checklist for Your SLR Search

This checklist, adapted from vom Brocke et al. (2015), helps you evaluate whether your search meets the standards of rigor expected in a systematic review. Use it at three stages: before you search, while searching, and after completing your search.

Before the Literature Search

During the Literature Search

After the Literature Search

Reference: vom Brocke, J. et al. (2015). Standing on the shoulders of giants: Challenges and recommendations of literature search in information systems research. Communications of the Association for Information Systems, 37. doi:10.17705/1CAIS.03709

Writing a Protocol

Overview

A protocol is a written plan that specifies, in advance, exactly how you intend to conduct your systematic literature review. It is not complete until all 11/12 sub-sections below have been worked through and recorded. Writing the protocol is not a bureaucratic hurdle; it is the mechanism that makes your review transparent, reproducible, and defensible to examiners, supervisors, and future readers.

The protocol has two functions. First, it forces you to make every methodological decision before you are influenced by seeing results. Second, it creates a timestamped record of those decisions so that any deviation from the plan is visible and requires justification.

Do not begin searching until your protocol is complete and your supervisor has reviewed it.

What a Protocol Contains

The table below lists all required sections. Each is explained in detail in the sub-sections that follow.

Section What it records
Title Working title of the review
Background Brief rationale: why this topic, why an SLR, why now
Research question Your structured question from the previous step, using PICO, SPIDER, or PCC
Eligibility criteria Explicit inclusion and exclusion rules
Search strategy Databases, search strings, supplementary methods
Screening process Phases, tools, screeners, conflict resolution
Quality appraisal Tool selected and how scores will affect inclusion
Data extraction Fields to be collected and who will extract
Synthesis method Narrative, thematic, or meta-analytic approach
Timeline Planned dates for each stage
Registration PROSPERO or OSF registration number, or a statement of why registration was not pursued
Protocol Amendments Any changes made to the protocol after searching begins: the date of each change, the section affected, the original wording, and the reason for the change. This section is blank at submission and completed during the review process. All amendments must be disclosed in the thesis methods chapter.

For a thesis-level SLR, the protocol will typically run two to four pages. A downloadable template covering all sections is available here.

Why a Protocol Matters

The core risk in any literature review is unconscious bias: the tendency to favor studies that confirm what you already expect to find. A pre-registered protocol addresses this directly by committing you to your methods before you have seen the results. Specifically, a protocol:

Eligibility Criteria

Eligibility criteria are the explicit rules that determine which studies are included in or excluded from your review. They are derived directly from your research question: each element of your PICO, SPIDER, or PCC framework suggests at least one criterion.

Criteria are divided into two types:

Common Criterion Categories

Category Example inclusion criterion Example exclusion criterion
Publication date Published between January 2015 and December 2025 Published before 2015
Language Written in English or German Written in any other language
Document type Peer-reviewed journal articles and conference papers Editorials, opinion pieces, book reviews, dissertations
Study design Empirical studies (qualitative, quantitative, or mixed-methods) Purely conceptual or theoretical papers
Geographic scope Studies conducted in EU member states Studies conducted outside Europe
Population/context Studies focused on SMEs Studies focused exclusively on large corporations
Relevance Studies directly addressing the phenomenon of interest Studies mentioning the topic only incidentally

Why Criteria Must Be Pre-Specified

Criteria written after you have seen the results of your search are retrospective and therefore biased. If you find yourself wanting to exclude a specific study because it complicates your synthesis, that is a signal to engage with it more carefully, not to rewrite a criterion. Any change to criteria after searching begins is a protocol amendment and must be documented.

Testing Your Criteria

Before finalizing your criteria, test them against five to ten records from a preliminary search: a mix of obviously relevant, obviously irrelevant, and borderline papers. If you cannot apply the criteria consistently to this small sample, they need further specification before you proceed to full screening.

Search Strategy

Your search strategy records exactly how you will find the literature. It has three components.

Databases: List every database you will search. For business and management research, the standard set is Business Source Ultimate (EBSCO) and JSTOR. Google Scholar may be used supplementarily for grey literature. The rationale for including each database should be noted briefly (coverage of the discipline, access to specific journal types, etc.).

Search strings: Document the complete Boolean search string you will use in each database. Strings are built from the concepts in your research question framework, with synonyms connected by OR and concepts connected by AND. If strings vary between databases due to different controlled vocabularies, record each variation. Full guidance on constructing strings is provided in Conducting Your Search.

Supplementary methods: Document any additional search methods beyond database searching. The most important of these is snowballing, a technique in which you trace citations forward and backward from a confirmed set of relevant papers. Backward snowballing examines the reference lists of included studies to find earlier relevant work; forward snowballing uses citation databases (Google Scholar, Scopus, Web of Science) to find later papers that have cited an included study. Snowballing is particularly valuable in management research, where relevant work may be published in practitioner journals or conference proceedings not fully indexed in major databases.

Phase 1. Extraction of relevant literature.png
"Phase 1 of the Bandara et al. SLR process: extraction of relevant literature" by Hasan Koç is licensed under CC BY-NC-SA 4.0, based on Bandara et al., 2015.

Screening Process

The screening process section of your protocol records how you will apply your eligibility criteria to the records returned by your search. It should specify:

Full guidance on conducting screening is provided in Screening the Results.

Quality Appraisal Approach

The quality appraisal section of your protocol specifies how you will assess the methodological rigor of included studies. Record:

This decision must be made before appraising any study. Deciding post-hoc to exclude low-quality studies after seeing their findings is a form of bias. Full guidance on quality appraisal is provided in Appraise Study Quality.

You will also be asked, at the end of your review, to evaluate the rigor of your own process using the self-assessment rubric on Evaluating Your Own SLR Process.

Data Extraction and Synthesis Method

Data extraction: Specify the fields you will extract from each included study and the format of your extraction form (typically a spreadsheet). At minimum, record: author, year, country, methodology, sample, key findings, theoretical framework, limitations, and quality appraisal rating. A blank copy of your extraction form should be included as a thesis appendix. Full guidance is provided in Extract Data.

Synthesis method: State whether you will use narrative synthesis, thematic synthesis, or meta-analysis, and briefly justify the choice in relation to your research question and expected study types. Even a one-sentence commitment ("findings will be synthesised narratively using thematic grouping") is sufficient at protocol stage. Full guidance on synthesis approaches is provided in Synthesise and Report.

Timeline

Provide estimated completion dates for each stage. A systematic review takes significantly longer than most students anticipate; building in buffer time is essential.

Stage Planned completion date
Protocol finalised and supervisor-approved  
Database searches completed  
Title/abstract screening completed  
Full-text screening completed  
Quality appraisal completed  
Data extraction completed  
Synthesis and write-up completed  

Protocol Registration (Optional)

Registering your protocol with an external repository creates a timestamped, publicly accessible record of your planned methods. This is optional for most student theses but is increasingly expected in academic publishing and demonstrates a high standard of rigor.

PROSPERO

PROSPERO (International Prospective Register of Systematic Reviews), hosted by the University of York, accepts reviews from health, social science, education, welfare, and business contexts. Registration is free and requires an ORCID iD. Note that PROSPERO does not accept scoping reviews; use OSF for those.

OSF (Open Science Framework)

The OSF, maintained by the Center for Open Science, accepts protocol registrations for any discipline with no topic restrictions. It is the more flexible option for management, design, or interdisciplinary business research, and accepts scoping reviews.

When Registration Is Not Required

Registration is not a formal requirement for most taught or research master's theses. If you do not register, state this explicitly in your methods chapter and give a brief reason (for example, the review is a thesis component rather than a standalone publication). Do not simply omit the topic.

Amendments to the Protocol

Any change made to the protocol after searching begins must be recorded as a formal amendment. For each amendment, document:

Amendments are not a sign of failure; they are a sign of transparency. What is not acceptable is changing methods silently to accommodate inconvenient results. Include the amendments log as an appendix in your thesis.

Common Mistakes to Avoid

Conducting Your Search

Overview

This step translates the search strategy you documented in your protocol into actual database queries, records the results systematically, and prepares a clean, deduplicated set of references for screening. Precision and documentation at this stage are critical: every decision you make must be recorded so that your search can be reported transparently in your final thesis.

Before You Begin

Confirm the following are in place before opening any database:

When your search is complete, use the Search Quality Self-Assessment Checklist to verify the search meets the standards of a rigorous systematic review. The checklist covers what to confirm before, during, and after searching.

Set Up Your Search Logbook

A search logbook is a running record of every search action you take. It is distinct from your protocol: the protocol records what you planned to do; the logbook records what you actually did. Both are required for a transparent, reportable review.

Your logbook should record, for every search:

Field Example
Database Business Source Ultimate (EBSCO)
Date of search 2026-02-23
Search string used ("sustainability reporting" OR "CSR disclosure") AND ("SME*" OR "small firm*") AND (Europe*)
Filters applied Peer-reviewed; 2015–2025; English
Number of results 347
Notes Reran with "non-financial reporting" added; 412 results

A simple spreadsheet works well for this purpose. The logbook feeds directly into the PRISMA flow diagram you will produce during write-up, so keep it current throughout.

Search Each Database

Execute your search strings in the order listed in your protocol. The recommended databases for business and management research at this institution are listed below, with notes on their particular strengths.

Database Strengths Access
Business Source Ultimate (EBSCO) Largest business-specific database; covers management, finance, marketing, HRM, economics; includes trade publications alongside peer-reviewed journals Via Research Database Index
JSTOR Strong for older foundational literature (pre-2010); humanities and social sciences including business history Via Research Database Index
Google Scholar Useful for supplementary grey literature and thesis searches; not suitable as a primary database due to lack of advanced filtering and inconsistent coverage Free; use supplementary only

Search each database independently. Do not rely on a single database regardless of how many results it returns; coverage varies significantly between databases, and a study that appears in JSTOR may not be indexed in Business Source Ultimate, and vice versa.

Constructing Effective Search Strings

If your strings from the protocol stage need refinement when you arrive at a database interface, follow these principles.

Boolean Operators

Three operators control how search terms are combined:

Phrase Searching

Enclose multi-word concepts in quotation marks to search for the exact phrase rather than the individual words. Example: "knowledge management" rather than knowledge management.

Truncation and Wildcards

Most databases support truncation with an asterisk (*) to capture variant word endings:

Check each database's documentation, as wildcard characters vary: EBSCO uses * and ?; Web of Science uses *, ?, and $.

Controlled Vocabulary

Many databases use a subject thesaurus to index articles with standardised terms regardless of the words an author used. Using these terms improves recall significantly:

Combining controlled vocabulary terms with free-text keywords in the same search string gives the best coverage. Example: (DE "employee engagement") OR ("employee engagement" OR "work engagement" OR "job involvement")

For more information on constructing search queries, review Advanced Search Techniques and Making the most of Generative AI.

An Example

Research question: How do sustainability reporting practices in European SMEs influence investor decision-making?

Concept Synonyms and variants
Sustainability reporting "sustainability reporting", "CSR disclosure", "non-financial reporting", "ESG reporting", "integrated reporting"
European SMEs "SME*", "small firm*", "small business*", Europe*, "European Union"
Investor decision-making "investor behavior", "investment decision*", "capital allocation", "shareholder*"

Combined string:

("sustainability reporting" OR "CSR disclosure" OR "non-financial reporting" OR "ESG reporting")
AND
("SME*" OR "small firm*" OR "small business*")
AND
(Europe* OR "European Union")
AND
("investor behavior" OR "investment decision*" OR "capital allocation")

Testing and Iterating Your Search

Before committing to a final string, run test searches to calibrate your results.

A useful calibration technique is to take three to five papers you already know are relevant to your topic and verify that your search string retrieves them. If a known-relevant paper is not found, revise the string before proceeding.

Example: Iterating a Search

Initial search: "government branding" AND communication
Database: Web of Science
Results: 1,856 hits (too many to screen)

Refinement 1: Added language and document type filters (English, journal articles only)
Results: 1,626 hits

Refinement 2: Added date range filter (2000–2021)
Results: 1,553 hits (manageable for screening)

Documented in logbook: Three iterations recorded with reasons for each refinement.

This iterative approach is normal and expected; document each iteration in your search logbook rather than only recording the final string.

Export and Deduplicate References

Once you are satisfied with your search strings and have run them across all databases, export all results and combine them into a single reference set.

Exporting from Databases

Export records in RIS format (also called .ris or "citation export"), which is compatible with Zotero and all major screening tools. Export the full record including abstract, author, year, journal, and DOI. Do not export title-only records.

Importing into Zotero

  1. In Zotero, create a new collection named for your review (e.g., "SLR — Sustainability Reporting SMEs")
  2. Import each database export file: File → Import → select your .ris file
  3. Repeat for each database export
  4. All records will now appear together in the collection

Deduplication

The same article will often appear in multiple databases. Duplicates must be removed before screening begins, as screening the same paper twice distorts your counts and PRISMA numbers.

Record the total number of records before and after deduplication in your search logbook. Both figures are required for the PRISMA flow diagram.

Snowballing

Database searching alone may miss relevant studies published in venues not fully indexed or using terminology that differs from your search string. Snowballing addresses this by tracing citations forward and backward from a confirmed set of relevant papers.

SLR Process by (Wohlin 2014).png
 "Snowball-centred SLR process" by Hasan Koç is licensed under CC BY-NC-SA 4.0, based on Wohlin (2014).

Backward Snowballing

Review the reference lists of your included studies to identify earlier work that was not retrieved by your database search. This is particularly valuable for foundational studies that established key concepts in your topic area.

Forward Snowballing

Use citation databases (Google Scholar, Web of Science, Scopus if available) to identify later papers that have cited an included study. This captures recent work that builds on established findings.

Illustration

SnowballPaper.png
"Iterative backward and forward snowballing across six iterations" by Hasan Koç is licensed under CC BY-NC-SA 4.0, based on Wohlin (2014).

 

When to Apply Snowballing

Snowballing is not a replacement for systematic database searching; it is a supplement applied after your initial screening phase when you have a confirmed set of relevant studies. Record all snowballing activity in your search logbook: the source paper, the direction (forward or backward), and the number of additional records identified.

For an indepth discussion of snowballing, see:

Checking Grey Literature

Depending on your topic, relevant evidence may exist outside peer-reviewed journals. Grey literature includes reports from industry bodies, government agencies, NGOs, think tanks, and working papers. For business research, relevant sources include:

Grey literature is generally searched manually rather than by Boolean string. Record any sources checked and the date of search in your logbook, even if they yield no results.

Common Mistakes to Avoid

Screening the Results

Overview

Screening is the process of applying your pre-specified inclusion and exclusion criteria to the deduplicated set of references produced earlier, in order to identify the studies that will form the basis of your review. It proceeds in two sequential phases: first by title and abstract, then by full text. Each phase reduces the total set further; only studies that pass both phases are included in your final review.

Screening is the step most vulnerable to unconscious bias. The discipline of applying your criteria consistently, rather than on a case-by-case intuitive basis, is what separates a systematic review from an informal one.

Prepare for Screening

Before beginning, confirm the following:

Calibration

Before screening the full dataset, test your criteria against a small sample of twenty to thirty records drawn randomly from your reference set. Apply the criteria independently, then compare decisions. This exercise helps surface edge cases and sharpen your application of the criteria.

Phase 1: Title and Abstract Screening

In the first phase, you review the title and abstract of every record in your deduplicated set and make a binary decision: include (proceed to full-text screening) or exclude (remove from the set, with reason recorded).

Decision Rules

Recording Exclusions

For every excluded record, note the primary reason for exclusion using the categories from your inclusion/exclusion criteria. Example reasons:

These reasons feed directly into the PRISMA flow diagram. You do not need to record a reason for every individual exclusion; recording the category count is sufficient (e.g., "247 excluded: wrong topic; 43 excluded: outside date range").

Phase 2: Full-Text Screening

Records that pass Phase 1 are retrieved in full and assessed against the complete set of inclusion and exclusion criteria. Full-text screening is more demanding than title/abstract screening because you are working with the entire paper and must make a definitive inclusion decision.

Retrieving Full Texts

Assessing Full Texts

Read at minimum the abstract, introduction, methods section, and conclusion of each paper. You do not need to read every paper cover to cover at this stage; the goal is to confirm eligibility, not to extract data. Focus on:

Record a clear reason for every full-text exclusion. At this phase, vague reasons such as "not relevant" are insufficient; specify which criterion was not met.

Screening Tools

Rayyan (Recommended for most students)

Rayyan (rayyan.ai) is a free, web-based screening tool designed specifically for systematic reviews. Key features:

To get started: create a free account at rayyan.ai, create a new review, and upload your .ris export files. Rayyan deduplicates on import.

Spreadsheet (Fallback Option)

An Excel or LibreOffice Calc spreadsheet with one row per reference, columns for title, abstract, Phase 1 decision, Phase 2 decision, and exclusion reason is a fully acceptable approach for smaller datasets (under 500 records). It requires more manual discipline but has no access barriers.

Produce the PRISMA Flow Diagram

At the conclusion of screening, compile the following counts from your logbook and screening tool:

  1. Total records identified across all databases
  2. Total records after deduplication
  3. Records excluded at title/abstract screening (with reason categories)
  4. Full texts sought
  5. Full texts not retrievable
  6. Full texts excluded (with reason categories)
  7. Studies included in the final review

These numbers populate the PRISMA 2020 flow diagram, a standardised visual representation of the screening process. A pre-formatted Word version of the PRISMA 2020 flow diagram is available here. Complete it as you go; do not attempt to reconstruct the numbers from memory at write-up stage.

Common Mistakes to Avoid

Appraise Study Quality

Overview

Quality appraisal is the systematic assessment of the methodological rigor of each study included after screening. It answers the question: how much confidence can we place in the findings of this study? Appraisal does not judge whether a study is interesting or relevant (screening already established relevance); it judges whether the study was conducted in a way that makes its findings trustworthy.

Quality appraisal is mandatory in a rigorous SLR. Omitting it means you treat a poorly designed survey and a well-designed longitudinal study as equally credible evidence, which undermines the validity of your synthesis.

What Quality Appraisal Assesses

Appraisal criteria vary by study type, but the core questions are consistent across tools:

No study is perfect. The goal of appraisal is not to exclude everything with weaknesses but to provide an honest account of the evidence base and to weight your synthesis accordingly.

Decide Before You Appraise

Two decisions must be made in your protocol and applied consistently here:

Will quality scores affect inclusion?

You have two options:

Either approach is defensible; what is not defensible is deciding after seeing the scores.

Who will appraise?

As with screening, appraisal by two independent reviewers with a conflict resolution process is the gold standard. For a thesis-level review, solo appraisal is acceptable but should be stated as a limitation in your methods chapter.

Selecting an Appraisal Tool

Choose your tool based on the study types in your included set. All three tools listed below are freely available with no registration required.

CASP Checklists

Access: casp-uk.net (direct PDF download, no registration)

The Critical Appraisal Skills Programme checklists are the most accessible entry point for students new to quality appraisal. Each checklist is short (ten to twelve questions with yes/no/can't tell responses) and includes guidance notes. Separate checklists exist for:

For most business and management SLRs, the qualitative checklist will be the primary tool. If your included studies are methodologically mixed, you will need to apply different checklists to different study types and note which checklist was used for each study in your data extraction form.

Mixed Methods Appraisal Tool (MMAT)

Access: mcgill.ca (free PDF download from McGill University)

The MMAT is the strongest choice when your included studies span multiple methodological types, since a single tool handles qualitative, quantitative descriptive, quantitative randomised, quantitative non-randomised, and mixed-methods studies consistently. Each category has five criteria, allowing cross-study comparison of quality scores within a heterogeneous dataset.

The MMAT does not produce a numerical score; instead, each criterion is rated yes, no, or can't tell. This is intentional: the authors explicitly caution against summing scores into a single quality number, as this can create false precision.

JBI Critical Appraisal Tools

Access: jbi.global (free PDF download, no registration)

The Joanna Briggs Institute tools are comparable in accessibility to CASP and provide thirteen separate checklists covering a wider range of study types, including prevalence studies, case reports, case series, and qualitative evidence synthesis. They are slightly more detailed than CASP and include more extensive guidance notes.

Conducting the Appraisal

Work through each included study using your chosen tool. For each study, complete the checklist and record:

A suggested format for recording appraisal results is a spreadsheet with one row per study and one column per checklist criterion, plus an overall rating column. This makes it easy to sort by quality rating and to identify patterns (for example, if most studies share a common weakness such as lack of reflexivity, this becomes a theme in your discussion).

A Practical Tip

Read the methods section of each paper carefully before completing the checklist. Authors do not always report methods in detail in the abstract or even the results section; insufficient reporting is itself a quality concern, but it is worth distinguishing between a study that did not address a criterion and one that did but failed to report it.

Reporting Quality Appraisal in Your Thesis

Quality appraisal results must be reported transparently in your methods chapter and referenced in your discussion. Standard practice is to:

  1. Name the tool(s) used and cite the source
  2. Present results in a summary table, with one row per included study and columns for each criterion or an overall rating
  3. Describe the overall quality of the evidence base in narrative: were most studies of moderate quality? Were there systematic weaknesses across studies (e.g., small sample sizes, single-country contexts)?
  4. Reference quality in your discussion: when interpreting conflicting findings, note whether higher-quality studies favor one conclusion over another

Avoid the common error of completing a quality appraisal table and then never mentioning it again. The appraisal should inform how confidently you present your conclusions.

Common Mistakes to Avoid

Extract Data

Overview

Data extraction is the process of systematically pulling the information you need from each included study and recording it in a standardised form. It bridges the gap between your screened, appraised set of studies and the synthesis you will conduct later. Consistent, thorough extraction is what makes synthesis possible: if you extract different information from different papers, you cannot meaningfully compare or combine them.

Extraction is not the same as reading for interest. You are not summarizing papers freely; you are completing a pre-designed form that captures the same fields from every study in the same way.

Design Your Extraction Form

Your extraction form should have been designed as part of your protocol. Review it now against your actual included studies and refine if necessary. Any changes at this stage count as a protocol amendment and should be documented.

Core Fields

Every extraction form for a business or management SLR should include the following fields as a minimum:

Field What to record
Study ID A unique reference number you assign (e.g., S01, S02) for use in tables and in-text citation during synthesis
Author(s) Last name and initials of all authors
Year Year of publication
Title Full title of the article
Journal/Source Journal name, conference, or report series
Country/Region Country where the study was conducted or the data originate
Study design/method Qualitative, quantitative, mixed-methods; specify further (e.g., semi-structured interviews, survey, case study)
Sample Size, type, and characteristics of the sample or dataset
Data collection period When data were collected (may differ from publication year)
Key findings A concise, accurate summary of findings relevant to your research question; use the authors' own language where possible
Theoretical framework Any theory the study draws on (relevant for deductive synthesis)
Limitations noted by authors As reported in the paper
Quality appraisal rating Transfer the overall rating from earlier appraisal
Notes Any observations relevant to synthesis (e.g., contradicts S04; uses unusual operationalisation)

Additional Fields by Research Type

Depending on your topic, you may also need:

Format of the Extraction Form

A spreadsheet (Excel or LibreOffice Calc) with one row per study and one column per field is the standard format and works well for most thesis-level reviews.

Advantages:

A blank version of the extraction form should be included as an appendix in your final thesis.

Conducting the Extraction

Work through each included study in order of your Study ID. For each paper:

  1. Read the full text carefully, focusing on the abstract, introduction, methods, results, and discussion sections
  2. Complete every field in the extraction form; leave no field blank (use "not reported" where the paper does not provide the information rather than leaving the cell empty)
  3. Record findings in your own words, except for key definitions or theoretical statements where the authors' precise language matters; note any direct quotations with page numbers
  4. Note any information that is ambiguous, inconsistent between sections of the paper, or that raises a question for synthesis

Handling Ambiguity

You will encounter papers where the methodology is not clearly described, findings are presented inconsistently, or the research question shifts between the introduction and the discussion. Record what is actually in the paper, note the ambiguity explicitly, and do not interpret charitably to fill gaps. Gaps in reporting are themselves evidence of methodological weakness and belong in your quality appraisal record.

Pilot Extraction

Before extracting all included studies, conduct a pilot on three to five papers. It tests whether your form captures the information you actually need for synthesis

After the pilot, review the form: are any fields consistently empty or impossible to complete? Are any fields producing inconsistent entries between reviewers? Revise the form before proceeding, and document any changes as a protocol amendment.

Maintaining an Audit Trail

Keep a running note of any decisions you make during extraction that go beyond straightforward form completion. Examples:

These notes protect you if your decisions are questioned during examination, and they support transparency if your review is ever published.

Preparing for Synthesis

Before moving to next step, review your completed extraction form as a whole:

A brief written memo at this stage, even half a page of notes, is a valuable precursor to synthesis. It helps you enter the next step with a sense of the landscape of the evidence rather than facing a blank page.

Common Mistakes to Avoid

Synthesize and Report

Overview

Synthesis is where the work of the review becomes an argument. Having identified, screened, appraised, and extracted data from your included studies, you now interpret what they collectively say in response to your research question. Reporting then translates that interpretation into a structured written account that meets the standards of academic transparency required for a thesis.

These two activities, synthesis and reporting, are treated together here because they are iterative: the structure of your synthesis shapes the structure of your report, and drafting the report often reveals gaps in the synthesis that require you to return to your notes.

Choose Your Synthesis Approach

Your synthesis method was specified in your protocol. The two principal options for business and management SLRs are narrative synthesis and meta-analysis. A third option, thematic synthesis, is increasingly common and sits between the two.

Narrative Synthesis

Narrative synthesis organises findings from included studies into themes or categories and describes patterns, relationships, contradictions, and gaps in discursive prose. It is appropriate when:

This is the most common synthesis approach in business and management research and is suitable for the majority of thesis-level SLRs.

Thematic Synthesis

Thematic synthesis, developed by Thomas and Harden (2008), applies a more structured coding procedure to the findings of included studies before organising them into themes. It is particularly well-suited to reviews of qualitative studies and connects directly to the deductive, inductive, and combined coding approaches described in the Bandara et al. (2015) framework. The process involves three stages:

  1. Line-by-line coding of the findings and conclusions sections of each included study
  2. Developing descriptive themes by grouping related codes
  3. Generating analytical themes that go beyond description to interpret what the evidence means in relation to your research question

Meta-Analysis

Meta-analysis pools numerical results from multiple quantitative studies using statistical methods to produce an overall effect size estimate. It is only appropriate when:

Meta-analysis is rarely appropriate at thesis level in business and management research; if your supervisor has suggested it, seek guidance early on statistical software (R, Stata, or JASP) and reporting requirements.

Conducting Narrative or Thematic Synthesis

The following steps apply to both narrative and thematic synthesis.

Step 1: Familiarize Yourself with the Evidence Base

Before coding, read all your extraction notes as a whole. Review the memo you wrote at the end of Extract Data. Note the overall shape of the evidence: how many studies, what methods, what contexts, what time period.

Step 2: Develop a Coding Framework

Decide whether you will code deductively, inductively, or using a combined approach.

Deductive, Inductive, and Combined Coding

Phase 2. Organization and preparation for analysis.png
"Phase 2 of the Bandara et al. SLR process: organisation and preparation for analysis, including coding approach selection" by Hasan Koç is licensed under CC BY-NC-SA 4.0, based on Bandara et al., 2015.

The Bandara et al. (2015) framework, widely used in information systems and management SLRs, defines three approaches to coding:

Deductive coding: You bring a pre-existing theoretical framework or model to the data and apply its categories to the findings of included studies. This approach is appropriate when your research question asks how a specific theory has been applied across contexts, or when you are testing whether empirical evidence supports a theoretical proposition.

Inductive coding: Codes emerge from the data without a predetermined structure. You read findings across studies and assign descriptive labels that capture what is being said, allowing themes to develop organically. This approach is appropriate for exploratory research questions where the conceptual landscape is not yet well defined.

Combined approach: Begin with a small set of deductive codes derived from your research question or theoretical framework, then allow additional codes to emerge inductively as you encounter concepts not anticipated by the initial framework. This is the most flexible approach and is common in thesis-level SLRs where the scope is narrower than a full mapping review but broader than a theory-testing study.

For most business thesis SLRs, a combined approach is practical and defensible. Document your starting framework explicitly in your protocol and note any inductively derived codes as you develop them.

Step 3: Code the Findings

Work through your extraction form, reading the key findings field for each study and assigning one or more codes. Use a simple coding log: a spreadsheet or table with Study ID, finding, and code assigned. Keep codes concise (two to five words) and descriptive at this stage.

Step 4: Develop Themes

Group related codes into broader themes. A theme should:

Aim for three to six themes for a typical thesis-level review. Fewer than three suggests over-aggregation; more than six suggests insufficient grouping.

Step 5: Interpret and Analyse

For each theme, write an analytical account that:

This is the intellectual contribution of your review. Do not simply list what each study found; explain what the body of evidence means.

Reporting Your Review

Your written report should follow the PRISMA 2020 reporting guidelines, which specify what information must be included and where. The standard structure for an SLR thesis chapter or standalone review paper maps onto the following sections.

Introduction

Methods

The methods section must be detailed enough for the review to be replicated. Include:

Results

Present results in three parts:

  1. PRISMA flow diagram: a visual account of records identified, screened, excluded at each phase, and finally included. The pre-formatted PRISMA 2020 Word template is available here.
  2. Characteristics of included studies: a summary table (one row per study) covering author, year, country, method, sample, and quality rating. This table belongs in the results section, not the appendix.
  3. Synthesis findings: your thematic or narrative synthesis, organised by theme, with in-text citations to included studies using your Study ID codes (e.g., S01, S07, S12).

Discussion

Conclusion

A brief section (one to two paragraphs) stating the main answer to your research question and its implications for research or practice. Do not introduce new evidence here.

The PRISMA Flow Diagram

The PRISMA 2020 flow diagram is a mandatory element of any SLR report. It visually documents the flow of records through the review process and allows readers to evaluate the thoroughness of your search and the basis for your inclusions.

The four stages represented in the diagram are:

  1. Identification: total records retrieved from each database, plus any records from supplementary sources (grey literature, snowballing, hand-searching)
  2. Screening: records after deduplication; records excluded at title/abstract screening with reason counts
  3. Eligibility: full texts assessed; full texts excluded with reason counts; full texts not retrievable
  4. Included: final number of studies included in the review

A pre-formatted Word version of the PRISMA 2020 flow diagram is available here. Complete the numbers from your search logbook and screening tool records; do not estimate.

Presenting Included Studies

In-Text Citation Convention

During synthesis, refer to included studies by their Study ID (e.g., S01) rather than by author and year, to distinguish them visually from other literature cited in the discussion. Provide a complete reference list of included studies as a separate appendix, clearly labelled "Included Studies," so that examiners can locate them independently of your general reference list.

The Characteristics Table

The summary table of included study characteristics is one of the most-read elements of an SLR report. Present it clearly and completely. At minimum, include: Study ID, author(s), year, country, methodology, sample, and your quality rating. If space allows, add a brief "key finding" column (one sentence per study).

Common Mistakes to Avoid

Evaluating Your Own SLR Process

This section is distinct from Appraise Study Quality, which assesses the rigor of the primary studies you have included. This section asks a different question: how rigorously did you conduct the review itself?

Overview

Quality appraisal tools such as CASP, MMAT, and JBI look outward: they help you evaluate the studies in your dataset. The checklist and scoring rubric on this page look inward: they help you evaluate your own review process against recognized best practices.

The items below are adapted from Petersen, Vakkalanka, and Kuzniarz (2015), who derived them from a systematic mapping study of how SLRs and systematic mapping studies are conducted in practice. Use this rubric in two ways:

This rubric was developed in the context of software engineering research. The core dimensions; motivating the review, search strategy, search evaluation, extraction/classification, and validity; apply equally to business and management SLRs. Items that refer to software-engineering-specific classification schemes may be skipped if they are not relevant to your discipline.

Part 1: Activities Checklist

The table below lists the 26 actions identified by Petersen et al. (2015) as relevant to a rigorous systematic review or mapping study. Work through each row and mark whether the action was taken (✓), partially taken (~), or not taken (✗). This produces a ratio score: count your ✓ marks and divide by 26 (or by the number of applicable items).

Phase Action Taken?
Motivate the review Motivate the need and relevance of the review  
  Define objectives and research questions  
  Consult with the target audience (e.g., supervisor, domain expert) to refine questions  
Search strategy Conduct a database search  
  Apply snowball sampling (backward and/or forward)  
  Conduct a manual search of key journals or conference proceedings  
Develop the search Use a structured framework (PICO, SPIDER, or PCC) to derive keywords  
  Consult a librarian or domain expert during search design  
  Iteratively refine the search string to improve coverage  
  Derive additional keywords from known relevant papers  
  Use thesauri, encyclopedias, or controlled vocabularies (e.g., MeSH, EBSCO subject headings)  
Evaluate the search Test the search against a set of known-relevant papers  
  Have an expert evaluate the search results  
  Check the web pages or profiles of key authors in the field  
  Conduct a test–retest to check consistency  
Inclusion and exclusion Define objective, pre-specified criteria for inclusion and exclusion  
  Involve a second reviewer; resolve disagreements systematically  
  Define and apply explicit decision rules for borderline cases  
Data extraction Define objective criteria for the extraction process  
  Blind or obscure information that could bias extraction  
  Involve a second reviewer; resolve disagreements in extraction  
  Conduct test–retest of extraction on a subset  
Classification Classify studies by research type (e.g., empirical, conceptual, review)  
  Classify studies by research method (e.g., case study, survey, experiment)  
  Classify studies by venue type (e.g., journal, conference, practitioner publication)  
Validity Discuss validity threats and limitations of the review process  

Part 2: Scoring Rubrics

After completing the checklist, use the rubrics below to assign a score to each of the five key dimensions. Record these scores in your methods chapter alongside a brief narrative.

Rubric 1: Motivating the Review

Score Label Description
0 Not described The review is not motivated and no objectives are stated
1 Partial Motivations and research questions are provided
2 Full Motivations and questions are provided and have been developed in dialogue with the target audience (supervisor, practitioners, or domain experts)

Rubric 2: Search Strategy

Score Label Description
0 Not described Only one type of search was conducted
1 Minimal Two search strategies were used
2 Full All three strategies were used: database search, snowball sampling, and manual search

Rubric 3: Evaluating the Search

Score Label Description
0 Not described No actions were taken to improve the reliability of the search or inclusion/exclusion process
1 Minimal At least one action was taken to improve either the reliability of the search or the inclusion/exclusion process
2 Partial At least one action was taken to improve both the search and the inclusion/exclusion process
3 Full All identified actions were taken

Rubric 4: Extraction and Classification

Score Label Description
0 Not described No actions were taken to improve extraction reliability or enable comparability between studies
1 Minimal At least one action was taken to increase extraction reliability
2 Partial At least one action to increase extraction reliability and studies were classified by research type and method
3 Full All identified actions were taken

Rubric 5: Study Validity

Score Label Description
0 Not described No threats or limitations are described
1 Full Threats and limitations of the review process are described

Interpreting Your Scores

No minimum threshold is formally established in the literature for general SLRs; the rubric is a diagnostic tool, not a pass/fail gate. Use the results as follows:

For more detail on designing and evaluating your search strategy, see the Search Quality Self-Assessment Checklist (adapted from vom Brocke et al., 2015), which provides granular guidance on the search phase specifically.