Evaluating Your Own SLR Process This section is distinct from  Appraise Study Quality , which assesses the rigor of the  primary studies you have included. This section asks a different question: how rigorously did you conduct the review itself? Overview Quality appraisal tools such as CASP, MMAT, and JBI look outward: they help you evaluate the studies in your dataset. The checklist and scoring rubric on this page look inward : they help you evaluate your own review process against recognized best practices. The items below are adapted from Petersen, Vakkalanka, and Kuzniarz (2015) , who derived them from a systematic mapping study of how SLRs and systematic mapping studies are conducted in practice. Use this rubric in two ways: During planning : as a checklist of actions to build into your protocol. Before submission: as a retrospective audit to identify gaps in your process and to disclose them transparently in your methods chapter. This rubric was developed in the context of software engineering research. The core dimensions; motivating the review, search strategy, search evaluation, extraction/classification, and validity; apply equally to business and management SLRs. Items that refer to software-engineering-specific classification schemes may be skipped if they are not relevant to your discipline. Part 1: Activities Checklist The table below lists the 26 actions identified by Petersen et al. (2015) as relevant to a rigorous systematic review or mapping study. Work through each row and mark whether the action was taken (✓), partially taken (~), or not taken (✗). This produces a ratio score: count your ✓ marks and divide by 26 (or by the number of applicable items). Phase Action Taken? Motivate the review Motivate the need and relevance of the review     Define objectives and research questions     Consult with the target audience (e.g., supervisor, domain expert) to refine questions   Search strategy Conduct a database search     Apply snowball sampling (backward and/or forward)     Conduct a manual search of key journals or conference proceedings   Develop the search Use a structured framework (PICO, SPIDER, or PCC) to derive keywords     Consult a librarian or domain expert during search design     Iteratively refine the search string to improve coverage     Derive additional keywords from known relevant papers     Use thesauri, encyclopedias, or controlled vocabularies (e.g., MeSH, EBSCO subject headings)   Evaluate the search Test the search against a set of known-relevant papers     Have an expert evaluate the search results     Check the web pages or profiles of key authors in the field     Conduct a test–retest to check consistency   Inclusion and exclusion Define objective, pre-specified criteria for inclusion and exclusion     Involve a second reviewer; resolve disagreements systematically     Define and apply explicit decision rules for borderline cases   Data extraction Define objective criteria for the extraction process     Blind or obscure information that could bias extraction     Involve a second reviewer; resolve disagreements in extraction     Conduct test–retest of extraction on a subset   Classification Classify studies by research type (e.g., empirical, conceptual, review)     Classify studies by research method (e.g., case study, survey, experiment)     Classify studies by venue type (e.g., journal, conference, practitioner publication)   Validity Discuss validity threats and limitations of the review process   Part 2: Scoring Rubrics After completing the checklist, use the rubrics below to assign a score to each of the five key dimensions. Record these scores in your methods chapter alongside a brief narrative. Rubric 1: Motivating the Review Score Label Description 0 Not described The review is not motivated and no objectives are stated 1 Partial Motivations and research questions are provided 2 Full Motivations and questions are provided and have been developed in dialogue with the target audience (supervisor, practitioners, or domain experts) Rubric 2: Search Strategy Score Label Description 0 Not described Only one type of search was conducted 1 Minimal Two search strategies were used 2 Full All three strategies were used: database search, snowball sampling, and manual search Rubric 3: Evaluating the Search Score Label Description 0 Not described No actions were taken to improve the reliability of the search or inclusion/exclusion process 1 Minimal At least one action was taken to improve either the reliability of the search or the inclusion/exclusion process 2 Partial At least one action was taken to improve both the search and the inclusion/exclusion process 3 Full All identified actions were taken Rubric 4: Extraction and Classification Score Label Description 0 Not described No actions were taken to improve extraction reliability or enable comparability between studies 1 Minimal At least one action was taken to increase extraction reliability 2 Partial At least one action to increase extraction reliability and studies were classified by research type and method 3 Full All identified actions were taken Rubric 5: Study Validity Score Label Description 0 Not described No threats or limitations are described 1 Full Threats and limitations of the review process are described Interpreting Your Scores No minimum threshold is formally established in the literature for general SLRs; the rubric is a diagnostic tool, not a pass/fail gate. Use the results as follows: In your methods chapter: Report your scores and briefly explain any dimension rated 0 or 1. A low score on a dimension is not automatically a fatal flaw, but it must be acknowledged as a limitation. In your discussion: Dimensions scored 0 (especially search strategy and validity) should be discussed explicitly when qualifying the strength of your conclusions. As a planning aid: If you are still in the protocol stage, any action not yet checked is a concrete item to build into your plan before searching begins. For more detail on designing and evaluating your search strategy, see the Search Quality Self-Assessment Checklist (adapted from vom Brocke et al., 2015), which provides granular guidance on the search phase specifically.