Evidence synthesis decision workshop / 5 MIN READ
Define which studies belong in each synthesis comparison
Explicit study membership decisions for each planned synthesis.
AI-assisted research and analysis for the SNS Creative Fieldbook
Sources checked 2026-11-07
Eligibility for a review does not make every result suitable for every comparison.
What you will leave with: Explicit study membership decisions for each planned synthesis.
Use this when
- A review includes interventions with different active ingredients or comparators.
- A proposed pooled result combines different outcome windows.
Not a general evidence matrix, a method for making incompatible outcomes equivalent or automatic approval to calculate a pooled estimate.
State the comparison before assigning studies
Cochrane Chapter 9, checked 7 November 2026, uses study-characteristic tabulation to decide which studies are sufficiently similar for each comparison. This lesson turns that principle into explicit membership decisions. It does not supply a universal similarity score or claim that sharing a topic makes results combinable.
Write the proposed comparison in operational terms: which activity is being contrasted with what alternative, for whom, using which outcome and observation window. Identify the decision that the result would inform. A broad review question can support several narrower comparisons. Do not start with the studies that happen to have convenient numbers and invent a common question afterward.
Create a separate destination for contextual evidence that informs the review but does not answer this comparison. That prevents an excluded synthesis contribution from disappearing from the review altogether. The distinction is useful when a study explains implementation problems yet supplies no comparable outcome for the proposed effect summary.
Make membership decisions on concrete differences
Use a fictional review of adult business-user instruction as a planning fixture. The proposed comparison is a short tutorial versus static instructions, with immediate task completion as the outcome. A tutorial-versus-static study belongs in that comparison if its other relevant conditions fit. A tutorial-plus-live-coaching study answers a different intervention question and should not enter automatically.
A second study may test the right instructional contrast but report only satisfaction. A third may measure task completion after several months of use. Give each an explicit disposition: contributes to the planned comparison, contributes to another defined comparison, remains contextual, or awaits clarification. Write the reason beside the decision. Do not use a blank cell to hide whether the study was forgotten or deliberately kept separate.
Preserve consequential distinctions during simplification
Review the proposed groups with a subject expert and an analyst before looking for a preferred summary. Ask which differences could change the meaning of the business decision. A comparison against no support differs from a comparison against an established help system. Calling both usual practice may erase the very alternative the organization is deciding whether to replace.
Keep enough study description to justify each membership choice without reproducing entire reports. When several variants can reasonably address a broader question, state why that broader question is useful and what detail the summary will lose. Do not broaden a group merely to increase its size. A small but coherent comparison can be more informative than a larger number that no longer describes an actionable choice.
Freeze the map before the synthesis readout
Have the accountable review team approve the membership map and retain the reasons for borderline dispositions. If new information changes a study’s classification, record the change and assess its consequences consistently across the set. Separate a factual correction from a decision to redefine the comparison. Neither should be hidden by silently editing the final table.
Give the analyst a list of included contributions for each comparison, plus the studies kept elsewhere and why. Check that no report contributes a result under a heading its actual intervention or outcome cannot support. Finish with the questions that remain unanswered rather than forcing a complete-looking grid. The artifact defines what each synthesis is allowed to mean; it does not settle statistical model choice, study quality or whether a pooled result is appropriate.
Make the reasoning concrete.
A fictional research team wants a headline about tutorials improving immediate task completion. Its eligible review contains tutorial-versus-static studies, tutorial-plus-coaching studies and satisfaction-only studies. A pooled headline would hide those differences.
Group by the word tutorial
Opening: Place every eligible study under the same effect heading.
Hypothesis: The evidence set looks larger.
Risk: The resulting question no longer matches the intended decision.
Assign explicit destinations
Opening: Keep the intended comparison separate and record where the other studies contribute.
Hypothesis: Each synthesis heading retains a defensible meaning.
Risk: Some important questions may remain supported by too little comparable evidence.
The team keeps the coaching and satisfaction contributions in separate destinations and leaves gaps visible. It does not discard those studies from the wider review.
Take the questions into your next brief.
Use these prompts in your own notes. Nothing is submitted from this page.
Comparison
What exact activity, alternative and outcome define this synthesis?
Contribution
Which result from each study could answer that question?
Disposition
Is each contribution included, elsewhere, contextual or unresolved?
Boundary
Which difference would change the business meaning?
Revision
What evidence would justify changing membership?
Watch for the wrong conclusion.
- Treating review eligibility as universal synthesis membership.
- Grouping by a shared topic word.
- Broadening the question only to increase the number of studies.
Your next action
Assign three invented study contributions to one explicit comparison and record a reason for every inclusion or alternative destination.
Sources & scope
Workflows and explicitly fictional adult business examples are editorial analysis. Methodological statements and their applications remain source-derived. Cochrane-specific requirements are not universal marketing mandates. No real research operations or publication are performed.
- Cochrane Handbook, Chapter 9: Summarizing study characteristics and preparing for synthesis →Primary technical documentation · Supports: Supports comparison-specific study grouping, not automatic pooling.