Practical framework / 5 MIN READ
Ask how missing outcomes could change the decision
Show the range implied by unknown results before relying on complete cases.
AI-assisted research and analysis for the SNS Creative Fieldbook
Sources checked 2026-11-06
When some outcomes are missing, the observed group may differ from the intended population. A sensitivity review makes that uncertainty explicit by asking how conclusions change under stated assumptions about the unobserved cases.
What you will leave with: Build a missing-outcome scenario sheet with transparent bounds, assumptions and a decision threshold.
Use this when
- An experiment or customer follow-up loses some outcome records.
- A favorable result is based only on people with complete data.
This is a sensitivity-planning exercise, not a replacement for valid design, sampling uncertainty or specialist missing-data analysis.
Count what is observed and what is unresolved
Define the target population, outcome and maturity window. Separate not-yet-mature outcomes from records that should be available but are missing. Preserve the original eligible or assigned denominator and show missingness by relevant group without assuming equal missingness means no bias.
Investigate the collection process. Missing outcomes may follow failed integrations, customer withdrawal or behavior affected by treatment. Use only permitted recovery methods and respect a person’s choice not to provide information. Do not infer an unknown outcome from an unrelated proxy merely to make the table complete.
Start with transparent outcome bounds
World Bank methodological guidance describes bounding and sensitivity approaches that vary assumptions about unobserved outcomes, noting that bounds can become wide. For a binary outcome, an introductory check assigns every missing case first to failure and then to success while keeping the original denominator.
Label this as a logical range for the observed dataset under those extremes, not a confidence interval. It does not include sampling uncertainty, measurement error or confounding. For continuous outcomes or complex designs, obtain analyst guidance rather than inventing plausible minima and maxima to produce a reassuring result.
Evaluate justified intermediate scenarios
If extreme bounds are too broad, ask what substantive evidence could justify narrower assumptions. Document scenarios in plain language, such as a stated success rate among missing cases, and show how the recommendation changes across them. Do not choose the assumption that preserves the preferred result and call it the estimate.
Compare groups under asymmetric missing-outcome scenarios where appropriate. The unobserved treatment cases need not behave like the unobserved control cases. Have the analyst assess whether imputation, weighting or another method is defensible, including what assumptions cannot be checked from the available data.
Tie the analysis to a decision and repair plan
Identify the assumptions under which the practical conclusion changes. If modest plausible differences overturn the recommendation, call the evidence fragile. If a useful direction survives stringent scenarios, state exactly which scenarios were considered without claiming all uncertainty has disappeared.
Choose a proportionate next step: repair collection, obtain permitted follow-up, limit rollout or remain inconclusive. Preserve the sensitivity sheet beside the headline result. Improving the next study’s outcome capture can be more valuable than adding elaborate adjustments to a dataset whose missingness remains poorly understood.
Make the reasoning concrete.
A hypothetical binary outcome has 100 eligible cases, with 20 successes among 80 observed cases and 20 missing. The observed-case rate is 25%, but the full-population rate lies between 20% and 40% under the two extreme assignments. These bounds do not include sampling uncertainty.
Use complete cases only
Opening: Report 25% as the population rate without qualification.
Hypothesis: The calculation uses known outcomes.
Risk: Missing cases are implicitly assumed to behave suitably for that conclusion.
Show assumptions and bounds
Opening: Report missingness and examine what decisions survive the 20%–40% logical range or justified scenarios.
Hypothesis: The owner can see how much depends on unknown outcomes.
Risk: Wide bounds may leave the decision unresolved.
Keep the full denominator and disclose the unknown outcomes. Ask the analyst to evaluate defensible assumptions and uncertainty before converting a sensitivity result into a population or causal claim.
Take the questions into your next brief.
Use these prompts in your own notes. Nothing is submitted from this page.
Population
Who belongs in the original outcome denominator?
Missingness
Why are outcomes unavailable and are they mature?
Bounds
What extremes follow from the outcome’s actual limits?
Scenarios
Which narrower assumptions have substantive support?
Decision
What changes the recommendation and what collection repair is possible?
Watch for the wrong conclusion.
- Treating complete cases as automatically representative.
- Calling logical bounds confidence intervals.
- Assuming equal attrition rates eliminate bias.
- Choosing missing-outcome assumptions after deciding the desired answer.
Your next action
Build a missing-outcome sensitivity sheet for one important result and identify the assumption at which the decision changes.
Sources & scope
Frameworks, worksheets and hypothetical examples are original editorial synthesis. Linked primary sources support only the stated concepts. Source checking is not an independent audit of effectiveness. No named human author or reviewer is claimed.
- David McKenzie, World Bank: If your follow-up survey has attrition, what should you do for your descriptive analysis? →Primary researcher methodological commentary · Supports: Explains bounding and sensitivity approaches for unobserved outcomes and why bounds can be wide. The hypothetical binary arithmetic is original and does not include sampling uncertainty or establish causal validity.