Marketing evidence review workshop / 6 MIN READ
Investigate the record moving the model before deciding to remove it
An inconvenient observation can be an error, a rare valid case or a sign that the model misses something important.
AI-assisted research and analysis for the SNS Creative Fieldbook
Sources checked 2026-11-07
A marketing regression can depend heavily on a small part of its evidence. Influence diagnostics help locate that dependence; they do not authorize deleting records until the preferred answer appears. Pair the diagnostic with a provenance investigation and an explicit decision about the population the model is meant to represent.
What you will leave with: An influential-observation review ledger with verified provenance, bounded sensitivity and a documented disposition.
Use this when
- A sales or campaign planning regression changes materially when a few observations are omitted.
- An analyst flags unusual records before presenting a recommendation.
This is a review of regression evidence, not a universal diagnostic cutoff or a command to clean production data. Qualified analysts choose methods appropriate to the model, dependence structure and intended use.
Distinguish unusual position from actual influence
Ask the analyst what triggered the flag and which conclusion it could affect. Keep the outcome residual, unusual predictor combination and effect on the fitted model separate. A record can look extraordinary on a business chart without dominating the model; a less visually dramatic record may matter to a particular coefficient or prediction. The review should name the affected conclusion rather than label a customer or campaign bad data.
NIST distinguishes leverage, unusual predictor values, from influence, changes in fitted parameters when an observation is omitted. It explicitly notes that high leverage need not imply high influence. Use those concepts to request an appropriate diagnostic explanation, not to adopt a universal threshold from an unrelated example or to treat every flag as a deletion instruction.
Investigate provenance before choosing a remedy
Trace a flagged record to its allowed source and processing history. Check units, period boundaries, duplicate aggregation, revisions and whether it belongs to the intended population. Ask the business owner whether an unusual event actually occurred. Preserve the original value and the evidence behind any proposed correction. An extreme but genuine launch period should not become a data-entry error merely because the model struggles to describe it.
University of Wisconsin’s SSCC guidance recommends investigating influential observations using subject knowledge and documenting how they are handled. Apply that principle through a short disposition record: verified error, valid in-scope event, out-of-scope observation or unresolved provenance. Keep sensitive customer details out of the shared review when an aggregate event description will answer the question.
Review sensitivity without shopping for a preferred answer
Have the analyst compare the relevant model outputs under justified alternatives, while preserving a clearly identified primary analysis. State in advance what the comparison is meant to diagnose. If one valid event changes the sign or operational recommendation, report that dependence openly. It may call for narrower use, a better model or more evidence, not an automatic declaration that either fitted answer is the truth.
Keep the unit of sensitivity aligned with the data structure. Several rows may belong to one campaign, region or customer history; ask the analyst whether row-by-row checks answer the actual dependency question. Do not repeatedly remove flagged records until the remaining result crosses a desired threshold. A diagnostic exploration must remain distinguishable from the final analysis and cannot erase the work required to justify a causal claim.
Record the disposition and its consequence
For a verified processing error, route an authorized correction through the controlled data workflow and rerun the affected checks. For a valid but unsupported operating regime, state where the model cannot yet guide decisions. For an excluded population, document the scope change rather than presenting the narrower result as universal. If provenance remains unresolved, explain its decision consequence instead of assigning the most convenient category.
Keep the review ledger beside the model version and recommendation. Include what changed, why, who verified the underlying facts and what sensitivity remains. Preserve enough information to reproduce the decision without distributing unnecessary records. The useful outcome is a transparent account of which evidence supports the model and where it is fragile, not a cleaner-looking chart produced by making difficult observations disappear.
Make the reasoning concrete.
A fictional regional retailer fits a relationship between promotion spend and sales. One genuinely recorded reopening campaign combines unusual spend with a changed store footprint and strongly affects the recommendation. The team suggests deleting it because it makes the channel estimate less reassuring.
Remove the difficult campaign
Opening: Exclude the reopening period because the preferred coefficient becomes clearer.
Hypothesis: The presentation may look more stable.
Risk: The omission is chosen for the desired result rather than a defensible target-population decision.
Investigate the operating regime
Opening: Verify the record and ask the analyst whether the current model represents reopening conditions or needs a bounded scope.
Hypothesis: The recommendation can state which business setting its evidence actually supports.
Risk: The available data may not support one pooled recommendation for both ordinary and reopening periods.
The retailer documents the real event and reviews the model’s intended scope. Any justified alternative analysis remains visible; the team does not rewrite the source record or hide the sensitivity.
Take the questions into your next brief.
Use these prompts in your own notes. Nothing is submitted from this page.
Flag
Which diagnostic and business conclusion motivated the review?
Provenance
What evidence establishes the record’s value, units and population membership?
Alternative
Which justified comparison tests the decision’s dependence?
Disposition
Is this a verified error, valid case, scope mismatch or unresolved record?
Consequence
How does the disposition change the model’s permitted recommendation?
Watch for the wrong conclusion.
- Equating every high-leverage point with an erroneous observation.
- Removing records until the preferred result appears.
- Changing population scope without changing the claim.
Your next action
For one influential observation, write the provenance evidence and proposed disposition before viewing the preferred result of excluding it.
Sources & scope
Workflows, worksheets and hypothetical examples are original editorial analysis. Sources support only explicitly attributed statistical principles. Qualified analysts own method selection and validation. No measured business result or named human reviewer is claimed.
- NIST Dataplot: Regression Diagnostics →Statistical reference guidance · Supports: Supports the distinction between leverage and influence. No numeric diagnostic rules or source code are adopted.
- University of Wisconsin SSCC: No Outlier Effects →Statistical reference guidance · Supports: Supports investigating observations with subject knowledge and documenting their treatment. The business review ledger is original analysis.