Practical learning lesson / 7 MIN READ
Use geography to ask what the campaign added
Local targeting and a local sales increase do not by themselves reveal how many outcomes advertising caused.
AI-assisted research and analysis for the SNS Creative Fieldbook
Sources checked 2026-11-05
Compare planned changes across suitable geographic units while controlling the design, timing and operational differences that can distort the result.
What you will leave with: Write a credible geographic test brief and recognize when the business lacks the units, data or conditions needed for causal learning.
Use this when
- A retailer wants to evaluate local media or store activation.
- User-level tracking is unavailable or inappropriate.
- A campaign’s effect is being inferred from a simple before-and-after chart.
This is a design briefing lesson, not a statistical implementation guide. Specialist review is needed for power, assignment, estimation and uncertainty.
Specify the intervention and the outcome
Write the exact difference between treatment and comparison areas: additional media spend, a new channel or a local activation. Keep the offer and service conditions comparable unless they are explicitly part of the treatment. Choose an outcome the business can measure consistently across regions, such as completed transactions or contribution over a defined period. A platform’s attributed conversions may not capture the total local response or substitution between channels.
Google’s geographic-experiment research describes randomly assigning non-overlapping regions to treatment and control and implementing those conditions through geographic targeting. This is different from comparing whichever city performed best with whichever performed worst. Your design should answer what changed because of the intervention, relative to an explicit counterfactual. Define the unit of assignment before anyone sees the result, and distinguish it from individual customers inside that unit.
Choose regions that can support a useful comparison
Inspect historical outcome patterns, scale, seasonality, store availability and planned operational changes. Comparable pre-period behavior is useful, but matching alone cannot guarantee that the regions would continue similarly. Avoid letting one enormous market dominate the test or assigning all already-growing markets to treatment. Randomization can reduce selection bias when enough suitable regions exist. A specialist should evaluate balance and the minimum effect the design can reasonably detect.
When randomization is constrained, a matched-market approach may be possible, but its assumptions become especially important. Google’s matched-markets research explicitly discusses these constraints and dependence on modeling assumptions. A small business with two very different shops may have a learning pilot rather than a strong incrementality experiment. Naming that limitation early prevents a precise-looking percentage from receiving more confidence than the design warrants.
Plan for movement across boundaries
People commute, shop online, share content and travel across areas. Media also spills beyond intended borders. Map likely contamination before choosing the regions. If a national influencer post is part of the campaign, a supposedly untreated area may still receive the message. If customers buy in one location after seeing an ad in another, agree how outcomes will be assigned and what uncertainty remains in that assignment.
Coordinate with store and media teams so unrelated changes are logged: stock-outs, openings, road closures, price changes, weather disruption and competitor promotions can affect outcomes. Do not quietly remove an inconvenient region after looking at results. Predefine exclusions and document exceptional deviations. Geographic targeting is an implementation tool; it is not evidence that exposure was perfectly isolated or that the audience complied with the intended treatment.
Agree on analysis and timing before launch
Set the pre-period, test period, any lag window, primary outcome and analysis method before the intervention starts. Ask an analyst whether the expected signal is large enough relative to normal volatility to support the decision. If the answer is no, change the scope or treat the exercise as operational learning. Extra decimal places cannot rescue a test with insufficient information, severe imbalance or broken delivery.
Keep the interpretation tied to the tested treatment and setting. A positive result from adding local media to an existing plan does not estimate the value of every channel or the effect of doubling spend nationally. Report a range of plausible effects and the design’s limitations. Separate evidence of no meaningful improvement from an inconclusive result that cannot distinguish a useful gain from noise. Those findings imply different next decisions.
Read local campaign cases as design questions
Cadbury’s Shah Rukh Khan-My-Ad used location to promote nearby small businesses. Small Business Saturday organizes support for local businesses within a wider program. These make useful examples for asking where treatment might vary and where spillover might occur. Their public descriptions are not randomized geographic experiments. A local mechanism, national publicity or a reported change in sales should never be silently converted into an incremental causal estimate.
Finish your brief with practical responsibilities: who controls the intervention, who monitors delivery, who maintains the outcome data and who analyzes the result without changing the rules. Explain what the business will do if the result is positive, negative or inconclusive. A geographic test earns its cost when it changes a real allocation decision or resolves an uncertainty, not merely when it produces a presentation-friendly uplift figure.
Make the reasoning concrete.
A retailer considers eight areas for a local media test. Two share a major commuter catchment, one will close for refurbishment and one has incomplete sales records.
Exercise
Write the treatment, outcome, comparison rule and three likely threats. Explain why comparing this month’s sales with last month’s sales would not isolate advertising.
Interpretation
No uplift is supplied because the example contains no observed test data.
Resolve contamination and data issues before assignment. Four apparently usable areas may still be too few or too uneven for the desired precision; seek design review rather than promising a conclusive result.
Take the questions into your next brief.
Use these prompts in your own notes. Nothing is submitted from this page.
Treatment
What exactly changes, and what remains comparable?
Regions
Which units can be independently targeted and measured?
History
Do prior patterns and planned operations support a credible comparison?
Spillover
How might people, media or purchases cross boundaries?
Analysis plan
What precision, timing, exclusions and estimator are agreed before launch?
Decision
What follows from a positive, negative or inconclusive result?
Read with a question in mind.
Cadbury Celebrations / Mondelēz
Shah Rukh Khan-My-Ad →
Where does localization become a possible treatment boundary?
WPP documents nearby-store targeting and retailer-generated versions, useful for thinking about exposure and spillover.
What it does not prove: The published campaign account does not provide a randomized geographic counterfactual.
American Express OPEN
Small Business Saturday →
How could a national program contaminate a local comparison?
The official program connects local participation with a nationwide annual event, illustrating the difficulty of defining an untouched area.
What it does not prove: Participation or a before-and-after sales comparison is not an estimate of incremental campaign impact.
Watch for the wrong conclusion.
- Choosing treatment areas because they are already growing faster.
- Ignoring national publicity or cross-border shopping.
- Dropping inconvenient regions after seeing outcomes.
- Reporting an inconclusive result as proof of no effect.
Your next action
Write a one-page geographic test brief and ask a qualified analyst whether the available regions can answer the actual spending decision.
Sources & scope
Original editorial synthesis with linked campaign evidence. Exercises and calculations are hypothetical, not historical campaign facts. No named human author or reviewer is claimed.
- Google Research: Measuring Ad Effectiveness Using Geo Experiments →Primary research abstract · Supports: 2011 research abstract: random assignment of non-overlapping regions to treatment and control using geo-targeted advertising.
- Google Research: A Time-Based Regression Matched Markets Approach for Designing Geo Experiments →Primary research abstract · Supports: 2018 research abstract: constraints on randomization, matched-market selection and dependence on modeling assumptions.
- WPP: #NotJustACadburyAd and Shah Rukh Khan-My-Ad →Primary agency-network case · Supports: 2020/2021 distinction, mechanic, distribution and reported output/view totals.
- US Small Business Administration: Small Business Saturday →Primary official program-partner page · Supports: American Express founded the initiative in 2010; SBA cosponsorship began in 2011. Confirms the continuing US annual program without importing current activity into launch-year results.