Do not call an invitation effect the effect of attendance

Name the assigned action and the received experience

Record what was randomized, what the customer actually received and what counts as participation. An invitation, a reminder and access to an exclusive benefit are different interventions. Keep assignment records even when a person ignores the offer or participates through another route.

J-PAL distinguishes intention-to-treat effects of assignment from local effects for people whose participation changes because of assignment. Its guidance identifies relevance, independence, monotonicity and an exclusion restriction among the important conditions for the latter interpretation. These conditions are additional causal requirements, not facts established merely by labeling the analysis instrumental variables.

Preserve the assignment comparison before asking a harder question

Analyze the randomized groups as assigned for the question the design directly supports, using the appropriate planned method. Do not drop non-attendees or move them into the comparison group. Attendance can reflect motivation, available time or prior interest, so an attendee-versus-non-attendee contrast loses the original randomized comparison.

Ask whether the effect of the invitation is already the relevant business decision. A marketer deciding whether to run the invitation program may care about its total effect under realistic take-up. A desire for a larger attendance effect is not itself a reason to replace that estimand. Report assignment, delivery and participation distinctly so stakeholders understand which intervention the number describes.

Examine every route from encouragement to the outcome

If participation effects are needed, ask a qualified analyst whether the proposed instrument changes participation sufficiently and whether its assignment is credibly independent of other outcome causes. A weak relationship between encouragement and participation can make the estimate unstable. More elaborate modeling does not repair an invitation that barely changes take-up.

Inspect the exclusion restriction in the actual message. An invitation containing a discount, persuasive product information or a special sales contact may influence purchases even if the person never attends. That direct route makes it harder to attribute the invitation-induced outcome difference only to attendance. Review monotonicity and spillover concerns as well, explaining their meaning for this intervention rather than treating them as a checklist of statistical tests.

Keep the local interpretation and remaining uncertainty visible

Under the required assumptions, the participation estimate concerns people induced to participate by this particular encouragement, not necessarily all attendees or all customers. Changing the message, eligibility or access route may change that group. Do not present the result as a universal return from attending any webinar or using any trial.

Have the analyst show the participation difference, outcome difference, uncertainty and relevant sensitivity analysis. Distinguish evidence that an assumption fails from an assumption that cannot be fully verified from the observed data. If the direct invitation effect is unavoidable, retain the valid assignment question and avoid overstating the participation claim. The useful outcome is a correctly named effect with a credible path to identification, not the biggest number obtainable by dividing two differences.

Make the reasoning concrete.

A fictional software company randomizes a webinar invitation that includes a discount code. Purchases rise among invitees, and an analyst proposes attributing the increase to webinar attendance even though non-attendees can use the code.

The team retains the randomized invitation comparison and does not relabel the discount-enabled result as the pure effect of attending. Any later design gets a separately reviewed intervention and analysis plan.

Take the questions into your next brief.

Use these prompts in your own notes. Nothing is submitted from this page.

1. Assignment

   What exactly was randomized?

2. Take-up

   How did assignment change actual participation?

3. Direct path

   Could the encouragement affect outcomes without participation?

4. Assumptions

   Which causal conditions require substantive justification?

5. Population

   Whose effect does the proposed estimate describe?

Watch for the wrong conclusion.

• Comparing attendees with non-attendees as though attendance were randomized.
• Ignoring persuasive content or discounts in the encouragement.
• Calling a local complier effect the effect for every customer.

Sources & scope

Workflows, worksheets and fictional examples are original editorial analysis. Named research guidance supports identified method principles. This is design-review preparation for a qualified analyst, not an estimation recipe or a claim of observed business results.

• J-PAL: Data analysis, noncompliance and local average treatment effects →Primary research-method guidance · Supports: Assignment versus participation estimands and assumptions for local effects under imperfect compliance. The invitation example and review workflow are original analysis.