Randomize the unit that can actually receive a different experience
Account for shared accounts, households and repeated visits before choosing an assignment method.
Read the lesson →THE CREATIVE FIELDBOOK / SNS MEDIA TECH
Practical lessons
529 source-led reads, 6 per page. These numbered pages work without JavaScript. Search every study and lesson →
Check who went missing before reading the experiment winner
Compare observed assignment counts with the intended allocation, then trace discrepancies through the data pipeline.
Read the lesson →Watch a running test without turning every glance into a verdict
Separate operational monitoring from the statistical rule used to declare a winner.
Read the lesson →Count the chances you gave yourself to find a winner
Variants, outcomes and audience slices all create opportunities for chance findings.
Read the lesson →Test the testing system with two identical experiences
Use an A/A exercise to challenge assignment, reporting and analysis before relying on a business result.
Read the lesson →Ask whether the effect survives the first encounter
Examine exposure age and calendar context before projecting a short launch response into the future.
Read the lesson →