Reconcile every selected case before naming a survey response rate

Start from the selected sample rather than the response export

Reconcile the versioned selected-case list with the permitted fieldwork records. Identify the study’s eligibility reference date and who can approve final classifications. A response export often omits everyone who never opened the questionnaire, so it cannot by itself establish the denominator. Retain one restricted internal reference per sampled case while keeping unnecessary contact details out of the review table.

Name the intended measure before calculating it. Response, cooperation and completion can describe different aspects of fieldwork. Ask the analyst which definition applies to this design and what evidence each category requires. Keep the platform’s operational counters as useful inputs, but do not let a dashboard label determine the methodological meaning of the rate.

Assign dispositions from evidence, retaining unknown eligibility

AAPOR’s named-person email guidance does not treat an undelivered invitation as automatic evidence of ineligibility. It requires clear evidence for ineligible classification and preserves cases whose eligibility is unknown.

Create an approved mapping from available operational evidence to final categories. Distinguish a verified eligible person who declines from a case for which eligibility was never established. Preserve the researcher’s documented complete and partial interview rules rather than assuming that a platform’s recorded status supplies them. Give ambiguous cases a review route and keep the basis of each decision inspectable without publishing personal explanations.

Check that every sampled case has one final disposition at the agreed cutoff and that the totals reconcile. Repeated delivery attempts are not additional sampled people. Conversely, a case with no returned questionnaire must not disappear merely because the analytical dataset has no row for it. Keep uncertainty visible instead of assigning the category that makes the resulting rate look strongest.

Calculate a named rate with a justified unknown-case treatment

Have the analyst apply the selected definition to the final counts and preserve the calculation. AAPOR’s rate definitions differ in their treatment of partial interviews and unknown eligibility. Where an eligible proportion is estimated for unknown cases, its basis must be documented rather than chosen to raise the rate.

Check the arithmetic independently using a small fictional count table. Verify that the numerator and denominator use the same selected sample, cutoff and disposition version. If two legitimate rate definitions are reported, label both rather than presenting their difference as a change in fieldwork performance. Do not apply an estimate for unknown eligibility borrowed from an unrelated project without a defensible methodological basis.

Reconcile the rate table with the final-case ledger, not only the totals in a presentation. Retain cases assigned unknown eligibility even when their contribution enters through an analyst-approved estimate. This preserves the distinction between observed status and an assumption used in the calculation.

Report the rate without turning it into a quality certificate

Publish the named rate, applicable design and enough methodological context to interpret it. Include the selected and achieved counts and material unknown-eligibility limitations. Explain any changes caused by corrected dispositions or a revised cutoff rather than quietly replacing the historical percentage. Keep the supporting ledger restricted; methodological transparency does not require disclosure of participant identities.

Use the rate as one part of a broader research assessment. A high figure does not prove that answers are unbiased, and a low figure does not by itself measure the direction or size of bias. The completed artifact makes the selected sample’s outcomes auditable and the calculation reproducible. Decisions about data retention, further contact and substantive analysis remain separate and require their own approved plans.

Make the reasoning concrete.

A fictional probability survey emails a fixed list of named customers. The platform reports a high completion percentage among starters, while some invitations return undelivered and those people’s eligibility is unresolved.

The team publishes the defensible calculation and does not claim that the rate alone establishes survey quality.

Take the questions into your next brief.

Use these prompts in your own notes. Nothing is submitted from this page.

1. Sample

   Which selected-case list and cutoff define the accounting?

2. Evidence

   What supports each final disposition?

3. Unknowns

   Which eligibility states remain unresolved?

4. Formula

   Which named rate and justified assumptions are used?

5. Report

   What counts and limitations accompany the percentage?

Watch for the wrong conclusion.

• Calling starter completion the selected sample’s response rate.
• Treating bounced email as automatic ineligibility.
• Choosing unknown-case assumptions to improve the headline.

Sources & scope

Workflows, worksheets and fictional examples are original editorial analysis. Sources support only specifically attributed principles. Platform guidance can change; legal requirements depend on jurisdiction and context. No measured results or named human reviewer are claimed.

• AAPOR: Standard Definitions, tenth edition (2023) →Primary survey methodology standard · Supports: Named-person email/web eligibility dispositions and named response-rate treatment of partials and unknown eligibility; scoped to the read sections.