← Reading listCHECK WHICH QUESTION EVIDENCE ACTUALLY LEAVES THE COURSE
Text

Learning delivery workshop / 6 MIN READ

Check which question evidence actually leaves the course

Build a synthetic question-to-report fixture that distinguishes absent evidence from an incorrect answer.

Published by SNS Media Tech

AI-assisted research and analysis for the SNS Creative Fieldbook

Sources checked 2026-11-07

A course can show a rich quiz while reporting only a final score. Before promising question-level analysis, verify which interactions the package actually sends and how the receiving system represents answered, missing and repeated responses.

What you will leave with: Build a synthetic question-to-report fixture that distinguishes absent evidence from an incorrect answer.

Use this when

  • A course supplier promises question-level reporting.
  • An LMS report omits quiz detail or shows repeated question entries.

Not question writing, learner surveillance or a transport retry contract.

Specify the evidence needed for the reporting decision

Rustici’s Engine interaction-data guidance, updated in June 2026 and checked on 7 November 2026, explains that a visible quiz does not prove the course reports interactions. It describes activity-level interaction records and warns that available fields vary by learning standard and content. This lesson uses a SCORM 2004 4th Edition package accepted into the specified Engine deployment.

Choose a narrow reporting question, such as whether a particular misconception recurs in an approved practice exercise. List the fields needed to interpret it: activity context, question reference, response representation and result. Do not collect every available field just because the platform exposes it. The acceptance requirement should explain what the report needs and what evidence would still be insufficient.

Use a small fixture with deliberately different response states

Prepare synthetic cases containing a correct response, an incorrect response and an unanswered question. Add a repeated interaction if the course permits returning to the question. Record the expected question reference and activity for each case, using only fictional answers. These cases reveal more than a single fully completed perfect quiz.

Inspect what arrives in the receiving report. Rustici notes that unreported interactions do not supply a complete question bank. Keep a separate fixture log of what the synthetic learner encountered and submitted. For each absent row, reconcile that log with the intended reporting coverage before assigning an interpretation. The absence is a question to investigate, not permission to invent an incorrect answer.

Decode the response without losing its context

Confirm how the actual export represents each response type. A choice code can be meaningful only when the analyst knows which question and option mapping it belongs to. A readable question description may be absent in one supported format, so the handover may need a separate approved reference. Do not assume that a code such as A always means the same option after question shuffling or a content revision.

Rustici describes courses that append revisited interactions and others that update an existing record. Test which behavior the actual package uses. Write the reporting question before selecting a retained response: does the team need the first interpretation, the revised answer or another approved measure? Record what evidence the observed history can support. If the needed history is unavailable, narrow the analysis instead of reconstructing it from assumptions.

Accept the extraction path as well as the course

Follow the fixture through the report or authorized extraction stakeholders intend to use. Rustici locates interaction detail within activity records, so retain the relevant activity context. Compare the output with the fixture ledger row by row and document where an expected field disappears. This separates the acceptance decision from a premature accusation of supplier data loss and gives the responsible owner a reproducible discrepancy to investigate.

Retain a field-coverage note with the package revision, standard edition and receiving implementation. Explain missing fields and repeat-response treatment before anyone uses the output to judge learners or redesign questions. Keep real learner data out of QA artifacts and limit production analysis to its approved purpose. Acceptance means the report can support the specified question-level interpretation, not that a final score implies complete interaction evidence or that every unanswered item proves a knowledge gap.

Make the reasoning concrete.

A fictional creative-training course offers different question branches for media buyers and designers. A synthetic media-buyer record contains no response for a design-only question. The analyst initially interprets that absence as a knowledge failure rather than checking which branch the fixture encountered.

Fill missing rows with failure

Opening: Assume the report contains the full question bank and assign zero to everything absent.

Hypothesis: A complete-looking table is easy to produce.

Risk: Unreported or unencountered questions become invented wrong answers.

Verify reporting coverage

Opening: Trace answered, unanswered and revisited fixture cases through the actual extraction.

Hypothesis: The analyst can distinguish available evidence from missing evidence.

Risk: Some questions cannot support the planned analysis without a better reporting contract.

The team documents coverage and response-history behavior before using the interaction report, without manufacturing scores for absent records.

Take the questions into your next brief.

Use these prompts in your own notes. Nothing is submitted from this page.

  1. Purpose

    Which question-level decision requires these fields?

  2. Coverage

    When does the package emit or omit an interaction record?

  3. Context

    Can each response be interpreted with its activity and question reference?

  4. Repeat

    Does a revisited question append history or update a record?

  5. Extraction

    Does the intended report expose the required detail without losing nested activities?

Watch for the wrong conclusion.

  • Assuming a visible quiz guarantees interaction reporting.
  • Treating missing rows as incorrect answers.
  • Calling every repeated question record a transport duplicate.

Your next action

Run correct, incorrect, unanswered and revisited synthetic cases and document exactly what the intended report exposes.

Sources & scope

Workflows, worksheets and fictional examples are original editorial analysis, not supplier implementation instructions or measured business results. Sources support only the specifically attributed platform facts. Supplier controls, contracts and local requirements can change; verify the actual placement and current provider guidance before implementation.