Reproduce the class scores before accepting the headline average

Freeze the classification question and evidence

The scikit-learn reference defines macro averaging across per-label metrics, weighted averaging by true-label support, and micro aggregation from pooled counts. F1 combines precision and recall harmonically. Record the operation, not just its label.

Choose a bounded editorial task such as routing invented document requests to the correct help category. Write down the category definitions, which categories belong in the report and whether each request receives exactly one label. Preserve the same reference labels and predictions for every calculation. Do not allow the reporting comparison to quietly become a new labeling exercise or a different test population.

Ask both report owners for the calculation specification, not merely a screenshot labeled F1. Record the library version, included label list, treatment of undefined values and rounding stage. If any category has no usable evidence, resolve that case explicitly with the analyst. Do not hide it by deleting the row after seeing how it affects the headline.

Reconstruct a deliberately uneven three-category fixture

Use an original confusion matrix with true categories as rows and predicted categories as columns. The three rows are [54, 6, 0], [12, 15, 3] and [4, 3, 3]. Their row totals are 60, 30 and 10; the column totals are 70, 24 and 6. All 100 invented requests appear exactly once. The diagonal contains 72 correct assignments.

The three precisions are 54/70, 15/24 and 3/6; recalls are 54/60, 15/30 and 3/10. F1 values are 0.830769, 0.555556 and 0.375000. Keep unrounded working values. These original fictional counts are not measured performance for a real support system.

Check the smallest category before summarizing. It has only three correct assignments out of ten reference cases. The worksheet should retain the actual wrong destinations, not reduce every error to an anonymous total. A reviewer can then ask whether the routing mistake sends readers to an irrelevant article or prevents completion of the task. The score alone cannot make that operational judgment.

Expose an incorrect order of operations

Mean class F1 is 0.587108. Combining mean precision and mean recall harmonically gives 0.597617, not macro F1. The predictions are unchanged. This discrepancy therefore cannot establish model improvement.

For this fixture, support-weighted F1 is 0.702628; micro F1 is 0.720000. Label them separately. A report owner must choose a statistic that answers the agreed review question rather than selecting whichever calculation gives the highest score. Keep the class rows beside the selected headline so the aggregation cannot conceal the small-category failure.

Use the wrong-order calculation as a regression test for the report. Ask the technical owner to show that the production reporting specification matches the approved operation, then compare with a separate implementation. The fixture was independently reproduced with Python 3.12.14 and scikit-learn 1.8.0; the referenced online documentation identifies version 1.9.1. This verifies the tested arithmetic only. It does not validate the reference labels, establish population representativeness or authorize any automated handling of real customer enquiries.

Resolve the discrepancy without rewriting evaluation history

If a dashboard used the wrong operation, correct its definition and explain which historical comparisons are affected. Preserve the original prediction snapshot. Do not retrain a model merely to make two incompatible metrics agree, and do not recalculate a single favorable month while leaving older periods under another definition. The interpretation needs a consistent record of what was measured.

Finish with the matrix, per-category values, declared label set, exact operation, unrounded result and display rounding. Have the accountable owner sign off on the meaning before the number enters a public claim. Recheck the packet when category definitions, report libraries or missing-category handling change. The accepted outcome is a reproducible score calculation, not a universal declaration that one averaging rule is best or that the classifier is ready for deployment.

Make the reasoning concrete.

Two fictional help-center dashboards disagree despite receiving the same prediction snapshot. The reporting owners have not compared their calculation order.

The owner corrects the reporting discrepancy and preserves the original evaluation evidence. No model, label or live route changes.

Take the questions into your next brief.

Use these prompts in your own notes. Nothing is submitted from this page.

1. Scope

   Which fixed requests and categories belong to the evaluation?

2. Counts

   Can every row, column and correct assignment be reconstructed?

3. Operation

   At what stage are class results combined?

4. Difference

   Does a changed number reflect new predictions or only a changed calculation?

5. Meaning

   What operational question does the approved headline answer?

Watch for the wrong conclusion.

• Combining averaged precision and recall and calling it macro F1.
• Changing the included category set between reports.
• Treating a higher recomputed headline as model improvement.

Sources & scope

All business examples and numerical fixtures are original and fictional. Sources support the identified mathematical or implementation distinctions. Qualified analysts own model specification and inference. No causal result, model fit, actual customer data or operational change is claimed.

• scikit-learn: precision_recall_fscore_support →Primary software documentation · Supports: Supports metric and aggregation definitions; the matrix and calculations are original.