Denominator drift and three other ways creator media kit reports go wrong

Creator media kits measurement mistakes can make a tidy report answer the wrong question. The checks below concern how a creator portfolio's figures are defined and interpreted. They do not estimate how often these errors occur or name businesses alleged to have made them.

What to take away

How this list was selected

This non-ranked editorial list was researched on 6 September 2026 for creator-kit reporting in England. Its universe is measurement risks addressed in the cited UK official statistical, evaluation and advertising material. An entry is included where that material supports a distinct risk that can be translated into a portfolio reporting check.

The selection excludes software bugs, privacy-law determinations, alleged supplier misconduct and benchmark claims without comparable data. The order follows interpretation, from defining the figure to presenting a claim; it is not a severity ranking.

There are no affiliate links or paid inclusions in this draft. Review entries when their sources or the proposed measurement definitions change, and retain a correction record identifying the statement, evidence and resulting amendment.

Switching the denominator

ONS's quality definitions explain comparability across sources, methods and time. The portfolio application is to check whether a reported response proportion still uses the same eligible population when periods are compared.

Define the event as an organisation's first qualifying commissioning enquiry. The denominator is eligible recipient organisations in the declared kit cohort, with a fixed follow-up window.

Worked example, illustrative figures: a kit reports 12 qualifying enquiries from 300 eligible recipient organisations, a response proportion of 4.0 per cent.

A later check finds 50 of those recipients fall outside the declared cohort, so the corrected figures are 12 from 250, or 4.8 per cent. The label 'enquiries per eligible recipient' holds only if the same cohort rule applies to both periods. The corrected report states the revised denominator and the date of the change.

The enquiry owner should reconcile the message register with the distribution source and record duplicate, test and pre-existing-project exclusions, plus uncertain matches. Substituting visits or only respondents changes the question; do not preserve the old label.

Treating later enquiries as caused by the kit

The Magenta Book's evaluation guidance distinguishes assessment of impact from monitoring what happened. A house report should therefore separate the dispatch event, recorded interaction, received enquiry and agreed commission before considering an explanation.

Ask what evidence supports the link between the revision and the outcome, and what could have happened without it. A sequence of timestamps supplies chronology. It does not remove alternative explanations such as an existing relationship. Keep an unsupported causal statement out of the kit even when the underlying count is accurate.

Hiding missing observations behind precision

ONS explains non-sampling error, including non-response and processing problems. For portfolio records, that supports checking the scope of missing or misclassified information rather than assuming more decimal places improve the evidence.

Name the source owner and extraction date beside a coverage note. Distinguish an unavailable event stream from an observed zero. If some enquiries cannot be matched to the cohort, show them separately and explain the uncertainty. Do not attach a survey confidence interval to an administrative count without a justified method.

Removing the caveat from a commercial claim

CAP's misleading-advertising section addresses substantiation and qualifications for communications within its remit. It supports checking a claim's evidence and presentation, not assuming every internal spreadsheet falls under the Code.

Before moving a reporting statement into a media kit, retain the event definition, dates, England cohort rule and material exclusions that make it interpretable. A narrowed sample should not become a claim about all recipients through an abbreviated caption. Refer disputed advertising scope or wording to the Advertising Standards Authority.

Use the checks on an actual report

Give each proposed check an owner and a place to record its result. Compare the draft claim with the retained source extract, not merely another chart containing the same number. Keep unresolved coverage, classification and causal questions visible to the release decision-maker.

This list has not been applied to a live kit. A named human editor and measurement specialist must review it before publication, with qualified advertising review where needed. Correct a failed claim or withhold it; do not mark a report sound because its layout is consistent.

Before you act

Record each check as one line, using this coverage-note format: [check] | [source and extraction date] | [result] | [unmatched or missing] | [owner].

Common questions

What changes when a creator swaps the denominator in a response proportion?

Substituting visits or only respondents changes the question being answered, so the old label should not be preserved. The check is whether the reported proportion still uses the same eligible population when periods are compared, with the denominator defined as eligible recipient organisations in the declared kit cohort.

Why is a timestamp sequence not enough to prove the kit caused an enquiry?

A sequence of timestamps supplies chronology but does not remove alternative explanations such as an existing relationship. The Magenta Book guidance distinguishes assessment of impact from monitoring what happened, so the dispatch event, recorded interaction, received enquiry and agreed commission should be separated before any explanation is offered.

When should a reporting statement be kept out of a media kit?

Keep an unsupported causal statement out even when the underlying count is accurate. Before moving a statement into a kit, retain the event definition, dates, England cohort rule and material exclusions that make it interpretable, and refer disputed advertising scope or wording to the Advertising Standards Authority.