QA samples two percent and calls it quality

the sampling problem nobody defends

Ask a quality manager what share of interactions they review and the honest answer is usually between one and three percent, selected by convenience rather than design. Then ask what decision the resulting score drives. In most operations it drives coaching conversations and, occasionally, a termination. It does not drive process change, product change, or policy change, which is where the real defects live.

A two percent sample cannot detect a defect that occurs in one call in two hundred, and one call in two hundred at scale is a systemic failure. So the quality function is structurally blind to exactly the class of problem it exists to find, and it compensates by being very precise about greeting scripts and call closings.

what full-population review changes

Once every interaction can be reviewed by a model at negligible marginal cost, the question stops being which calls to sample and becomes which signals to extract. Not a score — signals. Policy exceptions granted. Promises made that the system cannot keep. Repeated confusion about the same product term. Sentiment collapse at a specific step in a specific workflow. These are operational defects, and they are visible only in aggregate across the full population.

This is a redefinition of the quality function rather than an efficiency gain on top of it. The output stops being an agent scorecard and becomes a defect register aimed at product, policy, and process owners. Most organisations are not organisationally ready for that, which is why they deploy AI QA and use it to score the same two percent faster.

the governance you still need

Automated scoring that touches employment decisions is a high-consequence use of a model, and it should be treated accordingly: documented rubric, human review of adverse outcomes, drift monitoring, and appeal. The technology is ready. The failure mode is not accuracy; it is deploying evaluative automation against people without the process a person would be entitled to.

a discipline built around a sampling constraint that no longer exists

The two-percent sample was never a methodological choice; it was a headcount constraint. A human reviewer can score a finite number of interactions per shift, and quality teams sized their sample to what the team could physically review, then reverse-engineered a rationale about representativeness after the fact. Ask a QA lead why the sample is two percent rather than five, and the honest answer is almost always some version of "that's what we can staff," not "that's what statistical confidence requires."

Statistically, a sample that small cannot detect a defect occurring in one interaction in two hundred with any reliability, and a defect at that frequency, at the volume most contact centers run, is thousands of customers a year experiencing the same failure. The quality function exists to catch exactly this class of problem and is, by its own sampling design, blind to it. It compensates, understandably, by becoming extremely good at catching the things a small sample can catch: greeting compliance, hold procedure, call closing language. Precision on the trivial, blindness on the material.

the decision the score never drives

Ask what happens after a low quality score and the answer in most organisations is a coaching conversation with the individual representative, occasionally a formal improvement plan, rarely a review of whether the underlying process, script, or policy set the representative up to fail. This is a reasonable allocation of authority — quality teams typically cannot change product or policy — but it means the review function is structurally incapable of fixing the defects most likely to be systemic rather than individual. A confusing refund policy generates the same low score, repeated across dozens of representatives, for years, and the fix that would actually work is never in the QA team's authority to make.

what changes at full-population review

Once every interaction can be reviewed automatically at a marginal cost close to zero, the constraint that shaped the entire discipline disappears, and the question changes completely. It stops being which two percent to sample and becomes which signals are worth extracting from one hundred percent of the population. Not a per-agent score — a defect register: policy exceptions granted outside guidelines, promises made that downstream systems cannot fulfil, the same product confusion recurring across hundreds of unrelated calls, sentiment collapsing at an identifiable step in a specific workflow regardless of which representative is handling it.

This is a different deliverable aimed at a different audience. The agent scorecard goes to a team lead. The defect register goes to the product owner, the policy owner, and the process owner, each of whom can fix something a coaching conversation never could. Most organisations that deploy AI-assisted QA today skip this reorientation entirely and simply use the model to score the same small sample faster, which multiplies the speed of a tool that was pointed at the wrong target from the start.

the governance this actually requires

Automated scoring that feeds coaching, performance ratings, or termination decisions is a consequential use of a model over people's employment, and it earns a correspondingly higher governance bar than a defect register does. That means a documented, versioned rubric that representatives can see in advance; human review of every adverse outcome before it takes effect; drift monitoring so a rubric interpretation does not silently shift over a quarter; and a real appeal path with a human on the other end of it. None of this is a reason to avoid full-population review — it is a reason to route the two use cases differently, with the defect register moving fast and the personnel-facing scoring moving carefully.

The organisations getting this right treat full-population QA primarily as a product and process instrument, with individual coaching as a secondary output rather than the entire point. That reordering is the actual transformation. The technology to review one hundred percent of interactions has existed and worked well for some time; the operating model willing to act on what it finds is still rare.