An API Supplier Scorecard: Eight Dimensions You Can Measure
Share
Most supplier scorecards are a list of adjectives with numbers attached. Communication is rated four out of five. Responsiveness is rated good. Nobody can explain after the meeting why one supplier scored higher than another, and next year the scores will be different for reasons nobody recorded.
A scorecard is only worth building if two people scoring the same supplier independently reach the same number. That constraint eliminates most of what companies measure and leaves a smaller set of things that can be counted from documents already on file.
The test a criterion has to pass
Three questions decide whether a dimension belongs on the card.
- Can it be computed from something in a folder, rather than from a memory of a phone call?
- Does a bad score predict a specific failure, rather than a general disappointment?
- Would a different person, given the same records, produce the same number within a point?
A criterion that fails the third question is a proxy for how well the two teams get along, which is a real thing and is not a quality metric.
The eight dimensions
| Dimension | What to count | Source document | Failure it predicts |
|---|---|---|---|
| 1. Document completeness | Documents received against documents requested, first time | Onboarding and requalification files | Slow response at the worst moment |
| 2. Document accuracy | Errors found per document set | Your own review record | Contradictions discovered during release |
| 3. Batch consistency | Spread of key results across the last ten batches | Certificates of analysis | Unexplained drift and rejected lots |
| 4. Change notification | Changes notified before effect, against changes discovered | Change records and audit findings | Unnotified change invalidating your file |
| 5. Deviation and CAPA closure | Median days from opening to verified closure | Quality agreement reports | Open items shipping as if closed |
| 6. Complaint response | Days to a substantive first reply | Complaint log | Weeks of silence during a live problem |
| 7. Delivery reliability | Shipments on the agreed date and within the agreed condition | Shipping records and logger data | Line stoppage and repeat excursions |
| 8. Technical answer quality | Answers that explain a cause, against answers that restate the result | Correspondence on technical queries | Inability to investigate when something shifts |
Eight is deliberate. A card with twenty dimensions gets filled in once and abandoned. A card with eight gets used, because the data is already in the quality file and takes an afternoon to extract.
Why the third and eighth dimensions carry the most weight
Most of the others measure responsiveness, which is visible and therefore managed. Two of them measure something less visible and more predictive.
Batch consistency is the closest thing to a leading indicator available to a buyer. A supplier whose assay results sit between 99.4 and 99.8 across ten batches is controlling something. A supplier oscillating between 99.1 and 100.6 within specification is passing, and passing is not the same as controlling. The method for reading that spread is set out in batch-to-batch consistency.
Technical answer quality separates suppliers who hold data from suppliers who hold documents. Ask where an impurity comes from and one supplier describes a step; the other sends the specification again. The first can investigate a shift. The second can only report one.
Weighting, and why one weighting is wrong
The weights should follow what a failure costs you, and that differs by use.
For a material going into a sterile product, change notification and batch consistency dominate, because a shift in either is expensive to discover late. For a material going into feed or a non-regulated application, delivery reliability and document completeness carry more weight, because the regulatory consequence of drift is smaller.
A single corporate weighting applied across all uses produces a score that is accurate for nobody. Two weightings from one card is a better arrangement than one weighting nobody trusts.
What to do with the number
A score is only useful if it changes a decision. Four thresholds do most of the work.
Above the top band: reduce oversight, not attention. The failure mode of a good supplier is being taken for granted.
Middle band: target the weakest dimension with a specific request rather than a general improvement plan. "Notify changes before effect" is actionable; "improve quality culture" is not.
Below the band: start the second source process before the current relationship fails, not after. The logic is in adding a second API source.
Any supplier with an unresolved critical audit finding: the total score is irrelevant. One critical finding is not averaged against seven good dimensions, and a card that allows it to be is a card that is about to approve something it should not.
Where the data comes from
All eight dimensions draw on records a buyer should already hold: certificates of analysis, the onboarding document set, complaint correspondence, shipping documents, and audit reports. If a dimension has no data, that is itself a finding about your own process rather than about the supplier.
Two supporting practices make the scores usable over time. Keep the raw extract, not only the score, so a number can be defended a year later. And rescore on a fixed cycle tied to requalification rather than when a problem appears, because scores collected only during problems are scores about problems.
For how audit findings feed dimension four and five, see remote versus onsite supplier audit. For the trend data behind dimension three, see annual product quality review.
Primary sources: ICH Q10 for the quality system expectations the card is measuring against, the FDA Q7 guidance for API-specific obligations, and PIC/S for how inspections classify findings.
Frequently asked questions
How many criteria should a supplier scorecard have?
Few enough to be filled in from existing records without a project. Eight dimensions is a workable number; beyond about a dozen, the card becomes an annual exercise that nobody uses between cycles.
Should subjective criteria be excluded entirely?
Not excluded, but kept separate. Judgements about how a supplier behaves in a crisis are valuable and are not reproducible scores. Mixing them into a numeric total destroys the total's usefulness.
How often should suppliers be rescored?
On a fixed cycle tied to requalification, with an interim rescore after a significant event. Scores collected only when something goes wrong describe the problem, not the supplier.
What if a supplier refuses to provide the underlying data?
Then the dimension scores at its lowest value, because absence of data is a result. A supplier unable to provide ten certificates of analysis is telling you something about how it maintains records.
Can the same scorecard be used for resellers?
Partly. Dimensions about manufacturing consistency do not apply, while document accuracy, change notification and response time apply more heavily, because a reseller's only product is the accuracy of what it passes on.
Does a good score remove the need for audits?
No. A score summarises what documents show. An audit looks for what documents do not show, and the two answer different questions about the same supplier.