A supplier delivered nineteen out of twenty orders on time. That sounds clear until you ask what counted as an order, whose clock was used and whether partial deliveries were treated as complete. If the twentieth shipment held the one part a workshop needed to open, the headline percentage might also miss the most important event of the month. A useful supplier scorecard keeps the arithmetic and the business consequence visible together.
Start with a short list of questions. Did the supplier deliver what was agreed, when it was needed? Did the goods meet the agreed standard? When something went wrong, did the response help resolve it? Keep enough evidence that a person who did not enter the figures can reconstruct them. A scorecard should help the buyer and supplier decide what to do next.
Choose the unit before choosing the target
An order, order line, shipment and item are different units. A single order might contain ten different products delivered on two trucks. Measuring shipments makes sense if the main question is whether scheduled trucks arrive. Measuring order lines can show whether each product arrived when it was needed. Neither measure tells you by itself whether an order was complete on the original date.
Imagine a small workshop that buys cartons, labels and protective inserts. It cannot ship a finished product if any of the three is missing. For one group of orders, it tracks whether every required line arrived on time and in full. For a regular single-item order, it can use a simpler measure. Document the choice beside the number. If you change it, avoid comparing the old and new percentages without an explanation.
Agree on the point that counts as delivery. The time a courier scans a package, the time a truck arrives at the gate and the time the receiving team records usable goods can differ. Use the event that matches the commitment, and record the evidence needed to establish it. If the buyer's receiving team is delayed, preserve both timestamps so the supplier is not judged on a delay it did not cause.
Preserve the original promise and later changes
Save the promised date and the agreed quantity before the delivery takes place. If either changes, keep both versions and note who requested and agreed to the change. A buyer may ask for a later delivery because there is no storage space. A supplier may ask for an extension because production is late. Both can produce the same new date in a system, but they tell different stories about what happened.
For example, a shipment was due on Tuesday. On Monday the buyer asks to receive it Thursday, and the supplier agrees. It arrives Thursday. A scorecard can record that the original date was changed by agreement at the buyer's request, and that the shipment met the revised promise. If the supplier requested the change because it could not make Tuesday, the same Thursday delivery may still meet a revised promise, while the original miss remains visible for planning.
A late change should not quietly remove an order from the denominator. State how cancellations, approved reschedules, and orders without complete evidence are treated. Keep a small exceptions list with dates and reasons. This can help distinguish a contractual dispute from a straightforward correction to an order record.
Calculate delivery and quality separately
Suppose twenty orders were due in a month. Eighteen arrived complete by their agreed deadlines. One arrived on time with half its quantity and the rest the following day. One arrived complete two days late. Under an order-level on-time-and-in-full definition, the result is eighteen of twenty, or 90 percent. Both incomplete and late orders miss the combined test. Listing them separately shows what needs investigation.
Quality needs its own denominator. If the receiving team checked 100 units and found four defective, the observed defect proportion is four out of 100 inspected. If 500 units arrived, it is not automatically four out of 500. State how the 100 were chosen. A sample deliberately taken from suspicious cartons gives different evidence from a sample taken according to a routine plan.
Use the inspection standard the parties agreed on, including how borderline cases and replacements are handled. Record the initial defect and the replacement outcome separately. A replacement can resolve a shortage while leaving useful evidence that the original delivery contained defective goods. Conversely, if a later inspection shows an apparent defect was a buyer-side handling problem, update the record and retain the explanation.
Record what happened after the miss
An account manager can answer an email in ten minutes and still leave the buyer without an arrival date. For that reason, separate acknowledgment from resolution. Record when the problem was reported, when the supplier acknowledged it, what remedy was offered and when the remedy actually worked. A tracking number for a replacement is evidence of a dispatch, not evidence that the workshop received usable stock.
In the example, the supplier might offer to send the missing cartons by courier. The buyer records the offer, then closes the immediate shortage when the cartons arrive and pass inspection. The buyer can keep a separate action open if the reason for the short shipment is still unknown. That action might be to compare the packing record with the approved order and test a change to the pick list. A closed shortage and a completed prevention action are different outcomes.
This is also a good place to acknowledge what the buyer controlled. A late drawing approval, a changed shipping address or a delayed inspection can affect the result. Keep those events visible next to the supplier's response. If both sides can see the same timeline, they can discuss what to change without first arguing over what happened.
Show the business consequence beside the rate
Two missed orders can carry very different consequences. A late label shipment may be absorbed by stock in the storeroom. A smaller shortage of protective inserts may stop all outbound orders for a day. Record the consequence in terms people can check: which work stopped, for how long, what had to be rescheduled and which cost records support any amount shown. If there was no observed interruption, record that as well.
Don't invent a monetary loss to make the scorecard look precise. An estimate of staff time is different from an invoice for an emergency delivery. If the team wants to compare both, label the estimate and explain the method. For example, a courier bill can be recorded at its actual amount, while two staff hours spent rearranging production can be recorded separately without assuming what those hours cost.
If you buy critical and routine items from one supplier, show the misses by item group or highlight the critical lines. That makes the consequence visible without weighting an entire month around one incident. A consistent rate lets you compare periods. A brief account of the incident lets you decide where to act.
Check that the evidence supports the number
A scorecard needs enough detail to reproduce the result, but it does not have to contain every purchase document. Keep a row for each unit being scored. Include an order or line reference, the agreed deadline, the received time and quantity, any exception, and a link or identifier for the underlying receiving record. For quality, link to the relevant inspection and any later disposition. Let reviewers see what came from a system and what someone entered manually.
Before the first review, take a handful of records from different categories. Pick a pass, a miss, a reschedule and an unknown if those categories exist. Ask a colleague to reproduce the classification using the agreed definitions. If two people disagree about whether a split shipment was in full, the problem may be the definition rather than the supplier. Resolve it and document the change before publishing the score.
Public purchasing guidance illustrates the value of this approach. The UK government's contract management principles discuss combining performance data with other evidence about outcomes and relationships. The US Federal Acquisition Regulation's contractor performance procedures call for evidence grounded in relevant contract and programme records. These are sources for the evidence principles here. A workshop should use the terms of its own agreement and the rules that apply to it.
Treat missing records as missing
Suppose two of the twenty orders have no usable receiving timestamp. You may be able to establish that eighteen orders met the target, but you cannot classify the other two as late solely from the missing data. You also cannot simply remove them and report eighteen out of eighteen as though the whole month had been measured. State the known results and the number pending verification. Give the person resolving the gap a deadline.
Missing data can be uneven. Perhaps urgent deliveries are more likely to be accepted at a side door and recorded the next morning. Perhaps a supplier sends paperwork consistently for large orders but rarely for small replacements. Before drawing a trend from the score, check whether the record-keeping changed. A new scanning process can improve the completeness of timestamps without changing the supplier's actual arrival times.
Show the number of orders alongside any rate. One miss out of five produces a very different percentage from one miss out of fifty, even though each represents a single event. The effect of either event depends on what was missing. If volume is low, a running log or a longer comparison period may help people see repeated problems without overreacting to one number.
Make the review end with an action
Send the supplier the disputed lines and the definitions before the meeting. Leave room for their receiving proofs, dispatch records or an agreed schedule change that your system did not capture. Correct factual mistakes and explain the correction. Keep a record of unresolved disagreements. The goal is to decide what to do with the best available evidence.
One useful review sequence is to identify the largest consequence, trace the issue to an order record and agree on a testable next step. For the missing inserts, the test could be to reconcile the next three packing lists with what the workshop actually receives. Name who owns the reconciliation, when it will happen and what evidence will close the action. The next review can then ask whether the test happened and what it showed.
Balance new actions with follow-up. If the same problem returns, check whether the prior action was completed, whether it addressed the right cause and whether the expected result was ever measured. If the evidence supports escalation, use the agreed route rather than treating the scorecard itself as a penalty notice. Keep operational feedback, formal notices and decisions about the relationship clearly distinguishable.
Start with the records you can maintain
A small buyer can begin with delivery, completeness, quality and resolution. Write down one definition and one evidence source for each. Add a short explanation of the most important exception and one action for the next period. Expand only when a new measure answers a question people actually use in decisions.
Set a regular cut-off for reporting. An order due on the last day of the month may still be unresolved when the report is drafted. Label the report provisional if it includes pending records, and state when those records will be checked. Retain the first version if the figures change so that anyone reviewing an earlier decision can see what information was available at the time.
When the numbers are ready, pick one order and trace it from the original promise to receipt, inspection and any follow-up. If the story holds together, the scorecard is doing something useful. It makes a percentage traceable to the orders and gives both sides a concrete place to start the next conversation.
For a second example of tracking decisions through incomplete evidence, compare claims with the underlying evidence.
Sources and research note
Sources checked November 3, 2026. UK and US public procurement material informs the general evidence practices discussed above. The workshop, figures and scorecard scenarios are conceptual examples. They do not describe a real supplier or prescribe contract terms.