Checking pick accuracy with a camera model

A logistics fleet operator runs temperature-controlled distribution for a food-service retail customer. At the pilot site, pickers fill plastic trays with wrapped chilled items for each store order, guided by a voice-picking system. A mispick reaches the store as a complaint, and complaints carry penalties.

The proof of concept asked whether an overhead camera could count each tray and flag a wrong pick before the order shipped. We presented the readout to the client in April 2023.

The pilot site's picking process was not defined well enough to drop a camera in without disrupting daily operations. We narrowed the scope to three questions: can a model learn to count the items and tell them apart, how accurate is picking today, and how much error could camera auditing remove.

The second question decides whether the first one matters. A camera that is right 99 times in 100 has nothing to catch if the pickers are already that good.

The picking baseline

We audited 166 store orders by hand after picking: 1,489 trays and 23,596 items, each checked against the ordered quantity and the packing rules. The audit found 309 errors.

Per item, pickers were right 98.7% of the time. Per order, which is what a store sees, 72.0% of orders were correct in full. Packing compliance was lower: 80.3% of trays met the rules, and 19.8% of orders met them in full. The most common violation was too many items in one tray.

A per-item rate near 99% still left more than a quarter of orders wrong. One mechanism spoils a whole order at once. Pickers sign on to an order by reading out its number, and a swapped digit can produce a valid number for a different order. A review of related issues found 125 to 130 similar incidents in the previous 12 months.

The camera model

We trained the model in four steps: trays holding one item type, then mixed trays that followed the packing rules, then a test with no intervention, then rounds of run, analyze, correct and repeat. The questions were whether it could count in a cold room, recognize each item, and separate items in near-identical clear wrappers.

Training used 6,761 images, and the model processes a tray in under a second.

Version	Orders	Orders exact	Items exact	Orders exact, as a bar
v1	44	65.91%	99.69%
v2	41	82.93%	99.86%
v3	85	88.24%	99.91%
v4	35	91.4%	99.86%
v5	85	94.1%	99.96%

Versions three and five ran on the same 85 orders, 1,473 trays and 23,166 items, so that step is a clean comparison: exact orders rose from 88.24% to 94.1%. The other versions ran on smaller, different samples.

Where the model failed

Three quarters of the model's errors came from trays that broke the packing rules, with items stacked so the camera could not see them. The model missed items it could not see, and it sometimes misclassified a half-hidden item.

The data pointed at the process before the hardware. A camera can audit only a tray it can see, and about one tray in five broke the rules that keep items visible.

• We recommended designing the picking process with live camera auditing built in, and offered a limited rollout on the current process as the alternative.
• We logged 19 improvement initiatives from the audit and scored each one on five factors: the customer's service principles, its SLA metrics, financial benefit, ease of implementation and future readiness. Ten of them fit inside the picking design.
• For the sign-on errors: scan a barcode on the order sheet instead of reading the number aloud, and later have the warehouse system feed each picker the next order.
• For overpacked trays: pickers guessed how many trays an order needed and overfilled them to save trips. Printing the tray count on the order sheet from the warehouse system attacks the violation that hides items from the camera.
• For look-alike items: ask the supplier for distinct colors or markings on each wrapper, which also shortens training for new items.

On its final test the model got 99.96% of item counts and 94.1% of order counts right, in a temperature-controlled room, at under a second per tray.

Order accuracy rose from the 72.0% baseline to 94.1% with camera auditing, a gain of 22.1 points, and item-level accuracy rose from 98.7% to 99.9%. The camera caught overages and shortages that hand picking let through to the store.

On those results we recommended building live camera auditing into a redesigned picking process, with a limited rollout on the current process as the alternative. The sources end at the readout, and I have no record of which option the client chose.

• Assess the process before scoping the technology. The pilot site's picking process differed from the site we visited first, and it had no defined pick process a camera could slot into, so the scope had to pivot after we arrived. A short process walk at the actual pilot site before the statement of work would have set that scope on day one.
• Run the baseline and the model on the same orders. The baseline audit and the final model test used different order samples, so the comparison carries sampling noise we never sized. A paired design with confidence intervals would size it.
• Hold one test set fixed across model versions. Three of the five versions ran on samples of 35 to 44 orders, so part of the version-to-version change is sample noise.