The Model Is the Easy Part

Every ML project I've shipped in a manufacturing environment has followed the same arc: the model took two weeks, and everything around the model took six months.

This isn't a complaint. It's the job. But nobody tells you that when you're learning ML from papers and Kaggle, where the dataset arrives clean and the metric is the finish line.

The 90% nobody benchmarks

In a fab, the interesting problems start before the first training run:

• The data lies. Sensor telemetry has gaps, timestamp drift between tools, and schema changes nobody announced. Half of "model performance" is data contract enforcement.
• The labels are political. Two process engineers will classify the same defect image differently, and both have twenty years of experience. Your ground truth is a negotiation.
• The baseline is a human expert. Not a majority-class classifier, a person who has seen every failure mode since the 200mm era. Beating them on a held-out set is easy. Earning their trust is not.

Feedback loops beat architectures

The single highest-leverage thing we built was not a better model. It was a review queue: every low-confidence prediction routed to an engineer, every engineer decision routed back into the training set.

That loop did three things no architecture change could:

1. It made the model better every week without a retraining project.
2. It gave engineers a way to correct the system, which made them allies instead of skeptics.
3. It generated the labeled data we wished we'd had on day one.

What I'd tell past me

Build the evaluation harness first. Build the feedback loop second. Build the model last: by then, you'll actually know what it needs to do.

The model is the easy part. The system is the product.