August 27, 2026
Hi Everyone,
If your company scores leads, predicts churn, sets prices, screens candidates, or flags fraud, an AI model is shaping those decisions every day. And there's a good chance nobody is checking whether it still works.
A survey of the people who build these systems found that less than 40% of models in production get checked at all, and the most common answer was companies checking fewer than one in five.
Today we're breaking down three questions to show whether the models your company relies on are still right, plus a one-page list to keep them that way.
Zillow's AI lost $304 million in one quarter
In 2021, Zillow bought thousands of homes at prices set by its valuation AI model. The market slowed that autumn, but the model kept paying boom prices.
By November, Zillow had written down $304 million, closed its home-buying business, and cut 2,000 jobs. CEO Rich Barton told investors the "unpredictability in forecasting home prices far exceeds what we anticipated."
The model never crashed or showed an error. Broken code fails the same day, but a model that no longer fits the world keeps producing answers, and the damage shows up later in your results, looking like a sales or pricing problem.
Most companies hit a smaller version of this. In McKinsey's latest State of AI survey, 51% of companies using AI reported at least one negative result, most often wrong outputs.
Three questions that catch a failing AI model
Banks employ whole teams for this. You don't need one — three questions asked once a quarter cover most of the same ground. Ask them in your next meeting with whoever runs your data work, or with the vendor behind a tool you've bought.
1. Which of our AI systems touch decisions about money, customers, or people, and who owns each one?
Include the tools you've bought, not only the systems you built. The lead score in your CRM, the fraud check in your payment provider, and the screening step in your hiring tool are all AI making decisions for you. A solid answer is a short list with one person's name next to every system.
2. How would we know if one of these systems started getting worse?
A solid answer names the number being watched and the level at which a person steps in.
Say your model scores leads for the sales team. The number is how often those top-scored leads become customers, and the line is the point where someone has to review the model, like conversion falling from 30% to 20%.
"We'd see it in the results" is not an answer, because by then the model has been wrong for months.
3. Can we explain why this system decided what it decided?
When a hiring tool turns down a candidate or a fraud check blocks a real customer's order, someone eventually asks why, and "the system decided" is not an answer you can give them. A solid answer is that someone can pull up the reason for any single decision. For example, “the order was blocked because the delivery address didn't match any previous order.”
Start the list today
The three questions tell you where you stand today, but the answers won't stay true. New tools arrive, owners change jobs, and numbers move. So write the answers on one page, listing each system with its owner, its number and line, and the date it was last checked.
Book 30 minutes a quarter with the owners to go through the page together, compare each number against its line, and update anything that changed.
For tools you've bought, add one step. A vendor can change the model behind your tool without telling you, and a wrong price or a wrongly rejected customer is still your problem. Send the vendor the same three questions before each renewal.
Go deeper
👉 McKinsey: The state of AI in 2025 — use it to see which AI problems other companies are hitting, and compare against your own list.
👉 MIT CISR: Minimum viable governance for generative AI — read this if someone on your board wants a heavier process, and you want the research case for a lighter one.
👉 Kalinowski et al.: Naming the pain in ML-enabled systems — skim it to see how rarely models get checked, even at companies that build them for a living.
Coming up tomorrow
In tomorrow's issue, you'll learn how to check your strategy's core assumptions each quarter, and when a change in them means the plan needs a real review.
See you tomorrow!