Records, failures and promises
Make telling the truth pay
A team that tells you what went wrong has done the company a favor and paid for it. Unless you change that, the favor stops.
The story
Six weeks of quiet.
Field Repair Dispatch was the first Hartwell team to run the shared document-reading base on live jobs. Six weeks in, a senior dispatcher noticed something odd. On compressors built before 2016, crews kept arriving without the right seal kit. The cause turned out to be small, a cousin of the stamp problem in the first chapter. On the old service-log scans, the model suffix sits inside a rubber stamp, and the base read the stamp as noise.
Dispatch fixed it themselves. A short local rule lifted the suffix out of the stamp before the base ever saw the page. Wrong-crew corrections fell from about one job in ten to about one in forty. Dan Whitfield was pleased, and he told nobody.
Maya Okafor found out from a spreadsheet. She had asked every team to keep a simple count of the times a person overrode the model, and Dispatch's count had a cliff in it. She asked Dan about it on a Tuesday. He did not pretend. He told her what the stamp did, what the rule did, and why he had kept quiet.
Last spring, Customer Support, back when it still ran its own bot, had told the old platform group that the bot failed in Portuguese. Within a month the team's quarter of compute was “put under review”. It never came back. Dan had a request waiting for the next review. “If I tell you,” he said, “you will want to look at the whole project. And I would rather have a patch than a review.”
By then it had cost something. Priya Natarajan's Warranty Claims team reads the same stamps, because many exclusions depend on the model suffix. They had spent about a week of two analysts re-checking some 200 claims by hand. They learned about Dispatch's fix over lunch, from a friend. It was the second invoice, again.
Maya did not blame Dan. Maya said afterwards that she spent that evening trying to find the villain and kept finding her own rules. “He did exactly what I would have done. He did the rational thing in the system I built.”
The problem
Honesty is a gift that costs the giver.
Most advice about “a learning culture” asks people to be braver. It rarely lasts. People are not brave on schedule, and a quarterly budget does not reward bravery.
Think about what a team knows when it hits a failure. It knows what broke, how to recognize it again, and what it tried. That is valuable to the next team. The finder pays for the diagnosis and pays again for the telling: a write-up, a meeting, and a risk that the sponsor looks harder at the project. The benefit goes to someone else. Any one team can reason that it is better off keeping quiet, and many do.
Notice that nobody in this story is acting in bad faith. When honesty is a private loss and a company gain, the shortfall is built into the arrangement. The fix is to rebuild the arrangement.
What the research shows
A record is worth what it can be trusted to teach.
The research makes three separate points, and they matter in this order.
First, a record helps only if it is accurate, relevant and findable. In the lesson on records, a raw activity log teaches nothing. A reviewed, validated and diagnosed failure can. Filing the same record twice adds no knowledge. A wrong label applied to a new case can leave an assessor worse off than starting from scratch. And a signature the company has never seen stays unseen, however many successes are filed about other things.
Second, telling the truth has to be worth the telling. The lesson on promises shows a team that hides a failure because admitting it would end the project and its future access to compute. Hiding is that team's best private choice. The rewards that change the choice are narrow ones. A diagnosed record earns a bounded, expiring chance to propose new work, and the new work must still qualify. The old work can still stop. The reward is for the lesson, never for the project's survival.
A promise like that is only as good as the cost of breaking it. In the lesson's model, a sponsor honors a promise only if breaking it costs more than the benefit of breaking it. Teams can read that sum as well as the sponsor can. A promise that is cheap to break gets little disclosure.
Third, a better option tomorrow can make people less candid today. In one worked case, a genuinely useful future option made each team's honesty less attractive now. Teams stopped sharing, and the company finished with 3 units of value where the candid version earned about 8. Teams think a round ahead. A smarter process does not guarantee honest inputs.
One more detail is worth knowing, because it is the kind that decides real cases. In the favorable worked example, the team tells the truth only if its record turns into usable expertise more than 17 times out of 18. Disclosure needs a reward that arrives reliably, not one that merely exists.
What the research does not show is that ACM does any of this better by being ACM. In every model, a capable local arrangement with the same records, rules and rewards does exactly as well. Any gain comes from a named rule: the reward, the promise and the diagnosis. So the question for a manager is which of those rules to switch on.
The framework
Who gains from telling?
When a failure comes in, ask two questions. Does telling help the team that found it? Does telling help the company? Two yes-or-no answers give four cells, and each cell has a different treatment.
Chapter 5 · Framework 7
Teams disclose what helps them. The lesson that helps only the company is ACM's real job.
Top left, orphan lessons. The stamp failure sat here. Dispatch lost compute and attention by telling. Warranty, Support and a dozen later projects would have saved a week. Nothing in the old setup paid the teller. This cell is why I think a company needs someone whose job is to move a lesson from a team that holds it to a team that needs it, and to make the move worth its while.
Top right, free disclosure. A team posts a useful evaluation set because it helps their own next release. Do nothing but make it easy to find. Most of the leadership attention should stay away from this cell.
Bottom left, noise. A log of a thousand routine runs helps nobody. Do not collect it. A record nobody will read is a cost with a filing system.
Bottom right, the protection trap. This is where “everything is P1” comes from, and where a team keeps quiet about a failure that would end its own project. The team benefits and the company does not. It is the cell to design against, and chapter 4 gave the other half of the answer: fund the question, not the label.
The four cells are my reading aid. They are built from the research's incentive results, but the matrix itself is not a theorem, and the placement of a real failure in a cell is a judgment call that you should argue about in the room.
What Maya did
Reward the lesson. Do not rescue the project.
Maya's first rule was bad, and I want to show you why, because the next manager will write the same one.
She announced a “failure notes” program. Teams would be recognized for every failure note they filed. In the first five weeks, 61 notes arrived. Four were usable. The rest were one-line apologies: “model was slow”, “bad scan”. Two teams filed several each week. They had found the cheapest way to be recognized. This is exactly what the research predicts about raw volume. Filing records is not teaching.
Her second rule was better, and it is the one I would copy.
- A note has four lines. What failed. How to recognize it. What was tried. What it would cost another team to avoid it.
- A note counts only when it has been tested. A reviewer reproduced it, or another team used it. Filed is not counted. Used is counted.
- A counted note earns one thing. The right to send one request of up to 150 GPU-hours through the fast lane, instead of the usual 50, good for 90 days. It is a seat, not a pass. The request still has to be sound, and the old project is still judged on its merits.
- The team's name stays on the note. Credit is cheap for the company and valuable to the people who get it.
- Helen signs the promise into the charter. If the sponsor can cheaply take a promise back, teams stop believing it. Helen wrote hers where her peers could read it, so breaking it would be visible.
Dan's team was the first to file. They filed the stamp failure with a one-page fix, and Warranty reproduced it in an afternoon. Dispatch earned the first fast-lane seat. A month later they used it for a modest request, which was approved by a quick review. Dan's note became the most reused document in the program's first year.
Conviction · my judgment, not a proof
Reward the lesson, never the project's survival.
- Why I hold it
- If admitting a failure also ends the project, the team's best choice is to hide it. If admitting a failure keeps a doomed project alive, you will pay to fund failures. The only reward that works for both is credit and a bounded chance at new work, with the old work still judged on its merits.
- What would change my mind
- If I saw teams filing diagnoses to earn the seat and then reusing it for work of poor quality, I would shrink the reward or make the review stricter. If fast-lane seats stopped getting used, I would conclude the reward is too small to matter.
- What the research says
- Lesson: Making the truth worth tellingLesson: What a record of failures is worth
Conviction · my judgment, not a proof
Count the notes that other people used, never the notes that were filed.
- Why I hold it
- A filed note is cheap to produce and easy to game. A used note proves that someone else learned from it. The first Hartwell program produced 61 notes and 4 useful ones. That ratio was the lesson.
- What would change my mind
- If a company could cheaply verify quality without anyone reusing the note, I would count the notes at review time. I do not know of a cheap way.
- What the research says
- Lesson: What a record of failures is worth
Conviction · my judgment, not a proof
A promise the sponsor can break cheaply is not a promise.
- Why I hold it
- Teams read the sponsor's incentives as clearly as the sponsor does. If breaking the promise costs nothing, they assume it will be broken. So put the promise in writing, name the person who owns it, and make the cost of breaking it visible to that person's peers.
- What would change my mind
- If teams disclosed freely under verbal promises for a year, I would drop the formality. I expect they will not, but that expectation is judgment, and the cheapest of my convictions to test.
- What the research says
- Lesson: Making the truth worth telling
The trap inside the cure
The ratchet: better options can make people quieter.
Sales managers know the pattern. Beat your quota and next year's quota goes up. The sensible response is to beat it by a little. Teams do the same thing with information. A team that shares a diagnosis may be teaching the company to decline requests like its own. If you later add a smarter way to choose among requests, the team's reasoning does not stop at today's request.
This is the result I find hardest to carry into practice, and the one I keep returning to. The company improves its judgment, and the team becomes less willing to supply the facts that judgment depends on. You cannot see it from inside. The numbers look fine until you notice the failure notes have stopped.
I do not have a clean rule for it. I have a habit. When Maya introduces a better way to decide, she asks one question first: does this change what a team gets for telling us something? If the answer is yes, she checks the reward before she changes the process.
The same logic explains why the answer cannot be purely technical. A better test or a faster review does not make anyone more honest. The change that does is to what honesty earns.
Where this fails
What I would not claim.
Everything in this chapter is my judgment about incentives, and I mark it once here so I do not have to keep saying so. The research tests these incentives in small, worked games with invented numbers. It does not say how often a real team hides a failure or how much a bounded seat is worth to a real manager. I think the fast-lane seat works because it is cheap for the company and it matters to the team. That is a prediction, and Hartwell's year is a story, not a trial.
There is also a limit I can state flatly. Telling the truth cannot be rewarded when the truth is a person's fault. A diagnosis about a model is one thing. A diagnosis about a careless colleague is another, and no seat in a fast lane will make someone volunteer it. Keep failure notes about things, not about people.
Finally, a tidy caution. A note that is accurate in one setting can mislead in another. Dispatch's fix for the stamp problem helped Warranty and would have hurt a team reading modern, unstamped scans. Every note needs a line that says where it stops applying. That is the fourth line, and the one most often left blank.
What to do
On Monday
- 01Ask each team lead for a count of the times people overrode the model last month. Look for a cliff. Ask about the cliff, kindly.
- 02Write your failure-note template in four lines: what failed, how to recognize it, what was tried, what it would cost another team to avoid it, plus a line on where it stops applying.
- 03Decide the one reward for a tested note and write it on a page. A fast-lane seat for new work is a good start. Do not promise to protect the project.
- 04Name the person who owns the promise, and tell their peers. Write down what happens if the promise is broken.
- 05Pick one request in your queue that was marked urgent. Ask what the team would lose by telling you what is wrong with it.