Lesson 6 of 10

Making the truth worth telling

Why would a team admit a dead end, and what has to be true for a promise to change that?

The problem

The week nobody wants to write up

The Field Repair Dispatch team spent a week finding out why a vendor tool kept misreading handwritten service logs. The answer is useful: the tool was never trained on handwriting, and no amount of prompt tuning fixes that. Written up in two pages, it would save the next team that tries the same tool a week of its own.

Dispatch has three good reasons not to write it up. The two pages cost time it does not have. The benefit goes to other teams. And the worst reason: an honest account of a dead end might make Dispatch's next request look weaker. If admitting failure costs you the next slot, silence is the rational choice.

Lesson 5 showed what such records are worth when they exist. This lesson asks what it takes for them to exist. The obvious answer is a promise: "share your diagnosis, and you will get a fair shot at the next slot." But a promise is only as good as the moment it must be kept. When the next slot comes up and a ready piece of outside work looks more valuable, will the sponsor still honor it? Teams can see that moment coming long before it arrives.

The intuition

Rewards, promises and the future

Without a shared arrangement, each team keeps its own lessons and nobody asks for them. Nothing can be broken because nothing was promised. Dead ends get paid for again and again, but quietly, and nobody's next request is at risk.

With a shared arrangement, the sponsor can offer a deal: a useful diagnosis earns a credit — a chance at the next slot for a new project that must pass its own check. Not the old project; the credit follows the people, not the dead end. That separation matters. It lets the company stop paying for a failed approach without teaching the team that admitting failure ends its access to new work.

Three frictions decide whether the deal works, and each needs game theory, because each is about what one side expects the other to do.

The review must tell useful diagnoses from useless ones. If it rewards every submission, you get more paperwork, not more lessons.
The promise must survive temptation. When the credit comes due and outside work is worth more than the new project, a sponsor free to change its mind will. Teams anticipate that, and the promise does nothing.
Tomorrow's choices feed back into today's honesty. A team deciding whether to share today asks what the shared knowledge will do to its own future. If better judgment closes a comfortable route to funding, the team has a reason to keep quiet — even when the company would gain.

The first two are a commitment problem: what a decision-maker can credibly promise. The third is a repeated game: the same players meet again, and today's move changes tomorrow's position.

The model

A credit, a promise and two rounds

1. Is the credit worth earning?

Two teams each hold a diagnosis that is either useful or not, and each knows which. Preparing a submission costs the team effort $\kappa$ . Review accepts a useful diagnosis with chance $\alpha_{\text{useful}}$ and a useless one with chance $\alpha_{\text{useless}}$ . An accepted diagnosis earns a credit; to use it, the team's new project must pass its own gate, with chance $p_{\text{new}}$ . The slot is worth $B$ to the team that gets it, and if both teams hold a usable credit, a coin decides. If the sponsor will honor credits, a team's gain from submitting is:

Definition · gain from submitting, if credits are honored

\text{gain from submitting} \;=\; \alpha\,p_{\text{new}}\left(1-\tfrac12\,\bar r\right)B \;-\; \kappa

In words: the chance the diagnosis is accepted, times the chance the new project passes its gate, times the chance of not having to share, times what the slot is worth, minus the effort. Here r̄ is the chance the rival also holds a usable credit.

We want useful diagnoses submitted and useless ones kept back, whatever the rival does. The worst case for a useful team is a rival that certainly holds a credit too; the best case for a useless one is a rival that holds none. So it is enough that:

Checked in Lean · only useful diagnoses come in

B\,\alpha_{\text{useless}}\,p_{\text{new}} \;<\; \kappa \;<\; \tfrac12\,B\,\alpha_{\text{useful}}\,p_{\text{new}}

In words: the effort must be small enough that a useful diagnosis is worth submitting even when the slot may be shared, and large enough that a useless one is not worth submitting even when it would win alone.

In the worked case, half the diagnoses are useful and review tells them apart perfectly ( $\alpha_{\text{useful}} = 1$ , $\alpha_{\text{useless}} = 0$ ). Half of new projects pass a sound gate ( $p_{\text{new}} = 1/2$ ), and a passing one is worth 3 to the company. The slot is worth 4 to a team, and preparing a submission costs it 1/2. The window runs from 0 to 1, and 1/2 sits inside it. If review accepted useless diagnoses as readily as useful ones, no effort level could separate them.

2. Will the promise be kept?

Now the hard moment. A credit has come due. The sponsor can honor it, giving the slot to a new project worth $V_{\text{new}}$ on average, or break it and give the slot to ready outside work worth $V_{\text{outside}}$ . In the interesting case the outside work is worth more, otherwise there is no temptation. Breaking the promise has a cost $z$ to the sponsor — lost trust, a rule it must formally overturn, a commitment written into the scheduler.

Checked in Lean · when the promise holds

\text{keep the promise} \iff z \;>\; V_{\text{outside}} - V_{\text{new}}

In words: the sponsor keeps the promise only if breaking it costs more than what the switch would gain. If the two are equal, the sponsor is indifferent and the model makes no prediction.

With outside work worth 4 and a new project worth 3, breaking the promise must cost more than 1. Below that, the sponsor would break it, teams know it, and nobody prepares a diagnosis. The arrangement then only adds its setup cost: the company earns 15/4 instead of 4.

Above it, the promise is believed. One useful diagnosis arrives on average, worth 2 to the company. Each team holds both a credit and a passing new project with chance 1/4, so the slot moves from outside work to a new project with chance 7/16 — and every such move gives up 1 unit. Counting every cost:

Checked in Lean · the whole ledger

4 + 2 - \tfrac14 - \tfrac14 - \tfrac14 - \tfrac14 - \tfrac{7}{16} \;=\; \tfrac{73}{16} \;>\; 4

In words: outside value, plus the useful knowledge, minus preparation, review, running the credits and setting up the arrangement, minus the value given up when a credit displaces outside work. The arrangement beats doing nothing by 9/16.

Raise the value of the outside work and two things happen. The promise gets harder to keep, because the temptation grows. And even a promise that is kept gets more expensive, because each honored credit gives up more. Working the ledger by hand, the kept promise beats having no arrangement only while outside work is worth less than 3 + 16/7, about 5.3.

Lesson 6 · Figure 1

A promise changes behavior only when breaking it costs more than the switch would gain.

Shown at What breaking the promise costs the sponsor: 2.00 illustrative units; What the ready outside work is worth: 4.00 illustrative units. Turn on JavaScript to move the values.

Above 1, the promise is believed. Teams with a useful diagnosis share it, the others do not, and the company earns about 4.56 against 4 without the program.

Company value with the promise: 73/16
Company value without it: 4
Cost of breaking needed to be believed: more than 1

Two teams, one future slot. Below the threshold teams keep their diagnoses; above it only useful ones come in. Lean checks 73/16, 15/4 and 4 at outside value 4, and the threshold rule in general. Move the outside value past about 5.3: the promise is still kept, and it no longer pays.

Model result · checked in Lean

3. What sharing does to the sharer's future

The last friction needs two rounds. In round one a team whose evidence does not support its request can claim the slot anyway or file its diagnosis. A filed diagnosis can move the shared memory into an expert state, where the reviewer judges better. In round two the sponsor picks the next project, and that choice depends on whether the memory is expert. A team's advantage from filing rather than claiming splits cleanly into today and tomorrow:

Checked in Lean · today plus tomorrow

\text{advantage of filing} \;=\; \Delta_{\text{today}} \;+\; \big(\pi_{\text{file}} - \pi_{\text{claim}}\big)\big(V_{\text{expert}} - V_{\text{start}}\big)

In words: what filing gains the team now, plus how much filing raises the chance that the memory becomes expert, times how much the expert state is worth to this team compared with the starting state.

In the favorable case, today says claim: claiming earns the team 1/4, filing costs it 1/64, so $\Delta_{\text{today}}$ = −17/64. Tomorrow says file: filing guarantees the expert state, worth 25/16 to the team next round, while claiming leaves it to the rival's choice, worth 41/32 on average. Tomorrow's term is 9/32, today's is −17/64, and filing wins by 1/64. Across both teams a diagnosis arrives 3 times in 4, and the company earns 131/16, about 8.2, over both rounds.

That 1/64 margin is thin. If a filed diagnosis transfers to the next decision only with chance $\tau$ , tomorrow's term shrinks to (9/32) $\tau$ , and filing stays best only while $\tau$ is above 17/18. And the sign of the last bracket is not guaranteed. If the expert state is worth less to the team — because a sharper reviewer will turn down the comfortable projects it used to win — then the future pushes against honesty. Teams claim, memory never becomes expert, and the company earns 3.

Here is the most surprising result in this series. Keep every old option and add one more: a new round-two project that, once the memory is expert, is worth more to the company than the old one. The sponsor will pick it when the time comes. But it pays the teams less: their expected value in the expert state falls to 13/16, below the 1 they get in the starting state. Teams see that coming. They stop filing in round one, the expert state is never reached, and the company's value falls from 131/16 to 3. A strictly better option, freely chosen, made everyone worse off.

Lesson 6 · Figure 2

Shown at What the new option is worth to the company if it works: 9.00 illustrative units. Turn on JavaScript to move the values.

Above 8, the sponsor would switch to the new option once expertise exists. It pays the teams less, so they stop filing diagnoses, expertise never forms, and the company earns 3 instead of about 8.19.

Company value, both rounds: 3
With the old menu only: 131/16

Model result · checked in Lean

What each letter stands for

$B,\ \kappa$: what the slot is worth to the team that gets it; the effort of preparing a submission (as in lesson 4)team's illustrative units
$\alpha_{\text{useful}},\ \alpha_{\text{useless}}$: the chance review accepts a useful diagnosis, a useless oneprobability
$p_{\text{new}}$: the chance a team's new project passes its own gateprobability
$\bar r$: the chance the rival also holds a usable creditprobability
$V_{\text{new}},\ V_{\text{outside}}$: what a passing new project is worth on average; what the ready outside work is worthillustrative units
$z$: what breaking the promise costs the sponsorillustrative units
$\Delta_{\text{today}}$: what filing rather than claiming gains the team nowteam's illustrative units
$\pi_{\text{file}},\ \pi_{\text{claim}}$: the chance the memory becomes expert if the team files, if it claimsprobability
$V_{\text{expert}},\ V_{\text{start}}$: what the expert state, the starting state, is worth to this team next roundteam's illustrative units
$\tau$: the chance a filed diagnosis transfers to the next decisionprobability

What Lean checks

Lean is a program that checks every step of a proof. Each result below is exactly as strong as its statement.

Checked in Lean

If credits are honored and the effort sits inside the window, a team with a useful diagnosis strictly gains by submitting and a team with a useless one strictly loses, whatever the rival does.

selective_dominance in AccessCommitment.lean

Show the Lean statement

theorem selective_dominance (g : Game) (hv : Valid g)
    (huseful : g.burden < g.reward*g.usefulEligibility/2)
    (hnonuseful : g.reward*g.nonusefulEligibility < g.burden)
    (r : Strategy) : 0 < submitUtility g true r ∧ submitUtility g false r < 0

Here "eligibility" is acceptance times the gate's pass chance. identical_review_cannot_strictly_separate shows that if review accepts both kinds equally often, no effort level does it.

Checked in Lean

The sponsor strictly prefers keeping the promise exactly when breaking it costs more than the outside work's advantage over the new project.

honor_strict_threshold in AccessCommitment.lean

Show the Lean statement

theorem honor_strict_threshold (posterior outside breach : Rat) :
    outside-breach < posterior ↔ outside-posterior < breach

This is an algebraic rearrangement; its value is in naming the right comparison. credible_selective_implementation joins it to the result above: under both conditions, "useful teams submit, others hold back" is the only equilibrium and the sponsor strictly keeps its word. primary_value checks 73/16, no_promise_values checks 4 and 15/4, and mandatory_cost_losses checks the two losing cases below. What Lean cannot check is whether a real organization has the authority, budget and staying power to make breaking a promise costly.

Checked in Lean

A team's total advantage from one action over another equals today's difference plus the change in the chance of reaching the expert state times what that state is worth to the team.

two_state_deviation_gain in RepeatedGame.lean

Show the Lean statement

theorem two_state_deviation_gain
    (truthNow otherNow truthP otherP expert novice : Rat) :
    (truthNow + truthP*expert + (1-truthP)*novice) -
      (otherNow + otherP*expert + (1-otherP)*novice) =
    truthNow-otherNow + (truthP-otherP)*(expert-novice)

future_rent_reverses_current_truth shows the bracket can be negative and overturn a positive present. learning_retention_threshold checks the 17/18 boundary, which concerns one candidate plan's margin, not which equilibrium occurs.

Checked in Lean

Every old round-two option stays available, one new option is added, and the company's two-round value under the resulting equilibrium is strictly lower than before.

genuine_menu_expansion_equilibrium_loss in RepeatedGame.lean

Show the Lean statement

theorem genuine_menu_expansion_equilibrium_loss :
    (∀ g, g ∈ oldExpertMenu → g ∈ expandedExpertMenu) ∧
    sponsor (training false) false true expandedTerminalSponsor pooling pooling <
      sponsor (training false) false true (terminalSponsor false) truth truth

This does not contradict planning: a planner who could commit to a complete plan in advance would never be hurt by an extra option. The loss comes from deciding later, after teams have already reacted. Behind these concrete results sits sequential_dominates_every_policy: if no single-step deviation pays at any reachable point, then no complete plan of deviations pays either. Lean checks the stage-by-stage equilibria for the worked case; that they are the only ones is checked by the tested model code across all 81 pure plan pairs per stage, and plans that randomize were not studied.

What it means

Reward the lesson, not the project. The credit pays for a useful diagnosis with a chance at a new, separately checked project. It never revives the old one. In the deliberately bad version where the promise protects the old project, the company funds work it already knows is negative and ends at −1. Better disclosure, worse allocation.

A promise is a cost structure, not a sentence. "We will reward honesty" changes nothing if breaking it is free when the moment comes. What changes behavior is making the switch costly or impossible: a rule written into the scheduler, a budget held in escrow, a public commitment that would be embarrassing to overturn. Each of these has its own setup cost, and the model counts it.

A credible promise can still be a bad deal. The model keeps the losing cases:

The outside work is too valuable. At 8, the kept promise earns 109/16, about 6.8, against 8 with no arrangement. The diagnoses arrive; they are not worth what they displace.
The process eats the gain. With a setup cost of 1 instead of 1/4, the arrangement earns 61/16, below 4.
Review rewards the wrong thing. If every submission is accepted, everyone submits; the company pays for useless diagnoses and earns 5/2.
The new gate is too lenient. If bad new projects pass half the time, a passing project is still worth 1 on average, but bad projects get the slot 13/64 of the time and the company earns 203/64, below 4.

Honesty depends on the future you are building for the honest. If better shared judgment makes a team's own future worse, that team has a reason to keep quiet, and no amount of exhortation fixes it. Design recognition, access and accountability together: the people who make the memory smarter must not be the first ones it turns down. Economists call the general version the ratchet effect: revealing what you can do can worsen your future terms.

More options are not free. For a single planner who can commit in advance, an extra option cannot hurt. In a game, it can. The sponsor's freedom to pick the better project later is exactly what made sharing unattractive now. Sometimes the right move is to commit to a route and give up the right to re-optimize it, and a sponsor who quietly keeps that right should expect teams to price it in.

The shared route can also lose by force. Requiring the favorable two-round arrangement at an extra cost of 7 leaves 131/16 − 7 = 19/16, below the 2 a safe route would earn (real_model_mandatory_cost_loss). If teams could choose, they would take the safe route. Cost, compulsion and incentives: these are the three ways a shared route loses, and this lesson has shown the last one in its purest form.

What the model leaves out. Two teams, independent evidence, honest reviewers, no collusion, no reputations, no appeals. Payoffs and transfer chances are assumed. A capable local or federated arrangement offering the same bargain gets the same result; the research index states that once as a design rule. What these models identify are the conditions a real arrangement must meet — not proof that any real arrangement meets them.

Public analogy

NASA Aviation Safety Reporting System and the FAA · 1975

Aviation built a version of this bargain in the 1970s. The FAA instituted its voluntary reporting program in April 1975, with NASA running the confidential reporting system. Under the FAA's advisory circular, a pilot who files a timely report of an inadvertent violation — not criminal, not an accident, no recent enforcement history, report sent within 10 days — faces neither a civil penalty nor a certificate suspension, even though the FAA may still find that a violation occurred. NASA removes identifying details from reports.

Read as an illustration of a promise made costly to break: the protection is a published regulatory rule, administered by a separate agency, with its conditions stated in advance. It is not evidence about AI teams.

ASRS: Immunity policies (citing FAA Advisory Circular 00-46F), 2026-10-21

Next: when the people change too

In lessons 4 to 6 the teams could only report what they knew; what they could do never changed. But the deepest way a shared capability compounds is through people: someone learns from the specialist, then teaches the next person, then no longer needs the help at all. Lesson 7 asks how support should step back as people learn, and when it should not.