Lesson 2 of 10

Better expertise, worse fit

When does a more expert shared route lose to a team’s own work?

The problem

The expert who reads the wrong contract

A shared route can be more expert than a team and still do that team's job worse. This lesson finds the exact point where that happens.

Go back to the pump company. Suppose it built a shared route for reading scanned documents. The people who run it have now seen thousands of pages from both teams. They know which scanners smear, which fonts the reader confuses and which tests catch a bad page. They make fewer slips than either team would alone.

But most of the pages they practiced on were manuals and service logs, because Field Repair Dispatch sends more of them. Warranty Claims reads something different: contracts, with exclusions buried in annexes. A shared reader tuned on manuals may handle a contract well on average and still be aimed slightly wrong for it. It may, for example, treat an exclusion as a footnote. Warranty cannot tolerate a missed exclusion. Dispatch, where a dispatcher checks every route anyway, can tolerate an occasional slip.

So which route should Warranty use: the shared reader, with more expertise and a worse fit, or its own, with less expertise and a perfect fit? In lesson 1 this was hidden inside one number, the saving per team $d - a$ . Here we open it up.

The intuition

Two kinds of error

Every error a document reader makes is one of two kinds. A slip is random: a smudge misread, a page skipped. Sometimes the slip goes one way, sometimes the other. Experience makes slips rarer. A mismatch is systematic: the method aims at slightly the wrong target, every time, in the same direction. More experience with the wrong target does not fix it. Only tuning the method to this job does.

Now the trade is easy to state. The shared route has more expertise, because it practices on every team's work: fewer slips. It also serves several jobs with one method: more mismatch for the job that differs most. The local route has the opposite profile: a perfect aim, more slips. Centralizing buys expertise and pays in fit. Keeping work local buys fit and pays in expertise.

Which effect wins? It depends on the exchange rate between slips and mismatch, and intuition is bad at exchange rates. We need a small model that puts both kinds of error on one scale. There is still only one decision-maker here, so this is a comparison, not yet a game. But the model also tells us what practice is worth, which lesson 3 needs to decide how much practice to buy.

The model

One scale for slips and mismatch

Measure expertise $x$ as the chance that a job is done without a slip, between 0 and 1. When a slip happens, it has size $\sigma$ and is equally likely to go either way. On top of that, the method's mismatch $b$ shifts every answer by the same amount. So each job ends with an error of $b - \sigma$ , $b$ or $b + \sigma$ , with chances $(1 - x)/2$ , $x$ and $(1 - x)/2$ .

How bad is an error? We use half the squared error, so that big errors hurt more than in proportion: missing an exclusion by a lot is far worse than missing it by a little. Average over the three outcomes and something tidy happens: the two kinds of error separate.

Checked in Lean · finite_loss_decomposition

E(b,x)=\tfrac12\left[\,b^2+\sigma^2(1-x)\,\right]

In words: expected error is a mismatch part plus a slip part. Expertise shrinks the slip part; it never touches the mismatch part.

Write the shared route's mismatch and expertise as $b_S$ and $x_S$ , and the local route's as $b_L$ and $x_L$ . The shared route makes smaller expected errors exactly when:

Checked in Lean · common_local_threshold

E_S<E_L\iff b_S^2-b_L^2\;<\;\sigma^2\,(x_S-x_L)

In words: the extra mismatch the shared route brings, squared, must be smaller than its expertise edge, priced at the size of a slip squared.

The exchange rate is $\sigma^2$ . When slips are large, as with smudged scans and handwritten logs, each point of expertise is worth a lot, and the shared route can afford more mismatch. When slips are small, expertise is worth little, and almost any mismatch hands the job back to the local team.

Expertise is built by practice. We use a simple learning curve. Start at expertise $x_0$ . After $h$ blocks of practice, expertise is

Assumption · bounds checked in Lean (practice_bounds)

x(h)=\frac{x_0+h}{1+h}

In words: each block closes part of the remaining gap to full expertise, and each block closes less than the one before.

This curve has one flaw worth knowing. Give it the same practice in two halves and it reports more learning than in one go, because $(1 + h/2)^2$ is larger than $1 + h$ . So we count practice once per episode, never split. A different curve, $1 - e^{-\eta h}$ for a learning speed $\eta$ , has no such flaw, but its answers are not exact fractions, and exact fractions are what let a computer check proofs without rounding. Which curve real teams follow is unknown. No learning speed has been measured.

What each letter stands for

$x$: Expertise: the chance a job is done without a slipbetween 0 and 1
$x_0$: Expertise before any practicebetween 0 and 1
$\sigma$: Size of a sliperror units
$b$: Mismatch: how far the method aims from what this job needserror units
$E$: Expected error: half the squared error, averaged over slipssquared error units
${}_S,\ {}_L$: The shared route and the local routelabels
$h$: Blocks of practicewhole blocks
$\eta$: Learning speed in the alternative curveper block

Lesson 2 · Figure 1

Past a certain mismatch, the team's own work wins.

Shown at Mismatch of the shared route on this job: 0.40 error units; Expertise of the shared route: 0.75 0 to 1; Expertise of the local route: 0.25 0 to 1. Turn on JavaScript to move the values.

Below a mismatch of 0.71, the shared route wins: its extra expertise removes more slips (0.25) than its mismatch adds (0.08).

Shared route expected error: 0.21
Local route expected error: 0.38
Largest mismatch the shared route can carry: 0.71

The local route fits its own job, so its error is flat. The shared route's error grows with the square of its mismatch. Where they cross, the expertise edge exactly pays for the mismatch. Slip size $\sigma = 1$ ; local mismatch 0.

Model result · checked in Lean

At the starting position, the shared reader is three times more likely to avoid a slip (expertise 3/4 against 1/4) and aims a little off (mismatch 0.4). It wins. Slide the mismatch right. At about 0.71 the two routes tie. Push on to 1 and the shared route's expected error is 5/8 against the local route's 3/8. Better expertise, worse fit. Then narrow the expertise gap and watch the break-even point move left: the less the shared route knows beyond the team, the less mismatch it can afford.

Lesson 2 · Figure 2

The noisier the work, the more mismatch expertise can pay for.

Shown at Size of a slip, $\sigma$ : 1.00 error units; Mismatch the local route keeps on its own job: 0.00 error units. Turn on JavaScript to move the values.

With slips of size 1.00, the warranty case sits above the curve: its mismatch costs more than an expertise edge of 0.5 saves, so it stays local.

Mismatch the shared route can carry at edge 0.5: 0.71
Expertise edge the warranty case needs: 1.00

Each curve is the break-even mismatch, the square root of $b_L^2 + \sigma^2(x_S - x_L)$ . Below the curve, the shared route wins; above it, the local route does. The gold point is the warranty case of figure 1: an expertise edge of 0.5 and a mismatch of 1.

Model result · checked in Lean

With slips of size 1, the warranty case sits above the curve: it stays local. Now make the documents noisier. At a slip size of about 1.41 (the square root of 2) the curve passes through the warranty case, and beyond it the shared route carries the job despite its mismatch. Nothing about the shared reader changed. What changed is how much a slip costs, and so how much expertise is worth.

What Lean checks

Lean checks the error model with exact fractions, line by line. The source file writes bc and bl for $b_S$ and $b_L$ , xc and xl for $x_S$ and $x_L$ , and s for $\sigma$ .

Checked in Lean

Averaging half the squared error over the three outcomes gives exactly the mismatch part plus the slip part, $\tfrac12 [b^2 + \sigma^2(1 - x)]$ .

finite_loss_decomposition in ExpertiseScarcity.lean

Show the Lean statement

theorem finite_loss_decomposition (b s x : Rat) :
    finiteLoss b s x = (b*b+s*s*(1-x))/2

Checked in Lean

The shared route has the smaller expected error exactly when $b_S^2 - b_L^2$ is less than $\sigma^2(x_S - x_L)$ .

common_local_threshold in ExpertiseScarcity.lean

Show the Lean statement

theorem common_local_threshold (bc bl s xc xl : Rat) :
    finiteLoss bc s xc < finiteLoss bl s xl ↔
    bc*bc-bl*bl < s*s*(xc-xl)

Checked in Lean

A concrete case where more expertise loses: expertise 3/4 with mismatch 1 makes a larger expected error than expertise 1/4 with no mismatch (5/8 against 3/8).

common_mismatch_reversal in ExpertiseScarcity.lean

Show the Lean statement

theorem common_mismatch_reversal :
    finiteLoss 1 1 (3/4) > finiteLoss 0 1 (1/4)

Checked in Lean

One step of the learning curve keeps expertise between 0 and 1, including the general form where part of the old expertise is forgotten first.

practice_bounds in ExpertiseScarcity.lean

Show the Lean statement

theorem practice_bounds (x q d : Rat)
    (hx : 0 ≤ x) (hx1 : x ≤ 1) (hq : 0 ≤ q)
    (hd : 0 ≤ d) (hd1 : d ≤ 1) :
    0 ≤ practice x q d ∧ practice x q d ≤ 1

In that last statement Lean's own letters differ from ours: its $q$ is the practice in one episode and its $d$ the share of old expertise kept. On this site, $q$ and $d$ keep their lesson 1 meanings.

These are exact algebra. What Lean does not check is whether real AI errors look like this. Real slips are not three-valued. Expertise surely does more than reduce slips; here we gave it exactly one job, on purpose, so that it gets no unexplained credit for speed or quality. Lean does not check that the learning curve is right, that practice transfers between teams' documents, or that mismatch can be measured. It also proves nothing about many rounds of learning: only that one step stays in bounds.

What it means

Expertise and fit are different currencies

A shared route earns its place on noisy work where slips are costly. On clean, well-defined work, a team's own tuned method can beat a more experienced shared one. Asking "who is more expert?" is the wrong question. Ask "how big are the slips here, and how far off is the shared method for this job?"

Mismatch is squared, so it is forgiving when small and brutal when large

A shared method that is almost right for a job costs almost nothing extra. One that is clearly wrong costs a lot, quickly. This is why one size rarely fits the team with the most unusual work.

The fix is a shared base plus local tuning

The shared route keeps the expertise; each team pays to bring its own mismatch down. That tuning is the adaptation cost $a$ from lesson 1, and it is why a shared route must budget for it rather than promise one model for everyone. The model also shows why Warranty's skepticism is reasonable: for its job, the shared reader must prove its mismatch is small, not just that its people are experienced.

Practice has a value we can now write down

Over the future jobs a shared route will serve, the error it saves by learning is a mismatch-free gain: $\tfrac12 \sigma^2(1 - x_0)$ times the size of that future work, times how much of the gap the practice closes. Lesson 3 uses exactly this as the value of shared practice.

When the shared route loses

It loses when its mismatch for this job is large, when slips are small anyway, when the local team is nearly as expert, or when local tuning to remove the mismatch is not funded. None of these is a failure of the people in the shared route. Each is a property of the job.

Honest limits

The numbers are chosen, not measured. The model has one job per route, one kind of slip and fixed expertise during the job. It does not include learning from the job itself, or teams that copy each other's fixes. Those belong to later lessons, and to the tests a real company would run.

Next: how much practice to buy

More expertise helps, and practice builds it. But practice is not free. Every block of specialist time or compute spent on shared practice is a block not spent on today's work. Lesson 3 asks how many blocks to fund, when the first block can lose money while the third makes it, and whether the practice should go to one deep expert or be spread across two.