Extracting the judgement that lives in one person's head

When a senior person leaves, the thing that walks out is usually called experience. The word is the reason most extraction projects disappoint. Experience sounds like a volume — something you could pour out by recording someone talking for a day. What actually leaves is a set of decisions taken at particular forks under particular conditions. The fork can be recorded. The person cannot.

The unit is a fork, not a practice

Write each entry as a fork. Five parts: the condition that opens it, the evidence the person looked at, the road taken, the road rejected and the reason it was rejected, and the event that would make the choice wrong.

The last two are what make the entry knowledge. A record of what the experienced person does is a conclusion with its premises removed, and a conclusion without premises cannot be checked — only believed. That is why a library of practices is worthless even when every line in it is accurate: accuracy is precisely the property you cannot verify about it from the outside. A reader who cannot see the rejected road also cannot tell a rule from a habit, and the difference only shows up the first time the conditions move.

Five annotations

Every entry carries five labels. They are cheap to add now and expensive to reconstruct later.

• Basis. The evidence used, not the conclusion reached. "She compares the printed spec against the last complaint" survives a change of product; "she knows which spec is wrong" does not.
• Scope. Which products, customers, regions or seasons it holds for. An entry that is true for one client and written as if it were general is worse than no entry, because it will be applied confidently somewhere else.
• Status. Current, unconfirmed, or retired. Retired entries get marked, not deleted: deleting them loses the reason the replacement exists, and the old rule survives in the next person's head anyway.
• Date. When it became true, tied to a version of the source — a catalogue edition, a policy revision, a supplier change. A date with no version is decoration.
• Recheck trigger. The event after which someone must read the entry again. This is the annotation that makes a document age visibly instead of silently.

The five are what separate a judgement from an anecdote. Without scope and recheck trigger, the entry looks equally valid forever; that is the same failure mode as a shadow spreadsheet that outlived the process it was built around.

Three relations, and one action each

When a new entry arrives, it stands in one of three relations to what is already there, and each has exactly one correct action.

Supplement. The new entry covers a case the old one did not. Both stay, and their scopes must not overlap; if you cannot draw the boundary in one sentence, you do not have a supplement, you have the next case.

Replacement. The old entry is retired by the new one. Mark who retired it and on what evidence, and keep it retrievable for a period, because the people who learned it need to see why it changed. Silent replacement is how an organisation ends up with two rules in circulation and no memory of the switch.

Conflict. Both entries are current and they disagree. This is not a documentation problem, it is a live risk: a retrieval system will choose one, and it will choose silently. Record who decides and by when. Until then both are marked unconfirmed and the question routes to a person. Writing a plausible answer into the gap is how a knowledge base starts producing confident errors.

Four ways to get it out

The methods are few. Record a role, transcribe it, and extract the decisions rather than the narration. Trace exceptional events backwards to the decision that should have caught them. Walk the process with the person and compare it against the written one. Ask for a handover log from someone who is about to leave.

Choose by the shape of the judgement, not by preference. Frequent and visible judgements yield to a walkthrough. Rare and decisive ones, the ones that matter most, are only visible in the exceptions, so they need backward tracing. A person about to leave gets one pass, so the method is the handover log whether or not it is the best fit. Recording a role also means handling customer data and internal calls, which is a permission question before it is a method question; settle it against the security boundary before the first session, not after the transcript exists.

The failure you will see on site

The library fills with one-line practices and the assistant recites them fluently in situations where the condition no longer holds. The signal is specific: the experienced person reads an answer and says "that's different" — and then cannot say why, quickly, in words. The difference was never written, so it exists only as a feeling of wrongness, which is not something a colleague can inherit.

The second signal is slower. The answers are right for a quarter, until a catalogue changes, and then they are wrong with no error message and no complaint from the system. Every entry that has no recheck trigger fails this way eventually.

The test

Take an entry and change one condition in it: a model number, a department, a quarter. Ask a reader who was not in the room whether it still holds. If they cannot answer, the entry is a practice and not a judgement. Then ask them which road was rejected. If they cannot name it, they can only imitate the person who left.

Before the extractor is handed over, the entries have to survive both questions, and the runbook that explains how entries get added, retired and rechecked has to be readable by someone who never met the person who supplied them.