Inspectable reasoning: when the regulator asks 'show your work'

Every credit-risk team has sat in a model validation meeting where the validator opens a decision tree - a CART or a gradient-boosted ensemble - and walks the splits one by one. Income above threshold A: go right. Tenure below threshold B: go right again. Five splits later the model arrives at a probability of default. The tree is not a metaphor; it is an inspectable object. The validator can point at a node, ask why the split was placed there, and trace the path that produced the final number. SR 11-7 ongoing monitoring is built on exactly this kind of artefact.

Extended-reasoning models produce an artefact with the same shape. The model expands a search tree of candidate intermediate steps, scores them, prunes the dead branches, and emits the surviving path as a structured trace. The labels on the nodes are different - step.classify, step.lookup, step.compute, step.cite - but the audit move is the same: open the tree, point at a node, walk the path. The trace is the work, in a form the validator can read.

The literacy bridge is the decision-tree audit, not the cartoon thought-bubble. The first replaces the second one-for-one.

Chain-of-thought prompting (Wei et al., arXiv:2201.11903, 2022) showed that asking a model to emit intermediate steps before the final answer materially improves accuracy on multi-step problems. Self-consistency (Wang et al., arXiv:2203.11171, 2022) sampled N reasoning paths and majority-voted the answer. Tree-of-thoughts (Yao et al., 2023) generalised further: maintain a search tree where each node is a partial reasoning state, expanded by BFS or DFS, with a value heuristic that prunes weak branches.

The 2024-2025 reasoning models industrialise this. OpenAI's o1 and o3 bake long internal computation into the model itself - reasoning tokens generated before the visible answer. DeepSeek R1 ships the reasoning channel as a first-class, separately-billed output. Anthropic's Claude exposes extended-thinking blocks that the application can render or summarise. In all three, the user-visible artefact is a structured trace: a sequence (or tree) of typed steps with token counts, branch scores, and pruning decisions attached.

Important framing, and the reason this article spells it out: the model does not think. The model emits a trace. The trace is correlated with the final answer, not its deterministic cause, and the providers themselves caution that emitted rationales can include post-hoc rationalisations. Treat the trace as an inspectable artefact under SR 11-7 - an audit object generated alongside the output, useful for validation and challenge - not as the model's interior monologue. That distinction is what makes the trace usable for regulated work in the first place. The model, its reasoning channel, and the audit pipeline all run inside the firm perimeter; DORA Art. 28-30 third-party risk applies to the inference stack itself, not to a black-box API on someone else's GPU.

EU AI Act Annex IV section 2 requires the model logic and the assumptions behind any high-risk AI output to be traceable. SR 11-7 independent validation requires a second team to be able to challenge how a model arrived at an answer. Both regimes assume the audit object exists. With a non-reasoning LLM, it does not - the model emits an answer and the rest is gone. With a reasoning model, the trace is the audit object: typed steps, branch scores, the choice points where the model expanded one candidate and discarded another.

Operationally this matters most for the slow, defensible workstreams - IC memos that have to survive a compliance review, suitability narratives, model-risk documentation, regulatory replies. The trace lets a human validator walk the steps and challenge any one of them before publication. The bright path through the tree becomes the citation chain; the pruned branches stay visible at low opacity as evidence of what the system considered and rejected.

EU AI Act Annex IV section 2(b) names "the description of the algorithms and any optimisation algorithms used" and section 2(g) names "the foreseeable risks and the mitigations." SR 11-7 demands traceable logic and independent validation. Both regimes are easier to satisfy when the model itself produces an inspectable artefact than when the validator has to reverse-engineer one.

Pair this with DORA: because the reasoning model runs inside the firm perimeter, the trace, the prompts that produced it, and the data the model consulted are all retained inside the controlled boundary. The third-party risk register for this capability is short. The artefact is preserved alongside the run record, signed with the run hash, and available to the validator without a foreign-API round-trip.

Do not describe the model as "thinking," "deciding," or "reasoning through" anything. The model emits a trace. The trace is structured text generated by a transformer; it is not interior cognition and presenting it as such triggers the AI-fluff filter in every institutional buyer in under ten seconds. Anthropomorphising the trace also obscures the actual audit move - the trace is useful because it is inspectable, not because it is a window into a mind.

Do not visualise the trace as glowing brain shapes, sparkles, or pulsing thought bubbles. Render it as a tree of typed steps with token counts and confidence scores, the same way a credit-risk team renders a CART. Show the pruned branches at 20% opacity; that is the search-and-prune mechanism made visible. Do not collapse reasoning into a five-stage chevron flow either - that is the McKinsey "agentic workflow" slide, and it hides the structural point that reasoning is a tree, not a pipeline.

Do not confuse reasoning with retrieval. Chain-of-thought is reasoning depth; RAG is knowledge breadth. They are orthogonal, often combined, never substitutes. And do not market reasoning as "the model is more accurate" without naming the cost: reasoning tokens are billed, traces consume context, and the right design for any given workflow is the shortest trace that still passes validation.