Guardrails and structured output: three rails, not one shield.

Guardrails are not a shield. They are three different mechanisms applied at three different positions in the generation pipeline, each enforcing a different class of constraint. Conflating them is the first mistake every vendor deck makes.

Input rails are classifiers that screen prompts before they reach the model: PII detection, jailbreak detection, topic restriction, prompt-injection signatures. The reference implementations are themselves LLMs fine-tuned to emit a small categorical label - Llama Guard from Meta and Anthropic's Constitutional Classifiers being the public examples.

Decoding rails are token-level constraints applied during generation. The fundamental mechanism is logit masking: at each generation step, the logits over the vocabulary are intersected with a 'valid next tokens' set computed from a constraint - a regex, a JSON schema, or a context-free grammar. Tokens outside the valid set get their probabilities set to negative infinity in log space before softmax. Outlines compiles a JSON Schema down to a regex, then to a finite-state machine, and looks up valid next tokens by FSM state at each step. XGrammar, now the default in vLLM, SGLang, and TensorRT-LLM, operates at under 40 microseconds per token. llguidance from Microsoft parses at the byte level, independent of the tokeniser. The native Structured Outputs features from large model vendors implement the same idea inside the provider stack.

Output rails are post-generation validators: a Guard object wraps the call, runs validators (regex, classifier, semantic checks) on the response, then fails, retries, or repairs. NeMo Guardrails models the whole loop as five tiers - input, dialog, retrieval, execution, output - configured declaratively.

The FSM coalescence trick is worth knowing: when a state has only one outgoing transition, you skip the sample step entirely. Documented speed-ups of around 5x are typical.

The structural analogy that lands with PMs and compliance leads is the pre-trade compliance / post-trade surveillance pairing in trading systems. Pre-trade blocks the order at submission. Post-trade reviews what was executed. They are complementary, not redundant, and you would never replace one with the other.

Guardrails are the same idea applied to an AI workflow. The input rail is the pre-trade equivalent: it stops a prompt that violates policy before the model ever sees it. The decoding rail is the in-flight constraint: the output is forced to be valid JSON matching your firm's published schema, character by character, with zero chance of an off-schema field appearing in the downstream system. The output rail is the post-trade surveillance: it validates the completed response against business rules, classifies it for routing, and either passes, retries, or quarantines.

For a regulated buy-side workflow - portfolio commentary generation, factsheet drafting, KIID-to-internal-schema ingestion, compliance taxonomy enforcement - the decoding rail in particular is what removes a whole category of integration risk. If the schema is the contract, the FSM is what enforces it.

MiFID II Article 17 requires investment firms running algorithmic systems to have effective controls, kill-switches, and pre-trade limits. The supervisory expectation is layered controls with a defined position in the order pipeline. Treat guardrails as the AI-equivalent control set and the structural fit is immediate.

EU AI Act Annex IV section 2(b) requires documentation of the risk management measures applied to a high-risk AI system. A single shield icon is not a measure. Three rails, each named, each with its mechanism documented, is the artefact the technical file expects.

DORA Articles 6 to 8 require ICT risk management with documented controls and incident handling. The guardrail layer is an ICT control and has to live in that register. We deploy the constraint apparatus on infrastructure where prompts, masks, and validator outputs stay inside the client's tenancy and region - the rails that protect the model do not themselves become a data exfiltration channel.

Schema first. Every workflow starts from a Pydantic model or a JSON Schema published as a versioned artefact in the client's repository. That schema is the source of truth - the FSM is compiled from it, the output validator runs against it, the downstream system reads it. No drift between layers because there is one definition.

Decoding rail uses XGrammar or Outlines depending on the runtime. For high-throughput batch workloads we run XGrammar inside vLLM on the client's own GPUs. For lower-throughput interactive workflows on hosted models we use the provider's native structured-output feature with the same compiled schema. The mechanism is the same: logit mask, FSM advance, token coalescence where the state graph allows it.

Input rail layers a local PII detector plus a fine-tuned classifier for the firm's prohibited-topic list. Output rail layers a semantic validator (does the answer cite a source the firm approves?) and a deterministic validator (does the JSON parse, do the regex-typed fields match, are the numeric ranges in tolerance?).

Inference, classifiers, and validators all run on local or private-tenancy infrastructure - DORA Art. 28 to 30 third-party ICT risk is reduced because there is no third party. The constraint apparatus is part of the audit trail by construction: every prompt, every mask, every validator decision is logged with a stable id and retained on the client's storage.

Latency budget is non-negotiable. We measure the per-token overhead of the decoding rail against the unconstrained baseline and target the published XGrammar number. If a workflow cannot tolerate the mask cost, we move structure to the output rail and accept the retry loop - we do not silently drop the constraint.

Do not put a floating padlock with the word 'GUARDRAILS' on your architecture diagram. It erases the three-layer mechanism and replaces it with a feeling. Compliance leads will read it as marketing.

Do not use the 'AI brain in a cage' metaphor. It frames the model as adversary and misrepresents the buyer's risk position - the risk is that the model produces an off-schema output that breaks a downstream system, not that the model is hostile.

Do not draw guardrails as a single 'input -> filter -> output' pipe. It hides the token-level mechanism, which is the one doing the structural work no classifier can replicate.

Do not skip the output rail because the decoding rail forced valid JSON. Schema validity is not business validity - a perfectly-typed wrong answer is still wrong.

Do not send prompts, masks, or validator logs to a third-party guardrails SaaS without a DORA Art. 28 register entry. The constraint layer is in scope for ICT risk.