Guardrails and the security layer

Where a guardrail sits and what it checks

An LLM guardrail is a check that runs at one of three points: on the input before the model sees it, on the output before the user or your code sees it, or on an action an agent is about to take. Input checks look for injected instructions and personal data. Output checks look for policy breaches, leaked secrets and broken formats. Action checks decide whether a tool call is allowed at all.

The security layer holds these products (live products on record: 78), and the guardrails category holds the ones built for these checks. Some are libraries you call, some are proxies in front of the model, and some are models trained to classify text as safe or not.

No single check covers all of it, and a stack with checks at each point still fails sometimes. The aim is to make the common failures rare and the rare ones cheap: a refused request instead of a leaked record, a blocked tool call instead of a sent email.

Input checks: injection and data you should not send

Prompt injection is text that tries to override your instructions. It arrives from users, and more dangerously from content the system fetches: a web page, an email, a document in your RAG corpus. A classifier can flag likely attempts, but none catches every phrasing, so treat detection as one layer, never the only one.

The other input check is about what you send out. If prompts go to a hosted model, personal data, credentials and customer secrets in them leave your infrastructure. Redacting or masking them before the call is cheaper than explaining the leak, and procurement teams will ask whether you do it.

Separate instructions from data wherever the model allows it. Put your rules in the system prompt, mark fetched content clearly as content, and never paste a retrieved document into the place where instructions go. It does not stop injection, and it removes the easiest form of it.

Output checks: format, policy and leaks

The simplest output guardrail is a schema check. If your code expects JSON with certain fields, validate it and retry or fail cleanly when it does not parse. It is cheap, exact and catches output that would otherwise break the code downstream.

Policy checks come next: content your product must not produce, claims it must not make, topics it should refuse. These are often a second, smaller model scoring the output, which adds latency and cost to every call. Run them where the risk justifies it, not by default on every internal request.

Check outputs for leaks too. A model can repeat its system prompt, a passage from another customer’s documents, or a key that slipped into context. Pattern matching for secrets is crude, and it catches the obvious cases cheaply.

Agents raise the stakes: checks on actions

An agent that can send email, edit records or call an API turns a bad output into a bad action. The guardrail that matters most here limits what the agent may do: which tools it can call, with which arguments, on whose data, and which actions need a person to approve first.

Give each tool the narrowest credential that works. The MCP server pages record what each server reaches and which credential it wants (servers on record: 8,106). Read that before connecting one to an agent with access to anything you care about. A read-only key is a guardrail that cannot be talked out of its job.

Log every action an agent takes with the reason it gave, and make the irreversible ones wait for a person. Sending, deleting, paying and publishing are the obvious candidates. An approval step is slow, and those actions can afford it.

Testing that the checks work

A guardrail that has never blocked anything in testing may simply be unable to. Before you trust one, feed it the inputs it exists to catch: an injected instruction inside a fetched page, a prompt with a fake card number, an output that breaks your schema. Confirm each is caught and that the log says which check fired and why.

Keep those inputs as a test set and run it whenever the model, the prompt or the guardrail changes. Models change how they respond to the same attack, and a check tuned against one version can miss on the next. The evaluation category lists tools that can run such sets alongside your quality tests.

Measure the false positives as carefully as the catches. Run a sample of ordinary, harmless requests through the same checks and count how many are refused. A guardrail that blocks real customers will be loosened or removed by someone under pressure, and the gap it leaves will not be announced. Report both numbers together, every time the checks change, so nobody tunes one while ignoring the other.

What to ask of a guardrail product

Ask which of the three points it covers and how it is deployed: a library in your code, a proxy, or a hosted API that sees your prompts. The last option sends your data to one more party, which is worth knowing before the security review asks.

Ask what it adds to latency per call and whether checks can run in parallel with the model rather than before it. Ask how you tune it and how you see what it blocked, because a guardrail with false positives you cannot inspect will be switched off by the team it annoys. Then ask whether its decisions appear in your traces, so a blocked request can be explained later.