Start studying
Module 15 · AI Security Module contents
Module 15  ·  Card 4 / 16beginner

Insecure Output Handling

Insecure output handling is trusting what a model returns and passing it straight into another system.

A response rendered as HTML without escaping can carry a script into your user's browser.

The same response dropped into a SQL query, a shell command or a file path is the same class of bug.

The model is best treated as an untrusted user who can be talked into writing anything.

Escape, parameterise and validate model output exactly as you would input from a stranger.

Learn in depth ↓
Module 15 · AI Security · 16 concepts
In depth  /  15 lines

Insecure Output Handling, in more detail

Insecure output handling is what happens when a model response is trusted by whatever consumes it next.

The model is not the vulnerability here, the code downstream of it is.

Think of the model as an untrusted user who can be persuaded to write absolutely anything.

Rendered as HTML without escaping, a response can carry a script that runs in your user's browser.

Concatenated into a SQL statement, it is an injection in the classic database sense.

Passed to a shell, it becomes remote code execution with your service account's permissions.

Used as a file path, it can walk out of the directory you intended with a few dot segments.

Returned as a URL and fetched by your server, it can reach internal addresses that were never meant to be public.

The pattern is always the same, and it is output crossing into a system that parses it as instructions.

The defences are the ordinary ones, applied in the ordinary places.

Escape on render, parameterise every query, and pass argument arrays rather than assembled shell strings.

Validate against an allowlist of shapes you expect instead of scanning for patterns you fear.

Parse structured output into a typed object and reject anything malformed rather than repairing it.

Give the downstream system the least privilege it can work with, so that a slip stays survivable.

Injection and insecure output handling are the same bug seen from opposite ends of the request.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
Your application renders the model's response as HTML. What is the risk?
AI Security

16 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close