Start studying
Module 15 · AI Security Module contents
Module 15  ·  Card 14 / 16intermediate

Secure Tool Use

Every tool you give a model is a capability an attacker may reach.

Grant the narrowest permissions each tool genuinely needs.

Validate arguments in your own code, never trusting the model's output.

Never pass model output straight into a shell, SQL query or filesystem path.

Require human approval before anything irreversible.

Learn in depth ↓
Module 15 · AI Security · 16 concepts
In depth  /  15 lines

Secure Tool Use, in more detail

Every tool you give a model is a capability an attacker may eventually reach.

The model can be persuaded, and a tool it can call is a tool they can call.

So the design question is what damage is possible, not whether persuasion will happen.

Grant each tool the narrowest permission it genuinely needs.

A tool that reads one table should not hold credentials for the whole database.

Validate every argument in your own code, since model output is untrusted input.

Never pass it straight into a shell, an SQL query, or a filesystem path.

Those are the same injection classes software has fought for decades, with a new source.

Use parameterised queries and allowlists rather than escaping.

Constrain by construction, so a file tool should resolve paths inside one directory only.

Require human approval before anything irreversible or externally visible.

Sending, spending, deleting and publishing are the obvious gates.

Log every call with its arguments and result, since that trace is your only account of what happened.

Rate limit tools as well as the model, because a loop can do damage quickly.

Prefer read-only tools, and give a write tool a narrow and specific job.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
Your agent can run shell commands. What is the most important safeguard?
AI Security

16 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close