Start studying
Module 15 · AI Security Module contents
Module 15  ·  Card 12 / 16advanced

Model Extraction

Model extraction steals a model's capability through its API.

The attacker queries it many times and trains a copy on the responses.

The clone approximates the original without any access to its weights.

It is theft of expensive intellectual property.

Rate limiting, query monitoring and output watermarking raise the cost of doing it.

Learn in depth ↓
Module 15 · AI Security · 16 concepts
In depth  /  15 lines

Model Extraction, in more detail

Model extraction steals a model's capability through its public interface.

The attacker never touches the weights and does not need to.

They send many queries and record the responses.

Those pairs become a training set for a copy.

The clone approximates the original at a fraction of the original cost.

This is distillation, used without permission.

It is theft of expensive intellectual property, and it is difficult to prove after the fact.

Access to probabilities rather than just labels makes it far more efficient.

That is one reason many APIs limit what they return.

Rate limiting raises the cost, since extraction needs volume.

Query monitoring can spot the pattern, as extraction traffic looks unlike normal use.

Systematic coverage of the input space is the giveaway.

Watermarking output allows a copy to be identified later.

Adding noise to outputs degrades extraction and also degrades your product.

None of these prevent it, and together they make it expensive enough to deter most attempts.

Want the full technical article? Wikipedia ↗
Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
An attacker queries your API thousands of times to train a copy of your model. What is this?
AI Security

16 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close