Data poisoning corrupts a model by tampering with its training data.
An attacker inserts crafted examples that create a hidden behaviour.
A backdoor can make a specific trigger phrase produce a chosen output.
Models trained on scraped web data are especially exposed.
Provenance checks, data validation and anomaly detection are the defences.
Data poisoning corrupts a model by tampering with what it learns from.
The attack happens before the model exists rather than at query time.
An attacker inserts crafted examples into the training data.
Those examples create a behaviour that is not visible in normal testing.
A backdoor pairs a trigger phrase with a chosen output.
The model behaves normally until the trigger appears, then does what the attacker wants.
Standard evaluation will not find it, since nothing is wrong until the trigger is used.
Models trained on scraped web data are the most exposed, since anyone can publish.
Buying expired domains that appear in training sets is a demonstrated approach.
Poisoning a small fraction of a corpus can be enough to install a backdoor.
Fine-tuning on user-submitted data is a direct and often overlooked route in.
RAG has a version of this too, since poisoning the index poisons the answers.
Defences are provenance checks, data validation and anomaly detection on training sets.
Curate sources for anything you train on, rather than trusting a large scrape.
Verify checksums for downloaded datasets and pretrained weights, since supply chains are attacked.
One question on this card. Every option explains itself once you pick, right or wrong.
16 concepts in this module, then a graded test with every answer explained.