Start studying
Module 13 · AI Evaluation Module contents
Module 13  ·  Card 8 / 18intermediate

Faithfulness

Faithfulness asks whether the answer is actually supported by the context.

An unfaithful answer states things the source documents never said.

It is measured by checking each claim against the retrieved text.

An answer can be faithful and still unhelpful if it dodges the question.

It is the primary defence against hallucination in RAG systems.

Learn in depth ↓
Module 13 · AI Evaluation · 18 concepts
In depth  /  15 lines

Faithfulness, in more detail

Faithfulness asks whether the answer is supported by the context it was given.

It is not asking whether the answer is true in general.

An answer can be perfectly true and still unfaithful, if the source never said it.

That distinction matters, since the model was told to use the documents.

An unfaithful answer means the model drew on its training data or invented something.

Either way, the citation and the audit trail are worthless.

It is measured by breaking the answer into individual claims.

Each claim is checked against the retrieved text for support.

The score is the fraction supported, and this is usually done by a model.

It is the primary defence against hallucination in a RAG system.

It is also the metric to watch when retrieval quality drops.

A model handed weak context will fill the gaps rather than admit them.

High faithfulness with low relevance means the model is quoting safely and unhelpfully.

Both matter, so a system optimised for faithfulness alone becomes evasive.

Requiring inline citations makes faithfulness far cheaper to check.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
What does faithfulness measure?
AI Evaluation

18 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close