Ask the same question several times at a non-zero temperature and you get several reasoning paths.
Self-consistency takes the answer that most of those paths agree on, and discards the rest.
A single chain can go wrong in many ways, but wrong answers rarely agree with each other.
It reliably lifts accuracy on maths and multi-step reasoning, at several times the cost.
It needs an answer that can be compared for equality, so it fits problems with a short final result.
Chain-of-thought gets a model to write out its steps, and that single chain is then trusted.
The weakness is obvious once stated: one arithmetic slip early on ruins everything after it.
Self-consistency treats a single chain as one noisy sample rather than as the answer.
Sample the same prompt several times at a temperature above zero and the paths diverge.
Different orderings, different intermediate quantities, different framings of the same problem.
Extract only the final answer from each path and take the most common one.
The reasoning that led there is discarded, which is why the answers must be comparable.
The insight behind it is that there are many ways to be wrong and usually one way to be right.
Errors scatter across different wrong values while correct reasoning keeps landing on the same number.
So the mode of the sampled answers is a considerably better estimate than any single sample.
Reported gains on grade-school and competition maths benchmarks were large when it was introduced.
The cost is linear in the number of samples, since five paths means five full generations.
It also only applies where equality of answers is well defined: a number, a label, a short string.
Two good essays never match exactly, so voting is meaningless for open-ended writing.
For those, an LLM judge scoring candidates replaces voting, at the cost of trusting the judge.
Ask the same question times at a non-zero temperature, take the answer that appears most often, and discard the reasoning that led there.
The chains differ. The answer, if the model knows it, tends not to, and that asymmetry is what the method exploits.
With and , the majority is right about 75 percent of the time, and the gain comes from nothing but repetition.
If , voting makes it worse.
That is the warning. Below a coin flip, the majority converges on the wrong answer faster than a single sample would.
Errors must also be independent for the count to mean anything. A model that fails the same way every time will agree with itself confidently and be wrong.
Gains rise steeply to about ten samples and flatten after twenty, so the usual choice is between five and twenty depending on what the answer is worth.
Ten samples cost ten times as much and take as long as the slowest one, which is why this is reserved for high-value questions rather than used everywhere.
Temperature 0 defeats it entirely. Identical samples cannot disagree, so there is nothing to count.
Free-form text breaks it too, because no two samples are identical.
The method needs answers that can be compared for equality, which makes it natural for arithmetic, multiple choice and extraction, and awkward for essays or code.
For open-ended output the usual substitute is to generate candidates and have a model pick the best, which is a judge rather than a vote and carries the judge's biases.
The vote share is the useful by-product.
A question where eight of ten samples agree is in a different state from one where three answers tie, and that is a better confidence estimate than the model's own stated certainty.
One question on this card. Every option explains itself once you pick, right or wrong.
17 concepts in this module, then a graded test with every answer explained.