Validation rules turn knowledge about what the data should look like into checks that run without a person.
They exist because the same problems return with every new file, and nobody can inspect every load by hand.
A rule that fails loudly is cheaper than a wrong number that reaches a report.
Range rules set limits, such as an age between 0 and 120 or a discount between 0% and 100%.
Set rules list the allowed values, such as order statuses limited to placed, shipped, delivered and returned.
Format rules check patterns, such as the shape of a UK postcode or an email address containing exactly one @ sign.
Uniqueness rules check that an identifier such as an order number appears only once.
Cross-column rules check relations, such as a refund never exceeding the original payment.
Cross-table rules check that every order points to a customer that exists, which catches broken joins early.
Each rule needs an action on failure: stop the whole load, hold back the failing rows, or let them in with a warning.
Rules that are too strict block good data whenever the business changes, such as when a new order status appears.
Rules that are too loose pass everything, so rules need review as the data and the business change.
Libraries such as Great Expectations and pandera let teams write rules as code and keep them with the rest of the project.
A record of how many rows failed each rule, over time, shows whether a source is getting better or worse.
The usual mistake is writing rules only after a bad number has been published, then never adding more.