Distributed processing exists because some jobs are too big or too slow for one machine.
The idea is to divide the data and the work, run the parts at the same time, and combine the answers.
Doing this well is hard, because machines are slow to talk to each other and sometimes fail.
Data is split into partitions, chunks that fit comfortably on one machine.
Narrow steps, such as filtering rows, work on each partition alone and are cheap.
Wide steps, such as grouping or joining, need rows with the same key on the same machine.
Moving rows to bring keys together is called a shuffle, and it is often the slowest part of a job.
Skew happens when one key has far more rows than others, leaving one machine doing most of the work.
Joining a huge table to a small one can avoid a shuffle by copying the small table to every machine.
MapReduce was an early, influential model: map each partition, shuffle by key, then reduce each group.
Failures are handled by recomputing lost partitions from their inputs, rather than restarting the job.
Adding machines speeds a job up only to a point, since coordination and shuffles grow too.
Cloud services make clusters easy to start and stop, which lowers the cost of occasional large jobs.
In practice, cutting shuffles and balancing partitions matters more than adding machines.
The usual mistake is distributing a job that is slow because of a bad query, which a cluster only makes expensive.