Start studying
Module 6 · Data Wrangling Module contents
Module 6  ·  Card 11 / 14intermediate

Groupby

Groupby splits a DataFrame into groups by the values in one or more columns, then summarises each group.

The pattern is split, apply, combine: split the rows, run a calculation on each group, and join the results.

For example, sales.groupby("region")["revenue"].sum() gives the total revenue for each region.

The agg method runs several calculations at once, such as the sum, mean and count of revenue per region.

Groupby is the pandas form of GROUP BY in SQL and of a pivot table in a spreadsheet.

Learn in depth ↓
Module 06 · Data Wrangling · 14 concepts
In depth  /  15 lines

Groupby, in more detail

Most summary questions ask for one figure for each of several groups, and groupby answers exactly that.

Calling groupby on its own shows nothing useful; it returns an object that waits for a calculation.

The split step works out which rows belong to which group.

The apply step runs the calculation once per group, and the combine step joins the results into a new table.

Built-in calculations such as sum, mean, count, min and max are vectorised and fast.

The count method skips blanks while size counts every row, so the two can differ for the same group.

Grouping by two columns, such as region and month, gives one result for each combination.

By default the group labels become the index, and as_index=False keeps them as ordinary columns.

The transform method returns a result as long as the original, such as each order's share of its region's total.

Rows with a blank group label are dropped by default, which can hide a whole group from the summary.

Passing dropna=False keeps those rows as a group of their own.

Using apply with a custom Python function is flexible but much slower than the built-in calculations.

Groupby on millions of rows stays fast as long as it sticks to built-in calculations.

In practice groupby is how most reports start: totals by month, averages by team, counts by status.

A common misconception is that groupby changes the table; it only builds summaries and leaves the rows as they were.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
An analyst wants each region's total revenue and its number of orders in one result. What should they use?
Data Wrangling

14 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close