Start studying
Module 6 · Data Wrangling Module contents
Module 6  ·  Card 8 / 14intermediate

Column Data Types

Every pandas column has a data type, or dtype, that decides what you can do with it.

Common dtypes are int64 for whole numbers, float64 for decimals, object for text, bool, and datetime64 for dates.

For example, if one row of a price column says unknown, the whole column is read as text and summing it joins the strings.

The to_numeric and to_datetime functions convert columns, and their coerce option turns entries that fail into blanks.

Checking the dtypes right after loading catches most problems before they turn into wrong totals.

Learn in depth ↓
Module 06 · Data Wrangling · 14 concepts
In depth  /  15 lines

Column Data Types, in more detail

A column's dtype is usually decided when the data is loaded, by pandas guessing from the values.

One bad value is enough to change the guess for the whole column.

A price column with the word unknown in one row becomes object, the general type pandas uses for text.

Object columns are slow and use far more memory, because each entry is a separate Python object.

A whole number column with any blanks becomes float64, because the standard integer type has no way to mark a blank.

pandas also offers nullable types such as Int64, with a capital I, which can hold blanks.

Dates read from a CSV file arrive as text unless parsed, so sorting them sorts alphabetically.

Converting them with to_datetime makes date maths work, such as the number of days between two orders.

Day and month order is ambiguous, so 03/04 can mean 3 April or 4 March, and the format should be stated.

Text with a few repeated values, such as region names, can use the category dtype.

A category column stores each label once and uses small codes for the rows, which can cut memory sharply.

Converting float64 to float32 halves the memory but loses precision, which matters for money.

Money is often stored as whole pennies in an integer column to avoid rounding drift.

Looking at the dtypes or calling info straight after loading is the habit that catches most of these issues.

The common mistake is trusting the totals before checking the types that produced them.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
After loading a sales file, summing the price column returns joined text such as 12.994.50 instead of a number. What happened?
Data Wrangling

14 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close