Start studying
Module 13 · Data Engineering Basics Module contents
Module 13  ·  Card 1 / 13beginner

Big Data Characteristics

Big data describes datasets too large, too fast or too varied for a single ordinary computer to handle well.

The usual summary is the 3 Vs: volume for size, velocity for speed of arrival, and variety for mixed formats.

For example, a ride-hailing app logs the location of every car every few seconds, millions of rows an hour.

Some lists add veracity, meaning how trustworthy the data is, and value, meaning whether it is worth keeping.

Data that fits comfortably in a laptop's memory rarely needs big data tools, which add cost and complexity.

Learn in depth ↓
Module 13 · Data Engineering Basics · 13 concepts
In depth  /  15 lines

Big Data Characteristics, in more detail

Big data is less about a fixed size than about data outgrowing the tools that normally handle it.

What counts as big changes as ordinary computers get more powerful.

The useful question is whether one machine can store and process the data in reasonable time.

Volume means sheer size, such as billions of rows or many terabytes.

Velocity means speed of arrival, such as thousands of events a second from sensors or apps.

Variety means mixed forms, such as tables alongside logs, images, audio and free text.

Volume pushes work onto many machines, velocity pushes towards stream processing, and variety pushes towards flexible storage.

Veracity is a reminder that bigger data is often messier, with more errors and gaps.

More data reduces random error but not bias, so a huge biased dataset still gives wrong answers.

Big data tools add costs: clusters to run, slower development and more ways for jobs to fail.

Many analyses that seem to need big data work fine on a sample or a summary table.

A single well-equipped machine can often handle tables with hundreds of millions of rows.

Columnar file formats and efficient tools push that limit further before a cluster is needed.

In practice, teams start on 1 machine and move to distributed tools only when the data truly demands it.

The common misconception is that more data always means better answers, when the quality of the data matters more.

Want the full technical article? Wikipedia ↗
Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
A team's sales data is 2 GB and fits easily in a laptop's memory. Someone proposes a cluster of machines to process it. What is the best response?
Data Engineering Basics

13 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close