In high-dimensional data, quantifying how large an observation is or how far apart two records sit requires norms.
A norm takes a vector as input and maps it to a single non-negative real number representing its magnitude.
Norms must satisfy three mathematical rules: positivity, scalability under scalar multiplication, and the triangle inequality.
The most familiar norm is the L2 norm, corresponding to standard Euclidean straight-line distance in physical space.
To calculate the L2 norm, you square each coordinate, sum those squares together, and take the square root of the total.
The L1 norm, often called the Manhattan or taxicab norm, sums the absolute values of the components without squaring.
It represents grid-like travel distance, like walking along city street blocks rather than flying directly as the crow flies.
In high-dimensional spaces, L2 distance can become less informative as distances between points begin to concentrate.
The distance between two observations u and v is simply the norm of their difference vector, u minus v.
Subtracting matching components and taking the L2 norm gives Euclidean distance between the two observations.
Taking the L1 norm of the difference gives Manhattan distance between those two observations.
In statistical modelling, L1 norms encourage sparsity by forcing minor parameters to become exactly zero.
L2 norms penalise large values heavily by squaring them, discouraging isolated extreme parameters.
A common mistake is comparing vector distances across columns that have completely different measurement units.
If one column is measured in centimetres and another in kilograms, scale differences will distort the computed norm.