JPEG is probably the most widely used image format in the world, and also one of the most elegant from a technical point of view. Understanding what happens inside the encoder lets you choose quality sensibly, recognise artifacts and know which operations damage a photo and which do not. Let's follow an image step by step.
1. From RGB to YCbCr
The first step separates brightness from color. RGB values are transformed into three components:
- Y (luma): how light the pixel is, in practice a black-and-white version of the image;
- Cb and Cr (chroma): how far the color departs from gray towards blue and towards red.
The transformation is reversible and in itself loses no information. It prepares the next step, exploiting the fact that the eye is far more sensitive to detail in brightness than in color.
2. Chroma subsampling
The Cb and Cr components are often stored at reduced resolution:
| Scheme | Color resolution | Typical use |
|---|---|---|
| 4:4:4 | full | graphics, text, maximum quality |
| 4:2:2 | half horizontally | some photo and video workflows |
| 4:2:0 | half horizontally and vertically | common setting for photos |
With 4:2:0 the color data shrinks to a quarter. On a photograph the difference is hard to see; on red text on a blue background, however, the edges smear. That is why screenshots saved as JPEG often show halos around colored lettering.
3. 8 × 8 pixel blocks
Each component is divided into blocks of 8 × 8 samples, which will be processed independently. Choosing small blocks makes the algorithm fast and light on memory; the price is that, at high compression, the boundaries between blocks become visible.
4. The discrete cosine transform (DCT)
Each block is transformed with the DCT, which expresses the 64 values as a sum of 64 basic "patterns," from a flat tone up to ever finer checkerboards. The result is a matrix of 64 coefficients:
- the top-left coefficient (called DC) represents the block's average value;
- the others (called AC) represent ever finer variations, that is, higher frequencies.
In an almost uniform patch of sky, nearly all the energy is concentrated in the DC coefficient and a few low-frequency AC ones; the rest are close to zero. The DCT is also reversible in theory: the loss does not happen here.
5. Quantisation: where information is lost
Each coefficient is divided by a value taken from a quantisation table and rounded to an integer. The tables use small divisors for low frequencies and large ones for high frequencies. Many high-frequency coefficients thus become exactly zero.
The "quality" parameter we see in programs is not part of the standard: it is a scale defined by each encoder that, in effect, scales the table values. That is why "quality 80" in two different programs does not necessarily give the same result.
6. Zigzag ordering and coding
The 64 quantised coefficients are read in a zigzag from low to high frequencies, so that the zeros end up grouped at the end. At that point:
- DC coefficients are coded as the difference from the previous block;
- AC coefficients are coded as pairs of "how many zeros, then which value," with a special symbol for "only zeros from here on";
- everything finally goes through entropy coding, usually Huffman.
Progressive JPEG organises the data in successive passes: first the low frequencies of the whole image, then the detail. While loading, a blurred version appears straight away and sharpens progressively, and the file is often slightly smaller too.
Artifacts and how to recognise them
- Blocking: visible 8 × 8 pixel squares in flat areas, due to heavy quantisation of the DC coefficient and the first AC ones.
- Ringing (or mosquito noise): halos and specks around sharp edges, because once high frequencies are removed an abrupt jump can no longer be represented.
- Color bleeding: imprecise colored edges, an effect of subsampling.
- Banding in skies: quantisation flattens subtle variations.
In my analysis work, quantisation traces are also a valuable clue: areas of an image showing block grids misaligned with the rest can reveal a paste from another photo.
Safe and harmful operations
| Operation | Effect |
|---|---|
| Opening and resaving as JPEG | new quantisation: additional loss |
| Rotating by 90° with "lossless" tools | no loss, if dimensions are multiples of the block size |
| Cropping with "lossless" tools | no loss, if the crop is aligned to blocks |
| Editing metadata only | no loss |
| Raising the quality of an already compressed JPEG | heavier file, no detail recovered |
Practical advice
- For photos to publish, medium-high quality with 4:2:0 is a good compromise; check by eye on flat areas and edges.
- For images with text or graphics, use 4:4:4 or, better, a lossless format.
- Keep the original file and generate each new version from it, not from the latest JPEG.
In short
- JPEG separates luma and chroma and reduces color resolution.
- Each 8 × 8 block is transformed by the DCT into frequency coefficients.
- Loss happens in quantisation; "quality" is a scale specific to each encoder.
- Blocking, ringing and color bleeding are the typical artifacts.
- Every resave adds loss: always work from the original.
