A stereo track at 44.1 kHz and 16-bit requires about 1,411 kilobits per second. A good compressed file uses a tenth of that or less and, in most listening situations, sounds almost indistinguishable. The trick is not compressing data better but deciding which data to throw away. That is the field of psychoacoustics, the study of how the ear and brain perceive sound.
The ear doesn't hear everything
Hearing has precise limits that codecs exploit systematically.
Threshold of hearing
The ear's sensitivity varies with frequency: it peaks in the region between roughly 2 and 5 kHz and drops considerably in the deep lows and extreme highs. A faint sound at 15 kHz may be inaudible, while the same level at 3 kHz is clearly heard.
Frequency masking
A loud sound makes quieter sounds at nearby frequencies inaudible. If a violin plays loudly at 1,000 Hz, a faint component at 1,100 Hz vanishes from perception. The ear analyzes sound in critical bands, frequency ranges within which sounds interfere with each other.
Temporal masking
Something similar happens over time: after a loud sound, for a few tens of milliseconds, quiet sounds are less perceptible. There is also a brief effect before the loud event, much shorter.
How an encoder works
Simplified, every modern lossy codec performs these steps:
- Splits the signal into short windows, a few milliseconds long.
- Transforms it into the frequency domain, typically with a transform called the MDCT, producing a band-by-band representation.
- Applies a psychoacoustic model that estimates, band by band, how much noise can be added without being heard.
- Quantizes the coefficients with variable precision: high where the ear is sensitive, very low where the signal is masked.
- Compresses the result with lossless coding.
The loss happens at step 4: quantization error is "hidden" below the masking threshold. The lower the bitrate, the more error must be introduced and the harder it is to hide.
Typical artifacts
When the bitrate is not enough, the noise surfaces in recognizable ways:
- Pre-echo: before an impulsive sound, such as a drum hit or a click, you hear a short burst of hiss. It happens because the error spreads across the whole window, including before the attack. Modern codecs reduce it by switching to shorter windows on transients.
- "Underwater" or metallic effect: frequency bands that appear and disappear from one window to the next.
- Cut-off highs: to save bits, the encoder discards the highest frequencies.
- Unstable stereo image: with joint channel coding, the stereo field can narrow or shift.
MP3, AAC and Opus
| Codec | Characteristics | Typical use |
|---|---|---|
| MP3 | Universal compatibility, less efficient than its successors | Existing archives, older devices |
| AAC | More efficient than MP3 at the same bitrate, very widespread | Video, streaming, smartphones |
| Opus | Open standard, excellent from voice to music, low latency | Calls, video conferencing, web |
MP3 (MPEG-1 Audio Layer III) made widespread digital music possible, but today it has been surpassed in efficiency. AAC is its successor in the MPEG world and the default audio format of many video containers. Opus is an IETF standard, open and royalty-free, that combines a mode optimized for speech and one for music; it is designed for very low latency and is therefore widely used in real-time communication.
Practical rules
- Don't re-encode a lossy file: converting an MP3 to AAC adds new artifacts on top of the existing ones. Always start from an uncompressed or lossless original.
- Do the final encode only once: if a platform re-encodes uploaded files, provide the highest quality it accepts.
- Leave headroom for peaks: decoding can produce samples slightly higher than the original; a master too close to 0 dBFS risks distorting after compression.
- Listen to the critical spots: applause, cymbals, sibilant voices and percussion instruments immediately reveal the limits of a bitrate.
In my work, before choosing a project's settings, I encode a few seconds of the hardest passages at different bitrates and compare them on headphones. It is a test that takes a few minutes and avoids choices based on numbers alone.
In short
- Lossy codecs discard information the ear does not perceive, thanks to masking.
- The signal is transformed into frequency and quantized with variable precision per band.
- At low bitrates you get pre-echo, metallic sound and cut-off highs.
- AAC and Opus are more efficient than MP3; Opus also excels in real time.
- Never re-encode files that are already lossy: always start from the original.
