In a live stream, latency is the time between an event in front of the camera and the moment the viewer sees it. For a concert streamed online, a few dozen seconds of delay bother nobody; for an auction, a lecture with audience questions or a video call, even two seconds is too much. Reducing latency always has a cost, in stability or complexity, so the first question to ask is: how much delay can I afford?
Where the delay comes from
Total latency is the sum of many small delays along the chain:
- capture: the camera and capture card add a few frames of delay;
- encoding: the encoder must analyse frames before compressing them; B-frames and look-ahead increase the delay;
- upload to the server: the contribution protocol, such as RTMP or SRT, adds a safety buffer;
- transcoding: the server creates the various qualities for adaptive streaming;
- segmenting: with HLS and DASH the video is split into segments, and a segment cannot be sent until it is complete;
- delivery: the CDN brings segments close to viewers;
- player buffer: the player holds a few segments in reserve so it does not stall if the network slows.
The last two points, segmenting and buffering, are almost always the heaviest in traditional streaming.
Latency bands
| Band | Order of magnitude | Typical technologies | Use |
|---|---|---|---|
| standard | tens of seconds | HLS and DASH with long segments | events, entertainment |
| reduced | a few seconds | LL-HLS, low-latency DASH, chunked CMAF | sport, chat interaction |
| near real time | under a second | WebRTC | video calls, auctions, remote production |
Contribution protocols
These carry the signal from the encoder to the server.
RTMP
RTMP was for a long time the standard for sending live streams to platforms and is still accepted almost everywhere. It runs over TCP, so when packets are lost it retransmits data and can build up delay. It has limits with modern codecs, although recent extensions exist to overcome them.
SRT
Secure Reliable Transport is an open-source protocol designed to carry video over imperfect networks, such as the public internet. It runs over UDP and recovers lost packets within a configurable latency window: the wider it is, the more robust the transmission, but the greater the delay. It supports encryption and is widely used in remote production.
WebRTC and WHIP
WebRTC was created for real-time communication in browsers. It is designed to keep latency to a minimum and, when the network worsens, prefers lowering quality to building up delay. The WHIP protocol standardises sending a WebRTC stream to a server, making it usable by broadcast encoders too.
Delivery protocols
These are how viewers receive the video.
Traditional HLS and DASH
With 6-second segments and a player buffering three of them, you easily reach 20-30 seconds of total delay. In return you get great stability, compatibility and scalability through CDNs.
Low-latency variants
LL-HLS and low-latency DASH split segments into smaller parts, which can be sent and played before the whole segment is complete. The player can thus stay much closer to the «live» point without giving up HTTP delivery.
WebRTC to the viewer
It offers the lowest latency, but scaling to a very large audience is more complex and expensive, because it does not rely on traditional CDN caching.
Settings that reduce delay
Even without changing technology there is room to act:
- shorter keyframe interval, for example 1 or 2 seconds, so segments can be shorter;
- shorter segments in the server configuration;
- encoding presets designed for low latency, with less look-ahead and no B-frames if needed;
- smaller player buffer, knowing it will become more sensitive to network fluctuations.
With x264 the -tune zerolatency option disables the tools that introduce delay, at the price of lower compression efficiency.
The unavoidable trade-off
Less buffer means less margin. A very low-latency stream on an unstable connection will more often show quality drops or interruptions. For an important event, measure the stability of the upstream connection, plan a backup connection and choose the highest latency compatible with the kind of interaction required.
In short
- Latency is the sum of capture, encoding, upload, segmenting, delivery and buffering.
- In traditional streaming, segments and the player buffer weigh most.
- RTMP and SRT carry the signal to the server; SRT is more robust on imperfect networks.
- LL-HLS and low-latency DASH get down to a few seconds; WebRTC below one second.
- Reducing latency reduces the margin against network problems.
