A modern video game draws scenes made of millions of triangles tens or hundreds of times per second. It manages this because the graphics card (GPU) always runs the same sequence of steps, the rasterization pipeline, on huge amounts of data in parallel. Knowing the pipeline's stages helps you understand where performance problems and many visual artifacts come from, even if you do not write graphics code.
Overview of the stages
- Vertex input: the GPU reads vertices and indices from memory.
- Vertex shader: transforms each vertex.
- Primitive assembly and clipping: vertices are grouped into triangles; anything outside the view is discarded or cut.
- Rasterization: each triangle is converted into the set of pixels it covers.
- Fragment shader: computes the color of each fragment.
- Tests and blending: depth, transparency and writing to the final image.
Some stages are programmable (the shaders), others are fixed, run by specialised hardware with a few settings.
The vertex shader
The vertex shader is a small program run once per vertex. Its minimum task is to transform the vertex position from object coordinates to clip coordinates by multiplying it by the Model-View-Projection matrix. It can also compute other data and pass it on to later stages: transformed normals, texture coordinates, colors.
Clipping and culling
After the vertex shader, triangles entirely outside the visible volume are discarded; those that cross it are cut. Then comes the perspective divide and the transformation to screen coordinates.
Backface culling often happens at this stage too: triangles seen from behind, determined by the order of their vertices on screen, are removed. For a closed object, roughly half the triangles face away, so the saving is considerable.
Rasterization
This fixed stage determines which pixels each triangle covers. For each pixel, the hardware checks whether its center falls inside the triangle, using three simple edge functions, one per side.
For each covered pixel a fragment is generated, a candidate pixel carrying data interpolated from the three vertices: depth, texture coordinates, normals, colors. Interpolation takes perspective into account; otherwise textures would look distorted on slanted surfaces.
The fragment shader
The fragment shader (also called the pixel shader) runs for every fragment. This is where the surface's appearance is computed: textures are read, lighting models applied, materials and effects combined. It is often the most expensive stage, because a high-resolution scene generates millions of fragments per frame, and dozens of operations may run on each.
An important concept is overdraw: if several objects overlap, the same pixel may be processed several times, wasting computation on surfaces that will later be hidden.
The depth test (z-buffer)
How does the GPU know which surface is in front? It uses the z-buffer (or depth buffer), an image the same size as the screen that stores, for each pixel, the depth of the nearest surface drawn so far.
For each new fragment:
- if it is closer than the stored value, the color is written and the z-buffer updated;
- if it is farther away, the fragment is discarded.
The advantage is that objects can be drawn in any order. Many GPUs run the depth test before the fragment shader when possible (early-z), avoiding computing the color of fragments that would be hidden anyway. That is why drawing opaque objects from nearest to farthest improves performance.
Transparency and blending
Transparent surfaces are a hard case: the z-buffer stores only one depth per pixel, so it cannot handle several semi-transparent layers. The usual solution is:
- draw all opaque objects first;
- sort transparent objects from farthest to nearest;
- draw them with alpha blending, without updating the z-buffer.
Per-object sorting is not always correct, for example with intersecting transparent objects: this is the source of many of the transparency errors seen in games, such as leaves or glass appearing in front of what they should cover.
Antialiasing
Rasterization samples each pixel at a single point, so triangle edges look jagged. The main techniques are:
| Technique | Principle | Cost |
|---|---|---|
| MSAA | several coverage samples per pixel, shader run once | medium |
| SSAA | the whole image computed at higher resolution and then reduced | high |
| FXAA and similar | a filter on the final image that finds and softens edges | low |
| TAA | combines samples from successive, slightly shifted frames | low-medium, possible ghosting |
Where performance is lost
In my work, when I analyse a slow scene, the first question is always which stage is the bottleneck:
- too many vertices: models too detailed for the distance they are seen from;
- too many fragments: high resolution, overdraw, many overlapping transparent surfaces;
- shaders too heavy: expensive calculations repeated for every pixel;
- too many draw calls: many small objects sent separately, which burden the CPU more than the GPU.
A simple test: lower the resolution. If performance improves a lot, the limit lies in the fragment stage.
In short
- The pipeline turns vertices into pixels through programmable stages (shaders) and fixed stages.
- The vertex shader transforms vertices; the fragment shader computes each fragment's color.
- Rasterization determines which pixels each triangle covers and interpolates vertex data.
- The z-buffer resolves visibility; transparency requires sorting and blending.
- To optimise, you must identify the stage acting as the bottleneck.
