The Screen-Space Toolkit

Chapter 22 introduced SSAO and screen-space reflections in passing. This chapter goes deeper: how the modern versions work, why they look better, and the one limitation they all share, that the screen only knows the front surface of what's visible. Despite that, nearly every real-time renderer runs several of these effects every frame, because they're cheap, need no extra scene data, and catch detail that probes and shadow maps miss.

Kernel SSAO (Chapter 22) scatters test points in a ball and counts how many are hidden. It's a crude estimate: it counts volume rather than directions, it needs many samples for a smooth result, and it darkens flat surfaces at grazing angles. Horizon-based ambient occlusion (HBAO, Louis Bavoil, Miguel Sainz and Rouslan Dimitrov, 2008) asks a better question. Stand on a pixel and look along one direction across the screen: how high does the surrounding geometry rise, as seen from here? That's the horizon in that direction. Everything below it is blocked, everything above it is open sky. Search a few directions, and you know how much of the hemisphere is visible.

// Horizon-based AO for one pixel at view-space position P with normal N.
var occlusion = 0.0;
for (var d = 0; d < DIRECTIONS; d++) {
  let angle = (f32(d) + noise) * PI / f32(DIRECTIONS);
  let dir = vec2f(cos(angle), sin(angle));
  for (var side = -1.0; side <= 1.0; side += 2.0) {
    var highest = 0.0;                               // sine of the horizon's elevation above the surface
    for (var k = 1; k <= STEPS; k++) {
      let S = positionAt(pixel + dir * side * radiusInPixels * f32(k) / f32(STEPS));
      let toS = S - P;
      let dist = length(toS);
      let fade = clamp(1.0 - (dist / RADIUS) * (dist / RADIUS), 0.0, 1.0);   // fade out toward the radius
      highest = max(highest, dot(toS / dist, N) * fade);
    }
    // the cosine-weighted share of this side of the slice that's below the horizon
    occlusion += (1.0 - sqrt(1.0 - highest * highest)) * 0.5;
  }
}
let ao = 1.0 - occlusion / f32(DIRECTIONS);
    

(This is the simple form the hero uses: it measures each horizon's height above the surface's own plane.)

HBAO+

The original HBAO is careful but slow. Each step along a direction depends on the highest horizon found so far, and the steps wander across the depth buffer, so neighboring pixels read far-apart memory and the GPU's caches help little. HBAO+ (Louis Bavoil, NVIDIA, 2014) kept the horizon idea and reorganized the work for speed:

• Deinterleaved rendering. The depth buffer is split into 16 quarter-resolution textures, one for each pixel position in a 4 × 4 block. Each of those is processed with its own single, fixed rotation of the sample directions, so neighboring pixels of one texture read neighboring texels and stay in cache. The 16 results are interleaved back into one full-resolution image: across each 4 × 4 block, every rotation appears exactly once.
• Simpler samples. Its widely used formulation drops the running horizon. Each sample counts on its own: how far it rises above the surface's plane (the cosine between the normal and the direction to the sample, minus a small bias), times a fade with distance, all averaged. That's cheaper and smoother, though it no longer tracks which samples hide which.
• A depth-aware blur. The 4 × 4 pattern of rotations is averaged away by a blur that skips neighbors at a different depth, so it doesn't smear the AO across edges.

It ran much faster than HBAO at full resolution, and became one of the most widely used AO techniques in games of the 2010s.

GTAO

Where HBAO+ traded accuracy for speed, ground-truth ambient occlusion (GTAO, Jorge Jimenez, Xian-Chun Wu, Angelo Pesce and Adrian Jarabo, 2016) went the other way and made the horizon idea exact. In each direction it takes the horizons on both sides and computes the cosine-weighted visible arc between them in closed form. Unlike the code above, which measures heights against the surface's own plane, it accounts for the normal leaning within each slice, and weights each slice by how much the normal points along it. With only two or three directions per pixel (spread over neighboring pixels and over frames with noise and temporal accumulation), it comes close to ray-traced AO, which is where the "ground truth" name comes from.

The horizon search gives a bonus: the direction halfway between the two horizons points toward where the open sky actually is. Averaged over the search directions, this is the bent normal, and looking up the sky or probe light along it, instead of along the surface normal, makes ambient light come from where it really can (Chapter 22's exercise). Some renderers also use the cone of visible directions to shadow specular reflections from probes: specular occlusion.

Compare each method with the reference in the "AO only" view. Kernel SSAO over-darkens beside objects, on the floor in front of the cubes and just under the table's edge, where its test points land behind things. HBAO, HBAO+ and GTAO don't, but all three come out a little lighter than the reference along the creases and around the cubes: the depth buffer holds only the front surfaces, and a few screen directions catch only part of what's there. HBAO+ is the lightest, and renderers scale its strength to taste. GTAO darkens small spots at the feet of the table legs and the pole: from the depth buffer, a thin pole looks like a wall reaching back forever. The GTAO paper softens this by guessing how thick things probably are. Raising the directions and steps changes surprisingly little; most of what's left is what the screen can't see, not too few samples.

Chapter 22's screen-space reflection marched the reflected ray in small fixed steps, testing the depth buffer at each. Small steps are slow; big steps skip thin objects. The standard fix is Hi-Z tracing (Yasin Uludag, 2014): build a mipmap chain of the depth buffer in which each texel holds the nearest depth of the four below it. A coarse level says "nothing in this whole block is closer than this". The ray can leap across a whole block whenever it stays in front of that depth, and only descends to finer levels near surfaces. A trace that would take hundreds of fixed steps takes a few dozen.

In a shader, the loop the demo steps through looks like this, in two dimensions. The only new work is finding where the ray leaves the current cell: whichever of the cell's sides, left/right or top/bottom, it reaches first.

// Hi-Z march. The ray starts at `start` (pixel x, pixel y, depth) and
// moves by `dir` per unit of t. A bigger depth is farther away.
// Level k of hiZ holds the nearest depth of each 2^k by 2^k block.
var lvl = 0;
var t = 0.0;
for (var i = 0; i < MAX_STEPS; i++) {
  let p = start + dir * t;
  if (any(p.xy < vec2f(0.0)) || any(p.xy >= u.screen)) {
    break;                              // left the screen: a miss
  }
  let size = f32(1u << u32(lvl));       // pixels per cell at this level
  let cell = floor(p.xy / size);
  // t where the ray leaves this cell, through the side it reaches first
  let side = (cell + step(vec2f(0.0), dir.xy)) * size;
  let tCross = (side - p.xy) / dir.xy;
  let tEnd = t + min(tCross.x, tCross.y) + 0.001;
  let zEnd = start.z + dir.z * tEnd;    // the ray's depth there
  let cellDepth = textureLoad(hiZ, vec2i(cell), lvl).r;
  if (zEnd < cellDepth) {
    // in front of everything in this cell: cross it, then climb
    t = tEnd;
    lvl = min(lvl + 1, MAX_LEVEL);
  } else if (lvl == 0) {
    return p.xy;                        // behind one pixel's depth: a hit
  } else {
    lvl--;                              // it might hit: look closer
  }
}
return vec2f(-1.0);                     // no hit

step(vec2f(0.0), dir.xy) is 1 on each axis the ray moves forward along and 0 on each it moves back along, so side is the cell's far edge in the ray's direction. (Renderers nudge dir.x and dir.y away from exactly zero first, to avoid dividing by it.) Like the demo, this assumes the ray heads away from the camera, so its deepest point in a cell is where it leaves. A reflected ray coming back toward the camera compares the depth where it enters the cell instead.

The hero's floor reflections use plain marching with growing steps for simplicity. Production SSR adds roughness (tracing a few jittered rays and blurring by the cone of a glossy lobe), fades out where rays leave the screen or point toward the camera, and falls back to reflection probes (Chapter 24) wherever it misses.

Screen-space global illumination (SSGI) follows the same pattern for diffuse light: from each pixel, march a few short rays in random directions over the hemisphere, and where one hits, pick up the color already lit there. It catches the near-field bounce that probes (Chapter 22) are too coarse for: the red glow on the floor beside the red wall, the green tint under the green block. It inherits SSR's blindness to anything off screen or hidden, so it's used to add detail on top of probes or other broad GI, not to replace them.

Contact shadows are the cheapest of all: from each pixel, march a short distance, a few centimeters to a meter, toward the light through the depth buffer. If the ray goes behind something, the pixel is in shadow. Shadow maps (Chapter 20) have texels far too coarse for the shadow under a coin or between fingers, or where a character's foot touches the ground; a contact-shadow ray gets them almost for free.

Here is the hero's contact-shadow loop. Its G-buffer stores each pixel's view-space position rather than a raw depth, so "depth" here is the z of that position.

// Contact shadow for one pixel: P is its view-space position, N its
// normal, L the direction toward the sun. March 14 steps of 3 cm.
let start = noise4(pix.xy) * 0.03;      // a different start per pixel
for (var k = 1; k <= 14; k++) {
  let Q = P + N * 0.02 + L * (start + f32(k) * 0.03);
  let px = toPixel(Q);                  // project the point to the screen
  if (!onScreen(px)) {
    break;
  }
  let S = posAt(px);                    // the surface the camera sees there
  // Shadowed if the point is behind that surface, but by less than 30 cm.
  // Deeper than that, the ray is probably passing behind it, not into it.
  if (S.w > 0.0 && Q.z < S.z - 0.02 && S.z - Q.z < 0.3) {
    lit = 0.0;
    break;
  }
}

The four helpers it calls are the hero's own. u.proj is the camera's projection matrix, u.screen.xy the screen's size in pixels, and posTex the G-buffer's position target:

// View space to pixel: project, divide by w, then map -1..1 to pixels
// (y flipped, since pixel rows count downward)
fn toPixel(v: vec3f) -> vec2f {
  let c = u.proj * vec4f(v, 1.0);
  return (c.xy / c.w * vec2f(0.5, -0.5) + 0.5) * u.screen.xy;
}

// Is this pixel inside the screen?
fn onScreen(px: vec2f) -> bool {
  return all(px >= vec2f(0.0)) && all(px < u.screen.xy);
}

// The view-space position stored at a pixel (w is 0 for sky); the clamp
// keeps the read inside the texture
fn posAt(px: vec2f) -> vec4f {
  let last = vec2i(u.screen.xy) - 1;
  return textureLoad(posTex, clamp(vec2i(px), vec2i(0), last), 0);
}

// A number from 0 to 1, different for each pixel of every 4 x 4 block,
// the 16 values spread evenly (the blur afterward hides the pattern)
fn noise4(p: vec2f) -> f32 {
  let b = array<f32, 16>(0.0, 8.0, 2.0, 10.0, 12.0, 4.0, 14.0, 6.0,
                         3.0, 11.0, 1.0, 9.0, 15.0, 7.0, 13.0, 5.0);
  let i = vec2u(p) % 4u;
  return (b[i.y * 4u + i.x] + 0.5) / 16.0;
}

The camera looks down the −z axis, so a smaller z is farther away: Q.z < S.z means the ray point is behind the visible surface. S.w is zero where nothing was drawn (the sky). The small offsets (0.02 along the normal, and 0.02 in the depth test) keep a surface from shadowing itself.

All of these produce noise when they use few rays, so renderers run them with a pattern of directions that differs pixel to pixel, then smooth it with a small blur that respects depth edges (as the hero does), or accumulate it over frames (Chapter 31).

Every screen-space effect reasons from a single layer of surfaces, the nearest ones, seen from the camera. So all of them fail the same ways: things off screen don't exist (turn the camera and reflections vanish at the edge); the backs of objects don't exist (a thin pole is assumed infinitely thick, so it occludes and reflects too much); and anything hidden behind something else doesn't exist. Renderers soften these failures with fades and fallbacks, and the newest ones hand the hard cases to ray tracing against the real scene (Chapter 36), keeping screen-space tracing as a fast first attempt that resolves most rays cheaply.

A heightmap is a depth buffer seen from above, so the same horizon search shades terrain beautifully. This playground computes horizon AO for every pixel of a procedural landscape. Try fewer DIRECTIONS (you can see each direction's streaks), a larger RADIUS (valleys darken broadly), or multiply the AO into a simple sun shading.

• Horizon-based AO (HBAO, GTAO) finds the highest horizon on each side of a pixel in a few screen directions, and integrates the visible, cosine-weighted sky between them; it beats kernel SSAO in quality per sample. The horizons also give a bent normal. HBAO+ trades some accuracy for speed: simpler per-sample math, and deinterleaved rendering that keeps the GPU's caches busy.
• Screen-space reflections march the reflected ray through the depth buffer; Hi-Z tracing uses a nearest-depth pyramid to take large steps over empty space.
• SSGI marches short hemisphere rays and gathers the lit colors they hit; contact shadows march a short ray toward the light for fine shadows that shadow maps miss.
• All of them see only the front, visible layer: they miss off-screen and hidden geometry, and treat everything as infinitely thick. Fades, fallbacks and ray tracing cover the gaps.

Chapter 34 turns from shading to the geometry side of the frame: letting the GPU decide what to draw.

1. Directional sunlight on the terrain. In the playground, add sun shading: estimate the slope from neighboring heights, light it with a sun direction, and multiply by the AO. How much does the AO add?

   Hint

   The normal is roughly normalize([h(x-1) - h(x+1), h(y-1) - h(y+1), 2]) (using the heights left/right and up/down). Lambert with a low sun, plus a little sky light times AO, gives a convincing relief map.

2. A horizon for shadows. The same marching, done in just one direction (toward the sun), tells whether the sun is above the horizon at each pixel. Turn the playground into a terrain shadow renderer.

   Hint

   March toward the sun's direction on the map, and compare each rise ÷ distance with the tangent of the sun's elevation. That's exactly a contact shadow, on a heightmap, and how some terrain renderers precompute their shadows.

3. Halos. In the hero, look at the floor next to the thin pole with horizon AO on. Why is there a dark streak beside it, and how would a "thickness" guess help?

   Hint

   The depth buffer has no idea the pole is thin, so its horizon extends as if a wall stood behind it. Assuming everything is at most, say, 30 cm thick, and ignoring samples that are much further behind the sampled surface than that, reduces the halo.