A finished look for model scenes

24 × 24 px

ShadowsOcclusionTAABloomTurn

Drag to look round, and turn each stage off to see what it adds. Shadows ground the houses, occlusion darkens the sky light under the trees and eaves, TAA smooths the roof edges in the loupe, and bloom spreads the lanterns' light.

A lighting and a set of instances draw glTF models lit by the sun and the sky. On their own they look flat. Every surface gets the whole sky, nothing seems to rest on the ground, and edges step and shimmer as the view turns. Four stages finish the picture:

• The sun’s shadows, in cascades fitted to the view, set each model on the ground.
• Ambient occlusion darkens the sky light where nearby surfaces hide the sky: under eaves, between trees and where a wall meets the grass.
• TAA smooths edges and thin parts by taking a sample at a different offset each frame.
• Bloom spreads light brighter than white, and the final pass tone maps and grades the picture.

The program splits into three files. look.ts holds the lighting and every pass from the prepass to the canvas, so another program can copy it whole. The package ships it and props.ts whole in docs/recipes/lit-models/:

• scene.ts createScene with the camera, the props, the look and the frame
• look.ts createLook with the lighting, the passes and the post-processing
• props.ts createProps with the mesh pool and the instances

The props

Put every model in one mesh pool and every piece in one set of instances. The set draws each mesh once, however many pieces use it, so a thousand houses cost what one does on the CPU:

props.ts

import { instances, meshPool, owner } from '@typegpu/kit';
import type { TgpuRoot } from 'typegpu';

/** A piece of the scene: a model by name, and its row's values. */
export interface Placement {
  readonly model: string;
  /** Its position, rotation, scale and any material field, such as `emission`. */
  readonly values: instances.Values;
}

/** One pool of every model and shape, and one set of instances placing them. */
export function createProps(
  root: TgpuRoot,
  models: Readonly<Record<string, meshPool.ModelData | meshPool.Shape>>,
  placements: readonly Placement[],
) {
  const own = owner();
  const pool = own(meshPool.create(root, meshPool.roomFor(Object.values(models))));
  // A shape, such as `meshes.cylinder()`, is one mesh with nothing to decode
  const add = (item: meshPool.ModelData | meshPool.Shape) =>
    'meshes' in item ? pool.add(item) : { meshes: [pool.add(item)], ready: Promise.resolve() };
  const added = new Map(Object.entries(models).map(([name, item]) => [name, add(item)]));
  const modelOf = (name: string) => added.get(name)!;

  // A model takes a row for each of its meshes
  const rows = placements.reduce((sum, { model }) => sum + modelOf(model).meshes.length, 0);
  const props = own(instances.create(root, pool, { capacity: rows }));
  for (const { model, values } of placements) props.add(modelOf(model), values);

  return {
    renderer: props.renderer(),
    /** Settles once every model's textures are on the GPU. */
    ready: Promise.all([...added.values()].map((model) => model.ready)),
    flush: () => props.flush(),
    destroy: own.destroy,
  };
}

• meshPool.roomFor sizes the pool for every model’s vertices and indices. A shape such as meshes.cylinder() goes in models beside the glTF models, as the figure’s lantern posts do.
• A row starts with its mesh’s material from the file. Values you give replace it, so a lantern’s glass takes an emission above 1 and glows through the bloom.
• Pieces with shaders of their own, such as turning sails or lit windows, take a kind in a second set from the same pool. The look draws its renderer the same way.

Each placement carries the values its row starts with:

/** A turn of `yaw` radians round +y, as a quaternion. */
const turned = (yaw: number) => [0, Math.sin(yaw / 2), 0, Math.cos(yaw / 2)] as const;

const placements: Placement[] = [
  { model: 'church', values: { position: [0, 0, 0] } },
  { model: 'house', values: { position: [2, 0, 0], rotation: turned(Math.PI / 3) } },
  { model: 'house', values: { position: [-2, 0, 0], rotation: turned(Math.PI), scale: 0.9 } },
  { model: 'tree', values: { position: [1, 0, 1.73], scale: 1.2 } },
  { model: 'lantern', values: { position: [1, 0, 0.58], emission: [6, 3.6, 1.8] } },
];

The lighting and the occlusion

Ambient occlusion reads a depth texture drawn before the scene shades. The look draws every solid prop into a half-resolution prepass, traces the occlusion from it and dims the sky light by it in the lighting’s ambient hook. The sun’s shadow fits two cascades to the view:

look.ts

/** Sky light, dimmed where nearby surfaces hide the sky. */
const occludedSky = (material: shading.Material, point: shading.Point) => {
  'use gpu';
  return shading.hemisphere(material, point) * ao.at(point.position);
};

/** The lighting and every pass from the prepass to the canvas, seen through `view`. */
export function createLook(
  root: TgpuRoot,
  surface: Surface,
  view: cameras.OrbitCamera,
  options: LookOptions,
) {
  const { stages } = options;
  const own = owner();
  // Half-resolution depth of the solid props, drawn first for the occlusion to read
  const early = own(
    targets.create(root, surface, {
      depth: { format: 'depth32float', scale: 0.5, sampled: true },
    }),
  );
  const prepass = passes.create(root, early, { depth: passes.depth }).pipe(view.inject());
  const occlusion = own(
    ao.create(root, view, {
      depth: () => early.textures.depth,
      temporal: true, // turns its directions each frame for TAA to average
      settings: { radius: options.crease, power: 3 },
      enabled: () => stages.occlusion,
    }),
  );
  const light = own(
    lighting.create(root, {
      shading: { ambient: tgpu.fn(occludedSky).pipe(occlusion.inject()) },
      shadow: {
        eye: view,
        cascades: 2,
        far: options.far,
        filter: 9,
        enabled: () => stages.shadows,
      },
    }),
  );
  light.settings.patch(options.light);

• Set far a little past the farthest prop the view shows. The cascades spread the same texels over far, so a longer reach blurs every shadow. See cascades fitted to the view.
• Set crease near the size of the gaps the occlusion should darken. The figure uses 0.7 world units for houses about one unit tall.
• The occlusion dims only the sky light. Sunlit faces keep their brightness, and the sun’s shadow keeps its own edge.
• For reflections of a sky panorama, call environment.occluded with the same ao.at in place of shading.hemisphere, as shading with an environment shows.

The picture

The scene shades into an rgba16float target through the lighting, offset by TAA’s jitter. TAA blends each frame with the last into a smooth picture, which the bloom and the final pass both read. Inside createLook, after the lighting:

look.ts

/** Cools the shadows and warms the lights after the tone curve. */
const dusk = (color: d.v3f, _uv: d.v2f) => {
  'use gpu';
  const light = std.saturate(tone.luminance(color) * 2);
  return color * std.mix(d.vec3f(0.84, 0.93, 1.14), d.vec3f(1.02, 1, 0.96), light);
};
  const screen = own(
    targets.create(root, surface, {
      hdr: { format: 'rgba16float', sampled: true },
      depth: { format: 'depth24plus', sampled: true },
    }),
  );
  const aa = own(
    taa.create(root, {
      source: () => screen.textures.hdr,
      depth: () => screen.textures.depth,
      camera: view,
    }),
  );
  // Before the pipelines that read `ao.at`, so a resize keeps them
  const shared = bindings.share(root, { ao: occlusion.part });
  const scene = passes
    .create(root, screen, { hdr: passes.color, depth: passes.depth }, { jitter: aa.jitter, shared })
    .pipe(view.inject())
    .beneath(light.inject());
  const glow = own(bloom.create(root, { source: aa }));
  const final = own(finish.create(root, { format: surface.format, source: aa, glow, grade: dusk }));

• TAA reprojects last frame’s picture by the camera’s motion and the scene’s depth, so its depth target is sampled. Props that hold still need nothing more.
• Props that move need motion of their own. Give the scene a motion: taa.motion target, TAA its texture as motion, and the moving set history: true.
• The bloom spreads only light above its threshold, 1 by default. A lit wall stays below it, and a lantern’s glass at emission 6 glows.
• grade runs inside the final pass, after the tone curve, on colours in display range. See grading.

Drawing and recording

The look adds each renderer to the three passes that draw it, and records the frame in the order each stage reads the one before. Inside createLook, after the final pass:

look.ts

  const shapes: passes.Drawn[] = [];
  const casters: passes.Drawn[] = [];
  const drawn: passes.Drawn[] = [];

  return {
    light,

    /** Draws `renderer` in the prepass, the sun's shadow and the scene. */
    add(renderer: instances.Renderer) {
      shapes.push(own(prepass.add(renderer)));
      casters.push(own(light.shadow.add(renderer)));
      drawn.push(own(scene.add(renderer)));
    },

    /** Applies `stages` to TAA and the final pass, which keep their own state. */
    apply() {
      aa.enable(stages.taa);
      final.settings.patch({ glowIntensity: stages.bloom ? GLOW : 0 });
    },

    initAsync: () => pipelines.compile([prepass, occlusion, light.shadow, scene, aa, glow, final]),

    record(frame: Frame) {
      early.fit();
      screen.fit();
      light.shadow.record(frame, casters); // a depth pass per cascade
      if (stages.occlusion) prepass.record(frame, shapes);
      occlusion.record(frame); // reads the prepass
      scene.record(frame, drawn); // reads the shadow and the occlusion
      aa.record(frame);
      if (stages.bloom) glow.record(frame); // reads TAA's picture
      final.record(frame, surface.view(frame));
    },

    destroy: own.destroy,
  };

own frees everything the look made, the last first, including the draws add built.

The scene

scene.ts composes the parts. It owns the camera, hands it to the look, and adds the props’ renderer:

scene.ts

import { cameras, owner, type Frame, type gltf, type Surface } from '@typegpu/kit';
import type { TgpuRoot } from 'typegpu';
import { createLook } from './look.ts';
import { createProps, type Placement } from './props.ts';

export interface SceneOptions {
  /** Each model as `gltf.read` returns it, by name. */
  readonly models: Readonly<Record<string, gltf.ModelData>>;
  readonly placements: readonly Placement[];
}

export async function createScene(root: TgpuRoot, surface: Surface, options: SceneOptions) {
  const own = owner();
  const view = own(
    cameras.orbit(root, { size: surface, target: [0, 0.2, 0.5], distance: 14, pitch: 0.6 }),
  );
  const props = own(createProps(root, options.models, options.placements));
  const look = own(
    createLook(root, surface, view, {
      far: 28,
      crease: 0.7,
      light: { sunDirection: [-0.75, 0.4, -0.45], sunColor: [1, 0.66, 0.4], sunIntensity: 1.6 },
      stages: { shadows: true, occlusion: true, taa: true, bloom: true },
    }),
  );
  look.add(props.renderer);
  look.apply();
  await Promise.all([props.ready, look.initAsync()]);

  return {
    view,
    frame(frame: Frame) {
      view.update(frame.dt);
      props.flush();
      look.record(frame);
    },
    destroy: own.destroy,
  };
}

A game adds its own parts beside the props, such as a picker for the piece under the pointer or lines for a selection outline. Each draws through the same look by its renderer, or records after look.record into the canvas.

What each stage costs

• Shadows draw every caster once per cascade. The map’s 2048-texel layers take 8 MB each in the default depth16unorm, and filter: 9 takes nine comparisons per shaded pixel. One cascade suits a view that shows a small board from one height.
• Occlusion draws every solid prop again into the half-resolution prepass, then traces and denoises at half resolution, a quarter of the pixels.
• TAA takes one full-screen pass and two rgba16float history textures. FXAA costs one pass on the finished 8-bit picture and keeps no history. It smooths steps in a still frame but leaves thin parts shimmering as the view turns, so prefer it on the weakest phones.
• Bloom draws six levels down and five up, each half the size of the one before.
• The final pass tone maps, adds the glow and grades in one full-screen pass.

A stage turns off without rebuilding anything. The look reads its stages every frame: the shadow and the occlusion through their enabled options, the prepass and the bloom by skipping their recording. TAA and the glow keep state of their own, so apply passes a change on with aa.enable and the final pass’s glowIntensity. The figure above is this look, with buttons that flip its stages.