Picker

star-map20 000 stars; the one under the pointer is picked on the GPU.

PauseOpen the full example

Hover a star to light it in the same frame through found(). result reaches the CPU frames later. The dashed ring shows reach around the pointer. Set nearest to eye to choose the frontmost star within reach. The picker’s rank reads this choice from a uniform. Click a star to select it.

Wisps move on the GPU, so the CPU needs their positions to find one under a finger. Reading all positions copies the whole set and adds a frame or two of delay. A picking kernel instead projects every candidate with your camera and selects the nearest within reach. It returns one index. Later kernels can use that index in the same frame, before it reaches the CPU.

const hits = picker.create(root, view, {
  capacity: 512,
  size: surface,
  position: (i) => {
    'use gpu';
    return d.vec3f(wisps.$[i]!);
  },
});

let tap: [number, number] | undefined;
canvas.addEventListener('pointerup', (event) => {
  tap = [event.offsetX / canvas.clientWidth, event.offsetY / canvas.clientHeight];
});

frames.create(root, (frame) => {
  const compute = frame.encoder.beginComputePass();
  // ... the steps that move the wisps
  if (tap) hits.pick(compute, ...tap);
  compute.end();
  hits.result.capture(frame);
  tap = undefined;

  const found = hits.result.take(); // undefined until a pick comes back
  if (found !== undefined) select(found);
});

• x and y run from 0 to 1 across the view from its top-left corner, as a host’s pointing and a camera’s rayAt take them. settings.reach is a distance in points. The picker reads size and its pixelRatio at each pick, as text does, so reach stays the same when a window moves between screens. Without a pixelRatio, a point is a pixel.
• A candidate is within reach when its centre is within reach points, plus its radius as it looks from the eye. One behind the eye, or one alive turns down, is never picked.
• pick writes the point and clears the result with queue writes, so a frame picks once; a second pick replaces the first.
• result is a readback of the candidate’s index, or null for nothing. Capture it after the pass that picks. Without hz, each tap comes back once, a frame or two later.

Which one

Several candidates can be within reach. rank is a 'use gpu' function that maps a picker.Candidate to a number from 0 to 1. The lowest rank wins. Ties go to the lower index. A candidate holds:

• index;
• gap, from the point to its centre on the screen, as a fraction of the screen’s diagonal;
• depth, 0 at the near plane and 1 at the far.

picker.byPointer, the default, ranks by gap, for wisps a finger covers. picker.byEye ranks by depth, the one in front, for a hover among stars that overlap on the screen.

A rank that reads a uniform switches between them without a new picker, as the star map’s nearest does:

const byEye = root.createUniform(d.u32, 0);
const hits = picker.create(root, view, {
  capacity: 512,
  size,
  position: (i) => {
    'use gpu';
    return d.vec3f(stars.$[i]!);
  },
  rank: (candidate) => {
    'use gpu';
    return std.select(picker.byPointer(candidate), picker.byEye(candidate), byEye.$ === 1);
  },
});

byEye.write(1); // the next pick keeps the star in front

radius, in world units, makes a big candidate easier to hit, and grows on the screen as it comes closer. Here guns are wider than wisps:

const hits = picker.create(root, view, {
  capacity: WISPS + 32,
  size,
  position: (i) => {
    'use gpu';
    if (i < WISPS) return wisps.$[i]!.xyz;
    return d.vec3f(guns.$[i - WISPS]!);
  },
  radius: (i) => {
    'use gpu';
    return std.select(0, 0.6, i >= WISPS);
  },
  alive: (i) => {
    'use gpu';
    return i >= WISPS || wisps.$[i]!.w > 0;
  },
});

The picker kernel compiles these callbacks. They can read cameras.current.$ and any resources they have fills for, such as Star.position.pipe(stars.inject()). Candidates in a table take its capacity, alive and span from stars.live(): picker.create(root, view, { ...stars.live(), size, position }).

In the same frame

hits.found() is the candidate the last pick found, or picker.none, in any shader created from root.pipe(hits.inject()). A kernel after pick sees this frame’s pick, so a tapped wisp lights up the frame the tap lands:

const mark = tgpu.computeFn({ workgroupSize: [1] })(() => {
  'use gpu';
  const wisp = hits.found();
  if (wisp === picker.none) return;

  marked.$[focus.$] = 0;
  marked.$[wisp] = 1;
  focus.$ = wisp;
});

const marking = root.pipe(hits.inject()).createComputePipeline({ compute: mark });

Hovering

For hovering, pick every frame and set hz to control readback. Only captures due at that rate copy the result. The star map reads at 20 Hz. result.latest holds the newest completed read, a frame or two behind. result.latestFrame identifies the frame that picked it. take() returns each completed read once.

When the pointer leaves, call hits.clear() and stop picking. found() returns none from then on, and the next capture reads null, without a dispatch.

On the CPU

For a few candidates already on the CPU, use the camera’s projection to compare their screen positions without a kernel.

Limits

• atomicMin selects the lowest packed key. The index uses the bits needed for capacity. Rank uses the remaining bits, capped at 24. It gets 23 bits at 512 candidates and 8 bits at 2^24. Ranks within one quantisation step can tie.
• A centre and a radius only approximate a tall or long candidate. A kernel of your own, such as a ray through boxes, can read found() beside its own result.
• A second pointer needs a picker of its own.