Testing Workers in CI

Worker code has a reputation for being untestable, and it is half deserved: the algorithm is ordinary JavaScript that any runner can test, while the boundary needs a real browser and behaves differently from anything else in the suite. Treating those as one problem is what produces a suite that is either useless or permanently flaky.

This topic sits under Debugging, Profiling & Production Optimization and covers the split: what to test where, how to keep message-based assertions deterministic, and the CI-specific failures — core counts, origins, timeouts, isolation headers — that make a green local run mean nothing.

The Four Layers, and Where Each Is Tested

The shape of a healthy suite follows from the table: the vast majority of tests never touch a thread, a small deliberate set proves the boundary works, and a few end-to-end tests prove the whole thing is wired together. Suites that struggle usually have the opposite distribution — everything is an integration test, so every test is slow and every failure is ambiguous.

Prerequisites

• A worker entry file that contains no logic. If the parser lives inside onmessage, it can only be tested through a thread. Extract first; the refactor is usually mechanical.
• A browser-backed runner for the boundary layer — Vitest’s browser mode, Web Test Runner, Karma, or Playwright’s component testing. Which one matters far less than having one.
• A pinned browser binary in CI, so an engine update cannot change the suite’s behaviour overnight.
• An HTTP origin for tests. file:// breaks worker construction in every engine; the runner must serve over http://localhost.
• Isolation headers in the test server if any test touches SharedArrayBuffer, or those tests fail with a constructor that does not exist.

Step 1 — Split the Entry From the Algorithm

// core/aggregate.ts — pure, importable anywhere, the bulk of the tests
export interface Summary { rows: number; sum: number; nulls: number }

export function aggregate(values: Float64Array): Summary {
  let sum = 0, nulls = 0;
  for (const v of values) { if (Number.isNaN(v)) nulls++; else sum += v; }
  return { rows: values.length, sum, nulls };
}

// aggregate.worker.ts — wiring only; nothing here needs its own unit test
import { aggregate } from './core/aggregate';

self.addEventListener('message', (event: MessageEvent<{ id: number; values: Float64Array }>) => {
  const { id, values } = event.data;
  try {
    self.postMessage({ id, ok: true, summary: aggregate(values) });
  } catch (error) {
    self.postMessage({ id, ok: false, error: (error as Error).message });
  }
});

Everything interesting is now in core/aggregate.ts, which any runner tests in milliseconds with no environment at all. The worker file has one behaviour worth a test — that it answers with the right id and converts a throw into an ok: false reply — and that test belongs to the boundary layer.

Step 2 — Test the Protocol Without a Thread

Request correlation, credit accounting, retry and timeout logic are pure state machines. Driving them through a MessageChannel gives a faithful async environment with no worker, no bundler, and no flakiness:

// protocol.test.ts — both ends of the real protocol, one thread
import { RequestClient } from '../src/request-client';
import { serve } from '../src/worker-entry-logic';

it('rejects the matching request when the responder reports an error', async () => {
  const { port1, port2 } = new MessageChannel();
  serve(port2, { aggregate: () => { throw new Error('boom'); } });   // the entry's logic
  const client = new RequestClient(port1);

  await expect(client.request({ kind: 'AGGREGATE', values: new Float64Array(4) }))
    .rejects.toThrow('boom');
});

This is the layer where message-shaped bugs actually live: an id that is never cleared from the pending map, a reply matched to the wrong request after a retry, a credit not returned on the error path. All of them are reproducible here, deterministically, in a millisecond.

Step 3 — Prove the Boundary Once Per Method

Boundary tests answer questions the layers above cannot: does this argument survive structured clone, does the transfer actually detach, does expose run before the first call, does an error arrive with a usable message.

// boundary.test.ts — runs in a real browser
import { workerApi } from '../src/worker-service';

it('moves the buffer rather than copying it', async () => {
  const values = new Float64Array([1, 2, 3, NaN]);
  const summary = await workerApi.aggregate(transfer(values, [values.buffer]));

  expect(values.byteLength).toBe(0);            // proves transfer, not clone
  expect(summary).toEqual({ rows: 4, sum: 6, nulls: 1 });
});

it('propagates a worker-side failure as a rejection', async () => {
  await expect(workerApi.aggregate(null as never)).rejects.toThrow();
});

One test per exposed method is enough. The goal is coverage of the boundary, not of the logic — the logic already has hundreds of fast tests behind it.

Step 4 — Remove Every Timing Assumption

The single largest source of worker-test flake is asserting on elapsed time. Machines differ, CI runners are slower and noisier, and a threshold that holds on a laptop fails on a shared runner under load.

// ✗ Flaky: assumes the worker is faster than 200 ms on every machine, forever.
worker.postMessage({ kind: 'RUN' });
await sleep(200);
expect(received).toHaveLength(1);

// ✓ Stable: waits for the event that actually indicates completion.
const done = new Promise<Result>((resolve, reject) => {
  const timer = setTimeout(() => reject(new Error('worker did not reply within 10 s')), 10_000);
  worker.addEventListener('message', function on(event: MessageEvent) {
    if (event.data.id !== id) return;                     // ignore unrelated traffic
    worker.removeEventListener('message', on);
    clearTimeout(timer);
    resolve(event.data);
  });
});
worker.postMessage({ kind: 'RUN', id });
expect(await done).toMatchObject({ ok: true });

The timeout still exists — a hung worker must fail rather than hang the suite — but it is a failure threshold set generously, not a success threshold set tightly. Ten seconds never fires on a healthy run and always fires on a deadlock.

Step 5 — Make the Environment Explicit

Two values must never be read from the environment during a test.

Pool size. navigator.hardwareConcurrency is 8 on a developer laptop and 2 on a typical CI container. A pool that sizes itself will run a different concurrency in each, so ordering-sensitive tests pass in one and fail in the other. Inject it: new Pool({ size: options.size ?? navigator.hardwareConcurrency }).

Worker URLs. A test that constructs a worker from a hard-coded path breaks whenever the bundler’s output layout changes. Construct through the same factory the application uses, so the test exercises the real resolution path.

Everything else — timeouts, retry counts, credit windows — should be parameters with production defaults, so a test can set them to values that make the behaviour observable in milliseconds rather than seconds.

Step 6 — Make Concurrency Deterministic

Pool and queue tests are where suites become mysteriously flaky, because the assertion usually depends on which worker picked up which job — an outcome that varies with core count and scheduling.

The fix is to make the interleaving explicit rather than incidental. With a stubbed worker whose reply timing the test controls, every ordering question becomes a decision instead of a race:

// pool.test.ts — the test decides who finishes first
const replies = new Map<number, (value: Result) => void>();
const pool = new Pool({ size: 2, create: () => new ControlledWorker(replies) });

const a = pool.run({ id: 1 });
const b = pool.run({ id: 2 });
const c = pool.run({ id: 3 });          // queued: both workers are busy

expect(pool.stats()).toMatchObject({ busy: 2, queued: 1 });

replies.get(2)!({ ok: true });          // the SECOND job finishes first, deliberately
await b;
expect(pool.stats()).toMatchObject({ busy: 2, queued: 0 });   // c was dispatched

Three properties are worth asserting this way, and none of them can be tested reliably against real threads: that a queued job is dispatched as soon as any worker frees up, that results are matched to their own requests when they complete out of order, and that a rejected job releases its worker rather than leaking a slot.

Pin the pool size in every such test. A pool that reads navigator.hardwareConcurrency produces busy: 8 on a laptop and busy: 2 on a runner, and the expectations above become environment-dependent — the single most common cause of “passes locally, fails in CI” in worker code.

Testing Performance Claims Without Testing Performance

A worker exists to keep the main thread free, so it is tempting to assert on timing. Wall-clock assertions are the wrong tool — they encode machine speed — but the underlying claim is testable if it is restated as a behavioural one.

Claim: “the main thread is not blocked during the computation.” Assert that it kept servicing frames, not that the job was fast:

test('the main thread keeps painting during a heavy job', async ({ page }) => {
  await page.evaluate(() => {
    (window as never as { frames: number }).frames = 0;
    const tick = () => { (window as never as { frames: number }).frames++; requestAnimationFrame(tick); };
    requestAnimationFrame(tick);
  });

  await page.getByRole('button', { name: 'Recalculate' }).click();
  await expect(page.getByTestId('summary')).toBeVisible();

  // A blocked main thread produces near-zero frames; a free one produces dozens.
  expect(await page.evaluate(() => (window as never as { frames: number }).frames)).toBeGreaterThan(20);
});

The threshold is deliberately loose. It distinguishes “the thread was free” from “the thread was blocked”, which is the architectural property under test, and it does not encode how fast the machine is — a slow runner simply produces fewer frames in the same wall clock, and 20 is far below what any unblocked thread manages.

Claim: “work happens off the main thread.” That is a structural assertion, checked by observing that a worker exists and executed, not by timing anything. The end-to-end techniques for it are in End-to-End Worker Tests with Playwright.

Claim: “this is faster than the previous implementation.” That is a benchmark, not a test. Run it separately, compare distributions rather than single runs, and never gate a pull request on a wall-clock number produced by a shared CI runner.

Verification & Measurement

Track suite duration by layer. If the browser-backed layer grows past a minute or two, tests have leaked upward out of the algorithm layer. The fix is to move them back down, not to parallelise harder.

Fail the build on unhandled rejections. A worker error that nobody awaited will not fail a test by default; it becomes an unhandled rejection that most runners print and ignore. Configure the runner to treat those as failures, or a broken error path can stay green indefinitely.

Re-run the suite N times before trusting a green. Flake is a probability, not a state. A --repeat-each=5 run on the boundary and integration layers, once per week in CI, surfaces the timing assumptions that a single run hides.

Choosing a Runner for the Boundary Layer

Four options are in common use, and the differences that matter for worker tests are narrower than the general comparisons suggest.

Vitest browser mode runs your existing Vitest suite in a real engine driven by Playwright or WebDriver. Its advantage for worker code is that Vite’s own worker resolution applies, so new URL('./w.ts', import.meta.url) behaves exactly as it does in the application — the same transform pipeline, the same output shape. If the project already builds with Vite, this is the path of least surprise.

Web Test Runner is bundler-agnostic and serves real modules over HTTP with esbuild transforms. It is a good fit when the application is not built with Vite, and its coverage of worker scope is respectable, but any bundler-specific worker spelling has to be reproduced in its configuration.

Playwright directly is the heaviest option per test and the only one that can assert on the built artefact — which makes it the right home for the integration layer rather than the boundary layer. Using it for both means paying browser-launch cost for tests that only needed a Worker object.

Karma still works and still runs everywhere, and it is the pragmatic answer for a legacy suite that already uses it. It is not worth adopting for new work.

Whichever you choose, the properties to insist on are the same three: tests are served over a real HTTP origin, the runner can send custom response headers (for cross-origin isolation), and the browser binary is pinned. A runner missing any of those will produce failures that look like application bugs and are not.

One decision that outranks the choice of tool: run the boundary suite against one engine on every commit and the others on a schedule. Engine differences in module workers and structured clone are real and worth catching, but catching them within an hour rather than within a minute is a perfectly good trade for a suite that stays fast enough to run before every push.

Failure Modes & Error Handling

A green suite that never exercised a worker. If the boundary project is skipped when its browser is missing, CI reports success having run only the fast layer. Fail the build when a project is skipped rather than treating it as an optional extra, or the most valuable tests quietly stop running.

A worker that never replies hangs the runner. Without a per-request timeout, the suite waits for the global test timeout, reports an unhelpful message, and leaves the thread running. Always fail fast with a message naming the request.

Workers that outlive their test. A worker created in a test and never terminated keeps running between tests, consuming CPU and occasionally answering a later test’s message. Terminate in afterEach, and assert that the count of live workers returns to its baseline.

Errors swallowed by the entry file. A try/catch around the whole onmessage handler that posts { ok: false } will hide a programming error as a soft failure. Distinguish expected failures from bugs, and let bugs propagate to self.onerror so Fixing Uncaught Exceptions in Dedicated Workers applies.

Retries that hide a real defect. A single retry is reasonable insurance against infrastructure noise; a suite that needs two or three is telling you the worker tests contain a race. Track which tests consume retries and treat a repeat offender as a bug report rather than as background noise — worker races almost never stay benign.

Cross-test state inside a warm worker. A worker reused across tests keeps its module state — caches, loaded datasets, counters. Either reset it explicitly through the API or create a fresh worker per test file, and prefer the former, since construction is the slow part.

Browser Compatibility for Test Runners

Chromium is the pragmatic default for the boundary layer because coverage and debugging inside worker scope work best there. Running the same suite against Firefox and WebKit is still worth doing — engine differences in module workers and structured clone are real — but as a scheduled cross-browser job rather than on every commit.