Dynamic vs Fixed-Size Worker Pools

Pick the wrong pool sizing strategy and you pay for it either in memory that idle workers hold hostage, or in response-time spikes when a burst arrives and there is no warm thread to serve it.

This decision sits at the centre of Worker Pool Management, part of the Web Workers Architecture & Communication reference, and it comes up the moment a working fixed pool — like the one built in Implementing a Simple Worker Pool in Vanilla JS — meets a workload that is idle most of the time. The two strategies are not better and worse; they trade the same resource in opposite directions, and the arrival pattern of your tasks decides which trade is the right one.

The Core Trade-off

Every worker costs roughly 2–5 MB of baseline V8 heap before your code allocates anything, plus its script and its in-flight data. A pool of 8 therefore starts at 16–40 MB. A fixed-size pool pays that bill upfront and never thinks about it again, buying sub-millisecond dispatch for every task. An elastic pool defers the bill, and pays instead in isolate startup latency on the first task of each burst.

Minimal Reproducible Example: An Elastic Pool

The smallest complete pool that grows on demand and shrinks when quiet. Three fields drive everything: a floor that is never evicted, a ceiling that bounds memory, and an idle timeout that decides how patient the pool is before releasing a worker.

// elastic-pool.ts
interface Task<T> {
  payload: unknown;
  resolve: (value: T) => void;
  reject: (reason: unknown) => void;
}

export class ElasticWorkerPool<T = unknown> {
  private readonly scriptUrl: string;
  private readonly minSize: number;
  private readonly maxSize: number;
  private readonly idleTimeoutMs: number;

  private idle: Worker[] = [];
  private busy = new Map<Worker, Task<T>>();
  private queue: Task<T>[] = [];
  private idleTimers = new Map<Worker, ReturnType<typeof setTimeout>>();

  constructor(
    scriptUrl: string,
    options: { min?: number; max?: number; idleTimeoutMs?: number } = {}
  ) {
    this.scriptUrl = scriptUrl;
    this.minSize = options.min ?? 1;
    this.maxSize = options.max ?? navigator.hardwareConcurrency;
    this.idleTimeoutMs = options.idleTimeoutMs ?? 10_000; // 10 s default

    // Pre-warm the floor so the first task never pays isolate startup.
    for (let i = 0; i < this.minSize; i++) {
      this.idle.push(this.spawnWorker());
    }
  }

  execute(payload: unknown): Promise<T> {
    return new Promise<T>((resolve, reject) => {
      this.queue.push({ payload, resolve, reject });
      this.drain();
    });
  }

  private drain(): void {
    while (this.queue.length > 0) {
      const worker = this.acquireWorker();
      if (!worker) break;                // At capacity — the queue waits.

      const task = this.queue.shift()!;
      this.busy.set(worker, task);
      worker.postMessage(task.payload);
    }
  }

  private acquireWorker(): Worker | null {
    if (this.idle.length > 0) {
      const w = this.idle.shift()!;
      clearTimeout(this.idleTimers.get(w));  // Re-use cancels a pending eviction.
      this.idleTimers.delete(w);
      return w;
    }

    const total = this.idle.length + this.busy.size;
    if (total < this.maxSize) {
      return this.spawnWorker();           // Grow: pay 5–15 ms once.
    }

    return null;                           // Ceiling reached — stay queued.
  }

  private spawnWorker(): Worker {
    const worker = new Worker(this.scriptUrl, { type: 'module' });
    worker.onmessage = (e: MessageEvent) => this.onComplete(worker, e.data as T);
    worker.onerror = (e: ErrorEvent) => this.onError(worker, e);
    return worker;
  }

  private onComplete(worker: Worker, result: T): void {
    const task = this.busy.get(worker);
    if (task) {
      task.resolve(result);
      this.busy.delete(worker);
    }
    this.returnToIdle(worker);
    this.drain();
  }

  private onError(worker: Worker, e: ErrorEvent): void {
    const task = this.busy.get(worker);
    if (task) {
      task.reject(new Error(e.message));
      this.busy.delete(worker);
    }
    worker.terminate();  // A crashed worker never goes back in the pool.
    this.drain();        // The next task simply grows a replacement.
  }

  private returnToIdle(worker: Worker): void {
    const total = this.idle.length + this.busy.size + 1;
    if (total > this.minSize) {
      // Above the floor: arm an eviction timer for this specific worker.
      const timer = setTimeout(() => {
        worker.terminate();
        this.idle = this.idle.filter(w => w !== worker);
        this.idleTimers.delete(worker);
      }, this.idleTimeoutMs);

      this.idleTimers.set(worker, timer);
    }
    this.idle.push(worker);
  }

  destroy(): void {
    for (const timer of this.idleTimers.values()) clearTimeout(timer);
    this.idleTimers.clear();
    [...this.idle, ...this.busy.keys()].forEach(w => w.terminate());
    this.idle = [];
    this.busy.clear();
    this.queue = [];
  }
}

Step-by-Step Walkthrough

this.minSize = options.min ?? 1;
this.maxSize = options.max ?? navigator.hardwareConcurrency;
this.idleTimeoutMs = options.idleTimeoutMs ?? 10_000;

Three knobs, and every behavioural difference between this pool and a fixed one comes from them. minSize is the warm floor: workers at or below this count are never evicted, so a floor of 1 already removes spawn latency from the first task of a burst. maxSize is the memory ceiling. idleTimeoutMs is the eviction window — how long a surplus worker may sit doing nothing before the pool releases its heap.

private acquireWorker(): Worker | null {
  if (this.idle.length > 0) { /* fast path: reuse, cancel eviction timer */ }
  const total = this.idle.length + this.busy.size;
  if (total < this.maxSize) {
    return this.spawnWorker();          // Grow inside the dispatch path.
  }
  return null;                          // At capacity: leave it queued.
}

Growth happens inline in the dispatch path rather than in a background monitor, which is what keeps the implementation honest: a task is never dropped and never silently delayed by a polling interval. The returned promise stays pending until either an idle worker frees up or a freshly spawned one accepts the payload. Note the ordering — the idle list is checked first, so a warm worker is always preferred to a new one even when the pool is far below its ceiling.

private returnToIdle(worker: Worker): void {
  const total = this.idle.length + this.busy.size + 1;
  if (total > this.minSize) {
    const timer = setTimeout(() => {
      worker.terminate();
      this.idle = this.idle.filter(w => w !== worker);
      this.idleTimers.delete(worker);
    }, this.idleTimeoutMs);
    this.idleTimers.set(worker, timer);
  }
  this.idle.push(worker);
}

This is the whole shrink path. Each worker returned above the floor arms its own eviction timer, keyed in idleTimers so it can be cancelled individually. If that worker is claimed again before the timeout fires, clearTimeout in acquireWorker cancels the eviction — the hysteresis that stops the pool tearing itself down between two clicks a second apart. Timers are per-worker rather than a single sweep interval, so eviction is staggered: a pool that grew to 8 for one burst releases workers one at a time as each individually goes cold, instead of collapsing to the floor in a single tick.

One deliberate omission: the pool never inspects the payload. Whether you post a structured-cloned object or hand over ownership of an ArrayBuffer with Transferable Objects & Zero-Copy is the caller’s decision, and it changes the economics — a pool whose per-task cost is dominated by cloning a 10 MB payload will not get faster by adding workers.

The Fixed-Size Baseline

For sustained CPU work — image pipelines, WebAssembly compute loops, CSV transforms — the elastic machinery earns nothing, because the pool is always at its ceiling anyway. Delete it:

// fixed-pool.ts
export class FixedWorkerPool<T = unknown> {
  private idle: Worker[] = [];
  private busy = new Map<Worker, { resolve: (v: T) => void; reject: (e: unknown) => void }>();
  private queue: Array<{ payload: unknown; resolve: (v: T) => void; reject: (e: unknown) => void }> = [];

  constructor(scriptUrl: string, size = navigator.hardwareConcurrency) {
    for (let i = 0; i < size; i++) {
      const w = new Worker(scriptUrl, { type: 'module' });
      w.onmessage = (e: MessageEvent<T>) => {
        const ctx = this.busy.get(w);
        if (ctx) { ctx.resolve(e.data); this.busy.delete(w); }
        this.idle.push(w);
        this.flush();
      };
      w.onerror = (e: ErrorEvent) => {
        const ctx = this.busy.get(w);
        if (ctx) { ctx.reject(new Error(e.message)); this.busy.delete(w); }
        this.idle.push(w);
        this.flush();
      };
      this.idle.push(w);
    }
  }

  run(payload: unknown): Promise<T> {
    return new Promise<T>((resolve, reject) => {
      this.queue.push({ payload, resolve, reject });
      this.flush();
    });
  }

  private flush(): void {
    while (this.idle.length > 0 && this.queue.length > 0) {
      const worker = this.idle.shift()!;
      const task = this.queue.shift()!;
      this.busy.set(worker, task);
      worker.postMessage(task.payload);
    }
  }

  destroy(): void {
    [...this.idle, ...this.busy.keys()].forEach(w => w.terminate());
    this.idle = [];
    this.busy.clear();
    this.queue = [];
  }
}

No timers, no grow path, no eviction bookkeeping — the dispatch loop is eight lines and the memory profile is a flat line at size × ~3 MB. That flatness is the feature: it is trivially reasoned about in a heap snapshot, and there is no configuration whose misjudgement can cause a latency spike in production.

Choosing Between Them

A fixed-size pool wins when:

• Load is continuous or near-continuous — video transcoding, real-time sensor processing, a batch job draining a large queue.
• Worst-case latency matters more than idle memory: trading dashboards, games, anything where a 15 ms spike at the head of an interaction is visible.
• Simplicity is worth paying for. Fewer moving parts means fewer failure modes, and no timer can misfire on a worker that was already terminated.
• You have already profiled spawn latency as a bottleneck and want it gone entirely.

An elastic pool wins when:

• Load is bursty and user-triggered — CSV exports, on-demand image resizing, a search-as-you-type index rebuild.
• The application runs where memory is scarce: low-end Android, browser extensions, or an embedded webview sharing a budget with a host app.
• Several independent features each want a pool. Four fixed pools of 8 is 32 idle isolates; four elastic pools with a floor of 1 is four.
• The gap between bursts reliably exceeds idleTimeoutMs, so shrinking actually happens rather than being cancelled every time.

Picking min, max and idleTimeoutMs

Instrument first: record performance.now() at every execute() call for a representative session and look at the distribution of the gaps between them. That histogram answers all three settings.

• maxSize — navigator.hardwareConcurrency - 1, clamped to 2–8, with a fallback of 4 when the property is missing. This is the same ceiling a fixed pool would use; elasticity changes when workers exist, not how many are useful.
• minSize — the number of tasks that must start instantly. For an interactive surface where one click triggers one job, 1 is enough; if a single interaction fans out into parallel work, set the floor to that fan-out width, commonly 2.
• idleTimeoutMs — at least twice the median inter-burst gap, and never below about 2 s. A timeout shorter than the gap between two interactions in the same user flow means every interaction pays spawn cost, which is the worst of both strategies: elastic latency with fixed-pool complexity.

Gotchas and Edge Cases

1. navigator.hardwareConcurrency reports logical cores, not physical ones

On an 8-core machine with hyper-threading it returns 16, and 16 compute-bound workers will fight over 8 physical execution units while adding 16 isolates’ worth of heap. On hybrid ARM designs it counts efficiency cores that contribute a fraction of a performance core’s throughput. Treat the value as an upper bound to clamp, not a recommendation: Math.min(8, Math.max(2, (navigator.hardwareConcurrency ?? 4) - 1)) is a defensible default for both strategies, and only a measurement on representative hardware can improve on it.

2. A crashed worker must be replaced, not returned

A worker that fires onerror on a fatal script error cannot be trusted with the next task. Both implementations above terminate it without pushing it back onto this.idle — but the consequences differ. The elastic pool self-heals, because the next acquireWorker call sees total < maxSize and grows a replacement. The fixed pool silently and permanently loses capacity, so it needs explicit respawn logic, and that logic needs the script URL kept on the pool object (a Worker instance does not expose its URL):

// In the fixed pool's onerror handler:
w.terminate();
this.idle = this.idle.filter(x => x !== w);
if (this.replacements++ < MAX_REPLACEMENTS_PER_MINUTE) {
  const replacement = new Worker(this.scriptUrl, { type: 'module' });
  // … attach the same onmessage / onerror handlers …
  this.idle.push(replacement);
}

Cap the replacement rate. A script that throws at module-evaluation time will otherwise turn the pool into a respawn loop that pegs a core.

3. Eviction destroys per-worker state

An evicted worker takes its globals with it: a warmed WebAssembly instance, a compiled regular expression cache, a memoisation table, an open IndexedDB handle. The next task lands on a blank isolate and re-does that setup, which can dwarf the 5–15 ms spawn cost — a 3 MB WebAssembly module can take 30–80 ms to instantiate. If workers hold expensive state, either raise minSize so the warm ones keep it, or drop elasticity entirely. The same applies to views over shared memory: a worker holding a SharedArrayBuffer view as described in SharedArrayBuffer & Atomics must be excluded from eviction, or the coordination protocol loses a participant mid-handshake.

4. Destruction must cancel eviction timers, and the queue must have a ceiling

Calling destroy() without clearing idleTimers leaves setTimeout callbacks scheduled against terminated workers; they fire seconds later, call terminate() on a dead handle, and mutate this.idle after teardown. The implementation above clears every timer before terminating — the same discipline that page teardown needs in Handling Worker Termination Gracefully in SPAs.

Neither pool caps queue.length. Once arrivals outrun maxSize, tasks accumulate in memory indefinitely, each holding its payload alive. Add backpressure so overload surfaces as an error the caller can retry or shed:

execute(payload: unknown): Promise<T> {
  const BACKPRESSURE_LIMIT = this.maxSize * 4;
  if (this.queue.length >= BACKPRESSURE_LIMIT) {
    return Promise.reject(new Error('ElasticWorkerPool: queue full'));
  }
  return new Promise<T>((resolve, reject) => {
    this.queue.push({ payload, resolve, reject });
    this.drain();
  });
}

Performance Note

Measured on Chrome 124, MacBook Pro M3 (10 cores), 10 000 tasks of roughly 1 ms synthetic CPU work each, module workers served from cache:

The rule of thumb that falls out: elasticity costs about 1–2 % of steady-state throughput and buys back roughly 80 % of idle memory — but it converts a flat p99 into a spiky one. Under sustained load the elastic pool is indistinguishable from the fixed one because it lives at its ceiling; its entire advantage is realised in the seconds when nothing is happening. So the question to answer before choosing is not “which is faster” but “how much of my session is idle, and does anyone notice 15 ms at the head of a burst?” On an interactive surface where bursts are separated by seconds of user think time, that spike hides inside the user’s own reaction time and the memory saving is free. In a batch pipeline, it is pure loss.