Image Processing in Workers

Pixel manipulation is the archetypal main-thread offender: a single loop over a full-screen image touches millions of array elements, cannot yield, and holds the event loop hostage until it finishes. This guide is a specialisation of High-Performance Computation Patterns, applied to raster work — filters, convolution kernels, colour-space conversion, resizing, and image encoding — and it covers the whole route: getting pixels out of a canvas, into a worker without copying, through a filter chain, and back onto the screen at 60fps.

The Frame Budget Problem

A 1920×1080 image is 2,073,600 pixels, or 8,294,400 bytes of RGBA data. A trivial grayscale pass touches every one of those bytes; a 3×3 convolution reads nine neighbours per channel, which works out to roughly 56 million multiply-accumulate operations for one pass over one 1080p frame. At 4K (3840×2160) both numbers quadruple: 33.2 MB of pixels and around 224 million operations.

On the main thread the symptom is unambiguous. The user drags a “blur radius” slider, the input handler runs the kernel synchronously, and the browser cannot paint, cannot run requestAnimationFrame, and cannot dispatch the next pointermove until the loop returns. A 60fps budget is 16.7 ms per frame; a single 4K blur pass measured on a mid-range laptop takes long enough to swallow several frames in a row, which the Performance panel renders as one fat yellow “Long Task” block with a red corner and the tooltip Forced reflow / long task. The slider stops tracking the cursor, and the whole UI feels broken even though the arithmetic is perfectly correct.

The fix is structural rather than algorithmic. Move the loop to a thread that has no paint responsibilities, and make sure the pixels get there without being copied.

The Round-Trip Architecture

A worker runs in an isolated scope with no window, no document, and no access to the DOM. That restriction is exactly what makes it safe for image work: nothing inside the worker can trigger layout or paint, so the browser is free to schedule it on another core while the compositor keeps running. Communication happens only through structured messages, which is why the interesting engineering in this topic is about the shape of the payload rather than the filter maths.

The main-thread half of the pattern is small. Spawn the worker once, keep it alive across interactions, and treat every message as one complete unit of work:

// main-thread.ts
type FilterSpec = { name: string; params?: Record<string, number> };

interface WorkerResult {
  type: 'PIPELINE_DONE' | 'PIPELINE_FAILED';
  buffer?: ArrayBuffer;
  width?: number;
  height?: number;
  error?: { name: string; message: string; stack?: string };
}

// A module worker keeps `import` available inside the worker scope.
const worker = new Worker(new URL('./image-worker.js', import.meta.url), { type: 'module' });

worker.addEventListener('message', (e: MessageEvent<WorkerResult>) => {
  if (e.data.type === 'PIPELINE_FAILED') {
    // Errors do NOT structured-clone usefully; see the failure-modes section below.
    console.error('Filter pipeline failed:', e.data.error?.message);
    return;
  }
  const { buffer, width, height } = e.data;
  ctx.putImageData(new ImageData(new Uint8ClampedArray(buffer!), width!, height!), 0, 0);
});

// Fires for script load failures and uncaught throws that escape the worker's own handler.
worker.addEventListener('error', (err) => {
  console.error('Worker thread fault:', err.message, err.filename, err.lineno);
});

Reserve a worker for work that consistently exceeds a frame budget. Spawning one costs roughly 1–5 ms (script fetch, parse, scope construction), and each message hop adds a fraction of a millisecond, so dispatching one message per mouse-move for a 2 ms operation makes things slower, not faster. Batch instead: coalesce slider input with requestAnimationFrame and send at most one job per frame, dropping any job still queued behind it.

Prerequisites

Before implementing the pipeline below, confirm the following:

• A canvas you control. getImageData() on a canvas that has ever drawn a cross-origin image without CORS throws a SecurityError — the canvas is tainted. Load source images with crossorigin="anonymous" and serve them with Access-Control-Allow-Origin.
• A same-origin (or CORS-enabled) worker script URL. The Worker constructor rejects cross-origin scripts outright; a cross-origin script fetched through a Blob URL loses useful error reporting.
• Module worker support if you use { type: 'module' } — Chrome 80+, Firefox 114+, Safari 15+. Below that, bundle the worker to a classic script. Bundling Module Workers with Vite and webpack covers the build configuration for both.
• A feature test for OffscreenCanvas if you plan to render or encode inside the worker, plus a main-thread fallback path.
• Cross-origin isolation — Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp — only if you intend to parallelise across multiple workers with SharedArrayBuffer & Atomics. The single-worker transfer path needs no special headers.
• A baseline measurement. Record the current main-thread cost with performance.now() before you start, or you will have no way to prove the rewrite helped.

Step-by-Step Implementation

Step 1 — Hand pixel ownership to the worker

getImageData() returns an ImageData whose data is a Uint8ClampedArray over an ArrayBuffer. That ArrayBuffer is transferable: putting it in the second argument of postMessage moves the allocation to the worker instead of copying it. The mechanics generalise to every large binary payload and are covered in depth in Transferable Objects & Zero-Copy.

// main-thread.ts
const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
const { width, height } = imageData;      // capture BEFORE transfer — dimensions are not in the buffer
const buffer = imageData.data.buffer;

worker.postMessage(
  { type: 'RUN_PIPELINE', buffer, width, height, filters: [{ name: 'grayscale' }] },
  [buffer]                                 // transfer list: ownership moves, nothing is copied
);

// imageData.data.byteLength === 0 from here on. Drop the reference so the
// detached wrapper object is collectable.

Step 2 — Structure the worker as a filter registry

Keep every filter a pure function over (data, width, height, params) that mutates the buffer in place. A registry keyed by name lets the main thread describe a pipeline declaratively, and lets you run the entire chain in one message handler — one pass over memory, no intermediate allocations, no extra hops.

// image-worker.js
const filters = {
  grayscale(data) {
    for (let i = 0; i < data.length; i += 4) {
      // Rec. 601 luma weights — matches how the eye weights the channels.
      const luma = 0.299 * data[i] + 0.587 * data[i + 1] + 0.114 * data[i + 2];
      data[i] = data[i + 1] = data[i + 2] = luma;  // Uint8ClampedArray rounds and clamps for us
    }
  },
  invert(data) {
    for (let i = 0; i < data.length; i += 4) {
      data[i] = 255 - data[i];
      data[i + 1] = 255 - data[i + 1];
      data[i + 2] = 255 - data[i + 2];             // alpha at i+3 deliberately untouched
    }
  },
  threshold(data, _w, _h, { cutoff = 128 } = {}) {
    for (let i = 0; i < data.length; i += 4) {
      const v = data[i] > cutoff ? 255 : 0;
      data[i] = data[i + 1] = data[i + 2] = v;
    }
  }
};

self.onmessage = (e) => {
  const { buffer, width, height, filters: chain } = e.data;
  const pixels = new Uint8ClampedArray(buffer);

  try {
    for (const step of chain) {
      const fn = filters[step.name];
      if (!fn) throw new RangeError(`Unknown filter: ${step.name}`);
      fn(pixels, width, height, step.params);
    }
  } catch (err) {
    // Error objects clone unreliably across engines — flatten to plain data.
    self.postMessage({
      type: 'PIPELINE_FAILED',
      error: { name: err.name, message: err.message, stack: err.stack }
    });
    return;
  }

  self.postMessage({ type: 'PIPELINE_DONE', buffer, width, height }, [buffer]);
};

Notice that the worker does not call self.close(). Terminating after every job throws away a warm JIT and forces a fresh script parse on the next interaction. Keep the worker resident for the lifetime of the editing session, and terminate it only when the view unmounts.

Step 3 — Convolution, the pass that actually needs a worker

Point filters like grayscale are memory-bandwidth-bound and finish in a few milliseconds. Convolution is where the frame budget dies, because each output pixel reads a neighbourhood. Two optimisations matter more than anything else: read from a scratch copy so the kernel never sees pixels it has already written, and exploit separability — a Gaussian blur of radius r can be run as a horizontal pass followed by a vertical pass, turning (2r+1)² taps per pixel into 2(2r+1).

// image-worker.js — separable box/Gaussian blur, one axis at a time
function blurAxis(src, dst, width, height, radius, horizontal) {
  const norm = 1 / (radius * 2 + 1);
  for (let y = 0; y < height; y++) {
    for (let x = 0; x < width; x++) {
      let r = 0, g = 0, b = 0;
      for (let k = -radius; k <= radius; k++) {
        // Clamp at the edges instead of wrapping, or the blur bleeds across the image.
        const sx = horizontal ? Math.min(width - 1, Math.max(0, x + k)) : x;
        const sy = horizontal ? y : Math.min(height - 1, Math.max(0, y + k));
        const si = (sy * width + sx) * 4;
        r += src[si]; g += src[si + 1]; b += src[si + 2];
      }
      const di = (y * width + x) * 4;
      dst[di] = r * norm; dst[di + 1] = g * norm; dst[di + 2] = b * norm;
      dst[di + 3] = src[di + 3];               // preserve alpha
    }
  }
}

filters.blur = (data, width, height, { radius = 4 } = {}) => {
  const scratch = new Uint8ClampedArray(data);   // one allocation, reused across both passes
  blurAxis(data, scratch, width, height, radius, true);   // horizontal
  blurAxis(scratch, data, width, height, radius, false);  // vertical, back into the original
};

Step 4 — Tile the image so progress is reportable

A single 4K pass can still run for a noticeable stretch inside the worker. Nothing on screen freezes, but the user gets no feedback and you cannot cancel. Splitting the image into horizontal bands lets the worker post progress after each band and check a cancellation flag between them.

// image-worker.js
const TILE_ROWS = 128;
let currentJob = 0;

self.addEventListener('message', (e) => {
  if (e.data.type === 'CANCEL') { currentJob++; return; }  // invalidate the in-flight job
  if (e.data.type !== 'RUN_TILED') return;

  const job = ++currentJob;
  const { buffer, width, height, radius } = e.data;
  const pixels = new Uint8ClampedArray(buffer);

  for (let top = 0; top < height; top += TILE_ROWS) {
    if (job !== currentJob) return;                        // a newer job arrived; abandon this one
    const rows = Math.min(TILE_ROWS, height - top);
    // Halo: kernels read `radius` rows outside the band, so widen the read window.
    blurBand(pixels, width, height, top, rows, radius);
    self.postMessage({ type: 'PROGRESS', done: top + rows, total: height });
  }

  self.postMessage({ type: 'PIPELINE_DONE', buffer, width, height }, [buffer]);
});

Step 5 — Encode to WebP inside the worker

Encoding is CPU-heavy and, unlike filtering, has a native implementation the worker can call directly. OffscreenCanvas.convertToBlob() runs the browser’s own encoder off the main thread and returns a Blob you can upload straight away.

// encode-worker.js
self.onmessage = async (e) => {
  const { pixels, width, height, quality } = e.data;

  const surface = new OffscreenCanvas(width, height);
  const ctx = surface.getContext('2d');
  ctx.putImageData(new ImageData(new Uint8ClampedArray(pixels), width, height), 0, 0);

  // NOTE: an unsupported `type` silently falls back to image/png per spec —
  // read blob.type back rather than assuming you got what you asked for.
  const blob = await surface.convertToBlob({ type: 'image/webp', quality });
  self.postMessage({ type: 'ENCODED', blob, actualType: blob.type });
};

Blobs are cloneable but not transferable — the browser passes a reference to the underlying byte store rather than serialising the bytes, so returning a 2 MB WebP Blob is cheap. If you are caching encoded output for reuse, Caching Computed Results with the Cache API shows how to keep the result out of memory entirely.

Step 6 — Let the worker own the surface with OffscreenCanvas

Everything above keeps the canvas on the main thread and ships pixels back and forth. When the worker is producing every frame — a live camera filter, a paint tool, an animated visualisation — hand it the surface instead and stop round-tripping altogether.

// main-thread.ts
const canvas = document.getElementById('render-target') as HTMLCanvasElement;

if (typeof OffscreenCanvas === 'undefined' || !canvas.transferControlToOffscreen) {
  mountMainThreadFallback(canvas);            // 2D context on the main thread
} else {
  const offscreen = canvas.transferControlToOffscreen();
  // The surface is transferable; after this call the main thread can never draw to it again.
  worker.postMessage({ type: 'INIT_RENDER', canvas: offscreen }, [offscreen]);
}

Inside the worker, drive the loop with the worker scope’s own requestAnimationFrame — it is throttled to the display refresh exactly like the main thread’s, so there is no need to ping-pong “next frame” messages:

// canvas-worker.js
let ctx = null;
self.onmessage = (e) => {
  if (e.data.type !== 'INIT_RENDER') return;
  ctx = e.data.canvas.getContext('2d');
  const draw = () => { renderFrame(ctx); self.requestAnimationFrame(draw); };
  self.requestAnimationFrame(draw);
};

Data-Transfer Strategy: Clone, Transfer, Share or ImageBitmap

Image work offers four distinct ways to get pixels across the thread boundary, and picking the wrong one is the most common reason a “worker-optimised” pipeline ends up slower than the synchronous version it replaced.

Transfer is the right default. The buffer is produced by one thread, consumed by another, and never needed on both sides at once — which is precisely the ownership model transfer encodes. The measured difference is not subtle: transferring is a pointer handoff whose cost does not grow with the payload, while cloning is a memcpy plus allocation that scales linearly and blocks the sender for the duration.

ImageBitmap is the right choice when the worker draws rather than reads. createImageBitmap() works inside workers, decodes the source off the main thread, accepts resizeWidth/resizeHeight for a free downscale during decode, and produces a GPU-friendly object that drawImage() can blit without a pixel-by-pixel upload. If your pipeline is decode → resize → draw, never touch ImageData at all:

// thumbnail-worker.js
self.onmessage = async ({ data: { blob, size } }) => {
  const bitmap = await createImageBitmap(blob, {
    resizeWidth: size, resizeHeight: size, resizeQuality: 'high'
  });
  const surface = new OffscreenCanvas(size, size);
  surface.getContext('2d').drawImage(bitmap, 0, 0);
  bitmap.close();                                   // release GPU/decoded memory immediately
  const thumb = await surface.convertToBlob({ type: 'image/webp', quality: 0.8 });
  self.postMessage({ thumb });
};

SharedArrayBuffer earns its complexity only under parallelism. Because every thread sees the same bytes, four workers can filter four bands of one 4K frame with no handoff at all, coordinated by Atomics.add handing out tile indices. The price is real: the document must be cross-origin isolated with Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp, which also constrains every third-party asset you embed, and you must reason about visibility and ordering yourself. For a single filter job it is strictly worse than a transfer.

Metadata is the exception to all of this. EXIF blocks, colour profiles, and pipeline descriptors are small structured objects; clone them normally alongside the transferred buffer. Only when the metadata itself becomes large — thousands of detection boxes, per-pixel annotations — is it worth the encoding work described in Data Parsing & Serialization.

Verification & Measurement

A worker rewrite is only a win if you can show the numbers. Measure the whole round trip, not just the filter:

1. Record performance.now() immediately before the first postMessage, and again inside the message handler after ctx.putImageData() returns. The gap is what the user actually waits for.
2. Have the worker report its own compute time in the result message. Round trip minus compute is your transfer and scheduling overhead; at 1080p with transfers it should stay well under a millisecond in each direction.
3. Assert imageData.data.byteLength === 0 right after postMessage. A non-zero length proves the buffer was cloned, not transferred — usually because it was omitted from the transfer list.
4. Open the Chrome Performance panel and confirm the compute block appears on a Worker track, not on Main. If the long task is still on Main, the pixels are being copied rather than moved, or the filter never left the main thread. Profiling Worker CPU Usage with the Chrome Performance Tab walks through reading that track.
5. Check for dropped frames with the Frames track while dragging a filter control. The pass condition is that no frame exceeds 16.7 ms during continuous interaction.
6. Verify output correctness with a fixed test image and a checksum over the result buffer, so a “faster” refactor cannot silently change the pixels.

Failure Modes & Error Handling

Worker errors do not surface where main-thread errors do, and image pipelines add their own class of failure on top. Attach both handlers on the worker object and both global handlers inside the worker scope:

// image-worker.js — catch everything that would otherwise vanish
self.addEventListener('error', (e) => {
  self.postMessage({ type: 'PIPELINE_FAILED', error: { name: 'Error', message: e.message } });
});
self.addEventListener('unhandledrejection', (e) => {
  const r = e.reason;
  self.postMessage({
    type: 'PIPELINE_FAILED',
    error: { name: r?.name ?? 'UnhandledRejection', message: String(r?.message ?? r), stack: r?.stack }
  });
});

Error objects are cloneable in modern engines but their stack is not reliably preserved across all of them, so flatten to a plain { name, message, stack } object before posting — the pattern is generalised in Structured Error Serialization Across Threads. For retries, note that a transferred buffer is gone: a failed job cannot simply be re-sent, because the main thread no longer owns the pixels. Either re-read them from the canvas (getImageData again) or have the worker return the buffer on the failure path too.

Browser Compatibility

The practical floor for the transfer-based pipeline is very low — transferable ArrayBuffer support has been universal for a decade. OffscreenCanvas is the gating feature for worker-side rendering and encoding, so keep the main-thread fallback until your analytics say Safari 16.3 and Firefox 104 are gone from your traffic.

Going Further

Decoding is usually the largest single block of main-thread time in an upload flow, and it is entirely avoidable. Resizing Images with createImageBitmap in Workers covers decoding and downscaling in one step on the worker thread, the options that control quality and orientation, bitmap transfer, and the memory ceiling that decides how large a pool can safely be.