Scaling workers

Each worker runs one test at a time. Runs queue in Redis, and any free worker takes the next one, so the number of worker replicas is the number of tests that run in parallel. Add workers when runs wait in the queue; give the browser services more memory when browser runs slow down or time out.

Add workers

Docker Compose

Set WORKER_REPLICAS in .env (default 2) and apply it:

.env

WORKER_REPLICAS=4

docker compose up -d

Or scale once from the command line, without changing .env:

docker compose up -d --scale worker=4

Render

Raise numInstances on plainwright-worker in render.yaml (default 2), or change the instance count in the dashboard. Move plainwright-mcp to a bigger plan to match.

Railway

Raise the worker service’s replica count in its settings, and give playwright-mcp more memory.

Fly.io

fly scale count 4 -a plainwright-worker
fly scale memory 4096 -a plainwright-mcp

Every worker shares the same queue, database and artifact storage, so any number of replicas is safe. Use the same environment on all of them; in particular the same SECRET_KEY.

Size the rest of the stack

Component	How it scales	Rule of thumb
Workers	Horizontally: one run at a time each.	The knob for parallel runs. A worker mostly waits on the AI model and the browser, so it needs little CPU.
Browser services (playwright-mcp, -firefox, -webkit)	Vertically. Every worker shares them; each run gets its own isolated browser session.	About 0.5 GB of RAM and half a CPU per concurrent browser run, and keep shm_size at 2 GB or more.
Backend	Horizontally. It’s stateless, and migrations are locked so several can start at once.	One is plenty for most teams. More than one host needs object storage for artifacts.
Scheduler	Exactly one.	Two schedulers enqueue every scheduled test twice.
Postgres	Vertically.	Run history (test_runs, eval_results) grows steadily; a small instance handles thousands of runs a day.
Redis	Vertically, but stays small: it holds job payloads and live events.	Must never evict keys: maxmemory-policy noeviction.

Only some test kinds use a browser: UI tests, site quality checks, and conversation and red-team tests that talk to a chat widget, plus the AI Crawler in the backend. API tests, LLM and RAG evals, MCP server and A2A agent tests, and conversation tests against an HTTP or A2A target don’t touch the browser services, so they scale with workers alone.

Don't run two copies of a browser service behind one name

A browser run keeps one MCP session open for its whole duration: a long-lived stream plus requests that must reach the same server. Replicas behind one host name can split those requests and lose the session. Give the single browser service more memory and CPU instead.

Time limits

• A run’s job may take 300 seconds per trial, so a run with 3 trials gets 900 seconds.
• The scheduler checks every 60 seconds for runs still pending or running past that limit plus 60 seconds of grace, and marks them error with “Run did not complete — its worker stopped or it exceeded the time limit”. That’s what happens to a run whose worker was stopped or crashed mid-test.
• A UI test also stops with an error after 30 agent steps, one model call and tool call each (“Max tool calls reached”). Split very long journeys into several tests.

Scale workers down while the queue is idle. A worker that is stopped mid-run loses that run, and the reaper later marks it as an error.

Watch the queue

The backend’s /metrics endpoint reports the queue directly:

Metric	Healthy	Act when
plainwright_queue_depth{queue="test_runs"}	Near 0 between batches	It stays above about 20 for 10 minutes: add workers.
plainwright_queue_workers{queue="test_runs"}	Equals your replica count	It’s 0: no worker is listening, so nothing runs.
plainwright_runs_in_progress{status="running"}	At most your replica count	It’s stuck at the replica count while depth grows: workers are saturated.
plainwright_run_duration_seconds_last_24h{stat="p95"}	Stable	It climbs for browser kinds: the browser service is short of memory or CPU.

Without Prometheus, check the queue length straight from Redis:

docker compose exec redis redis-cli LLEN rq:queue:test_runs

See Monitoring for the full metric list and alert examples.

Cost and rate limits

More workers means more AI calls at the same moment. Paid providers bill per token, so a burst of parallel runs spends faster, and providers with low rate limits (free tiers in particular) start answering 429. Runs then fail with “The AI provider rate-limited the request”. Raise the provider limit, use a different model for the burst, or keep fewer workers. The server-wide caps MAX_TRIALS, MAX_EVAL_ROWS_PER_RUN and MAX_CONVERSATION_TURNS bound how much one run can spend; see Configuration and Trials, thresholds and cost.

Next steps

• Monitoring: metrics, alerts and logs.
• Docker Compose: services and resources on one machine.
• Retries and flaky tests: automatic retries also take worker time.