Parallel Runs, Sharding &amp; Reporting Interview Questions for Playwright Testing

Show answer

A worker is a separate operating system process that runs test files. Each worker starts its own browser, and tests inside a worker run one after another, while different workers run at the same time. Workers do not share memory, so you cannot pass variables between them. Worker-scoped fixtures are created once per worker, which is why expensive setup is often placed there.

How many workers does Playwright use by default, and how do you change it?

Show answer

By default Playwright uses about half of the logical CPU cores on the machine. You can set workers in playwright.config as a number or a percentage string such as '50%', or override it on the command line with --workers=4. Many teams use process.env.CI ? 2 : undefined so CI runners with few cores are not overloaded while laptops use their full power.

How do you use more than one reporter in the same run?

Show answer

// playwright.config.ts export default defineConfig({ reporter: [ ['list'], ['html', { open: 'never', outputFolder: 'playwright-report' }], ['junit', { outputFile: 'results/junit.xml' }], ], });

What does the JUnit reporter produce and why do CI tools need it?

Show answer

It writes an XML file in the JUnit format, with test suites, test cases, durations and failure messages. Most CI servers and test management tools read this format to show test counts, failures and history in their own dashboards. Set outputFile so the path is predictable, and publish that file as a test result in the pipeline.

What does the github reporter do?

Show answer

The github reporter prints failures in the format GitHub Actions understands, so each failed test appears as an annotation on the right line of the pull request diff. It is usually combined with the html or blob reporter, for example reporter: process.env.CI ? [['github'], ['html']] : 'list'. It saves reviewers from digging through raw logs.

What does --repeat-each do?

Show answer

--repeat-each=N runs every selected test N times in the same run, spread across workers. It is useful for checking a new or fixed test for flakiness before merging, for example npx playwright test checkout.spec.ts --repeat-each=20. Combined with several workers it also surfaces tests that collide with each other over shared data.

How do you open an HTML report downloaded from a CI artifact?

Show answer

Unzip the artifact and run npx playwright show-report followed by the folder path, for example npx playwright show-report ./playwright-report. This starts a small local server, which is needed because traces load through it. Opening index.html directly from disk often fails to show traces. You can also upload a single trace zip to the online trace viewer.

What does the open option of the HTML reporter control?

Show answer

open decides whether the report opens in a browser after the run: 'always', 'never' or 'on-failure', which is the default. In CI you set it to 'never' so the process does not try to start a browser or hang waiting. Locally on-failure is convenient. You can also set the PLAYWRIGHT_HTML_OPEN environment variable instead of editing config.

What is the difference between the list, line and dot reporters?

Show answer

All three print to the terminal. list prints one line per test with its status and is the default locally. line shows a single updating line with progress and prints only failures in full, which suits long runs. dot prints one character per test, the most compact output, and is the default on CI. Choose based on how much log noise you want.

What is the difference between workerIndex and parallelIndex?

Show answer

workerIndex is a unique number for every worker process started during the run, so it keeps growing when a worker is restarted after a failure. parallelIndex is a slot number from 0 to workers minus 1, and a restarted worker reuses its slot. Use parallelIndex, also exposed as TEST_PARALLEL_INDEX, to pick a fixed test account or database per slot.

Write a fixture that gives each parallel worker its own test account.

Show answer

import { test as base } from '@playwright/test'; export const test = base.extend<{}, { account: { email: string; password: string } }>({ account: [async ({}, use, workerInfo) => { const i = workerInfo.parallelIndex; await use({ email: `qa.user${i}@example.test`, password: process.env.QA_PASSWORD!, }); }, { scope: 'worker' }], });

Why does Playwright start a new worker after a test fails?

Show answer

A failed test may leave the browser or worker-scoped fixtures in a bad state, such as an open dialog or corrupted login. To protect the remaining tests, Playwright throws that worker away and starts a fresh one for the next tests. This means worker fixtures run again and beforeAll hooks may run more than once, which is normal and something your setup must tolerate.

Write a GitHub Actions job that runs the suite in four shards.

Show answer

jobs: e2e: runs-on: ubuntu-latest strategy: fail-fast: false matrix: shard: [1, 2, 3, 4] steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 - run: npm ci && npx playwright install --with-deps - run: npx playwright test --shard=${{ matrix.shard }}/4 --reporter=blob - uses: actions/upload-artifact@v4 with: name: blob-${{ matrix.shard }} path: blob-report

What is the blob reporter and why does it exist?

Show answer

The blob reporter writes a compact archive of everything about a run, including results, attachments and traces, into the blob-report folder. It is meant for machines rather than people. Each shard produces one blob, and npx playwright merge-reports later combines them into a single HTML, JUnit or JSON report. Without it, every shard would produce its own separate HTML report.

Which methods does the Playwright Reporter interface provide?

Show answer

The main ones are onBegin with the config and full suite, onTestBegin and onTestEnd for each test, onStepBegin and onStepEnd for steps, onStdOut and onStdErr for output, onError for errors outside tests and onEnd with the overall result. onEnd can be async, so you can send results to an API before the process exits. You register the reporter by path in the reporter config.

What does the reportSlowTests option do?

Show answer

reportSlowTests makes Playwright list test files that took longer than a threshold at the end of the run. You set max, the number of files to show, and threshold in milliseconds, for example { max: 5, threshold: 60000 }. It is a cheap way to find files worth splitting, because one slow file can hold up a worker or a whole shard.

CI runners have 2 CPU cores and workers is set to 8. Tests start timing out randomly. Why?

Show answer

Eight browsers compete for two cores and limited memory, so pages render slowly, actions hit timeouts and the app under test may also be overloaded. More workers than cores rarely helps. Set workers to match the runner, such as 2, and scale out with shards on more machines instead. Watch CPU and memory metrics in CI to confirm.

Parallel tests create users with the same email and fail with 'already exists'. How do you fix it?

Show answer

Generate unique data per test, for example by adding testInfo.testId, the parallel index or a random suffix to the email, or by using a data builder that creates a fresh user through the API. Avoid hard-coded shared records in test files. Clean up created users in a fixture teardown so the database does not fill up over time.

How do you run one group of tests serially while the rest of the suite runs in parallel?

Show answer

Put the group in its own project with fullyParallel: false and limit it to one worker, which recent versions allow with a workers setting on the project, or run it in a separate CI step with --project=serial --workers=1. Inside a file, test.describe.configure({ mode: 'serial' }) keeps order. Use project dependencies if the group must run after the parallel tests.

How do you add build details such as the commit and environment to the test report?

Show answer

Set the metadata field in playwright.config, for example metadata: { commit: process.env.GIT_SHA, env: process.env.TEST_ENV }, which reporters can read and recent HTML reports show. Inside tests use testInfo.annotations.push for per-test details like a ticket id. Custom reporters can read config.metadata in onBegin and include it in messages or uploads.

What does the JSON reporter output and how would you use it?

Show answer

The JSON reporter writes the full suite tree with every test, its status, retries, duration, errors and attachment paths into one file. Teams parse it in scripts to post a summary to chat, to collect flaky tests that passed on retry, or to push results to a database for trends. Set outputFile so the JSON does not mix with console output.

What does outputDir contain and why does it matter in parallel runs?

Show answer

outputDir, test-results by default, holds per-test artifacts such as screenshots, videos and traces. Playwright creates a separate subfolder for each test and retry, so parallel workers do not overwrite each other's files. testInfo.outputPath gives you a safe path inside it for your own files. It is cleared at the start of every run, so do not store anything permanent there.

Tell me about a time you reduced pipeline time with parallel runs or sharding.

Show answer

Situation: our regression suite took 55 minutes, so developers skipped it. Task: bring it under 15. Action: I removed shared test data so fullyParallel could be turned on, split the three slowest files, and ran four shards with blob reports merged into one HTML report. Result: the suite finished in 12 minutes and it became a required check on every pull request.

Two test files edit the same global settings page and break each other when run in parallel. How do you handle it?

Show answer

The best fix is to remove the shared resource, for example by giving each test its own tenant or user whose settings it can change. If that is impossible, put those tests in one file with test.describe.configure({ mode: 'serial' }), or in a separate project that runs with one worker after the main project. Document why, so the exception does not quietly spread.

How does sharding split tests when fullyParallel is on compared with off?

Show answer

With fullyParallel off, Playwright shards by test file, so one large file always lands on one shard and shards can be uneven. With fullyParallel on, it shards by individual test, giving much more even distribution. Either way the split is deterministic, so the same shard number always gets the same tests for a given set of files, which helps reproduce failures.

Write a minimal custom reporter that prints the titles of failed tests at the end.

Show answer

import type { Reporter, TestCase, TestResult } from '@playwright/test/reporter'; class FailedListReporter implements Reporter { private failed: string[] = []; onTestEnd(test: TestCase, result: TestResult) { if (result.status === 'failed' || result.status === 'timedOut') { this.failed.push(test.titlePath().join(' > ')); } } onEnd() { console.log(`Failed tests (${this.failed.length}):`); this.failed.forEach((t) => console.log(' - ' + t)); } } export default FailedListReporter;

One shard finishes in 4 minutes and another takes 15. How do you balance them?

Show answer

Check whether fullyParallel is off, because file-level sharding puts big files together. Turn it on if tests are independent, or split the largest files. Use reportSlowTests and the merged report's durations to find heavy files, and move slow setup into API calls or shared storage state. Also check that one shard is not getting all the tests of a slow project such as WebKit.

How would you build a flaky-test dashboard from Playwright results?

Show answer

Write a custom reporter or parse the JSON report after each CI run, and store per test: id, status, retry count, duration, branch and commit. A test that passed after a retry counts as flaky. Aggregate over the last few hundred runs to rank tests by flake rate and duration, and show trends. Use it to assign owners and quarantine tests above an agreed threshold.

How do you decide how many shards a suite should use?

Show answer

Start from the target wall-clock time and the total test time per machine. If one machine with its workers takes 40 minutes and the goal is under 10, you need about four or five shards, plus some margin for setup like installing browsers. Past a point each shard spends more time on setup than on tests, and the app or test data may become the bottleneck, so measure each change.

How do you rerun only the tests that failed in a sharded CI run?

Show answer

--last-failed works only on the machine that kept the .last-run.json file, so it does not help across shards. Instead collect the failed tests from the merged JSON report or the blob reports, then run them in a follow-up job by file and line or with --grep on their titles. Many teams simply rely on retries and treat any test needing a rerun as flaky.

Describe a time you improved how test results were reported to the team or managers.