Hard Playwright Testing Interview Questions

Show answer

Separate tests, page objects, fixtures, utilities, test data and config; use tags, projects, shared auth setup, linting, code review and a CI pipeline with sharding and reports.

How do you reduce the execution time of a 2000-test suite?

Show answer

Run in parallel with sharding, reuse authentication state, set up data through APIs, avoid unnecessary waits, run smoke on every commit and the full suite nightly.

How would you type the response of an API call in TypeScript?

Show answer

Define an interface for the payload, cast or validate the parsed JSON to it (for example with zod), and use that type in assertions to catch shape changes.

How do you create test data through the API before a UI test?

Show answer

In a fixture or beforeEach, call the API (request.post) to create the user or order, use it in the UI test, then delete it in teardown. It is faster and more reliable than creating data through the UI.

Tests are flaky in CI. How do you debug them?

Show answer

Turn on trace: 'on-first-retry', open the trace viewer, check network and timing, replace hard waits with web-first assertions, isolate test data and check for sharding conflicts.

How would you handle a very flaky element that appears after an animation?

Show answer

Rely on auto-waiting plus a web-first assertion on the final state, disable animations in config for tests and avoid fixed timeouts.

Design a Playwright framework for a 500-test suite.

Show answer

TypeScript + POM + custom fixtures, environment configs, API helpers for test data, tags like @smoke, sharding across CI machines, retries with traces, HTML/Allure reports, lint and review rules.

How would you design API test coverage for a new endpoint?

Show answer

Cover happy path, validation errors, auth and permissions, boundary values, idempotency, schema, performance and data cleanup.

What is expect(...).toPass and how is it different from expect.poll?

Show answer

toPass retries a whole block of code, including several actions and assertions, until it runs without throwing. expect.poll retries only one function that returns a value. Use toPass for flows like 'click refresh, then check the status' that must be repeated together. It has no timeout by default, so always pass a timeout to avoid a hung test.

How do you design a locator strategy for a team so tests survive UI redesigns?

Show answer

Set a written priority order: role with name, then label, placeholder or text, then test id, and CSS or XPath only as a last resort. Agree with developers to add data-testid on complex widgets. Keep locators inside page objects so a change is fixed in one place. Add a lint rule or review check against long CSS chains and nth-child selectors.

Developers keep changing CSS class names and 40 tests break every sprint. How do you fix this long term?

Show answer

Replace class-based selectors with role, label and text locators, and agree on data-testid for widgets without good semantics. Move all locators into page objects so future changes touch one file. Add a code review rule and a lint check that flags raw CSS selectors. Share a short locator guide with developers so they know which attributes tests depend on.

How do you write a custom matcher with expect.extend?

Show answer

expect.extend takes an object of matcher functions. Each receives the received value and arguments, and returns { pass, message, name, expected, actual }. For example a toHaveAmount matcher can read a locator's text, strip the currency symbol and compare numbers, retrying with expect.poll inside. Export the extended expect from a shared fixtures file so all tests use the same version.

A test checks a price that is formatted differently per locale. How do you make the assertion robust?

Show answer

Fix the locale for the test with the locale and timezoneId options so the format is predictable. Build the expected string with Intl.NumberFormat using the same locale, rather than hard-coding it. If you test several locales, run them as projects or parameterised tests. For business logic, also assert the raw number through the API response.

How do you combine fixtures from several files with mergeTests?

Show answer

When teams define fixtures in separate modules, such as dbTest and a11yTest, mergeTests(dbTest, a11yTest) returns one test object that has all their fixtures. mergeExpects does the same for custom matchers. This keeps fixture modules small and owned by different people. Watch for name clashes, because two modules defining the same fixture name will conflict.

A setup fixture creates expensive test data and suite time has doubled. How do you redesign it?

Show answer

Measure first with the report timings. Then create data through the API instead of the UI, move read-only shared data into a worker-scoped fixture, and seed common reference data once in a setup project. Keep per-test data only for things a test changes. Cache login with storageState. This usually cuts setup time sharply without losing isolation.

What is a fixture option and how do you define one?

Show answer

An option is a fixture whose value is configured rather than computed, declared with { option: true } and a default, for example defaultUser: ['standard', { option: true }]. Projects or test.use can override it, so one suite can run as admin in one project and as a viewer in another. Other fixtures can read the option to decide what to set up.

How do project dependencies differ from globalSetup?

Show answer

A setup project is a normal test file that other projects list in dependencies. It shows in the HTML report, can use fixtures, and records traces, so failures are easy to debug. globalSetup is a plain function that runs before everything without fixtures or traces. The docs now recommend project dependencies for login and data seeding.

How do you test a flow that needs two user roles, such as an admin and a customer, in one test?

Show answer

Create two browser contexts in the same test, each with its own storageState file, for example admin.json and user.json made by a setup project. Wrap them in fixtures like adminPage and userPage so tests stay short. The admin approves an order in one context and the customer sees the update in the other, with fully separate cookies.

A worker-scoped fixture holds shared data and some tests see stale values. How do you fix it?

Show answer

Worker fixtures live across many tests, so any test that changes their data affects the next ones. Keep worker fixtures read-only, such as a connection or a reference catalogue. Move anything a test modifies into a test-scoped fixture that creates fresh data. If sharing is needed for speed, reset the relevant state in the test fixture before use.

Describe a time a broken fixture failed many tests at once and how you handled it.

Show answer

Situation: one morning 300 tests failed in CI. Task: restore the pipeline quickly. Action: the trace showed every test failing inside an auth fixture after a login API change, so I fixed the fixture, added a dedicated setup test that fails fast with a clear message, and alerted the team. Result: the pipeline recovered within an hour and similar outages became easy to spot.

When should you avoid mocking, and how do you balance mocked and real tests?

Show answer

Avoid mocking in a small set of critical end-to-end journeys, such as signup, payment and checkout, where integration matters most. Use mocks for UI states, error handling and third-party services. Back the mocks with API contract tests so they match reality. A healthy split is many fast mocked component-level tests and a thin layer of full-stack tests.

Mocked responses drift from the real API and tests pass while production breaks. How do you prevent this?

Show answer

Generate mock data from the API schema or real recorded responses instead of hand-writing it. Validate mocks against the OpenAPI schema in a unit test. Add contract tests on the backend side and re-record HAR files on a schedule. Keep a few unmocked end-to-end tests that run after each deploy. Treat mock files as code with owners and reviews.

How do you mock WebSocket traffic in Playwright?

Show answer

Use page.routeWebSocket with a URL pattern. The handler gets a WebSocketRoute: you can reply to messages with ws.onMessage and ws.send without any server, or call connectToServer() to sit in the middle and change messages in both directions. This lets you test live dashboards and chat apps, for example pushing a fake price update.

A feature is controlled by a flag served from a config API. How do you test both states?

Show answer

Mock the config endpoint in a fixture option so each test chooses the flag value, for example featureFlags: { newCheckout: true }. Run the flow with the flag on and off and assert the different UI. Mocking avoids changing shared environment settings that would affect other testers. Also keep one test against the real config to catch wiring mistakes.

How do you mock a paginated endpoint that returns different data per page?

Show answer

await page.route('**/api/products?*', route => { const url = new URL(route.request().url()); const p = Number(url.searchParams.get('page') ?? 1); const items = Array.from({ length: 10 }, (_, i) => ({ id: (p - 1) * 10 + i })); return route.fulfill({ json: { items, hasNext: p < 3 } }); });

How can service workers interfere with network mocking?

Show answer

If the app registers a service worker, it may answer requests from its own cache, so page.route never sees them and mocks seem ignored. Set serviceWorkers: 'block' in the context or config options to turn them off when mocking. If you specifically test offline or PWA caching, keep them enabled and test that behaviour separately.

A test passes locally but fails in CI with a timeout on a click. How do you use the trace?

Show answer

Open the CI trace and go to the failed click. Check the action log, which shows what Playwright waited for, such as the element not being visible or another element intercepting pointer events. Compare before and after snapshots and the network tab for slow or failed calls. Common causes are a slower environment, a cookie banner or different screen size.

The suite has a 3 percent flaky rate. How do you find and fix the causes systematically?

Show answer

Collect data first: flaky results from the reports across a few weeks, grouped by test, browser and error message. Run the worst offenders with --repeat-each and traces on. Fix common causes such as missing awaits, shared data, fixed sleeps and order-dependent tests. Quarantine tests you cannot fix quickly with a tag and a ticket, and track the rate weekly.

What are the trade-offs of trace 'on' versus 'retain-on-failure' in CI?

Show answer

'on' records and keeps every trace, which costs time, disk space and artifact storage for thousands of passing tests. 'retain-on-failure' still records everything but deletes traces of passing tests, so cost is mainly runtime. 'on-first-retry' is cheapest but gives no trace for a first failure without retries. Most teams use on-first-retry with retries in CI.

A test fails only in WebKit. How do you investigate?

Show answer

Run it alone with --project=webkit --headed and record a trace. Compare it with the Chromium trace to see where behaviour differs, such as date input format, scrolling, font rendering or a missing browser API. Check the app's support matrix. If it is a real product bug, raise it, and skip the test for WebKit with a linked ticket until it is fixed.

Tests pass with one worker but fail with eight. How do you debug this?

Show answer

This usually means tests share something: the same user account, the same records or a global server setting. Run the failing file with --workers=8 and --repeat-each to reproduce it. Look at which tests run at the same time in the report. Fix it by giving each test unique data, using workerIndex in names, or isolating shared settings per worker.

Describe how you handled a flaky test that was blocking releases.

Show answer

Situation: a payment test failed randomly and blocked three release builds. Task: unblock releases without hiding a real problem. Action: I quarantined it with a tag and a ticket, kept it running in a separate job, and used traces from twenty runs to find a missing wait on a redirect. Result: it was fixed in two days and releases continued meanwhile.

How do contract tests differ from functional API tests?

Show answer

Functional tests check behaviour: given this input, the API returns the right data and status. Contract tests check that the shape agreed between consumer and provider, such as field names and types, has not changed. Playwright suits functional tests and simple schema checks. Dedicated contract tools help when many services evolve independently and need consumer-driven agreements.

An API test fails randomly because another test changes the same record. How do you fix it?

Show answer

Make each test own its data. Create the record inside the test or a fixture with a unique name, such as one using a timestamp or the worker index, and delete it in teardown. Avoid shared seed records for anything that gets modified. Shared read-only reference data is fine. This also lets tests run fully in parallel.

How do you test an async job API that returns 202 and finishes later?

Show answer

Start the job and assert the 202 status and the returned job id. Then poll the status endpoint with expect.poll, with sensible intervals and a timeout, until it reports completed. Finally fetch the result and validate it. Also test the failure path, for example invalid input leading to a failed status with a clear error message.

How do you keep API tests independent of environment data?

Show answer

Do not depend on records that happen to exist in a shared environment. Each test or fixture creates what it needs through the API and cleans up after. Read environment-specific values such as base URL and credentials from config. For reference data that must exist, seed it in a setup project and verify it before tests run.

How do you test that duplicate POST requests do not create duplicate orders?

Show answer

If the API supports an idempotency key header, send the same POST twice with the same key and assert one order exists and both responses return the same id. Send it with different keys and expect two orders. Also try concurrent requests with Promise.all to check race conditions. Without a key, document the risk to the team.

How does Playwright talk to browsers, and why does that make it less flaky?

Playwright CoreHard

Show answer

Playwright runs outside the browser and controls it through a persistent connection using each engine's own protocol, such as CDP for Chromium and patched protocols for Firefox and WebKit. It can watch DOM, network and events directly, so it waits on real state instead of polling from inside the page. This gives reliable auto-waiting, multi-tab and network control.

The app logs in through a third-party OAuth pop-up. How do you automate it?

Playwright CoreHard

Show answer

For most tests, avoid the pop-up: get a session through a test login API or a test identity provider and save it with storageState. Keep one test for the real flow, using const popup = page.waitForEvent('popup') before clicking, then fill the provider's form inside popup. Third-party pages can change and add captchas, so limit how often you depend on them.

Why use readonly and as const for test data?

Show answer

readonly and as const stop test data from being changed by accident, for example one test editing a shared user object and breaking the next. as const also keeps literal types, so ['admin', 'viewer'] as const gives the type 'admin' | 'viewer'. That lets TypeScript reject invalid roles at compile time instead of during a run.

What are discriminated unions and how can they model API responses?

Show answer

A discriminated union is a union of object types that share one literal field, such as status: 'ok' with data, or status: 'error' with a message. Checking the status field narrows the type, so TypeScript only allows data in the ok branch. In test helpers this forces you to handle the error case and prevents reading fields that do not exist.

How do you combine reports from sharded CI jobs?

Show answer

Use the blob reporter in each shard job, upload each blob-report folder as an artifact, then in a final job download them all into one folder and run npx playwright merge-reports --reporter html ./all-blob-reports. This gives one HTML report for the whole run instead of one per shard, which makes reviewing failures much easier.

Where should Playwright tests run in a delivery pipeline?

Show answer

Run a fast smoke set on every pull request against a preview or test environment, so feedback arrives in minutes. Run the full regression suite on merge to main or nightly, sharded across machines. After a deploy, run a few read-only production smoke checks. Match the suite size to the stage so pipelines stay fast but coverage is complete.

Ten engineers write tests and quality varies a lot. How do you keep the framework consistent?

Show answer

Write a short contribution guide with examples. Add ESLint with the Playwright plugin and the no-floating-promises rule, plus Prettier, enforced in CI. Require reviews from framework owners for shared fixtures and page objects. Provide generators or templates for new page objects, and run regular sessions to review flaky tests together.

How do you manage flaky tests without hiding real failures?

Show answer

Use retries in CI so one-off failures do not block merges, but treat any test marked flaky in the report as a defect. Track flaky tests on a dashboard, quarantine repeat offenders with a tag and an owner, and set a deadline to fix or delete them. Never retry without investigating, or real intermittent bugs will be missed.

Retries make the build green, but nobody notices flaky tests. How do you surface them?

Show answer

Playwright already marks tests that passed on retry as flaky in the HTML and JSON reports. Read the JSON report in a small script after each run, post flaky test names to the team channel, and fail the build if the flaky count crosses a threshold. Over time you can track trends per test and per area.

How can you run only the tests affected by a code change?

Show answer

Use npx playwright test --only-changed=origin/main. It looks at files changed against that git reference and runs the test files that changed or that import changed files. It suits fast pull request feedback in a large suite. It cannot see backend or config changes that tests do not import, so still run the full suite before release.

How do you upgrade Playwright safely in a large suite?

Show answer

Read the release notes for breaking changes and deprecations. Upgrade on a branch, update the browsers and the Docker image to the same version, and run the full suite on all projects several times. Compare visual snapshots, which often change after browser updates. Upgrade regularly in small steps rather than skipping many versions.

Describe how you mentored junior engineers on your test framework.

Show answer

Situation: three interns joined with little automation experience. Task: make them productive without lowering quality. Action: I ran short sessions on fixtures, locators and traces, paired with each of them on a first test, and gave detailed but kind review comments. Result: within a month they each shipped stable tests, and one later improved our reporting script.

Tell me about a disagreement with a developer over a failing test.

Show answer

Situation: a developer said a failing checkout test was flaky and asked to skip it before release. Task: decide based on evidence. Action: I shared the trace showing the total stayed wrong after a coupon was removed and reproduced it manually. Result: we agreed it was a real bug, it was fixed before release, and we started reviewing traces together.

Baselines are always rendered in Docker, yet the repo has separate Windows and Linux image files. How do you keep one baseline per browser?

Show answer

The default file name includes the platform, so a developer who updated baselines on Windows created a second set. Set snapshotPathTemplate in the config without the {platform} token, for example '{testDir}/__screenshots__/{testFilePath}/{arg}-{projectName}{ext}', delete the stray files and only update baselines through the Docker command or a CI job. Document the rule so it does not happen again.

A designer changes the primary button colour and 300 visual tests fail at once. How do you handle it?

Show answer

First confirm the change is intended and check a sample of diffs to make sure nothing else moved. Then regenerate baselines in the CI Docker environment in a separate commit, so reviewers can see only image changes. Longer term, reduce the count: most of those tests should not snapshot full pages. Keep a few page-level shots and move detail checks to component-level screenshots of key states.

How would you design a visual testing strategy for a large web application?

Show answer

Snapshot stable, high-value screens and shared components, not every page. Render baselines only in a pinned Docker image, mask or mock dynamic data, and disable animations. Keep visual tests in their own project or tag so they can run on pull requests touching UI code. Define owners for baseline reviews and keep thresholds strict in config. Combine with functional assertions, because pictures alone do not tell you why something broke.

An axe scan reports 40 violations on a legacy page. How do you add accessibility checks without blocking every build?

Show answer

Record the current violations as a known baseline, for example by snapshotting violation ids and targets, and fail only on new ones. Fix the worst issues first, such as missing labels and contrast on key flows, and shrink the baseline every sprint. Make new pages and components meet zero violations from day one. Report the trend to the team so the debt is visible rather than silently ignored.

How do you test that a modal dialog traps focus and returns it when closed?

Show answer

Open the dialog with a button and assert the dialog role is visible and focus moved inside it. Press Tab more times than there are controls and check with toBeFocused that focus never leaves the dialog. Press Escape, assert the dialog is hidden, and check that the opening button is focused again. Running the same test with Shift+Tab covers backward navigation as well.

Images load lazily and are sometimes missing from full-page screenshots. How do you make them stable?

Show answer

Scroll the content into view or remove loading=lazy for the test, then wait until every image has loaded, for example with expect.poll checking that all img elements report complete and a natural width above zero. Serving images from mocked routes removes network timing too. Only after that call toHaveScreenshot, which then waits for two identical frames.

When is visual regression testing a poor fit?

Show answer

It works badly for highly dynamic pages such as live feeds, personalised dashboards or content managed by editors, where most diffs are expected. It is also costly when the design changes every sprint, because baseline reviews eat time. In those cases use functional assertions, ARIA snapshots or component-level screenshots of fixed states instead of full-page pictures.

Describe how you convinced a team to adopt accessibility testing.

Show answer

Situation: the team saw accessibility as a later task. Task: I wanted checks in the pipeline. Action: I ran axe on our top five journeys and showed the results with one short screen reader demo, then proposed a non-blocking report first and blocking only for new violations. Result: within a quarter the main flows reached zero serious issues and the check became a required gate.

How do you test conditional GET requests that use ETag and If-None-Match?

Show answer

Call the endpoint once and read the etag header from the response. Call it again with the If-None-Match header set to that value and assert status 304 with an empty body. Then change the resource, repeat the conditional call and assert 200 with a new ETag. This proves caching works and that updates are not hidden behind stale responses.

An endpoint streams server-sent events. How would you test it with Playwright?

Show answer

The request fixture returns the full body only after the stream ends, so for a short finite stream you can read text() and parse each data line. For long-lived streams, test in the browser: open the page that uses EventSource, or open it inside page.evaluate, collect events into an array and assert with expect.poll. Mock the source with a route when you need precise timing.

Why do component tests run in two processes, and what limits does that create?

Show answer

The test code runs in Node while the component runs in the browser. When you write mount(<Button title='Save' />), Playwright serialises the JSX and props and rebuilds them in the browser bundle. So props must be plain data such as strings, numbers, arrays and objects, plus callbacks, which Playwright proxies back to Node. Class instances, complex objects or live references cannot cross that boundary directly.

A component under test needs a complex class instance as a prop and the test fails to serialise it. What do you do?

Show answer

Create a small test story component in a separate file that builds the class instance inside the browser and renders the real component with it. The test then mounts the story with simple props, such as an id or a scenario name. This keeps the data creation on the browser side of the boundary and also documents test scenarios as reusable stories.

The team wants to replace all end-to-end tests with component tests to save time. What do you advise?

Show answer

Agree to move detailed UI checks down to component tests, because they are faster and more precise. But keep a thin layer of end-to-end tests for critical journeys like signup, checkout and payment, since component tests use mocked data and cannot catch broken APIs, routing or deployment issues. Decide per risk, measure runtime savings, and track escaped defects after the change.

How do you test a component that reads from a global state store?

Show answer

Wrap the mounted component in the store provider inside beforeMount, and use hooksConfig to choose the initial state for each test, such as a logged-in user or an empty cart. Build the store in the browser side of index.tsx, not in the test file. Then interact with the component and assert on the rendered output rather than reading store internals.

A component test passes alone but fails when the full component suite runs. What do you check?

Show answer

Look for state that leaks between mounts: global stores or singletons created at module level in index.tsx, timers or listeners not cleaned up on unmount, and shared mock data mutated by one test. Make the providers create fresh state in beforeMount each time and reset mocks per test. Running with --repeat-each and a single worker often reproduces the leak quickly.

What are the risks of relying on an experimental testing feature in a large codebase?

Show answer

APIs may change between minor versions, so upgrades can break many tests at once, and documentation or community examples are thinner. To reduce risk, pin versions, wrap mount setup in shared helpers, keep stories simple and read release notes before upgrading. Also keep the end-to-end suite healthy so you are not dependent on one tool for all UI confidence.

Describe a time you had to decide which testing level a feature belonged to.

Show answer

Situation: a pricing table had many discount rules shown in the UI. Task: decide how to test them. Action: I proposed unit tests for the calculation, component tests for how each rule rendered, and one end-to-end test for buying a discounted plan. I explained the runtime and maintenance cost of each. Result: the team agreed, we covered 30 rules in under a minute, and the approach became our default.

Describe a time you designed a fixture layer that other teams adopted.

Show answer

Situation: three product teams each had their own login and data setup code. Task: give them one reliable layer. Action: I built shared worker-scoped auth fixtures, test-scoped data builders with automatic cleanup, and page object fixtures, published them as an internal package with examples and versioned releases. Result: two more teams adopted it within a quarter, duplicate setup code disappeared and onboarding time for new testers dropped from weeks to days.

A fixture mocks every /api call, but one test needs a different response for /api/cart. How do you override it?

Show answer

Register a new page.route for /api/cart inside that test. Handlers run in reverse order of registration, so the newer one runs first. Fulfil the cart request with the special data, and for anything else call route.fallback() so the request moves on to the fixture's general handler. This keeps the shared mock untouched and makes the override local and easy to read.

How do you verify that an API returns consistent data across two versions of an endpoint?

Show answer

Call both versions, for example /v1/orders/42 and /v2/orders/42, with the same test data. Map the fields that should match, since names may differ, and compare them with toEqual. Also assert that fields removed in v2 are really absent and new ones have correct types. Running this during a migration catches breaking changes before clients switch.

What is a generic type parameter constraint, such as T extends object?

Show answer

A constraint limits what types a generic can accept. function pick<T extends object, K extends keyof T>(obj: T, key: K) only accepts objects and keys that exist on them, so pick(user, 'emial') fails at compile time. In test helpers this catches typos in field names and keeps return types precise without using any.

Two test files edit the same global settings page and break each other when run in parallel. How do you handle it?

Show answer

The best fix is to remove the shared resource, for example by giving each test its own tenant or user whose settings it can change. If that is impossible, put those tests in one file with test.describe.configure({ mode: 'serial' }), or in a separate project that runs with one worker after the main project. Document why, so the exception does not quietly spread.

How does sharding split tests when fullyParallel is on compared with off?

Show answer

With fullyParallel off, Playwright shards by test file, so one large file always lands on one shard and shards can be uneven. With fullyParallel on, it shards by individual test, giving much more even distribution. Either way the split is deterministic, so the same shard number always gets the same tests for a given set of files, which helps reproduce failures.

Write a minimal custom reporter that prints the titles of failed tests at the end.

Show answer

import type { Reporter, TestCase, TestResult } from '@playwright/test/reporter'; class FailedListReporter implements Reporter { private failed: string[] = []; onTestEnd(test: TestCase, result: TestResult) { if (result.status === 'failed' || result.status === 'timedOut') { this.failed.push(test.titlePath().join(' > ')); } } onEnd() { console.log(`Failed tests (${this.failed.length}):`); this.failed.forEach((t) => console.log(' - ' + t)); } } export default FailedListReporter;

One shard finishes in 4 minutes and another takes 15. How do you balance them?

Show answer

Check whether fullyParallel is off, because file-level sharding puts big files together. Turn it on if tests are independent, or split the largest files. Use reportSlowTests and the merged report's durations to find heavy files, and move slow setup into API calls or shared storage state. Also check that one shard is not getting all the tests of a slow project such as WebKit.

How would you build a flaky-test dashboard from Playwright results?

Show answer

Write a custom reporter or parse the JSON report after each CI run, and store per test: id, status, retry count, duration, branch and commit. A test that passed after a retry counts as flaky. Aggregate over the last few hundred runs to rank tests by flake rate and duration, and show trends. Use it to assign owners and quarantine tests above an agreed threshold.

How do you decide how many shards a suite should use?

Show answer

Start from the target wall-clock time and the total test time per machine. If one machine with its workers takes 40 minutes and the goal is under 10, you need about four or five shards, plus some margin for setup like installing browsers. Past a point each shard spends more time on setup than on tests, and the app or test data may become the bottleneck, so measure each change.

How do you rerun only the tests that failed in a sharded CI run?

Show answer

--last-failed works only on the machine that kept the .last-run.json file, so it does not help across shards. Instead collect the failed tests from the merged JSON report or the blob reports, then run them in a follow-up job by file and line or with --grep on their titles. Many teams simply rely on retries and treat any test needing a rerun as flaky.

Describe a time you improved how test results were reported to the team or managers.

Show answer

Situation: managers only heard pass or fail and could not tell flaky tests from real bugs. Task: make results useful. Action: I added a custom reporter that posted a short summary to our team chat with failures, flaky tests and a report link, and a weekly trend of flake rate. Result: flaky tests dropped by half in two months because owners could see them.

How do you run tests against a preview deployment created for each pull request?

Show answer

Make baseURL come from an environment variable, such as process.env.BASE_URL, with a default for local runs. In the pipeline, wait for the preview deployment to be ready, read its URL from the deployment step and pass it to the test job. Add a quick health check request first, so a failed deployment is reported clearly instead of as hundreds of test failures.

An API receives a payload with an unknown extra field. How do you test what it should do?