QA Automation Lead Interview Questions &amp; Answers

Show answer

reportSlowTests makes Playwright list test files that took longer than a threshold at the end of the run. You set max, the number of files to show, and threshold in milliseconds, for example { max: 5, threshold: 60000 }. It is a cheap way to find files worth splitting, because one slow file can hold up a worker or a whole shard.

What does the DEBUG=pw:api environment variable do?

Debugging and Trace ViewerMedium

Show answer

Setting DEBUG=pw:api prints detailed logs of every Playwright API call to the terminal, including waits and retries inside actions. It helps when a test hangs and you need to see exactly which step is waiting and why. Other channels such as pw:browser show browser process output. It is noisy, so use it only while investigating.

How do you tag and filter tests?

Show answer

Add tags in the title (@smoke) or use the tag option, and run with --grep @smoke or --grep-invert.

CI runners have 2 CPU cores and workers is set to 8. Tests start timing out randomly. Why?

Show answer

Eight browsers compete for two cores and limited memory, so pages render slowly, actions hit timeouts and the app under test may also be overloaded. More workers than cores rarely helps. Set workers to match the runner, such as 2, and scale out with shards on more machines instead. Watch CPU and memory metrics in CI to confirm.

Retries make the build green, but nobody notices flaky tests. How do you surface them?

Show answer

Playwright already marks tests that passed on retry as flaky in the HTML and JSON reports. Read the JSON report in a small script after each run, post flaky test names to the team channel, and fail the build if the flaky count crosses a threshold. Over time you can track trends per test and per area.

Tests pass with one worker but fail with eight. How do you debug this?

Debugging and Trace ViewerHard

Show answer

This usually means tests share something: the same user account, the same records or a global server setting. Run the failing file with --workers=8 and --repeat-each to reproduce it. Look at which tests run at the same time in the report. Fix it by giving each test unique data, using workerIndex in names, or isolating shared settings per worker.

How do you mock WebSocket traffic in Playwright?

Network MockingHard

Show answer

Use page.routeWebSocket with a URL pattern. The handler gets a WebSocketRoute: you can reply to messages with ws.onMessage and ws.send without any server, or call connectToServer() to sit in the middle and change messages in both directions. This lets you test live dashboards and chat apps, for example pushing a fake price update.

Describe a time you had to decide which testing level a feature belonged to.

Component TestingHard

Show answer

Situation: a pricing table had many discount rules shown in the UI. Task: decide how to test them. Action: I proposed unit tests for the calculation, component tests for how each rule rendered, and one end-to-end test for buying a discounted plan. I explained the runtime and maintenance cost of each. Result: the team agreed, we covered 30 rules in under a minute, and the approach became our default.

Mocked responses drift from the real API and tests pass while production breaks. How do you prevent this?

Network MockingHard

Show answer

Generate mock data from the API schema or real recorded responses instead of hand-writing it. Validate mocks against the OpenAPI schema in a unit test. Add contract tests on the backend side and re-record HAR files on a schedule. Keep a few unmocked end-to-end tests that run after each deploy. Treat mock files as code with owners and reviews.

How does sharding split tests when fullyParallel is on compared with off?

Show answer

With fullyParallel off, Playwright shards by test file, so one large file always lands on one shard and shards can be uneven. With fullyParallel on, it shards by individual test, giving much more even distribution. Either way the split is deterministic, so the same shard number always gets the same tests for a given set of files, which helps reproduce failures.

How do you run one group of tests serially while the rest of the suite runs in parallel?

Show answer

Put the group in its own project with fullyParallel: false and limit it to one worker, which recent versions allow with a workers setting on the project, or run it in a separate CI step with --project=serial --workers=1. Inside a file, test.describe.configure({ mode: 'serial' }) keeps order. Use project dependencies if the group must run after the parallel tests.

Where should Playwright tests run in a delivery pipeline?

Show answer

Run a fast smoke set on every pull request against a preview or test environment, so feedback arrives in minutes. Run the full regression suite on merge to main or nightly, sharded across machines. After a deploy, run a few read-only production smoke checks. Match the suite size to the stage so pipelines stay fast but coverage is complete.

How can you run only the tests affected by a code change?

Show answer

Use npx playwright test --only-changed=origin/main. It looks at files changed against that git reference and runs the test files that changed or that import changed files. It suits fast pull request feedback in a large suite. It cannot see backend or config changes that tests do not import, so still run the full suite before release.

How do you reuse login state across tests?

Show answer

Log in once in global setup, save storageState to a JSON file and set use.storageState in the config so every test starts logged in.

Ten engineers write tests and quality varies a lot. How do you keep the framework consistent?

Show answer

Write a short contribution guide with examples. Add ESLint with the Playwright plugin and the no-floating-promises rule, plus Prettier, enforced in CI. Require reviews from framework owners for shared fixtures and page objects. Provide generators or templates for new page objects, and run regular sessions to review flaky tests together.

How do you combine reports from sharded CI jobs?

Show answer

Use the blob reporter in each shard job, upload each blob-report folder as an artifact, then in a final job download them all into one folder and run npx playwright merge-reports --reporter html ./all-blob-reports. This gives one HTML report for the whole run instead of one per shard, which makes reviewing failures much easier.

How does Playwright talk to browsers, and why does that make it less flaky?

Playwright CoreHard

Show answer

Playwright runs outside the browser and controls it through a persistent connection using each engine's own protocol, such as CDP for Chromium and patched protocols for Firefox and WebKit. It can watch DOM, network and events directly, so it waits on real state instead of polling from inside the page. This gives reliable auto-waiting, multi-tab and network control.

Design a Playwright framework for a 500-test suite.

Show answer

TypeScript + POM + custom fixtures, environment configs, API helpers for test data, tags like @smoke, sharding across CI machines, retries with traces, HTML/Allure reports, lint and review rules.

What can automated accessibility tools not catch?

Visual & Accessibility TestingMedium

Show answer

Scanners like axe find rule-based problems such as missing labels, bad contrast or invalid ARIA, which is roughly a third to a half of real issues. They cannot judge whether alt text is meaningful, whether focus order makes sense, whether error messages are understandable, or whether a screen reader user can actually finish the task. So automated scans are combined with keyboard tests and periodic manual checks with screen readers.

How do you integrate Playwright with Allure or other reports?

Show answer

Install the allure-playwright reporter, add it in the config reporters array and generate the report from the allure-results folder in CI.

How do you run tests against a preview deployment created for each pull request?

Show answer

Make baseURL come from an environment variable, such as process.env.BASE_URL, with a default for local runs. In the pipeline, wait for the preview deployment to be ready, read its URL from the deployment step and pass it to the test job. Add a quick health check request first, so a failed deployment is reported clearly instead of as hundreds of test failures.

Write an accessibility scan that checks only WCAG 2.1 A and AA rules.

Visual & Accessibility TestingMedium

Show answer

import { test, expect } from '@playwright/test'; import AxeBuilder from '@axe-core/playwright'; test('signup page has no WCAG A/AA issues', async ({ page }) => { await page.goto('/signup'); const results = await new AxeBuilder({ page }) .withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa']) .analyze(); expect(results.violations).toEqual([]); });

How would you design a visual testing strategy for a large web application?

Show answer

Snapshot stable, high-value screens and shared components, not every page. Render baselines only in a pinned Docker image, mask or mock dynamic data, and disable animations. Keep visual tests in their own project or tag so they can run on pull requests touching UI code. Define owners for baseline reviews and keep thresholds strict in config. Combine with functional assertions, because pictures alone do not tell you why something broke.

How do you test that a modal dialog traps focus and returns it when closed?

Show answer

Open the dialog with a button and assert the dialog role is visible and focus moved inside it. Press Tab more times than there are controls and check with toBeFocused that focus never leaves the dialog. Press Escape, assert the dialog is hidden, and check that the opening button is focused again. Running the same test with Shift+Tab covers backward navigation as well.

How do you write a custom matcher with expect.extend?

Locators and AssertionsHard

Show answer

expect.extend takes an object of matcher functions. Each receives the received value and arguments, and returns { pass, message, name, expected, actual }. For example a toHaveAmount matcher can read a locator's text, strip the currency symbol and compare numbers, retrying with expect.poll inside. Export the extended expect from a shared fixtures file so all tests use the same version.

The suite has a 3 percent flaky rate. How do you find and fix the causes systematically?

Debugging and Trace ViewerHard

Show answer

Collect data first: flaky results from the reports across a few weeks, grouped by test, browser and error message. Run the worst offenders with --repeat-each and traces on. Fix common causes such as missing awaits, shared data, fixed sleeps and order-dependent tests. Quarantine tests you cannot fix quickly with a tag and a ticket, and track the rate weekly.

What is a fixture option and how do you define one?

Fixtures and HooksHard

Show answer

An option is a fixture whose value is configured rather than computed, declared with { option: true } and a default, for example defaultUser: ['standard', { option: true }]. Projects or test.use can override it, so one suite can run as admin in one project and as a viewer in another. Other fixtures can read the option to decide what to set up.

Tell me about a disagreement with a developer over a failing test.

Show answer

Situation: a developer said a failing checkout test was flaky and asked to skip it before release. Task: decide based on evidence. Action: I shared the trace showing the total stayed wrong after a coupon was removed and reproduced it manually. Result: we agreed it was a real bug, it was fixed before release, and we started reviewing traces together.

How would you structure a large Playwright framework for a team?

Show answer

Separate tests, page objects, fixtures, utilities, test data and config; use tags, projects, shared auth setup, linting, code review and a CI pipeline with sharding and reports.

An axe scan reports 40 violations on a legacy page. How do you add accessibility checks without blocking every build?

Show answer

Record the current violations as a known baseline, for example by snapshotting violation ids and targets, and fail only on new ones. Fix the worst issues first, such as missing labels and contrast on key flows, and shrink the baseline every sprint. Make new pages and components meet zero violations from day one. Report the trend to the team so the debt is visible rather than silently ignored.

What does maxFailures do?

Show answer

maxFailures stops the run after a set number of test failures, for example maxFailures: process.env.CI ? 10 : undefined. If an environment is down and every test fails, the build stops early instead of burning an hour of CI time. Set it high enough that a few real failures still show their full picture.

Tests are flaky in CI. How do you debug them?

Show answer

Turn on trace: 'on-first-retry', open the trace viewer, check network and timing, replace hard waits with web-first assertions, isolate test data and check for sharding conflicts.

One shard finishes in 4 minutes and another takes 15. How do you balance them?

Show answer

Check whether fullyParallel is off, because file-level sharding puts big files together. Turn it on if tests are independent, or split the largest files. Use reportSlowTests and the merged report's durations to find heavy files, and move slow setup into API calls or shared storage state. Also check that one shard is not getting all the tests of a slow project such as WebKit.

Write a minimal custom reporter that prints the titles of failed tests at the end.

Show answer

import type { Reporter, TestCase, TestResult } from '@playwright/test/reporter'; class FailedListReporter implements Reporter { private failed: string[] = []; onTestEnd(test: TestCase, result: TestResult) { if (result.status === 'failed' || result.status === 'timedOut') { this.failed.push(test.titlePath().join(' > ')); } } onEnd() { console.log(`Failed tests (${this.failed.length}):`); this.failed.forEach((t) => console.log(' - ' + t)); } } export default FailedListReporter;

How do you rerun only the tests that failed in a sharded CI run?

Show answer

--last-failed works only on the machine that kept the .last-run.json file, so it does not help across shards. Instead collect the failed tests from the merged JSON report or the blob reports, then run them in a follow-up job by file and line or with --grep on their titles. Many teams simply rely on retries and treat any test needing a rerun as flaky.

Baselines are always rendered in Docker, yet the repo has separate Windows and Linux image files. How do you keep one baseline per browser?

Show answer

The default file name includes the platform, so a developer who updated baselines on Windows created a second set. Set snapshotPathTemplate in the config without the {platform} token, for example '{testDir}/__screenshots__/{testFilePath}/{arg}-{projectName}{ext}', delete the stray files and only update baselines through the Docker command or a CI job. Document the rule so it does not happen again.

Describe a time you improved how test results were reported to the team or managers.

Show answer

Situation: managers only heard pass or fail and could not tell flaky tests from real bugs. Task: make results useful. Action: I added a custom reporter that posted a short summary to our team chat with failures, flaky tests and a report link, and a weekly trend of flake rate. Result: flaky tests dropped by half in two months because owners could see them.

How do you manage test data and environments?

Show answer

Use environment variables or .env files, a baseURL in config, API calls for setup and cleanup, and unique data per run.

How would you handle a very flaky element that appears after an animation?

Locators and AssertionsHard

Show answer

Rely on auto-waiting plus a web-first assertion on the final state, disable animations in config for tests and avoid fixed timeouts.

What is the blob reporter and why does it exist?