Hard Automation Testing Interview Questions

Show answer

Headless mode or screen resolution, timing issues, environment data, missing dependencies, browser–driver version mismatch or parallel conflicts. Check logs and failure screenshots and make waits robust.

How would you automate a login flow with multiple user roles?

Show answer

Keep credentials in external data, parameterise the test with a data provider, reuse a login method in a page class, and clear sessions between roles.

Explain your CI/CD pipeline for tests.

Show answer

Code pushed to Git → Jenkins job triggers (webhook or schedule) → mvn clean test with the suite XML → reports and screenshots published → notifications on Slack/email → failures triaged before release.

How would you design a CI pipeline for nightly regression?

Show answer

Schedule the job, spin up environments or containers, run suites in parallel, store reports and screenshots, notify failures on Slack or email and track trends.

How do you handle a website that uses CAPTCHA?

Show answer

Do not automate it; ask for a test environment where CAPTCHA is disabled or use test keys, or a bypass token agreed with developers.

How would you design an automation framework from scratch?

Show answer

Maven project → POM with a BasePage → driver factory with ThreadLocal → config files → data-driven utilities (Excel/JSON) → TestNG or Cucumber runner → logging → Extent reports with screenshots on failure → Jenkins pipeline.

How do you decide what to automate?

Show answer

Automate stable, repetitive, high-risk and data-heavy flows (smoke, regression). Skip one-time tests, frequently changing UI and checks needing human judgement. Compare run frequency × manual time with build and maintenance cost.

How do you reduce flakiness in a CI pipeline?

Show answer

Stabilise waits and data, isolate tests, use retries cautiously, quarantine unstable tests, run in consistent containers and track flaky rates.

How do you calculate automation ROI?

Show answer

Compare effort saved by running automated tests repeatedly against the cost of building and maintaining scripts, counting execution time, defects found earlier and release speed.

How do you run Cucumber scenarios in parallel?

Show answer

With TestNG, extend AbstractTestNGCucumberTests and override scenarios() with @DataProvider(parallel = true), then set data-provider-thread-count. With JUnit 5, set cucumber.execution.parallel.enabled=true in junit-platform.properties. Each scenario must own its WebDriver, usually through a ThreadLocal or a per-scenario DI context, and test data must not clash. Parallelism is per scenario, not per step.

Your team has 600 Cucumber scenarios and step definitions have become a mess of duplicates. What do you do?

Show answer

First run a report of step usage to find duplicate and unused steps. Agree a step vocabulary per domain, merge near-identical steps using parameters, and delete unused ones. Move logic out of steps into page objects and service helpers. Add a review checklist and a dryRun gate in CI. Do this incrementally by feature area so the suite keeps running while you clean up.

Business stakeholders never read your feature files. Is BDD still worth it?

Show answer

Partly. Readable scenarios still help testers and developers, but if nobody discusses examples before coding, Cucumber is only an extra layer over normal tests. I would first try to restart three-amigos sessions for complex stories. If the business truly will not engage, I would keep Gherkin only for key business flows and write lower-level tests in plain TestNG to reduce maintenance.

How do you create a custom parameter type in Cucumber?

Show answer

Annotate a method with @ParameterType, giving a regex and returning a domain object. For example @ParameterType("admin|manager|viewer") public Role role(String name) { return Role.valueOf(name.toUpperCase()); } lets you write steps like "I log in as {role}". It keeps conversion logic in one place, makes steps type-safe and gives clearer errors when an invalid value is used.

What is the scope of a step definition class instance in Cucumber?

Show answer

Cucumber creates a new instance of every glue class for each scenario. Instance fields therefore reset between scenarios, which keeps scenarios isolated. This is why you should not rely on static variables for state: they survive across scenarios and threads, causing order-dependent failures and clashes in parallel runs. Share state through injected context objects instead.

An API test passes alone but fails when the suite runs in parallel. What could be wrong?

Show answer

Likely shared state. Common causes are static RestAssured settings changed by one test, shared test data like the same user being updated by two tests, token reuse after another test logged out, or rate limits. Fix by using per-test RequestSpecification instead of global statics, creating unique data per test, isolating tokens, and checking server rate-limit responses such as 429.

How do you design a REST Assured framework for many microservices?

Show answer

Create a base module with shared specs, auth handling, config per environment, filters for logging and reporting, and utilities. Each service gets its own client class wrapping endpoints and POJOs for its models. Tests call clients, never raw URLs. Use builders for test data, schema checks for contracts, tags to run per service, and run independent services in parallel in CI.

PageFactory with @FindBy or By locators: which do you prefer and why?

Show answer

Many teams now prefer plain By locators with explicit waits. PageFactory proxies look up elements lazily, which can hide timing problems and makes StaleElementReference harder to reason about, and its AjaxElementLocatorFactory waits are less flexible. PageFactory is fine and readable for stable pages. The key is consistency across the framework and always waiting for the right condition before acting.

What is the Screenplay pattern and how does it compare to POM?

Show answer

Screenplay models tests around actors who have abilities (browse the web, call an API), perform tasks (place an order) made of interactions, and ask questions to verify state. It scales better for large suites because tasks are small, composable classes, whereas page objects can grow huge. It has a steeper learning curve, so POM with components is enough for most teams.

How would you design a framework that supports both web and mobile using POM?

Show answer

Define page interfaces for business actions, such as LoginScreen.loginAs(). Provide web and mobile implementations using Selenium and Appium, chosen by a factory from configuration. Tests depend only on the interfaces. Share test data, config, reporting and API helpers. Accept that some flows differ by platform and keep those in platform-specific tests rather than forcing one abstraction.

You join a project where tests call driver.findElement directly everywhere. How do you move to POM?

Show answer

I would not rewrite everything at once. First create the base structure: DriverManager, BasePage and config. Then pick the most-used flows, such as login and navigation, create page objects for them and migrate tests touching those pages. Agree that new tests must use POM, and migrate old tests when they are next changed. Track progress so the team sees improvement in maintenance time.

Should page objects use inheritance or composition?

Show answer

Use a shallow inheritance only for a BasePage with common helpers. For shared UI parts, use composition: a page has a Header, a SideMenu and a Table component. Deep inheritance like BasePage to ListPage to ProductListPage to SaleProductListPage becomes fragile and hard to debug. Composition mirrors how modern UIs are built from components and keeps classes small.

How do you test page objects themselves, or keep them reliable?

Show answer

Page objects are tested indirectly through a smoke suite that exercises each page method at least once. Keep them simple with no logic beyond waiting and interacting. Add clear error messages in waits, log each action at debug level, and run the smoke suite on every pull request that changes framework code, so a broken page method is caught before regression runs.

How do you run tests on multiple browsers in parallel in Jenkins?

Show answer

Use a parallel block in the pipeline with a stage per browser, or a matrix directive with axis BROWSER values chrome, firefox and edge. Each branch runs mvn test -Dbrowser=${BROWSER}, ideally against Selenium Grid or containers so agents do not need local browsers. Publish results per branch with clear names so failures show which browser broke.

What is a Maven multi-module project and when would you use it for automation?

Show answer

A parent pom lists modules that share dependency versions through dependencyManagement and plugins through pluginManagement. In automation you might have core-framework, ui-tests, api-tests and performance modules. Teams can run or release modules separately, and shared code lives in one place. It is worth it when several teams or test types share the same framework.

How do you fix a Maven dependency conflict?

Show answer

Run mvn dependency:tree to see which versions are pulled in transitively. Maven picks the nearest version, which may not be the one you need. Fix by declaring the required version in dependencyManagement, excluding the unwanted transitive dependency with exclusions, or upgrading the library that brings the old version. Conflicts often appear as NoSuchMethodError or ClassNotFoundException at runtime.

Nightly Jenkins regression runs take 4 hours. How would you bring it down?

Show answer

Measure first: find the slowest tests and setup steps. Then run tests in parallel across Grid nodes or containers, replace UI setup with API calls, remove duplicate coverage, move checks down to API level, split the suite into shards across agents, and cache Maven dependencies. Aim for a smoke stage under 15 minutes on every commit and a parallel regression under an hour.

What is consumer-driven contract testing with Pact?

Show answer

In consumer-driven contracts, the consumer's tests record the requests it makes and the responses it expects into a pact file. The provider runs a verification against that pact to prove it still meets every consumer's needs. A Pact Broker stores contracts and can block deployments that break a consumer. It catches integration breaks without needing all services deployed together.

How do you test API security basics as an automation tester?

Show answer

Check that endpoints reject missing or invalid tokens with 401, block users from other users' data with 403 or 404, validate input to prevent injection, do not leak stack traces or sensitive fields, enforce HTTPS and set security headers. Automate these as negative tests in the regression suite and work with the security team for deeper penetration testing.

How do you manage test environments for a CI pipeline?

Show answer

Keep environment details in config files or environment variables per env, never in code. Prefer ephemeral environments created per pull request with containers, seeded with known data and destroyed afterwards. For shared environments, reserve them through the pipeline, reset data before runs and monitor health with a pre-test check so tests do not fail due to a down service.

A third-party payment API is slow and makes your tests time out in CI. What do you do?

Show answer

Separate concerns: for most functional tests, stub the payment API with WireMock returning success, failure and timeout responses, so tests are fast and deterministic. Keep a small set of integration tests against the sandbox, with longer timeouts and retries for network errors, run on a schedule. Report sandbox slowness separately so it does not block every pull request.

How do you test GraphQL APIs compared with REST?

Show answer

GraphQL usually has one endpoint and you send a POST with a query and variables. Tests check the data returned matches the requested fields, that errors appear in the errors array, often with status 200, and that authorization applies per field. Also test query depth limits and nested queries for performance. REST Assured works fine by sending the query as JSON body.

What metrics do you track for test automation in CI/CD?

Show answer

Useful metrics are pass rate per run, flaky test rate, average and slowest test duration, pipeline time to feedback, defects caught by automation versus escaped to production, automation coverage of critical flows and time spent on maintenance. Trend them over weeks. Metrics should drive action, such as fixing the top flaky tests, not just fill dashboards.

What is the @Factory annotation in TestNG?

Show answer

@Factory creates test class instances dynamically, each with different constructor data, so all tests in the class run once per instance. For example a factory can create LoginTests for chrome and firefox. It differs from DataProvider, which runs one method many times with different arguments. Factory is useful when a whole class should repeat for each configuration.

What is IAnnotationTransformer in TestNG?

Show answer

IAnnotationTransformer is a listener that changes @Test annotations at runtime before tests run. A common use is attaching a retry analyzer to every test automatically instead of adding retryAnalyzer to each method, or enabling and disabling tests from configuration. It must be registered in testng.xml or via service loader, not with the @Listeners annotation.

Your API token expires after 15 minutes and long REST Assured suites start failing with 401. What do you do?

Show answer

Centralise token handling in an auth helper that caches the token with its expiry time and refreshes it shortly before it expires, or in a filter that catches a 401, refreshes once and retries. Do not log in before every test, as that slows the suite and may hit rate limits. Make the helper thread-safe for parallel runs.

Developers changed a field from a number to a string and your tests did not catch it. How do you prevent this?

Show answer

Add JSON schema validation for key responses so type changes fail immediately, and deserialize into typed POJOs, which fail when types do not match. For services with multiple consumers, introduce contract tests so providers learn about breaks before deploying. Also ask for API changes to be shared in pull requests or a changelog that QA reviews.

Tell me about a time you led the design of a test framework for a new project.

Show answer

Situation: a new product had no automation and a three-month release target. Task: I was asked to set up the framework and guide two juniors. Action: I chose Java, TestNG, Selenium with POM and REST Assured, set up Maven, Jenkins and Allure, wrote conventions and reviewed every pull request early on. Result: we had 250 stable tests in CI before release, and the juniors could extend it independently.

Your retry analyzer makes the suite green but bugs are slipping into production. What is wrong?

Show answer

Retries are hiding real intermittent defects, such as race conditions, by treating them as flaky tests. Limit retries to one, report retried tests separately as flaky in the dashboard, and require a ticket for each one. Review retried tests weekly to find whether the cause is the test or the product. Retries should buy time to fix, not replace fixing.

Parallel TestNG runs open browsers that interact with each other's tests. How do you fix it?

Show answer

The WebDriver is shared, probably through a static field. Replace it with a ThreadLocal<WebDriver> in a DriverManager, create the driver in @BeforeMethod and call quit() and remove() in @AfterMethod(alwaysRun = true). Make page objects take the driver from the current thread. Also check test data, reports and loggers are thread-safe.

Your test framework leaks memory during long runs and Jenkins agents crash. How do you investigate?

Show answer

Check whether drivers are always quit, ThreadLocal values are removed, and static lists or maps keep growing, such as collecting screenshots or response objects. Enable heap dumps with -XX:+HeapDumpOnOutOfMemoryError and analyse with a tool like VisualVM or Eclipse MAT to see which objects hold memory. Fix the leak and tune Surefire's forked JVM memory with argLine if needed.

A utility method that parses dates from the UI breaks in parallel runs. What might be wrong?

Show answer

It probably uses a shared static SimpleDateFormat, which is not thread-safe, so parallel threads corrupt each other's parsing. Replace it with java.time DateTimeFormatter, which is immutable and thread-safe, and LocalDate or LocalDateTime for values. Generally, avoid shared mutable static objects in test utilities that run in parallel.

You need the same test to support Chrome, Firefox and Edge. How would you design driver creation in Java?

Show answer

Use a factory: a BrowserType enum and a DriverFactory.create(BrowserType) that returns the right WebDriver with its options, using a switch. Read the browser from a system property with a default. For remote runs, return RemoteWebDriver with the same options. Tests depend only on the WebDriver interface, so adding a new browser changes one class, following the open-closed principle.

You must read test data from a 50,000-row CSV for a data-driven test. How do you handle it efficiently in Java?

Show answer

Do not load everything as one giant DataProvider; that creates thousands of UI tests. First question whether all rows need UI runs; most can be validated at API level. Read the file with BufferedReader or Files.lines() as a stream, filter to the rows you need, and use a lazy Iterator<Object[]> DataProvider so rows load as tests run, keeping memory low.

A file download test passes locally but fails in a Grid node. How do you approach it?

Show answer

The file is saved on the remote node, not the machine running the test, so the test cannot find it locally. Options are enabling Selenium Grid's managed downloads with se:downloadsEnabled and fetching the file through the Grid API, sharing a mounted volume with the container, or verifying the download by requesting the file URL with an HTTP client and checking status, size and type.

Your UI regression suite has 15 percent flaky failures. How would you bring it under control?

Show answer

Measure flakiness per test over many runs and fix the worst first. Typical fixes are replacing sleeps with explicit waits, creating independent test data via API, avoiding shared logins, using stable locators, isolating parallel drivers and stubbing unstable third parties. Quarantine known flaky tests into a separate job so they do not block releases, and set a target like under 2 percent.

How do you test an application that requires login on every test without slowing the suite?

Show answer

Log in once through the API, capture the session cookie or token, and inject it into the browser with driver.manage().addCookie() after opening the domain, or set local storage with JavaScript. Then navigate directly to the page under test. Keep one real UI login test to cover the login page itself. This often saves several seconds per test.

How do you automate a two-factor authentication step in a test environment?

Show answer

Do not try to read SMS from a real phone. Ask developers for a test-only approach: a fixed OTP for test users, a bypass flag in non-production, or an API to fetch the latest OTP. For authenticator apps, store the test account's TOTP secret and generate codes in Java with a TOTP library. Keep production 2FA untouched and secrets in the CI credential store.

Tell me about a time you had to automate a very complex or unusual UI.

Show answer

Situation: our app had a drag-and-drop dashboard builder built on canvas. Task: automate the key layout flows. Action: I tried the Actions class, found canvas elements were not in the DOM, and worked with developers to expose test hooks and an API to verify saved layouts. Result: we automated the main flows reliably, combining a few UI actions with API checks.

How do you prioritise when the release is tomorrow and many automated tests are failing?

Show answer

Situation: 60 tests failed the evening before a release. Task: give a clear go or no-go input. Action: I grouped failures by root cause, found most came from one changed locator and one test data issue, fixed those first, and manually checked the three failures that pointed to real bugs. Result: I gave the lead a short report in two hours and one real defect was fixed before release.

How do you handle pressure when management wants 100 percent automation coverage?

Show answer

Situation: leadership set a goal to automate every manual test case. Task: set realistic expectations. Action: I analysed the cases, showed that some were one-time, visual or exploratory, and proposed covering critical and repetitive flows first with a clear ROI estimate. Result: we agreed on 80 percent of regression cases, delivered it on time, and kept exploratory testing manual.

Tell me about a time you had to choose between Selenium and another tool for a project.

Show answer

Situation: a new project team asked whether to keep Selenium or switch tools. Task: recommend an option with evidence. Action: I ran a two-week proof of concept automating the same five flows in both, compared stability, run time, team skills and Grid support, and shared the results. Result: we kept Selenium with Java for the existing team skills and Grid setup, and adopted better waits and parallel runs from the trial.

Write Appium Java code to perform a vertical swipe using W3C Actions.

Show answer

// Swipe up from 80% to 20% of the screen height Dimension size = driver.manage().window().getSize(); int x = size.width / 2; int startY = (int) (size.height * 0.8); int endY = (int) (size.height * 0.2); PointerInput finger = new PointerInput(PointerInput.Kind.TOUCH, "finger"); Sequence swipe = new Sequence(finger, 1); swipe.addAction(finger.createPointerMove(Duration.ZERO, PointerInput.Origin.viewport(), x, startY)); swipe.addAction(finger.createPointerDown(PointerInput.MouseButton.LEFT.asArg())); swipe.addAction(finger.createPointerMove(Duration.ofMillis(600), PointerInput.Origin.viewport(), x, endY)); swipe.addAction(finger.createPointerUp(PointerInput.MouseButton.LEFT.asArg())); driver.perform(List.of(swipe));

How does Appium automate iOS apps and what setup is required?

Show answer

Appium uses the XCUITest driver, which builds and installs WebDriverAgent on the simulator or device. WebDriverAgent is a small XCTest-based server that receives commands and drives the UI. You need a Mac with Xcode, the iOS SDK and command line tools. For real devices you also need a valid Apple developer team, provisioning profile and signed WebDriverAgent. Most iOS setup failures come from signing problems or Xcode and iOS version mismatches.

Your 200 mobile tests across six devices take too long to finish. How would you speed them up?

Show answer

Run devices in parallel with one thread per device, each with its own udid, systemPort or wdaLocalPort. Use deep links or API calls to reach the screen under test instead of navigating through the UI every time. Prefer noReset with explicit data cleanup over reinstalling the app. Replace XPath with accessibility ids. Split the suite so pull requests run a small smoke set on emulators while the full device matrix runs nightly.

In a hybrid Android app, driver.getContextHandles() returns only NATIVE_APP. Why?

Show answer

The webview is probably not debuggable. Android only exposes a webview to automation when the app calls WebView.setWebContentsDebuggingEnabled(true), which developers usually enable only in debug builds, so ask for a debug or test build. Also wait until the webview screen is actually loaded before reading contexts. If the app uses Chrome custom tabs, that is a separate Chrome process, not a webview. Finally check that a compatible Chromedriver is available to Appium.

How do you run Appium tests in parallel on several Android devices?

Show answer

Give each device its own session: unique appium:udid, a separate appium:systemPort for UiAutomator2 and, for hybrid apps, a separate chromedriverPort. Store the driver in a ThreadLocal so threads never share it. Pass device details through TestNG parameters or a DataProvider and run with parallel="tests", one test tag per device. One Appium server can handle several sessions, or use a Selenium Grid 4 with Appium nodes or a cloud device farm for larger matrices.

How do you test deep links in a mobile app with Appium?

Show answer

On Android use the mobile: deepLink command with the URL and package, for example driver.executeScript("mobile: deepLink", Map.of("url", "shop://product/42", "package", "com.example.shop")). On iOS simulators driver.get with the custom URL opens it, though Safari may show a confirm dialog to handle. Then assert that the correct screen and data are shown. Deep links are also a great shortcut to reach deep screens quickly in other tests.

How would you design one mobile framework that serves both Android and iOS?

Show answer

Keep tests platform-neutral and push differences into page objects. Use Appium's @AndroidFindBy and @iOSXCUITFindBy annotations, or separate locator files per platform, behind one screen class. Create the driver through a factory that reads platform from configuration. Ask developers to add the same accessibility ids on both platforms so most locators are shared. Put gestures and waits in a common base screen, and tag tests that apply to only one platform.

Your company is debating a cloud device farm versus an in-house device lab. How do you advise?

Show answer

A cloud farm gives hundreds of device and OS combinations, parallel capacity and no hardware maintenance, but costs per minute, adds network latency and may raise data privacy concerns. An in-house lab is cheaper over time for a small fixed set of devices and gives full control, but needs USB hubs, OS updates, battery care and someone to maintain it. Many teams keep a few local devices for debugging and use the cloud for wide coverage.

What can testers do with Chrome DevTools Protocol support in Selenium 4?

Show answer

Selenium 4 exposes CDP through the DevTools interface and the newer cross-browser BiDi APIs. Testers can capture console logs and JavaScript errors, intercept or mock network requests, emulate slow networks or geolocation, add basic authentication headers and collect performance metrics. CDP is Chromium-specific and tied to browser versions, so for long-term code prefer WebDriver BiDi features where available, and keep CDP usage behind a small helper class.

What is ITestContext in TestNG and how can you use it to share data?

Show answer

ITestContext represents one test tag from testng.xml and is passed to listeners and configuration methods. It holds the suite, included methods and results, and has setAttribute and getAttribute to share values, such as an auth token created in @BeforeTest and read by later classes. Because it is shared by every thread in that test tag, store only thread-safe or read-only data there, not WebDriver instances during parallel runs.

What are the main components of Selenium Grid 4?

Show answer

Grid 4 is split into the Router, which receives all requests, the Distributor, which assigns new sessions to nodes, the Session Map, which records which node owns each session, the New Session Queue, which holds requests waiting for a free slot, the Event Bus, which lets components communicate, and Nodes, which run the browsers. In standalone or hub mode these run in one process, but they can also be deployed separately to scale.

What is the New Session Queue in Selenium Grid 4 and what happens when all slots are busy?

Show answer

When a test requests a session and no node slot matches, the request waits in the New Session Queue instead of failing immediately. The distributor picks it up when a matching slot frees. If it waits longer than the session request timeout, which is 300 seconds by default, the client gets a SessionNotCreatedException. Tuning this timeout and matching thread count to slot count prevents long waits and timeouts in CI.

How does Selenium Grid decide which node gets a new session?

Show answer

Each node advertises slots with stereotypes, which are capability sets like browserName chrome, a browser version and platformName linux. The distributor compares the requested capabilities with those stereotypes and picks a free matching slot, preferring nodes with lower load. If nothing matches, the request stays in the queue until timeout. Custom capabilities in the stereotype let you route tests to special nodes, such as nodes with a particular extension.

What is Dynamic Grid in Selenium 4?

Show answer

In Dynamic Grid the node does not run browsers itself. It is configured with a Docker socket and a list of browser images, and for each new session it starts a fresh browser container, runs the test and then removes the container. Every test gets a clean browser, and capacity grows without pre-starting many nodes. It can also record video per session. The trade-off is container start-up time for each session.

How does running Selenium Grid on Kubernetes with autoscaling differ from Docker Compose?

Show answer

Docker Compose runs a fixed number of nodes on one machine, which is simple but limited to that host's CPU and memory. On Kubernetes, typically installed with the Selenium Helm chart, nodes are pods spread across a cluster, and KEDA can scale node pods up based on the queue length and back to zero when idle. It suits large teams with bursty CI demand but needs cluster knowledge and monitoring.

Your UI tests run noticeably slower on the Grid than on your laptop. How do you find the cause?

Show answer

Every command now travels over the network, so chatty code with many findElement calls or getText in loops suffers most. Check latency between test runner and Grid, ideally keeping them in the same network or region. Check node resources: overloaded CPU or many sessions per container slow browsers. Video recording and VNC add overhead. Reduce round trips by batching checks, using JavaScript for bulk reads and avoiding implicit waits combined with explicit ones.

You need 2,000 UI tests to run on every pull request in under 20 minutes. How would you design the infrastructure?

Show answer

First shrink the UI suite by moving checks to API and unit level. Then run the rest on an autoscaling Grid, for example Kubernetes with KEDA, sized so tests divided by slots times average duration fits the budget. Make tests independent with API-based setup and ThreadLocal drivers. Shard tests across CI agents, fail fast on smoke tests first, and track flaky tests separately. Pin image versions and monitor queue time.

How do you test an API that supports multiple versions at the same time?

Show answer

First learn how versions are selected: URL path like /v1 and /v2, a header, or a query parameter. Keep a request specification per version and run the shared contract tests against each supported version, plus version-specific tests for new fields or changed behaviour. Check that old versions still return their original response shape so existing clients do not break, and that deprecated versions send the agreed deprecation headers or errors.

Your pipeline runs the full test suite on every commit, even for documentation-only changes. How do you optimise it?

Show answer

Use path filters so commits that touch only docs or markdown skip the test stages. For code changes, map modules to their test suites and run only the affected ones on pull requests, with the full regression on merge to main or nightly. Cache Maven dependencies and Docker layers. Keep a fast smoke stage first so failures appear early. Track pipeline time as a metric so the optimisation does not quietly regress.

How are throughput, response time and concurrent users related?

Show answer

Little's Law links them: concurrent users equal throughput multiplied by the time each user spends per request, which is response time plus think time. For example, 50 requests per second with 2 seconds response and 8 seconds think time needs about 500 users. If response time rises while users stay fixed, throughput falls. This helps you size thread counts from a target throughput and spot when the system has saturated.

How do you set up distributed load testing in JMeter?

Show answer

One controller machine coordinates several worker machines that generate the load. On each worker start jmeter-server with the same JMeter and Java versions, place CSV files locally and open the RMI ports in the firewall. On the controller list the workers in remote_hosts in jmeter.properties, or pass -R host1,host2, and run in non-GUI mode with -r or -R. Remember that thread counts multiply per worker, so 100 threads on 4 workers gives 400 users.

What is the difference between open and closed workload models in load testing?

Show answer

In a closed model a fixed number of users loop through the scenario, so when the server slows down, users wait and the arrival rate drops, which can hide problems. In an open model new requests arrive at a set rate regardless of response time, like real internet traffic, so a slow server builds a queue. JMeter thread groups are closed by default; the Open Model Thread Group or throughput timers approximate open workloads.

Write a JSR223 Groovy PostProcessor that reads a JSON response and stores a value as a JMeter variable.

Show answer

// JSR223 PostProcessor, language: groovy import groovy.json.JsonSlurper def body = prev.getResponseDataAsString() def json = new JsonSlurper().parseText(body) if (json?.orderId) { vars.put('orderId', json.orderId.toString()) } else { prev.setSuccessful(false) prev.setResponseMessage('orderId missing in response') }

How do you run JMeter tests in a Jenkins pipeline and fail the build when an SLA is breached?

Show answer

Run JMeter in non-GUI mode in a pipeline stage, ideally inside a Docker image with a pinned JMeter version, and archive the JTL and HTML report. Enforce SLAs with Duration Assertions or a JSR223 check, or parse the JTL in a script that compares 95th percentile and error rate with thresholds and exits non-zero. The Jenkins Performance plugin can also mark builds unstable or failed based on thresholds and trends.

Response times are fine with 50 users, but at 300 users the error rate jumps to 20 percent. How do you analyse it?

Show answer

Look at which requests fail and their error codes: 503 or 504 suggest overloaded servers or gateways, connection refused points to exhausted connection pools or ports. Find the user count where errors start by running a stepped ramp-up. Correlate with server metrics such as CPU, thread pools, database connections and GC. Confirm the load generator itself was not overloaded. Report the saturation point and the bottleneck with evidence, not just the error percentage.

JMeter runs out of memory during a large load test. What do you do?

Show answer

First remove or disable GUI listeners like View Results Tree and run in non-GUI mode. Save results in CSV format with only needed fields instead of XML with response data. Increase heap through the HEAP environment variable, such as -Xms2g -Xmx4g, within the machine's RAM. Use Groovy instead of BeanShell, avoid huge regex extractions, and if one machine still cannot generate the load, distribute it across several workers.

Your load test showed good results, but production slowed down at the same traffic. What could explain the difference?

Show answer

The test may not have matched reality. Common gaps are unrealistic think time or user journeys, repeated test data giving high cache hits, a smaller database than production, missing third-party calls that were stubbed, a test environment with different hardware or configuration, and load generated from one region. Review production logs and analytics to rebuild the workload model, use production-like data volumes and repeat the test.

A stakeholder asks whether the system can handle 10,000 users. How do you plan the performance test?

Show answer

First clarify what 10,000 users means: concurrent sessions, users per hour or requests per second, and which journeys they perform. Use production analytics to build the workload mix and think times, then agree on SLAs like 95th percentile under two seconds and errors under one percent. Prepare a production-like environment and data, plan baseline, load, stress and soak runs, set up monitoring, and share a report with clear limits and bottlenecks.

Response time grows slowly over a six-hour soak test while load stays constant. What does it suggest?

Show answer

A steady rise under constant load usually indicates a resource leak, such as memory not being freed, growing caches, unclosed database connections or threads, or log and temp files filling disk. Check heap usage and garbage collection trends, connection pool counts and open file handles over time. A heap dump comparison can show which objects grow. Share the evidence with developers, since such issues rarely appear in short tests.

What is the difference between ThreadLocal and a synchronized block in Java?