Regression Models Interview Questions for Machine Learning
25 Regression Models questions with answers for machine learning roles — 6 easy, 14 medium, 5 hard.
About Regression Models
Regression predicts continuous values such as price, demand or delivery time. Learn linear regression assumptions, OLS, interpretation of coefficients, R-squared, MAE, RMSE, Ridge, Lasso and Elastic Net, polynomial features, residual diagnostics, quantile regression and the basics of time series forecasting with lag features, all with scikit-learn and statsmodels examples.
Sheet progressStart practicing
3 QUESTIONS FREESign up to unlock the full question bank and track your progress.Unlock every question and answer, then continue practising company-wise and package-wise.
Linear regression assumes a linear relationship between features and target, independent errors, constant error variance (homoscedasticity), normally distributed errors for valid inference, and little multicollinearity among features. Violations mainly hurt confidence intervals and coefficient interpretation more than raw predictions. Check them with residual plots, Q-Q plots, the Durbin-Watson test and VIF.
R-squared is the share of variance in the target explained by the model, 1 - SS_res / SS_tot. It never decreases when you add features, even useless ones. Adjusted R-squared penalises extra features: 1 - (1 - R2)(n - 1)/(n - p - 1), so it only rises if a new feature improves fit more than expected by chance. R-squared can be negative on test data if the model is worse than predicting the mean.
MAE is the average absolute error, easy to explain and robust to outliers. MSE averages squared errors and punishes large errors heavily, which suits cases where big misses are costly. RMSE is the square root of MSE, so it is in the same unit as the target. If RMSE is much higher than MAE, a few large errors are present. MAPE is useful for percentage errors but fails near zero.
04
How do you interpret a coefficient in a multiple linear regression?
A coefficient is the expected change in the target for a one-unit increase in that feature, holding other features constant. For example, a coefficient of 2,500 on 'area in square metres' means each extra square metre adds about 2,500 to price. Interpretation breaks down with multicollinearity or when features are scaled, so report units clearly and check stability.
05
Fit a linear regression in scikit-learn and print RMSE and R-squared.
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
import numpy as np
lr = LinearRegression().fit(X_train, y_train)
pred = lr.predict(X_test)
print('RMSE', np.sqrt(mean_squared_error(y_test, pred)))
print('R2', r2_score(y_test, pred))
print(dict(zip(X_train.columns, lr.coef_)))
Newer scikit-learn versions also provide root_mean_squared_error directly.
06
How do you check regression residuals visually in Python?
OLS chooses coefficients that minimise the sum of squared residuals between actual and predicted values. It has a closed-form solution, beta = (X^T X)^-1 X^T y, when X^T X is invertible. For very large or ill-conditioned data, gradient descent or a pseudo-inverse is used instead. Squaring errors penalises large mistakes heavily, which makes OLS sensitive to outliers.
08
What is the difference between Ridge, Lasso and Elastic Net regression?
All add a penalty on coefficient size to reduce overfitting. Ridge uses an L2 penalty (sum of squares), shrinking coefficients toward zero but rarely to zero, and handles multicollinearity well. Lasso uses L1 (sum of absolute values) and can set coefficients exactly to zero, performing feature selection. Elastic Net mixes both, which is useful when there are groups of correlated features.
09
What is polynomial regression and what risk does it carry?
Polynomial regression adds powers and interaction terms of features, such as x squared and x1 times x2, so a linear model can fit curves. It is still linear in its coefficients. The main risk is overfitting at high degrees, plus wild behaviour outside the training range. Use cross-validation to choose the degree, scale features, and combine it with Ridge regularisation.
10
What is heteroscedasticity and why does it matter?
Heteroscedasticity means the variance of residuals changes with the level of the prediction, such as errors growing for expensive houses. Predictions remain unbiased, but standard errors and p-values become unreliable. Detect it with a residuals versus fitted plot or the Breusch-Pagan test. Fixes include log-transforming the target, weighted least squares, or robust standard errors.
11
How do you approach a time series forecasting problem differently from normal regression?
Time series data is ordered and autocorrelated, so random splits leak future information. I use time-based splits or TimeSeriesSplit, create lag and rolling-window features from past values only, and handle trend and seasonality. Baselines like last value or seasonal naive come first. Models range from ARIMA and Prophet to gradient boosting on lag features.
12
Your house price model's residuals grow larger for expensive houses. What do you do?
That is heteroscedasticity, typical for price data that is right-skewed. I would model log(price) instead of price, which turns absolute errors into roughly percentage errors and stabilises variance. Then I would convert predictions back with exp and check residual plots again. Evaluating with RMSLE or MAPE also fits better when relative error matters more than absolute error.
13
A demand forecast is consistently under-predicting during festivals. How would you fix it?
The model lacks festival information. I would add features for festival dates such as Diwali or regional holidays, days-to-festival, and past festival uplift for the same item. If the business cost of stock-outs is higher than over-stock, I would use a quantile loss for a higher percentile. Then validate specifically on past festival periods, not just the average error.
14
Two models have RMSE 120 and 125, but the second has much lower MAE. Which do you choose?
It depends on error cost. The first model makes fewer big errors, while the second is better on typical cases but has some large misses. If large errors are expensive, such as stock-outs of costly items, choose the first. If typical accuracy matters more, choose the second. I would also check error distributions by segment and discuss with stakeholders before deciding.
15
How do you tune the alpha of Ridge regression using cross-validation?
RidgeCV tests many alphas efficiently; scale features first.
from sklearn.linear_model import RidgeCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
import numpy as np
m = make_pipeline(StandardScaler(), RidgeCV(alphas=np.logspace(-3, 3, 30)))
m.fit(X_train, y_train)
print(m[-1].alpha_)
LassoCV and ElasticNetCV work the same way.
16
How do you create polynomial features with regularisation in a pipeline?
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
from sklearn.linear_model import Ridge
from sklearn.pipeline import make_pipeline
model = make_pipeline(
PolynomialFeatures(degree=2, include_bias=False),
StandardScaler(),
Ridge(alpha=1.0)
)
model.fit(X_train, y_train)
Tune degree and alpha together with GridSearchCV to avoid overfitting.
17
Use statsmodels to get p-values for regression coefficients.
import statsmodels.api as sm
Xc = sm.add_constant(X_train)
ols = sm.OLS(y_train, Xc).fit()
print(ols.summary())
The summary shows coefficients, standard errors, p-values, confidence intervals, R-squared and diagnostics. scikit-learn focuses on prediction and does not report p-values, so statsmodels is preferred for inference.
18
Tell me about a time you had to explain a model's error to a business team.
Situation: a sales forecast missed by 18 percent in one month. Task: explain why to the category team. Action: I broke the error down by region and found most came from a sudden promotion not in our data. I showed a simple chart and proposed adding a promotion calendar feed. Result: the team shared promotion plans in advance, and the next quarter's error fell below 8 percent.
19
How do you handle disagreement with a senior colleague about model choice?
I keep it about evidence. In one project a senior preferred a complex model while I proposed regularised regression. I suggested we compare both on the same cross-validation folds and also measure inference time and explainability. The complex model was only 1 percent better, so we chose the simpler one. I made sure to credit his idea of the extra features, which we kept.
20
What is regularisation in general and why does it reduce overfitting?
Regularisation adds a constraint or penalty that discourages overly complex models, such as large weights in linear models, deep trees or too many boosting rounds. It trades a little extra bias for a larger drop in variance, so test error falls. Examples include L1 and L2 penalties, dropout in neural networks, early stopping, pruning and limiting tree depth.
21
Why does L1 regularisation produce sparse models while L2 does not?
The L1 constraint region is a diamond with corners on the axes, so the optimal point often lands on a corner where some coefficients are exactly zero. The L2 region is a circle with no corners, so coefficients shrink smoothly but stay non-zero. In gradient terms, L1 applies a constant push toward zero regardless of size, while L2's push becomes tiny as weights get small.
22
When would you use Huber or quantile regression instead of OLS?
Huber regression uses squared loss for small errors and absolute loss for large ones, so it resists outliers while staying smooth. Quantile regression predicts a chosen percentile, such as the 90th percentile of delivery time, which is useful for setting safe promises or prediction intervals. Gradient boosting libraries support both losses for non-linear versions.
23
Lasso dropped a feature the business considers very important. How do you handle it?
Lasso often keeps only one of a group of correlated features, so the dropped feature may be redundant rather than useless. I would check correlations, try Elastic Net which keeps correlated groups, and compare cross-validated performance with the feature forced in. I would explain the trade-off clearly; if the accuracy difference is small, keeping the feature for trust and interpretability is reasonable.
24
You need prediction intervals, not just point estimates, for delivery time. How?
Options are quantile regression, training separate models for the 10th and 90th percentiles with GradientBoostingRegressor(loss='quantile') or LightGBM; conformal prediction, which wraps any model and gives intervals with guaranteed coverage; or bootstrapping. I would check calibration: about 80 percent of actual values should fall within an 80 percent interval on a holdout set.
25
How do you create lag and rolling features for a sales forecasting model in pandas?
Sort by item and date, then shift so features use only past values.
df = df.sort_values(['item', 'date'])
g = df.groupby('item')['sales']
df['lag_7'] = g.shift(7)
df['roll_mean_28'] = g.transform(lambda s: s.shift(1).rolling(28).mean())
The shift before rolling prevents the current day's sales from leaking into its own feature.
Frequently asked questions
How important is Regression Models in Machine Learning interviews?
Regression predicts continuous values such as price, demand or delivery time. Learn linear regression assumptions, OLS, interpretation of coefficients, R-squared, MAE, RMSE, Ridge, Lasso and Elastic Net, polynomial features, residual diagnostics, quantile regression and the basics of time series forecasting with lag features, all with scikit-learn and statsmodels examples.
How should I use the Regression Models sheet?
Go top to bottom — easy to hard. Answer before opening each answer, tick what you solved and save misses to Last-Minute Revision.