Data Wrangling with dplyr and tidyr Interview Questions for R Programming

The tidyverse is a collection of R packages that share a common design and grammar for data science. Core packages include readr for importing, dplyr for transformation, tidyr for reshaping, ggplot2 for visualization, purrr for functional programming, stringr for text, forcats for factors and lubridate for dates. library(tidyverse) loads the core set. Its consistent function names and the pipe make analysis code readable, which is why most R analytics teams use it.

dplyr has a small set of verbs for data transformation: filter() keeps rows that meet conditions, select() picks columns, mutate() creates or changes columns, arrange() sorts rows, summarise() reduces groups to summary rows and group_by() defines groups for the next operations. Others include distinct(), rename(), relocate(), count() and slice(). Each takes a data frame as the first argument and returns a data frame, so verbs chain naturally with the pipe.

Tidy data follows three rules: each variable is a column, each observation is a row and each type of observational unit forms a table. For example, a sales sheet with separate columns for Jan, Feb and Mar is not tidy, because month is a variable spread across columns. Reshaping it into month and sales columns makes it tidy. Tidy data works directly with dplyr, ggplot2 and modelling functions, reducing reshaping effort later.

I load both datasets and standardise the key, for example trimming spaces and matching data types for transaction_id. Then anti_join(finance, report, by = 'transaction_id') lists transactions in finance but not in my report, and the reverse anti_join shows extras. I examine the missing rows for patterns such as a date range, status or filter in my code. Usually the cause is a timezone cut-off or an excluded status.

mutate() adds or modifies columns and keeps the same number of rows, so each row gets a value. summarise() collapses each group into one row of summary values. For example, group_by(city) %>% mutate(city_avg = mean(sales)) adds the city average to every row, useful for comparing each row with its group, while group_by(city) %>% summarise(city_avg = mean(sales)) returns one row per city.

%>% comes from the magrittr package and |> is the native pipe built into R since version 4.1. Both pass the left side as the first argument of the right side, so df |> filter(x > 1) equals filter(df, x > 1). The native pipe is faster and needs no package, but it uses _ as a named placeholder only, while %>% supports the dot placeholder anywhere. New code increasingly uses |>, and both work with dplyr.

left_join() keeps all rows of the left table and adds matching columns from the right, inner_join() keeps only matching rows, right_join() keeps all rows of the right table and full_join() keeps all rows from both, filling NA where there is no match. Keys are given with by = join_by(customer_id) or by = c('id' = 'cust_id') for different names. Left joins are most common when enriching a fact table with lookup details.

Window functions return one value per row while looking at other rows in the group. Common ones are lag() and lead() for previous or next values, row_number(), min_rank() and dense_rank() for ranking, cumsum() and cummean() for running totals, and ntile() for buckets. For example, group_by(product) %>% arrange(month) %>% mutate(prev = lag(sales)) gives the previous month's sales per product, which is the basis for growth calculations.

I first aggregate to one row per product and month with group_by() and summarise(). Then I group by product, arrange by month and use mutate(growth = (sales - lag(sales)) / lag(sales) * 100). The first month for each product becomes NA, which is correct. I make sure months with zero sales exist using tidyr::complete(), otherwise lag() compares non-consecutive months. Finally I round and format growth for reporting.

I group by customer_id and keep the row with the latest updated_at using slice_max(updated_at, n = 1, with_ties = FALSE), then ungroup. An alternative is arrange(desc(updated_at)) followed by distinct(customer_id, .keep_all = TRUE). I check that updated_at is a proper date-time and has no NAs, since missing timestamps would be dropped or mis-sorted. Finally I confirm that n_distinct(customer_id) equals the number of rows.

First I investigate whether they are data errors, such as an extra zero, or genuine high earners. I flag outliers with the IQR rule, values beyond Q1 - 1.5 IQR or Q3 + 1.5 IQR, or with percentiles. Errors I fix or remove with documentation. Genuine values I keep, but I report the median alongside the mean, or winsorize at the 1st and 99th percentiles for modelling. The choice depends on the business question.

I group by the relevant category, such as product category, and replace NAs with that group's median: group_by(category) %>% mutate(price = coalesce(price, median(price, na.rm = TRUE))) %>% ungroup(). Group medians are more realistic than a global value. I also add a flag column marking imputed rows so models and reviewers can see them. If a whole group is missing, I fall back to the overall median, and for important features I consider model-based imputation.

They are filtering joins: they keep rows of the left table based on matches in the right table but add no columns. semi_join() keeps rows that have a match, and anti_join() keeps rows that have no match. For example, anti_join(customers, orders, by = 'customer_id') lists customers who never ordered. They are also safer than inner joins for filtering, since they never duplicate rows when the right table has repeated keys.

A grouped data frame stays grouped after mutate() and some other verbs, so later operations silently run per group. For example, a mutate(share = sales / sum(sales)) meant to compute share of grand total would compute share within each group instead. Calling ungroup() after grouped work, or using .groups = 'drop' in summarise() or the .by argument for one-off grouping, prevents these subtle bugs in long pipelines.

Situation: I received three years of vendor invoices from different branches in Excel files with inconsistent headers, merged cells and dates in four formats. Task: build one clean table for spend analysis. Action: I used readxl to read all sheets, janitor::clean_names() to standardise headers, lubridate to parse dates and a mapping table for vendor name variants. I wrote validation checks for totals. Result: a clean dataset of 180,000 rows that finance now refreshes monthly.

I can keep dplyr syntax but change the backend. dtplyr translates dplyr code to data.table, which is much faster in memory. duckdb with dbplyr runs the pipeline in an embedded columnar database, often handling data larger than RAM. arrow lets me query Parquet files with dplyr verbs lazily. I would also filter and select early, avoid row-wise operations and profile the pipeline to find the slow step before rewriting anything.

dbplyr is a dplyr backend that translates dplyr verbs into SQL and runs them inside a database. You connect with DBI, reference a table with tbl(con, 'orders') and write normal dplyr code. Queries are lazy: nothing runs until you call collect(), and show_query() prints the generated SQL. This lets analysts work with large warehouse tables without pulling all rows into R, though some R functions have no SQL translation.

dplyr uses data masking, so you can write column names without quotes. Inside your own function, a column passed as an argument is not found automatically because R looks for a variable with that name. Embracing the argument with {{ }} tells dplyr to use the column the caller supplied. For column names stored as strings, you use .data[[var]]. For new column names, use the glue syntax '{{var}}_mean' := with the walrus operator.

Situation: I shared a retention analysis showing improvement, but a week later I noticed I had excluded customers with missing signup dates, who were mostly churned users. Task: correct it before decisions were made. Action: I informed my manager immediately, reran the analysis with proper handling, explained the impact clearly and added a missing-value check to my template. Result: the corrected numbers were used, and my manager appreciated the transparency.