Data Transformation with dplyr

dplyr is the cornerstone of the tidyverse and the standard toolkit for data transformation in modern R. Its consistent "verb" functions and the pipe operator make wrangling data readable and fast.

1. The Pipe Operator

The pipe (|> native, or %>% from magrittr) chains operations left-to-right, avoiding deeply nested function calls:

# Without pipe (hard to read)
summarise(filter(df, age > 25), avg = mean(salary))

# With pipe (reads top to bottom)
df |>
  filter(age > 25) |>
  summarise(avg = mean(salary))

2. select() — Choose Columns

library(dplyr)

df |> select(name, age)          # keep specific columns
df |> select(-age)               # drop a column
df |> select(starts_with("sale")) # pattern-based selection

3. filter() — Choose Rows

df |> filter(age > 30)
df |> filter(age > 30 & department == "Sales")
df |> filter(department %in% c("Sales", "Marketing"))

4. mutate() — Create or Modify Columns

df |> mutate(bonus = salary * 0.10)
df |> mutate(
  seniority = ifelse(age > 40, "Senior", "Junior")
)

5. summarise() — Aggregate Data

df |>
  summarise(
    avg_salary = mean(salary),
    max_age = max(age),
    total = n()
  )

6. group_by() — Combine with summarise() for Grouped Stats

This is where dplyr becomes truly powerful — the split-apply-combine pattern:

df |>
  group_by(department) |>
  summarise(
    avg_salary = mean(salary),
    headcount = n()
  ) |>
  arrange(desc(avg_salary))

7. Other Essential Verbs

df |> arrange(desc(salary))
df |> rename(full_name = name)
df |> distinct(department)

employees |> left_join(departments, by = "dept_id")

8. A Full Pipeline Example

df |>
  filter(age >= 25) |>
  mutate(bonus = salary * 0.10) |>
  group_by(department) |>
  summarise(avg_total_pay = mean(salary + bonus)) |>
  arrange(desc(avg_total_pay))

9. Production-Ready Checklist

• ✅ Learn the core verbs — select, filter, mutate, summarise, arrange.
• ✅ Master group_by() + summarise() — the split-apply-combine workhorse.
• ✅ Use the pipe operator — for clean, readable multi-step transformations.
• ✅ Know your join functions — for combining multiple tables.