hdfs dfs -mkdir -p /user/student/sales
hdfs dfs -put sales_2026.csv /user/student/sales/
hdfs dfs -ls /user/student/sales
hdfs dfs -du -h /user/student/sales      # size on HDFS
hdfs dfs -cat /user/student/sales/sales_2026.csv | head -5

CREATE EXTERNAL TABLE sales (
  order_id   STRING,
  city       STRING,
  amount     DOUBLE,
  order_date DATE
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION '/user/student/sales'
TBLPROPERTIES ('skip.header.line.count'='1');

SELECT city, SUM(amount) AS revenue
FROM sales
GROUP BY city
ORDER BY revenue DESC;

Before joining, can I...
☐ Navigate folders and run commands in a Linux terminal?
☐ Write SELECT queries with WHERE, GROUP BY and JOIN?
☐ Write a Python function and read a CSV file?
☐ Spend 8-10 hours a week on practice?
If any box is empty, ask about a foundations module first.

-- SQL drill: top 3 cities by revenue
SELECT city, SUM(amount) AS revenue
FROM sales
GROUP BY city
ORDER BY revenue DESC
LIMIT 3;

# Python drill: count rows per city
import csv
from collections import Counter
with open("sales_2026.csv") as f:
    print(Counter(row["city"] for row in csv.DictReader(f)))

from pyspark.sql import SparkSession, functions as F

spark = SparkSession.builder.appName("sales-batch").getOrCreate()

sales = spark.read.csv("hdfs:///user/student/sales/", header=True, inferSchema=True)
daily = (sales
    .withColumn("order_date", F.to_date("order_date"))
    .groupBy("order_date", "city")
    .agg(F.sum("amount").alias("revenue"), F.count("*").alias("orders")))

(daily.write.mode("overwrite")
      .partitionBy("order_date")
      .parquet("hdfs:///user/student/output/daily_sales"))
spark.stop()

# Create a topic and send a few test events (Kafka 3.x CLI)
kafka-topics.sh --create --topic orders --bootstrap-server localhost:9092 \
  --partitions 3 --replication-factor 1

kafka-console-producer.sh --topic orders --bootstrap-server localhost:9092
> {"order_id": "A1", "city": "Gurugram", "amount": 1200}

kafka-console-consumer.sh --topic orders --from-beginning \
  --bootstrap-server localhost:9092

Questions to ask at the Gurugram campus
☐ Which modules are in the current big data syllabus?
☐ How many hours are hands-on lab time?
☐ Where do labs run — shared cluster, Docker or cloud?
☐ What projects will I complete and keep?
☐ Batch timings: weekday / weekend options?
☐ Current fees and payment options?
☐ What placement support is offered, and how does it work?

Module                 Covered?  Hours  Hands-on?
Linux + SQL foundations ____     ____   ____
HDFS / cloud storage    ____     ____   ____
Hive / Spark SQL        ____     ____   ____
PySpark                 ____     ____   ____
Kafka / streaming       ____     ____   ____
Airflow / scheduling    ____     ____   ____
Final project           ____     ____   ____

My goal: ____
My current skills: ____
Course fit (1-5): ____
Schedule fit (1-5): ____
Project quality (1-5): ____
Open questions: ____
Decision and date: ____