MongoDB vs Old Tools What Changed for Data Engineers

-- Relational model

CREATE TABLE users (
  id SERIAL PRIMARY KEY,
  name TEXT NOT NULL,
  email TEXT UNIQUE
);

CREATE TABLE addresses (
  id SERIAL PRIMARY KEY,
  user_id INT REFERENCES users(id),
  city TEXT,
  country TEXT
);

-- Fetching a user requires a JOIN
SELECT u.name, a.city
FROM users u
LEFT JOIN addresses a ON a.user_id = u.id;

// Document model

{
  "_id": ObjectId("..."),
  "name": "Aarav",
  "email": "aarav@example.com",
  "addresses": [
    { "city": "Noida", "country": "India" },
    { "city": "Delhi", "country": "India" }
  ],
  "preferences": {
    "newsletter": true,
    "theme": "dark"
  }
}

// Fetching a user is a single document read
db.users.findOne({ email: "aarav@example.com" })

// No joins needed for embedded data

// Sharding a collection in MongoDB

sh.enableSharding("appdb")

sh.shardCollection(
  "appdb.orders",
  { "customer_id": "hashed" }
)

// Result:
// Documents are distributed across shards
// based on the shard key

// Choose a shard key carefully:
// - High cardinality
// - Even distribution
// - Query-aligned

When sharding is actually needed:

- Data is growing faster than one server can handle
- Write throughput is high
- You need regional distribution
- You have an application that grows fast

When it is NOT needed:

- Small or medium datasets
- Single region
- Stable traffic
- Early-stage product

Start simple.
Add sharding when you have to,
not because it sounds modern.

-- Top 5 customers by total order value

SELECT
  c.name,
  SUM(o.amount) AS total
FROM customers c
JOIN orders o ON o.customer_id = c.id
WHERE o.created_at >= CURRENT_DATE - INTERVAL '30 days'
GROUP BY c.name
ORDER BY total DESC
LIMIT 5;

// Same query in MongoDB

db.orders.aggregate([
  {
    $match: {
      created_at: {
        $gte: new Date(Date.now() - 30 * 24 * 3600 * 1000)
      }
    }
  },
  {
    $group: {
      _id: "$customer_id",
      total: { $sum: "$amount" }
    }
  },
  { $sort: { total: -1 } },
  { $limit: 5 },
  {
    $lookup: {
      from: "customers",
      localField: "_id",
      foreignField: "_id",
      as: "customer"
    }
  },
  { $unwind: "$customer" }
])

Aggregation pipeline mental model:

Stage 1: $match    → filter documents
Stage 2: $group    → aggregate
Stage 3: $sort     → order results
Stage 4: $limit    → trim
Stage 5: $lookup   → join

Each stage is a transformation.
Pipeline reads top to bottom.
Like SQL but written as data, not text.

This is powerful for data engineers
who need to build complex pipelines.

// MongoDB index example

db.orders.createIndex({ customer_id: 1 })
db.orders.createIndex({ created_at: -1 })

// Without indexes, queries scan entire collections
// Just like SQL without indexes

// Check query plan
db.orders.find({ customer_id: 42 }).explain("executionStats")

// Look for:
// - "stage": "IXSCAN" → uses index
// - "stage": "COLLSCAN" → full scan, slow

Data quality is still your job:

- Consistent field names
- Consistent data types
- No orphan references
- No duplicated data without reason

MongoDB's flexible schema means
the database will not stop you
from writing bad data.

That is the trade-off.
You gain flexibility.
You lose automatic enforcement.

If you need strict validation,
use MongoDB schema validation
or keep a relational tool.

Ask before you choose a database:

□ Does my data have a stable, predictable shape?
  Yes → relational
  No  → document or flexible

□ Do I need strong transactions?
  Yes → relational
  No  → either

□ Will my data grow beyond one server?
  Maybe → MongoDB or sharded SQL
  No    → single-node SQL is fine

□ Do I have a team comfortable with SQL?
  Yes → SQL is safer
  No  → MongoDB may be easier

□ Do I need complex reporting and joins?
  Yes → SQL or columnar
  No  → MongoDB

Do not choose a tool because it is popular.
Choose it because it fits the workload.

A realistic decision flow:

1. Start with PostgreSQL.
   It handles 90% of applications well.

2. Move to MongoDB when:
   - Data has nested / variable shape
   - You need flexible document storage
   - You want built-in horizontal scaling

3. Move to a warehouse (BigQuery, Snowflake)
   when analytics becomes the primary workload.

Most data engineers end up using:
- Relational for transactional systems
- MongoDB or a document DB for flexible storage
- A warehouse for analytics

Not one tool — a toolbox.