Why DevOps Engineers Must Learn TensorFlow

# Export a SavedModel — the artefact a DevOps engineer actually handles.
# This is what a data scientist hands over after training.

import tensorflow as tf

# Load (or define) a model
model = tf.keras.applications.MobileNetV2(weights="imagenet")

# Export as SavedModel — the production format
model.save(
    filepath="saved_models/mobilenet/1",   # /1 is the model version
    save_format="tf",                       # tf = SavedModel
    include_optimizer=False,                # smaller artefact
)

# The resulting directory:
#   saved_models/mobilenet/1/
#     saved_model.pb           ← computation graph
#     variables/
#       variables.data-00000-of-00001
#       variables.index
#     assets/

# What a DevOps engineer does with this:
# - Containerise it (see next tab)
# - Version it in a registry (like Docker images)
# - Deploy it via TF Serving
# - Monitor the served predictions

# The version number (/1) is important: TF Serving and Kubernetes
# use it to do rolling updates and rollbacks, exactly like a
# container image tag.
#
# What this proves in an interview:
# - You understand that a model is a versioned artefact
# - You know the difference between "trained" and "served"
# - You can point to the specific directory a deploy pipeline reads

# Serve with TensorFlow Serving — the production server.
# This is the exact command a DevOps engineer ships.

# Run TF Serving in Docker, mounting the model directory:
docker run --rm -p 8501:8501 \
  -v "$PWD/saved_models/mobilenet:/models/mobilenet" \
  -e MODEL_NAME=mobilenet \
  tensorflow/serving:2.15.0

# What just happened:
# - TF Serving loaded /models/mobilenet/
# - It picked the highest-numbered version (1 in this case)
# - It exposed two endpoints:
#     http://localhost:8501/v1/models/mobilenet     (REST)
#     grpc://localhost:8500                         (gRPC)
#
# Predict via REST:
curl -X POST http://localhost:8501/v1/models/mobilenet:predict \
  -d '{"instances": [[...]]}'

# Serve a new version — just mount a new numbered directory:
#   saved_models/mobilenet/2/
# TF Serving will hot-reload it automatically. No restart needed.
#
# Rollback — mount the previous version and restart:
docker run --rm -p 8501:8501 \
  -v "$PWD/saved_models/mobilenet/1:/models/mobilenet/1" \
  -e MODEL_NAME=mobilenet \
  tensorflow/serving:2.15.0

# What this proves in an interview:
# - You know that "deploying a model" means "running TF Serving"
# - You understand model versioning at the filesystem level
# - You can explain how a rollback works without retraining

# Kubernetes Deployment for TensorFlow Serving
# A production-shaped ML deployment — health checks, resources, model version.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mobilenet-serving
spec:
  replicas: 2
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: mobilenet-serving
  template:
    metadata:
      labels:
        app: mobilenet-serving
    spec:
      containers:
        - name: tf-serving
          image: tensorflow/serving:2.15.0
          ports:
            - containerPort: 8501  # REST
            - containerPort: 8500  # gRPC
          env:
            - name: MODEL_NAME
              value: mobilenet
          resources:
            requests:
              cpu: "500m"
              memory: "1Gi"
            limits:
              cpu: "2000m"
              memory: "2Gi"
          # TF Serving exposes a status endpoint on :8501
          readinessProbe:
            httpGet:
              path: /v1/models/mobilenet
              port: 8501
            initialDelaySeconds: 20
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /v1/models/mobilenet
              port: 8501
            initialDelaySeconds: 60
            periodSeconds: 10
          volumeMounts:
            - name: model-volume
              mountPath: /models/mobilenet
      volumes:
        - name: model-volume
          persistentVolumeClaim:
            claimName: mobilenet-models

# What a DevOps engineer adds beyond the raw command:
# - Zero-downtime rolling updates (maxUnavailable: 0)
# - Health checks so broken pods don't serve traffic
# - Resource limits so one model doesn't starve the cluster
# - A PVC so the model version is managed outside the image
#
# This is exactly the type of work a "DevOps engineer who knows
# TensorFlow" does — it is infrastructure, not model training.