Why Every Cloud Engineer Should Learn TensorFlow in 2026

import tensorflow as tf

(x_train, y_train), _ = tf.keras.datasets.mnist.load_data()
x_train = x_train / 255.0

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(28, 28)),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation="relu"),
    tf.keras.layers.Dense(10, activation="softmax"),
])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"])
model.fit(x_train, y_train, epochs=2, batch_size=128)

# Export a SavedModel for TensorFlow Serving (version folder "1")
model.export("models/digits/1")

# Inspect what the exported model expects and returns
saved_model_cli show --dir models/digits/1 --tag_set serve \
  --signature_def serving_default
ls -R models/digits/1

docker run --rm -p 8501:8501 \
  -v "$PWD/models/digits:/models/digits" \
  -e MODEL_NAME=digits \
  tensorflow/serving
# Model status
curl localhost:8501/v1/models/digits

import json, numpy as np, requests

image = np.zeros((28, 28)).tolist()       # replace with a real image
payload = {"instances": [image]}
r = requests.post("http://localhost:8501/v1/models/digits:predict",
                  data=json.dumps(payload), timeout=10)
print(r.status_code, np.argmax(r.json()["predictions"][0]))

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: digits-serving
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: digits-serving
  minReplicas: 1
  maxReplicas: 5
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: { type: Utilization, averageUtilization: 65 }

ML cost guard checklist
[ ] Every ML resource tagged with team and project
[ ] Budgets and alerts per project
[ ] Training instances stop when jobs finish
[ ] Notebook instances have idle shutdown
[ ] Serving starts on CPU; GPU only if latency needs it

apiVersion: apps/v1
kind: Deployment
metadata:
  name: digits-serving
spec:
  replicas: 1
  selector:
    matchLabels: { app: digits-serving }
  template:
    metadata:
      labels: { app: digits-serving }
    spec:
      containers:
        - name: tfserving
          image: tensorflow/serving
          env:
            - { name: MODEL_NAME, value: digits }
          ports: [{ containerPort: 8501 }]
          readinessProbe:
            httpGet: { path: /v1/models/digits, port: 8501 }
          resources:
            requests: { cpu: "500m", memory: "1Gi" }
          volumeMounts:
            - { name: models, mountPath: /models/digits }
      volumes:
        - name: models
          persistentVolumeClaim: { claimName: digits-models }

apiVersion: v1
kind: Service
metadata:
  name: digits-serving
spec:
  selector: { app: digits-serving }
  ports:
    - { name: rest, port: 8501, targetPort: 8501 }

Runbook: new model version
1. Data science team uploads models/digits/2 to storage
2. Sync to the volume (or object storage path)
3. TF Serving loads version 2 automatically
4. Check /v1/models/digits shows version 2 AVAILABLE
5. Monitor latency and errors for 30 minutes
6. Rollback: remove version 2 folder, version 1 serves again