MLOps & the Model Lifecycle

Shipping an AI feature is not the finish line. Models, prompts, data, retrieval indexes, and user behavior all change. MLOps is the discipline that keeps those changes reproducible, observable, and reversible.

In applied AI engineering, the “model” in production is rarely just weights. It is a bundle: model ID, prompt version, retrieval index, safety checks, eval set, serving config, and rollout policy.

The mental model

Every production answer should be traceable to a release manifest.

release = {
    "model": "support-router-v3",
    "base_model": "gpt-4o-mini-2026-06-01",
    "prompt": "support_router@2026-07-10",
    "retrieval_index": "help_docs@sha256:91b2",
    "eval_set": "support_golden@42",
    "guardrails": "default@7",
}

If a user reports a bad answer, you should be able to reconstruct what code, prompt, model, data, and index produced it. Without that, debugging becomes folklore.

Version everything that changes behavior

Versioning applies beyond code:

• Data — raw sources, labels, cleaning scripts, train/validation/test splits.
• Models — base model, fine-tuned adapter, quantization, serving parameters.
• Prompts — template text, few-shot examples, output schema, tool list.
• Indexes — embedding model, chunking config, source document versions.
• Evals — datasets, rubrics, judge prompts, thresholds.

A prompt change can be as behavior-changing as a code change. Treat it that way.

Reproducibility

Reproducibility means a past result can be explained and, when necessary, rerun. For AI systems, capture:

• request ID and timestamp,
• model and prompt versions,
• sampling parameters,
• retrieved chunk IDs and scores,
• tool calls and results,
• output and validation outcomes,
• user-visible decision or action.

You may not get bit-for-bit identical model output from a hosted API, but you can still make the system auditable.

CI for ML and prompts

CI should run more than unit tests. For AI changes, add:

• Schema tests — prompts still produce valid structured output.
• Golden-set evals — quality does not regress on representative cases.
• Slice checks — rare classes, long documents, adversarial inputs, languages.
• Cost/latency checks — token budgets and expected model calls stay bounded.
• Retrieval checks — known questions retrieve known supporting chunks.

The goal is not perfect prediction. The goal is catching regressions before users become the test suite.

Safe rollout

Ship model changes gradually:

1. Shadow — run the new model on real traffic without showing its output.
2. Canary — send a small percentage of users to the new path.
3. A/B test — compare business and quality metrics against control.
4. Ramp — increase traffic only while guardrail metrics stay healthy.
5. Rollback — return to the previous manifest quickly if metrics degrade.

Rollouts should be controlled by configuration, not emergency code patches.

Monitoring drift and quality

AI systems degrade for reasons ordinary services do not: data distribution shifts, new user intents, stale documents, new prompt-injection patterns, or upstream model changes.

Monitor both system and quality signals:

• latency, errors, retries, cost per request,
• token counts and context truncation rate,
• retrieval empty-result and low-score rates,
• judge or rubric scores on sampled traffic,
• user feedback, escalation rate, task completion,
• slice-level metrics for important customer groups.

Pattern recognition

Variations

Worked problems

rollout_steps = [0.0, 0.02, 0.10, 0.25, 0.50, 1.0]

def should_rollback(metrics):
    return (
        metrics["unsupported_claim_rate"] > 0.03
        or metrics["p95_latency_ms"] > 4_000
        or metrics["escalation_rate_delta"] > 0.02
    )

def shares(counts):
    total = sum(counts.values())
    return {k: v / total for k, v in counts.items()} if total else {}

def drift_alert(baseline_counts, current_counts, threshold=0.10):
    base = shares(baseline_counts)
    cur = shares(current_counts)
    alerts = []
    for label in sorted(set(base) | set(cur)):
        delta = cur.get(label, 0.0) - base.get(label, 0.0)
        if abs(delta) >= threshold:
            alerts.append((label, base.get(label, 0.0), cur.get(label, 0.0), delta))
    return alerts

baseline = {"billing": 200, "technical": 500, "account": 300}
current = {"billing": 380, "technical": 420, "account": 200}
print(drift_alert(baseline, current))

from dataclasses import dataclass
from datetime import datetime

@dataclass(frozen=True)
class PromptVersion:
    name: str
    version: str
    template_sha: str
    status: str  # draft, shadow, canary, production, archived
    created_by: str

@dataclass(frozen=True)
class InferenceLog:
    request_id: str
    prompt_name: str
    prompt_version: str
    model_id: str
    input_tokens: int
    output_tokens: int
    created_at: datetime

The classic pitfall