Small systems / careful observations
A compact notebook for infrastructure health, model routing and the work between deployments.
Illustrative lab: all hosts and measurements on this site are fictional, synthetic demo data — not live telemetry.
Inference gateway
Anthropic · Claude
Routes short interactive requests. A compact worker pool keeps queue latency predictable; long-running jobs go elsewhere.
Batch processing
DeepSeek
Processes overnight classification and summarization batches. Jobs checkpoint between chunks rather than holding one large request open.
Evaluation sandbox
GLM
Runs multilingual evaluation suites against pinned prompts. Results are reviewed before any routing policy changes.
Routing / October 10, 2026
Separate provider selection, retry policy and queue ownership before drawing conclusions from one fast answer.
Observability / October 7, 2026
A running process can still serve a slow or unusable endpoint. Keep the probes close to what the user actually needs.
Maintenance / October 3, 2026
Capacity planning has to include retries, log rotation and the next deployment, not just the happy-path request.