Building an internal agent platform with Kubernetes primitives

In this talk, Nihal explores how his team built an internal platform to deploy, run, and scale AI agent services using standard Kubernetes primitives. The platform was designed to handle high-variance workloads through predictable infrastructure patterns.

The session demonstrates a lightweight Kubernetes platform with standardized service templates, autoscaling driven by request volume and queue depth, and built-in observability. Nihal shares lessons from operating this platform in production and how it evolved to improve reliability.

Key takeaways:
• Designing lightweight internal platforms
• Kubernetes primitives that mattered most in practice
• Tuning autoscaling and rollouts for latency-sensitive services
• Adapting the platform based on real production failures