Recorded on
with Aishwarya Srinivasan, Arvind Narayanamurthy
Open models are getting dramatically larger and more capable, which makes serving them efficiently a systems problem of its own. This session explores modern inference techniques used to make state-of-the-art open models practical in production, covering caching, kernels, quantization, speculative decoding, and disaggregated inference, with lessons from the recent Kimi K3 launch and its production-scale vLLM integration.
Learn the techniques that improve latency, throughput, memory efficiency, and cost when serving large open models.
Explore caching, quantization, optimized kernels, disaggregated serving, & speculative decoding for production workloads
See how recent inference optimizations helped bring the 2.8T-parameter Kimi K3 model into production efficiently.
Open models are getting dramatically larger and more capable, which makes serving them efficiently a systems problem of its own. This session explores modern inference techniques used to make state-of-the-art open models practical in production, covering caching, kernels, quantization, speculative decoding, and disaggregated inference, with lessons from the recent Kimi K3 launch and its production-scale vLLM integration.
Developer Relations @ Nebius · Nebius
Sujee Maniyam is an AI Developer Advocate at Nebius with deep experience across AI, distributed systems, data engineering, and cloud infrastructure. A developer educator, author, and open-source contributor, he regularly speaks and teaches on open models, AI infrastructure, inference optimization, and building production-ready AI systems.
Hosted by Aishwarya Srinivasan and Arvind Narayanamurthy.
AI Advisor / AI Educator
Aishwarya Srinivasan is the Co-Founder of The Gen Academy and a Developer Relations Lead at Nebius. Her experience spans Fireworks AI, Microsoft for Startups, Google Cloud and IBM, where she worked with developers and businesses on practical AI and machine-learning solutions. She holds a postgraduate degree in Data Science from Columbia University and teaches AI engineering, applied machine learning, generative AI and LLMOps.
Founder @ The Gen Academy | AI Solutions Architect | Ex - Adobe, Microsoft, IBM
Arvind brings a cross-company perspective on AI at scale, built across Microsoft, IBM, and Adobe, where he led data science teams and developed production-grade machine-learning solutions. He has built and deployed enterprise AI systems and focuses on translating AI concepts into practical, production-ready solutions. He holds a Master's from Carnegie Mellon University.
