We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic.
By clicking "I accept", you consent to our use of cookies.

Serving LLMs on Kubernetes

Riproduci il video

Evento:
AI Conf 2025
Lingua:
Inglese

Tag

  • AI Generativa
  • Artificial Intelligence
  • Container
  • Kubernetes
  • LLM
  • MLOps
  • Platform Engineering

Speaker

Serving LLMs on Kubernetes

What are the key hurdles in running Large Language Models (LLMs) efficiently on Kubernetes? This session is crafted for MLOps and Platform Engineers seeking effective strategies for LLM integration. It will provide an overview of the current landscape for LLM deployment options, discussing the suitability of Kubernetes for these models.

The talk will dissect the complexities associated with the size, tuning, and scaling of LLMs, and explore technologies such as KServe, vLLM, KubeFlow Model Registry, and Ray