sample-genai-on-eks-starter-kit

A comprehensive toolkit for deploying production-ready Generative AI infrastructure on Amazon EKS. Includes pre-configured components for: 🚀 AI Gateway (LiteLLM) 🤖 LLM Serving (vLLM, SGLang, Ollama) 📊 Vector Databases, 🔍 Embedding Models (TEI) 📈 Observability (Langfuse, Phoenix) etc. Fast-track your GenAI deployment with Kubernetes

aws-samples/sample-genai-on-eks-starter-kit

View on GitHub ↗

A comprehensive toolkit for deploying production-ready Generative AI infrastructure on Amazon EKS. Includes pre-configured components for: 🚀 AI Gateway (LiteLLM) 🤖 LLM Serving (vLLM, SGLang, Ollama) 📊 Vector Databases, 🔍 Embedding Models (TEI) 📈 Observability (Langfuse, Phoenix) etc. Fast-track your GenAI deployment with Kubernetes

sample-genai-on-eks-starter-kit ranks 336th of 397 in Production Serving & Deployment, actively maintained, last commit 2 days ago.

Star trajectory

16 days tracked
2026-11-06102 stars today2026-11-22
Momentum

- in 30d, below the Production Serving & Deployment median of +1.3%.

Maintenance

Last commit 2 days ago, fresher than the median of 3d.

How it compares in Production Serving & Deployment

sample-genai-on-eks-starter-kit
102
stars
- 30d
commit 2 days ago
Production Serving & Deployment median
272
stars
+1.3% 30d
commit ~3d ago
Leader · vllm
97.3k
stars
+2.2% 30d
commit today

Ranked 336th of 397 by stars.

Ranked by shared tags, then keywords, then stars
  1. Open-source sandboxes where coding agents build and deploy. Spin up isolated environments where Claude Code…

    +4 (+1.3%) in 30d · Committed 4d ago · Python · Other license

    Shares Production Serving & Deployment, Ship to Users, Observability & Tracing

  2. SmarterRouter: An intelligent LLM gateway and VRAM-aware router for Ollama, llama.cpp, and OpenAI. Features…

    +11 (+7.1%) in 30d · Committed 196d ago · Python · MIT

    Shares LLM Gateway & Routing, Production Serving & Deployment, Observability & Tracing

  3. Open-source Token as a Service (TaaS) platform on Kubernetes — deploy LLMs, price by model × GPU, and…

    Committed yesterday · Go · Apache-2.0

    Shares Production Serving & Deployment, LLM Gateway & Routing, Observability & Tracing

  4. 🦎 A lightweight, streaming-native, multi-protocol gateway for AI, APIs, and real-time data. Route LLM…

    -28 (-1.6%) in 30d · Committed today · Java · Other license

    Shares LLM Gateway & Routing, Production Serving & Deployment, Observability & Tracing

  5. The production framework for Predictive and Generative AI. Serve any model as an API in one line, with…

    +2 (+0.7%) in 30d · Committed yesterday · Python · Apache-2.0

    Shares Production Serving & Deployment, LLM Gateway & Routing, Ship to Users

  6. This Guidance demonstrates how to streamline access to numerous large language models (LLMs) through a…

    +0 (+0.0%) in 30d · Committed 9d ago · HCL · MIT-0

    Shares LLM Gateway & Routing, Production Serving & Deployment, Ship to Users

  7. Self-hosted, Docker-first OpenAI-compatible LLM gateway with provider routing, fallbacks, API keys, and usage…

    +0 (+0.0%) in 30d · Committed 108d ago · Python · Other license

    Shares LLM Gateway & Routing, Observability & Tracing, Ship to Users

  8. Open-source infrastructure for the full inference lifecycle: deploy, observe, scale, optimize, and safely…

    -22 (-41%) in 30d · Committed today · Go · Apache-2.0

    Shares Production Serving & Deployment, Observability & Tracing, LLM Gateway & Routing

  9. Open-source LLM gateway in Rust. One endpoint for OpenAI, Anthropic, Gemini and every other vendor. Any SDK…

    +843 (+590%) in 30d · Committed today · Rust · Apache-2.0

    Shares LLM Gateway & Routing, Production Serving & Deployment, Observability & Tracing

  10. MCPCAN is a centralized management platform for MCP services. It deploys each MCP service using a container…

    +3 (+0.4%) in 30d · Committed 233d ago · Go · Other license

    Shares LLM Gateway & Routing, Production Serving & Deployment, Observability & Tracing

Language
JavaScript
License
MIT No Attribution
Contributors
18
Forks
65
Watchers
3
Open issues
19
Releases
7
Created
2025-07-04
Stars this week
+0

You've seen where sample-genai-on-eks-starter-kit stands today.

Get the moves before everyone else: the 5 Production Serving & Deployment repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain sample-genai-on-eks-starter-kit?

Claim this page to add your own call-to-action.

Claim this page