Feature Request: LLM-D Integration with Kubernetes in AI Gateway

[devfloor9]

There is a request for examples demonstrating how LLM-D can be tightly integrated with Kubernetes in AI Gateway. Adding such examples would be beneficial for users looking to implement this integration.

Suggested Example Components:

• Kubernetes deployment manifests for LLM-D
• Configuration for AI Gateway to interact with LLM-D pods
• Resource allocation best practices
• Scaling configurations
• Service mesh integration patterns

[krapie]

I think we can now add an llm-d demonstration along with a writing guide for EKS in the llm-d infra-providers docs itself.

• docs: Introduce Amazon EKS as a well-lit infra provider llm-d/llm-d#370
• https://aws.amazon.com/blogs/machine-learning/introducing-disaggregated-inference-on-aws-powered-by-llm-d/
• https://github.com/llm-d/llm-d/pkgs/container/llm-d-aws
• https://github.com/llm-d/llm-d/tree/main/docs/infra-providers