Skip to main content

Getting Started with Envoy AI Gateway

Welcome to the Envoy AI Gateway getting started guide!

This guide will walk you through setting up and using Envoy AI Gateway, a tool for managing GenAI traffic using Envoy.

Guide Structure​

This getting started guide is organized into several sections:

  1. Prerequisites

    • Setting up your Kubernetes cluster
    • Installing required tools
    • Setting up Envoy Gateway

  2. Installation

    • Installing Envoy AI Gateway
    • Configuring the gateway
    • Verifying the installation

  3. Basic Usage

    • Deploying a basic configuration
    • Making your first request
    • Understanding the response format

  4. Connect Providers

    • Setting up OpenAI integration
    • Configuring AWS Bedrock
    • Managing credentials securely

Quick Start​

If you're familiar with Kubernetes and want to get started quickly, run these commands to install Envoy Gateway, Envoy AI Gateway, and deploy a basic configuration:

helm upgrade -i eg oci://docker.io/envoyproxy/gateway-helm \
  --version v0.0.0-latest \
  --namespace envoy-gateway-system \
  --create-namespace

helm upgrade -i aieg oci://ghcr.io/envoyproxy/ai-gateway/ai-gateway-helm \
  --version v0.0.0-latest \
  --namespace envoy-ai-gateway-system \
  --create-namespace

kubectl apply -f https://raw.githubusercontent.com/envoyproxy/ai-gateway/main/examples/basic/basic.yaml

kubectl wait --timeout=2m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available
kubectl wait --timeout=2m -n envoy-ai-gateway-system deployment/ai-gateway-controller --for=condition=Available

Make a request​

Check out Making a Request in the Basic Usage Guide

tip

For detailed instructions and explanations, start with the Prerequisites section.

Need Help?​

If you run into any issues: