Skip to main content
Version: latest

Installation

This guide will walk you through installing Envoy AI Gateway and its required components.

Installing Envoy AI Gateway​

The easiest way to install Envoy AI Gateway is using the Helm chart. First, install the AI Gateway Helm chart; this will install the CRDs as well. Once completed, wait for the deployment to be ready.

helm upgrade -i aieg oci://docker.io/envoyproxy/ai-gateway-helm \
    --version v0.0.0-latest \
    --namespace envoy-ai-gateway-system \
    --create-namespace

kubectl wait --timeout=2m -n envoy-ai-gateway-system deployment/ai-gateway-controller --for=condition=Available

Installing CRDs serperately​

If you want to manage the CRDs separately, install the CRD Helm chart (ai-gateway-crds-helm) which will install just the CRDs:

helm upgrade -i aieg-crd oci://docker.io/envoyproxy/ai-gateway-crds-helm \
    --version v0.0.0-latest \
    --namespace envoy-ai-gateway-system \
    --create-namespace

After the CRDs are installed, you can install the AI Gateway Helm chart without re-installing the CRDs by using the --skip-crds flag.

helm upgrade -i aieg oci://docker.io/envoyproxy/ai-gateway-helm \
    --version v0.0.0-latest \
    --namespace envoy-ai-gateway-system \
    --create-namespace \
    --skip-crds

Configuring Envoy Gateway​

After installing Envoy AI Gateway, apply the AI Gateway-specific configuration to Envoy Gateway, restart the deployment, and wait for it to be ready:

kubectl apply -f https://raw.githubusercontent.com/envoyproxy/ai-gateway/main/manifests/envoy-gateway-config/redis.yaml
kubectl apply -f https://raw.githubusercontent.com/envoyproxy/ai-gateway/main/manifests/envoy-gateway-config/config.yaml
kubectl apply -f https://raw.githubusercontent.com/envoyproxy/ai-gateway/main/manifests/envoy-gateway-config/rbac.yaml

kubectl rollout restart -n envoy-gateway-system deployment/envoy-gateway

kubectl wait --timeout=2m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available

Note that the redis configuration is only used for the rate limiting feature. If you don't need rate limiting, you can skip the redis configuration, but you need to remove the relevant configuration in the config.yaml file as well.

Verify Installation

Check the status of the pods. All pods should be in the Running state with Ready status.

Check AI Gateway pods:

kubectl get pods -n envoy-ai-gateway-system

Check Envoy Gateway pods:

kubectl get pods -n envoy-gateway-system

Next Steps​

After completing the installation: