Skip to main content
Version: latest

Supported OpenAI API Endpoints

The Envoy AI Gateway provides OpenAI-compatible API endpoints for routing and managing LLM/AI traffic. This page documents which OpenAI API endpoints are currently supported and their capabilities.

Overview​

The Envoy AI Gateway acts as a proxy that accepts OpenAI-compatible requests and routes them to various AI providers. While it maintains compatibility with the OpenAI API specification, it currently supports a subset of the full OpenAI API.

Supported Endpoints​

Chat Completions​

Endpoint: POST /v1/chat/completions

Status: ✅ Fully Supported

Description: Create a chat completion response for the given conversation.

Features:

  • ✅ Streaming and non-streaming responses
  • ✅ Function calling
  • ✅ Response format specification (including JSON schema)
  • ✅ Temperature, top_p, and other sampling parameters
  • ✅ System and user messages
  • ✅ Model selection via request body or x-ai-eg-model header
  • ✅ Token usage tracking and cost calculation
  • ✅ Provider fallback and load balancing

Supported Providers:

  • OpenAI
  • AWS Bedrock (with automatic translation)
  • Azure OpenAI (with automatic translation)
  • Any OpenAI-compatible provider (Groq, Together AI, Mistral, etc.)

Example:

curl -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "Hello, how are you?"
      }
    ]
  }' \
  $GATEWAY_URL/v1/chat/completions

Embeddings​

Endpoint: POST /v1/embeddings

Description: Create embeddings for the given input text.

Features:

  • ✅ Single and batch text embedding
  • ✅ Model selection via request body or x-ai-eg-model header
  • ✅ Token usage tracking and cost calculation
  • ✅ Provider fallback and load balancing

Supported Providers:

  • OpenAI
  • Any OpenAI-compatible provider that supports embeddings

Models​

Endpoint: GET /v1/models

Description: List available models configured in the AI Gateway.

Features:

  • ✅ Returns models declared in AIGatewayRoute configurations
  • ✅ OpenAI-compatible response format
  • ✅ Model metadata (ID, owned_by, created timestamp)

Example:

curl $GATEWAY_URL/v1/models

Response Format:

{
  "object": "list",
  "data": [
    {
      "id": "gpt-4o-mini",
      "object": "model",
      "created": 1677610602,
      "owned_by": "openai"
    }
  ]
}

Provider-Endpoint Compatibility Table​

The following table summarizes which providers support which endpoints:

ProviderChat CompletionsEmbeddingsNotes
OpenAI✅✅
AWS Bedrock✅🚧Via API translation
Azure OpenAI✅🚧Via API translation
Google Gemini✅✅Via OpenAI-compatible API
Groq✅❌Via OpenAI-compatible API
Grok✅❌Via OpenAI-compatible API
Together AI⚠️⚠️Via OpenAI-compatible API
Cohere⚠️⚠️Via OpenAI-compatible API
Mistral⚠️⚠️Via OpenAI-compatible API
DeepInfra✅✅Via OpenAI-compatible API
DeepSeek⚠️❌Via OpenAI-compatible API
Hunyuan⚠️⚠️Via OpenAI-compatible API
Tencent LLM Knowledge Engine⚠️❌Via OpenAI-compatible API
Google Vertex AI🚧🚧Work-in-progress: issue#609
Anthropic on Vertex AI🚧🚧Work-in-progress: issue#609
SambaNova✅✅Via OpenAI-compatible API
  • ✅ - Supported and Tested on Envoy AI Gateway CI
  • ⚠️️ - Expected to work based on provider documentation, but not tested on the CI.
  • ❌ - Not supported according to provider documentation.
  • 🚧 - Unimplemented, or under active development but planned for future releases

What's Next​

To learn more about configuring and using the Envoy AI Gateway with these endpoints: