Skip to content

Gateway Implementations

This project has several conformant Gateway implementations:

Agentgateway supports both standalone and Kubernetes deployment modes.

Gateway conformance status is governed by the Conformance Report Deprecation Policy. Implementations that have not submitted a successful Gateway profile report for the current minor release or either of the two previous minor releases may be removed from conformant implementation listings until they submit an accepted report.

Alibaba Cloud Container Service for Kubernetes

Alibaba Cloud Container Service for Kubernetes (ACK) is a managed Kubernetes platform offered by Alibaba Cloud. The implementation of the Gateway API in ACK is through the ACK Gateway with Inference Extension component, which introduces model-aware, GPU-efficient load balancing for AI workloads beyond basic HTTP routing.

The ACK Gateway with Inference Extension implements the Gateway API Inference Extension and provides optimized routing for serving generative AI workloads, including weighted traffic splitting, mirroring, advanced routing, etc. See the docs for the usage.

ACK Gateway with Inference Extension v1.6.1-apsara.3 passes the Gateway profile for Gateway API Inference Extension v1.5.0. See the conformance report.

Istio

Istio is an open source service mesh and gateway implementation. It provides a fully compliant implementation of the Kubernetes Gateway API for cluster ingress traffic control. For service mesh users, Istio also fully supports east-west (including GAMMA) traffic management within the mesh.

Istio supports the Gateway API Inference Extension for inference-aware routing. See the Istio documentation for installation and configuration instructions. Istio v1.28.0 or later is recommended to ensure compatibility with the InferencePool API.

Agentgateway

Agentgateway is a high-performance, Rust-based AI gateway for LLM, MCP, and A2A workloads that can also serve as a Gateway API and Inference Gateway implementation.

It can run as a standalone binary or in Docker on a local machine or server without Kubernetes, or be deployed on Kubernetes for cluster-based environments, or within your llm-d infrastructure to improve accelerator (GPU) utilization for AI inference workloads.

Higress

Higress is a cloud-native API gateway built on Istio and Envoy. It supports the Gateway API Inference Extension for inference-aware routing and provides API management and AI gateway capabilities. Higress is an open source CNCF project.

Source code, documentation, and issue tracking are available in the Higress repository.

NGINX Gateway Fabric

NGINX Gateway Fabric is an open-source project that provides an implementation of the Gateway API using NGINX as the data plane. The goal of this project is to implement the core Gateway API to configure an HTTP or TCP/UDP load balancer, reverse-proxy, or API gateway for applications running on Kubernetes. You can find the comprehensive NGINX Gateway Fabric user documentation on the NGINX Documentation website.