Envoy

For conventional HTTP APIs, limiting the number of requests per minute is usually enough. For large language model APIs, however, limiting only the request count is far from sufficient: one request may consume only a few dozen tokens, while another may consume tens of thousands. Therefore, a practical AI gateway usually needs to support all of the following: RPM (Requests Per Minute): the number of requests per minute. TPM (Tokens Per Minute): the number of tokens per minute. Separate metering by tenant, API key, model, or plan. Shared quotas across multiple Envoy instances. Protection against quota multiplication caused by per-instance counters under high concurrency. Consistent 429 Too Many Requests responses and remaining-quota information. This article explains how to build a distributed AI rate limiting system with envoyproxy/ratelimit, Redis, and Envoy. ...

On November 11, 2025, the official Kubernetes blog formally announced that the Ingress NGINX project has entered the Retirement phase and will cease maintenance entirely in March 2026. This move marks the official entry of Kubernetes cluster ingress and traffic management into the Gateway API era. For teams currently using Ingress NGINX, this is not just a technical upgrade, but a risk management task that needs to be planned as soon as possible. ...