Tpm

For conventional HTTP APIs, limiting the number of requests per minute is usually enough. For large language model APIs, however, limiting only the request count is far from sufficient: one request may consume only a few dozen tokens, while another may consume tens of thousands. Therefore, a practical AI gateway usually needs to support all of the following: RPM (Requests Per Minute): the number of requests per minute. TPM (Tokens Per Minute): the number of tokens per minute....