Yes — read the message carefully: it says requests per minute, not tokens. There are two limits and the dashboard chart shows tokens. If you fire 60 small requests in the first ten seconds of a minute you are out of requests while barely touching tokens. Two fixes, either works: batch small prompts…
Jump to postMy usage dashboard says I am at about 20% of my tokens-per-minute limit, but I still get 429 Too Many Requests in bursts every few minutes.
{"error": {"type": "rate_limit", "message": "Requests per minute exceeded (60/60)."}}Am I hitting a different limit than the one on the dashboard?
Yes — read the message carefully: it says requests per minute, not tokens. There are two limits and the dashboard chart shows tokens. If you fire 60 small requests in the first ten seconds of a minute you are out of requests while barely touching tokens.
Two fixes, either works:
batch small prompts into one request, or
spread them out with a token bucket on your side.
The x-ratelimit-remaining-requests header tells you exactly where you are.
Oh. I was batching per user and had 60 users hitting at the top of each minute because of a cron. Jittering the cron fixed it.
The headers trick is underrated. We log x-ratelimit-remaining-requests on every call and alert at 10%.