Streaming responses stop after exactly 60 seconds behind nginx

I am streaming completions through an nginx reverse proxy. Locally everything works. In production, every stream that runs longer than 60 seconds is cut off — the client just sees the connection close, no error event.

location /v1/ {
    proxy_pass http://app:8080;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
}

Request ids from today: req_8f2a…c91, req_17be…004. Python SDK 2.4.1. Is there a server-side limit I am missing?

Sixty seconds is suspicious. That is nginx's default proxy_read_timeout, not ours.

Lukas is right, and there is a second one that bites people. You need both:

proxy_read_timeout 600s;
proxy_buffering off;

Without proxy_buffering off nginx holds the chunks until its buffer fills, so the stream looks stuck and then arrives all at once. With the default read timeout it closes the upstream after 60s of what it thinks is silence.

Without proxy_buffering off nginx holds the chunks until its buffer fills, so the stream looks stuck and then arrives all at once.

That was it. Both lines, reloaded nginx, and a 7-minute generation came through cleanly. Thank you both!

Could this go in the streaming guide? The deployment section never mentions proxies at all.

Added to the docs backlog. Thanks for the nudge, @aminata.