[Updated] Correctly setup proxy config - [Original] Image generation on cpu times out

[Ahmedsaed]

Bug Report

Description

Bug Summary:
Image generation on cpu times out.

Steps to Reproduce:

1. Run a Automatic1111 or comfy instance on the cpu
2. connect it to open-webui
3. have a model generate the prompt and click on the image generation button
4. The open-webui will timeout while the model is generating the image.

Expected Behavior:
Wait a bit longer or provide a setting to control the timeout.

Actual Behavior:
It just times out.

Environment

• Open WebUI Version: v0.1.124

• Operating System: Ubuntu Server

• Browser (if applicable): Chrome

Reproduction Details

Confirmation:

• I have read and followed all the instructions provided in the README.md.
• I am on the latest version of both Open WebUI and Ollama.
• I have included the browser console logs.
• I have included the Docker container logs.

Installation Method

Installed through Docker.

Additional Information

This issue is not entirely similar to #1611 but I believe a fix could solve both of them since they are related to timeouts

[Ahmedsaed]

Update: Upon further inspection of the logs, I identified that the error was a 504 Gateway Timeout, indicating that the proxy, not the open-webui, was timing out.

I confirmed that everything works correctly when directly accessing the website without a proxy. To fix the problem, I modified the Nginx proxy configuration as follows:

Solution:

proxy_connect_timeout 3000;
proxy_send_timeout    3000;
proxy_read_timeout    3000;
send_timeout          3000;

This configuration increases the timeout to 3000 seconds (50 minutes).

But I have also noticed another problem with using a proxy. The typewriter effect where the Ollama server responds with tokens progressively is not functioning as expected. Instead of seeing a smooth, incremental display of text, there's a delay followed by a sudden chunk of multiple words. This is observed in the logs where multiple responses are received together, suggesting that the proxy is bundling the responses and sending them in batches.

The problem is caused by Nginx's default behavior of buffering proxied responses. To achieve the typewriter effect, buffering needs to be disabled so that responses are streamed immediately as they are received from the backend server.

Solution:

# Disable buffering
proxy_buffering off;
proxy_cache off;

[Ahmedsaed]

I believe including these information in the docs would be helpful

[skrenes]

You may want to consider using or at least auditing https://github.com/nginx-proxy/nginx-proxy. In addition, for uploading, you may want to consider adding

client_max_body_size 100M;
proxy_request_buffering off;

[Ahmedsaed]

Thanks for the suggestion. I haven't used nginx-proxy before, and I don't believe it would be the right choice for me for the following reasons:

• Usually, I have other services running on non-Docker containers, and I won't be able to set up a proxy for them since port 80 is taken by nginx-proxy. I prefer to handle my proxy setup myself or at least use projects like nginx-proxy-manager, which lets me have full control.
• I am not sure how using nginx-proxy would solve the problem unless it automatically adds the configuration I mentioned above.
• I would need to edit all compose files to include VIRTUAL_HOST env variable.

I don't see how using another Nginx tool would solve the problem since it lies in the configuration itself. Any Nginx setup with the right configuration would work.

[skrenes]

For your situation then the second part of my initial suggestion applies (audit the configuration generated by nginx-proxy). You could do the same thing with nginx-proxy-manager, which may have better conf files since it also has /etc/nginx/conf.d/include/block-exploits.conf.

Probably like you, I'm using a nginx container with conf files that I am manually managing and there are many corner cases that pop up (like Safari not supporting TLS 3 on websockets only) or special forwarding rules that protect against exploits. While I like the perception of control and the stability, I see the value of having an external project take care of updating conf files.

I've tried to keep as much as possible in containers on my server, so I'm now looking into switching over to using nginx-proxy-manager and fail2ban in conjunction with Cloudflare's proxy: https://blog.lrvt.de/fail2ban-with-nginx-proxy-manager/