Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
menu search
person
Welcome To Ask or Share your Answers For Others

Categories

Spent all week on this one and tried every related stackoverflow post. Thanks for being here.

I have an Ubuntu VM running nginx with reverse proxies pointing to various docker daemons concurrently running on different ports. All my static sites work flawlessly. However, I have one container running an expressjs app.

I get responses after restarting the server for about an hour. Then I get 502 Bad Gateway. A refresh brings the site back up for approx 5 seconds until it permanently goes down. This is reproducible.

  • The docker container has express listening on 0.0.0.0:8090 inside the container

  • The container is running

    02e1917991e6   docker/express-site       "docker-entrypoint.s…"   About an hour ago   Up About an hour   127.0.0.1:8090->8090/tcp   express-site
    
  • The 8090 port is EXPOSEd in the Dockerfile.

  • I tried other ports.

  • When down, I can curl the site from within the container when inspecting.

  • When down, curling the site from within the VM yields

    curl: (52) Empty reply from server
    
  • Memory and CPU usage within the container and within the VM barely reach 5%.

  • Site usually has SSL but tried http as well.

  • Tried various nginx proxy settings (see config below)

  • Using out-of-the box nginx.conf

  • Considering that it might be related to a timeout or docker network settings...

My site-available config file looks like:

server {
   server_name example.com www.example.com;

   location / {
        proxy_pass http://127.0.0.1:8090;
        #proxy_set_header Host $host;
        #proxy_buffering off;
        #proxy_buffer_size 16k;
        #proxy_busy_buffers_size 24k;
        #proxy_buffers 64 4k;
   }
    listen 80;
    listen [::]:80;
    #listen 443 ssl; # managed by Certbot
    #ssl_certificate /etc/letsencrypt/live/www.example.com/fullchain.pem; # managed by Certbot
    #ssl_certificate_key /etc/letsencrypt/live/www.example.com/privkey.pem; # managed by Certbot
    #include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    #ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

Nginx Error Log shows:

2021/01/02 23:50:00 [error] 13901#13901: *46 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: ***.**.**.***, server: example.com, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8090/favicon.ico", host: "www.example.com", referrer: "http://www.example.com"

Anyone else have ideas?

question from:https://stackoverflow.com/questions/65545516/nginx-reverse-proxy-to-docker-502-bad-gateway

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
1.4k views
Welcome To Ask or Share your Answers For Others

1 Answer

Didn't get much feedback, but I did more research and the issue is now stable so I wanted to post my findings.

I have isolated the issue with the docker container. Nginx works fine with the same app running on the VM directly.

I updated my docker container image from node:12-alpine to node:14-alpine. The site has been up for 42 hours without issue.

If it randomly fails again, then it's probably due to load.

I hope this solves someone's issue.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
...