socket.io - 使用 Traefik 或 NGINX 在 Docker Swarm 上正确处理 Socket.io

Question

我正在使用使用 Docker Swarm 部署的 Socket.IO 在 Node.js 中处理一个应用程序，我想要应用程序服务的多个实例的选项。但是，当有多个实例时，应用程序会失败。失败涉及浏览器中每个 Socket.IO 消息的错误，应该在消息中发送的数据永远不会到达等。
Docker Stack 文件有四个服务

Node.js 应用程序

在多节点 Socket.IO 服务中处理 Socket.IO 和 session 所需的 REDIS 实例——是的，我已经阅读了关于此的 Socket.IO 文档，实现了 connect-redis SessionStore，并使用 socket.io-redis做多节点Socket.IO

数据库(MySQL)

反向代理 - 我使用过 NGINX 和 Traefik

在 Socket.IO 中有一个例行的保活请求，例如 /socket.io/?EIO=3&transport=polling&t=NLjcKJj&sid=X5UnuTjlYNJ4N8OsAAAH 上的 GET .此请求在反向代理的日志文件中可见，并由应用程序处理。 Engine.IO 的调试输出表示它收到了这些请求。
具体来说:

2020-10-28T05:06:02.557Z Net read redis:6379 id 0
2020-10-28T05:06:02.557Z socket.io:socket socket connected - writing packet
2020-10-28T05:06:02.557Z socket.io:socket joining room X5UnuTjlYNJ4N8OsAAAH
2020-10-28T05:06:02.557Z socket.io:client writing packet {"type":0,"nsp":"/"}
2020-10-28T05:06:02.557Z socket.io:socket joined room [ 'X5UnuTjlYNJ4N8OsAAAH' ]
2020-10-28T05:06:02.656Z engine intercepting request for path "/socket.io/"
2020-10-28T05:06:02.656Z engine handling "GET" http request "/socket.io/?EIO=3&transport=polling&t=NLjcKJj&sid=X5UnuTjlYNJ4N8OsAAAH"
2020-10-28T05:06:02.656Z engine setting new request for existing client
2020-10-28T05:06:02.655Z engine intercepting request for path "/socket.io/"
2020-10-28T05:06:02.655Z engine handling "POST" http request "/socket.io/?EIO=3&transport=polling&t=NLjcKJh&sid=X5UnuTjlYNJ4N8OsAAAH"
2020-10-28T05:06:02.655Z engine unknown sid "X5UnuTjlYNJ4N8OsAAAH"
2020-10-28T05:06:02.774Z engine intercepting request for path "/socket.io/"
2020-10-28T05:06:02.774Z engine handling "GET" http request "/socket.io/?EIO=3&transport=polling&t=NLjcKLI&sid=X5UnuTjlYNJ4N8OsAAAH"
2020-10-28T05:06:02.774Z engine unknown sid "X5UnuTjlYNJ4N8OsAAAH"
2020-10-28T05:06:02.775Z engine intercepting request for path "/socket.io/"
2020-10-28T05:06:02.775Z engine handling "POST" http request "/socket.io/?EIO=3&transport=polling&t=NLjcKLJ&sid=X5UnuTjlYNJ4N8OsAAAH"
2020-10-28T05:06:02.775Z engine setting new request for existing client
2020-10-28T05:06:02.775Z socket.io:client client close with reason transport close
2020-10-28T05:06:02.775Z socket.io:socket closing socket - reason transport close
2020-10-28T05:09:14.955Z socket.io:client client close with reason ping timeout
2020-10-28T05:09:14.955Z socket.io:socket closing socket - reason ping timeout

日志消息说 engine unknown sid "X5UnuTjlYNJ4N8OsAAAH"似乎很重要。它说 session ID 未知。但是 session 是使用 REDIS 在节点之间共享的。因此，令人困惑的是为什么 session 是未知的，因为它们应该使用 connect-redis 共享。 .
另一个重要的事情是浏览器中的日志记录。
在 JavaScript 控制台中，会持续报告这些消息:

WebSocket connection to 'ws://DOMAIN-NAME/socket.io/?EIO=3&transport=websocket&sid=h2aFFkOvNZtFc1DcAAAI' failed: WebSocket is closed before the connection is established.
Failed to load resource: the server responded with a status of 400 (Bad Request)

最后一个报告为 http://DOMAIN-NAME/socket.io/?EIO=3&transport=polling&t=NLjf5hB&sid=h2aFFkOvNZtFc1DcAAAI然后，对于这些请求，我看到响应正文是:

{
    "code": 1,
    "message": "Session ID unknown"
}

这显然与 unknown sid一致早点留言。我认为这意味着连接正在关闭，因为服务器认为 session ID 不正确。
在我对此进行的研究中，我了解到在 Docker Swarm 中，流量以循环方式分布——即 Docker Swarm 充当循环负载平衡器。在这种情况下，Socket.IO 的成功途径是实现粘性 session 。
我在某处读到 NGINX 中的粘性 session 支持不适用于这种情况，而 Traefik 可以支持这种情况。
在 NGINX 中，我有这个代理配置:

location / {

    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy false;

    proxy_pass http://todos;
    proxy_redirect off;

    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
}

upstream todos {
    ip_hash;

    server todo:80  fail_timeout=1s max_fails=3;
    keepalive 16;
}

这并没有改变行为 - 仍然 unknown sid等等。因此我已经切换到 Traefik，并且我在 Traefik 的这一方面找不到文档时遇到了麻烦。这是我第一次使用 Traefik，FWIW。我能够使用 Lets Encrypt 实现 HTTPS，但不能使用粘性 session 。
为了配置 Traefik，我使用了命令行参数和 Docker 容器标签，以便整个配置都在 Docker Stack 文件中。

    traefik:
        image: traefik:v2.0
        restart: always
        ports:
            - "80:80" # <== http
            - "8080:8080" # <== :8080 is where the dashboard runs on
            - "443:443" # <== https
        deploy:
            replicas: 1
            labels:
                #### Labels define the behavior and rules of the traefik proxy for this container ####
                - "traefik.enable=true" # <== Enable traefik on itself to view dashboard and assign subdomain to view it
                - "traefik.http.routers.api.rule=Host(`monitor.DOMAIN-NAME`)" # <== Setting the domain for the dashboard
                - "traefik.http.routers.api.service=api@internal" # <== Enabling the api to be a service to access
                - "traefik.http.routers.api.entrypoints=web"
            placement:
                constraints:
                    - "node.hostname==srv1"
        command:
            - "--providers.docker.swarmmode=true"
            - "--providers.docker.endpoint=unix:///var/run/docker.sock"
            - "--providers.docker.watch=true"
            - "--log.level=DEBUG"
            - "--accesslog=true"
            - "--tracing=true"
            - "--api.insecure=true" # <== Enabling insecure api, NOT RECOMMENDED FOR PRODUCTION
            - "--api.dashboard=true" # <== Enabling the dashboard to view services, middlewares, routers, etc...
            - "--providers.docker=true" # <== Enabling docker as the provider for traefik
            - "--providers.docker.exposedbydefault=false" # <== Don't expose every container to traefik, only expose enabled onesconfiguration file
            - "--providers.docker.network=todo_webnet" # <== Operate on the docker network named web
            - "--entrypoints.web.address=:80" # <== Defining an entrypoint for port :80 named web
            - "--entrypoints.web-secured.address=:443" # <== Defining an entrypoint for https on port :443 named web-secured
            - "--certificatesresolvers.mytlschallenge.acme.tlschallenge=false" # <== Enable TLS-ALPN-01 to generate and renew ACME certs
            - "--certificatesresolvers.mytlschallenge.acme.email=E-MAIL-ADDRESS@DOMAIN-NAME" # <== Setting email for certs
            - "--certificatesresolvers.mytlschallenge.acme.storage=/letsencrypt/acme.json" # <== Defining acme file to store cert 
            - "--certificatesresolvers.mytlschallenge.acme.httpChallenge.entryPoint=web"
        volumes:
            - /home/ubuntu/letsencrypt:/letsencrypt # <== Volume for certs (TLS)
            - /var/run/docker.sock:/var/run/docker.sock # <== Volume for docker admin
        networks:
            - webnet

    todo:
        image: robogeek/todo-app:first-dockerize-redis
        # ports: 
        #    - "80:80"
        networks:
            - dbnet
            - webnet
            - redisnet
        deploy:
            replicas: 2
            labels:
                #### Labels define the behavior and rules of the traefik proxy for this container ####
                - "traefik.enable=true" # <== Enable traefik to proxy this container
                - "traefik.http.routers.todo.rule=Host(`DOMAIN-NAME`)" # <== Your Domain Name goes here for the http rule
                - "traefik.http.routers.todo.entrypoints=web" # <== Defining the entrypoint for http, **ref: line 30
                - "traefik.http.routers.todo.service=todo"
                - "traefik.http.services.todo.loadbalancer.healthcheck.port=80"
                - "traefik.http.services.todo.loadbalancer.sticky=true"
                - "traefik.http.services.todo.loadbalancer.server.port=80"
                - "traefik.http.routers.todo-secured.rule=Host(`DOMAIN-NAME`)" # <== Your Domain Name goes here for the http rule
                - "traefik.http.routers.todo-secured.entrypoints=web-secured" # <== Defining the entrypoint for http, **ref: line 30
                - "traefik.http.routers.todo-secured.service=todo"
                - "traefik.http.routers.todo-secured.tls=true"
                - "traefik.http.routers.todo-secured.tls.certresolver=mytlschallenge" # <== Defining certsresolvers for https
                # - "traefik.http.routers.todo-app.middlewares=redirect@file" # <== This is a middleware to redirect to https
                # - "traefik.http.routers.nginx-secured.rule=Host(`example.com`)" # <== Your Domain Name for the https rule 
                # - "traefik.http.routers.nginx-secured.entrypoints=web-secured" # <== Defining entrypoint for https, **ref: line 31
        depends_on:
            - db
            - redis
        dns:
            - 8.8.8.8
            - 9.9.9.9
        environment:
            - SEQUELIZE_CONNECT=models/sequelize-mysql-docker.yaml
            - SEQUELIZE_DBHOST=db
            - SEQUELIZE_DBNAME=tododb
            - SEQUELIZE_DBUSER=dbuser
            - SEQUELIZE_DBPASSWD=PASS-WORD-HIDDEN
            - REDIS_ENDPOINT=redis
            - NODE_DEBUG=redis
            - REDIS_PASSWD=PASS-WORD-HIDDEN
            - DEBUG=todos:*,ioredis:*,socket.io:*,engine
        command: [ "./wait-for-it.sh", "-t", "0", "db:3306", "--", "node", "./app.mjs" ]

Answer 1

在 Traefik 论坛上我发现了这个:https://community.traefik.io/t/sticky-sessions-dont-work/1949
根据讨论，我添加了以下内容 label到 todo容器:

- "traefik.http.services.todo.loadbalancer.sticky.cookie.name=StickySessionCookie"

现在它运行良好，从 1 个容器扩展到 4 个容器，并且运行良好。