docker - 如何使用 docker swarm 部署 elasticsearch？

Question

我使用docker-machine创建了3个虚拟机，有:

NAME       ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER     ERRORS
cluster    -        virtualbox   Running   tcp://192.168.99.101:2376           v18.09.5   
cluster2   -        virtualbox   Running   tcp://192.168.99.102:2376           v18.09.5   
master     -        virtualbox   Running   tcp://192.168.99.100:2376           v18.09.5 

然后我在 master 机器上创建了一个 docker swarm:

 docker-machine ssh master "docker swarm init ----advertise-addr 192.168.99.100"

并在cluster和cluster2中加入master:

docker-machine ssh cluster "docker swarm join --advertise-addr 192.168.99.101 --token xxxx 192.168.99.100:2377"

docker-machine ssh cluster2 "docker swarm join --advertise-addr 192.168.99.102 --token xxxx 192.168.99.100:2377"

docker node ls 信息:

ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
r4a6y9wie4zp3pl4wi4e6wqp8     cluster             Ready               Active                                  18.09.5
sg9gq6s3k6vty7qap7co6eppn     cluster2            Ready               Active                                  18.09.5
xb6telu8cn3bfmume1kcektkt *   master              Ready               Active              Leader              18.09.5

有部署配置swarm.yml:

version: "3.3"

services:
  elasticsearch:
    image: elasticsearch:7.0.0
    ports:
      - "9200:9200"
      - "9300:9300"
    environment:
      - cluster.name=elk
      - network.host=_eth1:ipv4_
      - network.bind_host=_eth1:ipv4_
      - network.publish_host=_eth1:ipv4_
      - discovery.seed_hosts=192.168.99.100,192.168.99.101
      - cluster.initial_master_nodes=192.168.99.100,192.168.99.101
      - bootstrap.memory_lock=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    networks:
      - backend
    deploy:
      mode: replicated
      replicas: 3
      #endpoint_mode: dnsrr
      restart_policy:
        condition: none
      resources:
        limits:
          cpus: "1.0"
          memory: "1024M"
        reservations:
          memory: 20M
networks:
  backend:
    # driver: overlay
    # attachable: true

我将 elasticsearch 镜像拉到虚拟机:

docker-machine ssh master "docker image pull elasticsearch:7.0.0"
docker-machine ssh cluster "docker image pull elasticsearch:7.0.0"
docker-machine ssh cluster2 "docker image pull elasticsearch:7.0.0"

在运行之前，我运行此命令修复一些 elasticearch Bootstrap 错误:

docker-machine ssh master "sudo sysctl -w vm.max_map_count=262144"
docker-machine ssh cluster "sudo sysctl -w vm.max_map_count=262144"
docker-machine ssh cluster2 "sudo sysctl -w vm.max_map_count=262144"

然后我运行 `docker stack deploy -c swarm.yml es，elasticsearch 集群无法工作。

docker-machine ssh master
docker service logs es_elasticsearch -f 

显示:

es_elasticsearch.1.uh1x0s9qr7mb@cluster    | {"type": "server", "timestamp": "2019-04-25T16:28:47,143+0000", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "elk", "node.name": "e8dba5562417",  "message": "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [192.168.99.100, 192.168.99.101] to bootstrap a cluster: have discovered []; discovery will continue using [192.168.99.100:9300, 192.168.99.101:9300] from hosts providers and [{e8dba5562417}{Jy3t0AAkSW-jY-IygOCjOQ}{z7MYIf5wTfOhCX1r25wNPg}{10.255.0.46}{10.255.0.46:9300}{ml.machine_memory=1037410304, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0"  }
es_elasticsearch.2.swswlwmle9e9@cluster2    | {"type": "server", "timestamp": "2019-04-25T16:28:47,389+0000", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "elk", "node.name": "af5d88a04b42",  "message": "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [192.168.99.100, 192.168.99.101] to bootstrap a cluster: have discovered []; discovery will continue using [192.168.99.100:9300, 192.168.99.101:9300] from hosts providers and [{af5d88a04b42}{zhxMeNMAQN2evKDlsA33qA}{fpYPTvJ6STmyqrgxlMkD_w}{10.255.0.47}{10.255.0.47:9300}{ml.machine_memory=1037410304, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0"  }
es_elasticsearch.3.x8ouukovhh80@master    | {"type": "server", "timestamp": "2019-04-25T16:28:48,818+0000", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "elk", "node.name": "0e7e4d96b31a",  "message": "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [192.168.99.100, 192.168.99.101] to bootstrap a cluster: have discovered []; discovery will continue using [192.168.99.100:9300, 192.168.99.101:9300] from hosts providers and [{0e7e4d96b31a}{Xs9966RjTEWvEbuj4-ySYA}{-eV4lvavSHq6JhoW0qWu6A}{10.255.0.48}{10.255.0.48:9300}{ml.machine_memory=1037410304, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0"  }

我猜集群形成失败可能是由于网络配置错误。我不知道如何修复它，我尝试了很多次修改配置，失败并再次失败。

Answer 1

试试看，这是有效的:) docker-compose.yml

version: "3.7"
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.2.0
    hostname: "{{.Node.Hostname}}"
    environment:
      - node.name={{.Node.Hostname}}
      - cluster.name=my-cluster
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
      - discovery.seed_hosts=elasticsearch
      - cluster.initial_master_nodes=node1,node2,node3
      - node.ml=false
      - xpack.ml.enabled=false
      - xpack.monitoring.enabled=false
      - xpack.security.enabled=false
      - xpack.watcher.enabled=false
      - bootstrap.memory_lock=false
    volumes:
      - elasticsearch-data:/usr/share/elasticsearch/data
    deploy:
      mode: global
      endpoint_mode: dnsrr
      resources:
        limits:
          memory: 4G
  nginx:
    image: nginx:1.17.1-alpine
    ports:
      - 9200:9200
    deploy:
      mode: global
    command: |
      /bin/sh -c "echo '
      user nobody nogroup;
      worker_processes auto;
      events {
        worker_connections 1024;
      }
      http {
        client_max_body_size 4g;
        resolver 127.0.0.11 ipv6=off;
        server {
          listen *:9200;
          location / {
            proxy_set_header Connection keep-alive;
            set $$url http://elasticsearch:9200;
            proxy_pass $$url;
            proxy_set_header  Host $$http_host;
            proxy_set_header  X-Real-IP $$remote_addr;
            proxy_set_header  X-Forwarded-For $$proxy_add_x_forwarded_for;
          }
        }
      }' | tee /etc/nginx/nginx.conf && nginx -t && nginx -g 'daemon off;'"

volumes:
  elasticsearch-data: