I am new to OpenTelemetry. I am trying to do setup using yaml files appraoch but facing 'connection refused' issue while connecting apm to elasticsearch.

我是OpenTelemeter的新手。我正在尝试使用YAML文件appraoch进行设置，但在将APM连接到ElasticSearch时遇到‘连接被拒绝’的问题。

What I have achieved so far:
a) Able to receive data in the OtelCollector.
b) Able to receive data in the Prometheus.

我到目前为止所取得的成就：a)能够在OtelCollector中接收数据。B)能够在普罗米修斯号上接收数据。

Issues facing:
a) Not able to connect APM with Elasticsearch.
enter image description here

面临的问题：a)无法将APM与Elasticearch连接起来。在此处输入图像描述

Stack:
Otel Collector
Elastic APM
Elasticsearch
Prometheus

堆栈：Otel Collector Elastic APM Elasticearch Prometheus

I am attaching all yml files for reference. Please help me in resolving issues and point out other mistakes I am doing right now.

我附上所有YML文件以供参考。请帮助我解决问题，并指出我现在正在做的其他错误。

docker-compose.yaml

Docker-compose.yaml

version: "3.7"
services:     
# APM Server   
  apm-server:
    image: docker.elastic.co/apm/apm-server:8.8.2
    volumes:
      - ./apm-server.docker.yml:/usr/share/apm-server/apm-server.yml
    cap_add: ["CHOWN", "DAC_OVERRIDE", "SETGID", "SETUID"]
    cap_drop: ["ALL"]
    ports:
      - "0.0.0.0:8200:8200"
    environment:
      host: 0.0.0.0:8200
    depends_on:
      - elasticsearch
    command: >
       apm-server -e
         -E apm-server.rum.enabled=true
         -E output.elasticsearch.hosts=['localhost:9200']
    networks:
      - icmnetworks  
# Elasticsearch     
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.8.2
    container_name: elasticsearch
    volumes:
      - ./elasticsearch.yaml:/usr/share/elasticsearch/config/elasticsearch.yml
    #volumes:
      #- /usr/share/elasticsearch/data
      #- C:\ElasticSearch\elasticsearch-7.17.0\data:/usr/share/elasticsearch/data
    ports:
      - "0.0.0.0:9200:9200"
      - "0.0.0.0:9300:9300"
    environment:
      transport.host: 127.0.0.1
      ES_JAVA_OPTS: -Xms512m -Xmx512m
    mem_limit: 1073741824
    ulimits:
      memlock:
        soft: -1
        hard: -1
    networks:
      - icmnetworks      
  prometheus:
    container_name: prometheus
    image: prom/prometheus:latest
    volumes:
      - ./prometheus.yaml:/etc/prometheus/prometheus.yml
    ports:
      - 9090:9090
    networks:
      - icmnetworks  
  otel-collector:
    image: otel/opentelemetry-collector-contrib:latest
    container_name: otel-collector
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
      - ./output:/etc/output:rw
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
      - 8888:8888   # Prometheus metrics exposed by the collector
      - 8889:8889   # Prometheus exporter metrics
      - 4317:4317   # OTLP gRPC receiver
      - 4318:4318   # OTLP http receiver
    depends_on:
      - prometheus
      - elasticsearch
    networks:
      - icmnetworks

networks:
  icmnetworks:
    driver: bridge

otel-collector-config.yaml

Otel-collector-config.yaml

receivers:
    otlp:
        protocols:
            grpc:
            http:

processors:
    batch:
  
exporters:
    file:
        path: /etc/output/logs.json
    prometheus:
        endpoint: "0.0.0.0:9090"
    otlp/elastic:
        endpoint: "apm-server:8200"
        tls:
            insecure: true
    logging:
        loglevel: debug

service:
    pipelines:
        traces:
            receivers: [otlp]
            processors: [batch]
            exporters: [logging, otlp/elastic]
        metrics:
            receivers: [otlp]
            processors: [batch]
            exporters: [logging, prometheus]
        logs:
            receivers: [otlp]
            processors: []
            exporters: [logging, otlp/elastic] #, file

apm-server.yaml

Apm-server.yaml

apm-server:
  host: "0.0.0.0:8200"
output.elasticsearch:
  hosts: ["localhost:9200", "0.0.0.0:9200"]
path.data: /usr/share/apm-server/data
path.logs: /var/log
logging.level: info
logging.to_syslog: true
logging.metrics.enabled: true
logging.files:
  path: /var/log/apm-server
  name: apm-server
  rotateeverybytes: 10485760 # = 10MB
  keepfiles: 7
  permissions: 0600
  interval: 0

elastisearch.yaml

Elastisearch.yaml

cluster.name: elasticsearchcluster
node.name: elasticnode
node.attr.rack: r1
path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/logs
network.host: 0.0.0.0
http.port: 9200
discovery.type: single-node
xpack.security.enabled: false
action.auto_create_index: .monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*,+u*,+o*
cluster.routing.allocation.disk.threshold_enabled: false
cluster.routing.allocation.disk.watermark.low: 700mb
cluster.routing.allocation.disk.watermark.high: 600mb
cluster.routing.allocation.disk.watermark.flood_stage: 500mb
cluster.info.update.interval: 1m

prometheus.yaml

Prometheus.yaml

scrape_configs:
  - job_name: 'otel-collector'
    scrape_interval: 10s
    static_configs:
      - targets: ['localhost:9090']

I tried configuring OpenTelemetry using yaml files.
Expectation: Data should be forwarded from Otel-Collector to elasticsearch.

我尝试使用YAML文件配置OpenTelemeter。预期：数据应从Otel-Collector转发到ElasticSearch。

I'll suggest a few modifications to what you have as follows

我将建议对您所拥有的内容进行一些修改，如下所示

1. Apm-server.yml:
   In the output.elasticsearch.hosts section, make sure you're not using localhost. Instead, use the service name from Docker Compose. So, it should be:

output.elasticsearch:
  hosts: ["elasticsearch:9200"]

2. docker-compose.yaml:
   It look like the otlp/elastic exporter is specified in the otel-collector-config.yaml file but not actually defined in the docker-compose.yaml.
   Perhaps you can add a new service for the OpenTelemetry Collector exporter and link it to the APM server.

otlp-exporter:
  image: otel/opentelemetry-collector-contrib:latest
  command: ["--config=/etc/otel-collector-config.yaml"]
  volumes:
    - ./output:/etc/output:rw
    - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
  ports:
    - 8200:8200 # Assuming APM server is listening on 8200
  depends_on:
    - apm-server
  networks:
    - icmnetworks

Finally docker-compose up -d will reveal if any of these is any good

最后，docker-compose up-d将显示这些选项中的任何一个是否有用