ruby-on-rails - 如何测试 ElasticSearch Logstash 和 Kibana

Question

我已经在我的 Debian 服务器上安装了 elasticsearch、logstash 和 kibana。问题是 Kibana 没有显示任何统计信息或日志。我不知道出了什么问题以及如何调试这个问题。当我测试每个组件(elasticsearch、kibana 和 logstash)时，一切看起来都正常。

Elasticsearch 测试

检查 elasticsearch-cluster 状态:

curl 'localhost:9200/_cluster/health?v'

{"cluster_name":"elasticsearch","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":71,"active_shards":71,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":71,"number_of_pending_tasks":0}

检查 Elasticsearch 节点状态:

curl 'localhost:9200/_cat/nodes?v'

host ip            heap.percent ram.percent load node.role master name    
ais  193.xx.yy.zz            6      10     0.05      d       *   Shathra

检查 elasticsearch-index 状态:

curl 'localhost:9200/_cat/indices?v'

health status index              pri rep docs.count docs.deleted store.size pri.store.size 
yellow open   countries            5   1        243          365    145.2kb        145.2kb 
yellow open   imports              5   1         26            7     49.6kb         49.6kb 
yellow open   categories           5   1          6            1     20.6kb         20.6kb 
yellow open   faculties            5   1         36            0     16.9kb         16.9kb 
yellow open   users                5   1       6602           29      1.8mb          1.8mb 
yellow open   cities               5   1        125            0     23.5kb         23.5kb 
yellow open   exam_languages       5   1        155            0     26.6kb         26.6kb 
yellow open   departments          5   1        167           70    166.4kb        166.4kb 
yellow open   examinations         5   1          4            0     14.1kb         14.1kb 
yellow open   certificates         5   1          1            0        3kb            3kb 
yellow open   .kibana              1   1          2            1       14kb           14kb 
yellow open   exam_centers         5   1          5            0     22.7kb         22.7kb 

检查 elasticsearch-service 状态:

$ service elasticsearch status

[ ok ] elasticsearch is running.

ElasticSearch 也可以在我的浏览器中从 localhost:9200 访问并且列出正确的索引。

/etc/nginx/sites-available/elasticsearch 文件 =>

server {
  listen 443;
  server_name es.xxx.yyy.com;
  ssl on;
  ssl_certificate /etc/elasticsearch/ssl/es_domain.crt;
  ssl_certificate_key /etc/elasticsearch/ssl/es_domain.key;
  access_log /var/log/nginx/elasticsearch/access.log;
  error_log /var/log/nginx/elasticsearch/error.log debug;
  location / {
    rewrite ^/(.*) /$1 break;
    proxy_ignore_client_abort on;
    proxy_pass http://localhost:9200;
    proxy_redirect http://localhost:9200 http://es.xxx.yyy.com/;
    proxy_set_header  X-Real-IP  $remote_addr;
    proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header  Host $http_host;
    auth_basic "Elasticsearch Authentication";
    auth_basic_user_file /etc/elasticsearch/user.pwd;
  }
}

server{
  listen 80;
  server_name es.xxx.yyy.com;
  return 301 https://$host$request_uri;
}

Kibana 测试

$ service kibana4 status

[ ok ] kibana is running.

/etc/nginx/sites-available/kibana 文件 =>

server {
  listen 443;
  server_name kibana.xxx.yyy.com;
  ssl on;
  ssl_certificate /opt/kibana/ssl/es_domain.crt;
  ssl_certificate_key /opt/kibana/ssl/es_domain.key;
  access_log /var/log/nginx/kibana/access.log;
  error_log /var/log/nginx/kibana/error.log debug;
  location / {
    rewrite ^/(.*) /$1 break;
    proxy_ignore_client_abort on;
    proxy_pass http://localhost:5601;
    proxy_redirect http://localhost:5601 http://kibana.xxx.yyy.com/;
    proxy_set_header  X-Real-IP  $remote_addr;
    proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header  Host $http_host;
    auth_basic "Kibana Authentication";
    auth_basic_user_file /etc/nginx/htpasswd.users;
  }
}

server{
  listen 80;
  server_name kibana.xxx.yyy.com;
  return 301 https://$host$request_uri;
}

Kibana 也可以在我的浏览器中从 localhost:5601 访问，没有任何问题。

Logstash 测试

$ sudo /etc/init.d/logstash status

[ ok ] logstash is running.

/etc/logstash/conf.d/01-ais-input.conf 文件 =>

input {
  file {
    type => "rails"
    path => "/srv/www/xxx.yyy.com/site/log/logstasher.log" 
    codec => json {
      charset => "UTF-8"
    }
  }
}

output {
  elasticsearch {
   host => 'localhost'
   port => 9200
  }
}

这些服务和配置文件有什么问题吗？每个组件看起来都运行良好，但我在 Kibana 界面中看不到任何内容。如何测试我的 ELK 堆栈？

Answer 1

您需要在 Kibana 中配置索引模式才能查看 elasticsearch 数据。

从浏览器打开 Kibana http://localhost:5601

点击设置

键入您现有的索引名称，然后单击创建。 (取消选中“索引包含基于时间的事件”选项，除非您的索引具有日志或任何基于时间戳的数据)

这样做，您必须能够看到所有的 elasticsearch 文档。