elasticsearch - 使用Elasticsearch的Logstash:我正在使用Logstash将数据提取到ES索引中。但是现在我希望logstash运行24/7

Question

    #file:db.conf

input { 
    jdbc { 
           jdbc_driver_library => ""
           jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver" 
           jdbc_connection_string => "jdbc:oracle:thin:@abcd.klm.uvw:1521/qtp1"
           jdbc_user =>"user_wew"
           jdbc_password => "password_wew"

           
           
                  statement => "select col1, col2, col3, col4, col5, col6, countid,max(version) as mv from master_object_table where version >:sql_last_value group by countid"

            schedule => "* * * * *"
           last_run_metadata_path => "C:/ES1/ELK_stack_7.4.2/logstash-7.4.2/logstash-7.4.2/Master_refresh_a.txt"

           use_column_value => true

            tracking_column => "version"

         } 
       }
       
        filter {
      mutate {
        convert => {
          "countid" => "string"
          }
          }
          }
       
output {
  elasticsearch { 
           hosts => "localhost:9200" 
           index =>"refresh_index_a"
           document_id =>"%{countid}"
           #document_type="_doc"
           
           }
 file {
         path => "C:\\ES1\\ELK_stack_7.4.2\\logstash-7.4.2\\logstash-7.4.2\\bin\\logstashESRecordsIngestionDetails_refresh_a.txt"
         
         codec => rubydebug
       }
           
  stdout {   codec => rubydebug }
}

上面是我的logstash配置文件设置。我要运行24/7的logstash，并且还要在运行该logstash的机器关闭的情况下运行，那么如何管理该Logstash，因为此Logstash正在将实时数据提取到ES索引中。请提出建议。 如果一台服务器关闭另一节点上的logstash，是否有任何方法可以继续工作。

Answer 1

按照documentation

Logstash is horizontally scalable and can form groups of nodes runningthe same pipeline. Logstash’s adaptive buffering capabilities willfacilitate smooth streaming even through variable throughput loads. Ifthe Logstash layer becomes an ingestion bottleneck, simply add morenodes to scale out. Here are a few general recommendations:

1. Beats should load balance across a group of Logstash nodes.
2. A minimum of two Logstash nodes are recommended for high availability.
3. It’s common to deploy just one Beats input per Logstash node, but multipleBeats inputs can also be deployed per Logstash node to exposeindependent endpoints for different data sources.