elasticsearch - 容器化WebApp的日志不会通过Serilog.Sinks.Elasticsearch发送到Elasticsearch

Question

我正在测试Serilog.Sinks.Elasticsearch库和ELK堆栈(Elasticsearch＆Kibana)的组合，以从我的ASP.NET Core 2.2应用程序中收集日志。 Web应用程序和ELK服务通过Docker进行了容器化。

通过docker-compose旋转所有容器时，一切都很好，但出于某种原因，Elasticsearch Sink不会将HomeController.cs中生成的日志推送到Elasticsearch。可以在http://localhost:9200/_cat/indices?v上检查，其中不会生成带有logstash- *的新索引。

有趣的是，当通过IIS(不在容器中)和容器中的ELK堆栈运行WebApp组合时，将生成日志并将其发送到Elasticsearch。我假设docker-compose.yml文件中的WebApp的端口或网络配置有问题。

我还尝试在appsettings.Development.json中使用elasticsearch(elasticsearch容器的实际名称)切换localhost，但没有任何效果。

有人可以帮助我解决这个问题吗？谢谢。

Docker-compose.yml文件:

version: '3.4'
services:
  elastic.serilog.web:
    image: ${DOCKER_REGISTRY-}elasticserilogweb
    build:
      context: .
      dockerfile: Elastic.Serilog.Web/Dockerfile

  elasticsearch:
   image: docker.elastic.co/elasticsearch/elasticsearch:7.2.0
   container_name: elasticsearch
   ports:
    - "9200:9200"
   volumes:
    - elasticsearch-data:/usr/share/elasticsearch/data
   environment:
    - discovery.type=single-node
    - xpack.security.enabled=false
    - xpack.monitoring.enabled=true
    - xpack.watcher.enabled=false
    - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
   networks:
    - docker-network

  kibana:
   image: docker.elastic.co/kibana/kibana:7.2.0
   container_name: kibana
   ports:
    - "5601:5601"
   depends_on:
    - elasticsearch
   environment:
    - ELASTICSEARCH_HOSTS="http://elasticsearch:9200"
    - XPACK_MONITORING_ENABLED=true
   networks:
    - docker-network

networks:
  docker-network:
    driver: bridge

volumes:
  elasticsearch-data:

Docker-compose.override.yml文件:

version: '3.4'
services:
  elastic.serilog.web:
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
    ports:
      - "80:7000"

Dockerfile:

FROM microsoft/dotnet:2.1-aspnetcore-runtime AS base
WORKDIR /app
EXPOSE 80

FROM microsoft/dotnet:2.1-sdk AS build
WORKDIR /src
COPY Elastic.Serilog.Web/Elastic.Serilog.Web.csproj Elastic.Serilog.Web/
RUN dotnet restore Elastic.Serilog.Web/Elastic.Serilog.Web.csproj
COPY . .
WORKDIR /src/Elastic.Serilog.Web
RUN dotnet build Elastic.Serilog.Web.csproj -c Release -o /app

FROM build AS publish
RUN dotnet publish Elastic.Serilog.Web.csproj -c Release -o /app

FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "Elastic.Serilog.Web.dll"]

Appsettings.Development.json:

{
  "Logging": {
    "LogLevel": {
        "Default": "Error",
        "System": "Error",
        "Microsoft": "Warning"
    }
  },
  "AllowedHosts": "*",
  "ElasticConfiguration": {
    "Uri": "http://localhost:9200/"
  }
}

Startup.cs:

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Serilog;
using Serilog.Exceptions;
using Serilog.Sinks.Elasticsearch;
using System;

namespace Elastic.Serilog.Web
{
    public class Startup
    {
        public IConfiguration Configuration { get; }

        public Startup(IConfiguration configuration, IHostingEnvironment hostingEnvironment)
        {
            var builder = new ConfigurationBuilder()
                .SetBasePath(hostingEnvironment.ContentRootPath)
                .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
                .AddJsonFile($"appsettings.{hostingEnvironment.EnvironmentName}.json", reloadOnChange: true, optional: true)
                .AddEnvironmentVariables();

            Configuration = builder.Build();

            var elasticUri = Configuration["ElasticConfiguration:Uri"];

            Log.Logger = new LoggerConfiguration()
               .Enrich.FromLogContext()
               .Enrich.WithExceptionDetails()
               .Enrich.WithMachineName()
               .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri(elasticUri))
               {
                   AutoRegisterTemplate = true,
               })
            .CreateLogger();
        }

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
        }

        public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
        {
             loggerFactory.AddSerilog();

            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                app.UseExceptionHandler("/Home/Error");
                app.UseHsts();
            }

            // app.UseHttpsRedirection();
            app.UseStaticFiles();
            // app.UseCookiePolicy();

            app.UseMvc(routes =>
            {
                routes.MapRoute(
                    name: "default",
                    template: "{controller=Home}/{action=Index}/{id?}");
            });
        }
    }
}

HomeController.cs

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Elastic.Serilog.Web.Models;
using Microsoft.Extensions.Logging;
using Serilog;

namespace Elastic.Serilog.Web.Controllers
{
    public class HomeController : Controller
    {
        ILogger<HomeController> _logger;
        public HomeController(ILogger<HomeController> logger)
        {
            _logger = logger;
        } 

        public IActionResult Index()
        {
            _logger.LogInformation($"oh hai there! : {DateTime.UtcNow}");

            Log.Error("Test Serilog!");

            try
            {
                throw new Exception("oops. i haz cause error in UR codez.");
            }
            catch (Exception ex)
            {
                _logger.LogCritical("ur app haz critical error", ex);
                _logger.LogError(ex, "ur code iz buggy.");

                Log.Information("Test Serilog!");

            }

            return View();
        }

        public IActionResult About()
        {
            ViewData["Message"] = "Your application description page.";

            return View();
        }

        public IActionResult Contact()
        {
            ViewData["Message"] = "Your contact page.";

            return View();
        }

        public IActionResult Privacy()
        {
            return View();
        }

        [ResponseCache(Duration = 0, Location = ResponseCacheLocation.None, NoStore = true)]
        public IActionResult Error()
        {
            return View(new ErrorViewModel { RequestId = Activity.Current?.Id ?? HttpContext.TraceIdentifier });
        }
    }
}

Answer 1

解决方案:

1.)确保docker-compose.yml中定义的所有服务都在同一网络中。在我们的情况下，我们必须将所有服务放在自定义的“docker-network”中。 (如果我们跳过定义自定义网络，则Docker将使用默认模式，它也处于桥接模式-因此，理论上所有容器仍将使用同一网络)。

2.)由于我们正在容器“localhost”中运行服务，因此与在appsettings.Development.json中寻址端口9200不再相关。我们需要将localhost更改为Elastic Search容器的实际名称-在我们的示例中为“elasticsearch”:
"ElasticConfiguration": {
"Uri": "http://elasticsearch:9200/"
}
经过这两个步骤之后，serilog-sinks-elasticsearch将所有日志推送到Elastic Search，并且进一步在默认的logstash- *索引中将在Kibana中可见。