c# - Azure Function HttpClient 不给出任何响应并不断重试

Question

我有一个在 Azure 中运行的函数应用程序。它是一个 HttpTrigger，由流分析触发，重试策略设置为 Drop。它采用 C# 编码，并通过 CI/CD 从 DevOps 进行部署。以下是一些配置。

FUNCTIONS_EXTENSION_VERSION: ~4
FUNCTIONS_WORKER_RUNTIME: dotnet-isolated
WEBSITE_RUN_FROM_PACKAGE: 1

该函数的目的是获取一些数据并使用 HttpClient(通过 HttpClientFactory 创建，超时时间为 15 秒)将其转发到不同服务器上的各种其他端点。到目前为止，一切都正常，直到最近添加了新服务器上的新端点。这个问题在某种程度上与这个新服务器有关，但我很难从我的终端获取任何有用的日志，而且整个函数在发送数据时表现得很奇怪。代码结束如下:

        log.LogInformation($"Preparing to send...");
        var content = new StringContent(jsonContent, Encoding.UTF8, "application/json");
        var httpRequestMessage = new HttpRequestMessage
        {
            Method = HttpMethod.Post,
            RequestUri = new Uri(uri),
            Headers = {
                { "Authorization", $"Bearer {token}" },
                { "Accept", "*/*" },
            },
            Content = content
        };
        var response = await _httpClient.SendAsync(httpRequestMessage);
        var responseString = await response.Content.ReadAsStringAsync();
        log.LogInformation($"Returned content: {responseString}.");

我使用 Application Insights 进行监控。 uri 参数根据发送数据的位置而有所不同，并且它适用于除 1 个端点之外的所有位置。当我尝试发送到这个“错误”端点时，这是我在 Application Insights 中看到的内容:

01:27:37 PM Trace: Preparing to send...
01:27:37 PM Trace: Start processing HTTP request POST https://... (URI parameter)
01:27:37 PM Trace: Sending HTTP request POST https://... (URI Parameter)
01:27:37 PM Trace: End processing HTTP request after 188.05ms - OK
01:27:37 PM Trace: Received HTTP response after 187.92ms - OK
01:29:17 PM Exception: Exception while executing function: Functions.MyFunction
01:29:17 PM Trace:Executed 'Functions.MyFunction' (Failed, Id=3d9afe5b-c58a-470d-8f54-0d98ca60deb7, Duration=100005ms
01:29:17 PM Exception: Exception while executing function: Functions.MyFunction

当我检查异常时，它说:

    Exception while executing function: Functions.MyFunction Result: Failure
Exception: System.AggregateException: One or more errors occurred. (A task was canceled.)
 ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Threading.Tasks.Task.GetExceptions(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionInvoker`2.<>c.<InvokeAsync>b__6_0(Task`1 t) in D:\a\1\s\src\DotNetWorker.Core\Invocation\DefaultFunctionInvoker.cs:line 32
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
   at System.Threading.Tasks.Task.ExecuteFromThreadPool(Thread threadPoolThread)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
--- End of stack trace from previous location ---

   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionInvoker`2.<>c.<InvokeAsync>b__6_0(Task`1 t) in D:\a\1\s\src\DotNetWorker.Core\Invocation\DefaultFunctionInvoker.cs:line 32
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionExecutor.ExecuteAsync(FunctionContext context) in D:\a\1\s\src\DotNetWorker.Core\Invocation\DefaultFunctionExecutor.cs:line 45
   at Microsoft.Azure.Functions.Worker.OutputBindings.OutputBindingsMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in D:\a\1\s\src\DotNetWorker.Core\OutputBindings\OutputBindingsMiddleware.cs:line 16
   at Microsoft.Azure.Functions.Worker.GrpcWorker.InvocationRequestHandlerAsync(InvocationRequest request, IFunctionsApplication application, IInvocationFeaturesFactory invocationFeaturesFactory, ObjectSerializer serializer, IOutputBindingsInfoProvider outputBindingsInfoProvider) in D:\a\1\s\src\DotNetWorker.Grpc\GrpcWorker.cs:line 167
Stack:    at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionInvoker`2.<>c.<InvokeAsync>b__6_0(Task`1 t) in D:\a\1\s\src\DotNetWorker.Core\Invocation\DefaultFunctionInvoker.cs:line 32
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionExecutor.ExecuteAsync(FunctionContext context) in D:\a\1\s\src\DotNetWorker.Core\Invocation\DefaultFunctionExecutor.cs:line 45
   at Microsoft.Azure.Functions.Worker.OutputBindings.OutputBindingsMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in D:\a\1\s\src\DotNetWorker.Core\OutputBindings\OutputBindingsMiddleware.cs:line 16
   at Microsoft.Azure.Functions.Worker.GrpcWorker.InvocationRequestHandlerAsync(InvocationRequest request, IFunctionsApplication application, IInvocationFeaturesFactory invocationFeaturesFactory, ObjectSerializer serializer, IOutputBindingsInfoProvider outputBindingsInfoProvider) in D:\a\1\s\src\DotNetWorker.Grpc\GrpcWorker.cs:line 167 

然后函数似乎会重试 Http 请求，我不明白为什么。 “准备发送”日志不存在，但“开始处理”日志会出现，并且每约 2 分钟就会失败一次。由于我的自定义日志消息不在这里，因此我认为是 HttpClient 以某种方式重试，而不是整个函数。

有人知道发生了什么事吗？为什么我的代码在尝试读取response.Content 时似乎卡住了？我如何进一步调查出了什么问题，以及为什么 HttpClient 像这样重试自身？如果 HttpClient 失败，我预计该函数会崩溃/停止。

编辑:更正日志上的时间戳以匹配持续时间。同时出现多条日志，我选错了。

Answer 1

很难提供帮助，因为我们无法访问您的环境，但我会尝试以下方法:

在我看来，日志的问题是因为它是一个异步操作，该日志并不代表内部/真正的异常。我建议您切换到同步操作并发送到此故障端点。一旦获得真正的异常/正确的修复，您就切换回异步操作。