stackdriver - 如何通过 Python 中的 Stackdriver Logging 将错误记录到 Stackdriver Error Reporting

Question

我已经使用 Stackdriver Logging 很长时间了，现在我还想利用错误报告。我更喜欢使用 Python 的日志记录机制，并在可能的情况下(出于各种原因)从日志文件中清除异常，而不是使用 error_reporting 库。话虽这么说，文档非常困惑。例如文档说: https://cloud.google.com/error-reporting/docs/setup/compute-engine#log_exceptions

First, install the fluent-logger-python library:

sudo pip install google-cloud-error-reporting --upgrade

这让我相信 google-cloud-error-reporting 是 fluence-logger-python 的一个分支或与之相关，但是，当我初始化 google-cloud-error-reporting 时，它直接调用 GCE 元数据服务器，而不是而不是连接到本地 fluidd。这两个不相关的软件包是否存在，或者文档是否错误或具有误导性？如果我将 JSON 格式的异常发送到 Fluentd 或由 Fluentd 监控的日志文件，错误报告会理解它们吗？

感谢您的澄清

Answer 1

文档有误。
TL;DR 您必须输出类似 https://cloud.google.com/error-reporting/docs/formatting-error-messages 的内容

这是我的解决方案:

#Parse raw log entries to expose severity field so that
#StackDriver log viewer can properly categorize (and so we can filter)
<source>
  @type tail
  path /var/log/conductor
  pos_file /var/log/td-agent/conductor.pos
  format multiline
  format_firstline /\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/
  format1 /^(?<message>(?<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}),\d*\s*[a-zA-Z_]*\s*(?<severity>[A-Z]*).*)/
  read_from_head true
  multiline_flush_interval 3s
  tag conductor.app
</source>

#Add hostname field
<filter conductor.app>
  @type record_transformer
  <record>
    hostname ${hostname}
  </record>
</filter>

#Filter and tag log entries of severity ERROR or CRITICAL
<match conductor.app>
  @type rewrite_tag_filter
  rewriterule1 severity ERROR|CRITICAL conductor.err
  rewriterule2 severity .+ conductor.info
</match>

#Process entries with tracebacks differently than those without
<match conductor.err>
  @type rewrite_tag_filter
  rewriterule1 message .*Traceback conductor.err.traceback
  rewriterule2 message .+ conductor.err.message
</match>

#Parse out the traceback
<match conductor.err.traceback>
  @type parser
  key_name message
  format multiline
  format1 /^(?<message>(?<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}),\d*\s*(?<log>[a-zA-Z_]*)\s*(?<severity>[A-Z]*).*(?
<traceback>Traceback .*))/
  tag conductor.err.traceback.report
</match>

#Format traceback reports
<filter conductor.err.traceback.report>
  @type record_transformer
  <record>
      serviceContext {
        "service": "${record[\"log\"]}"
      }
      message ${record["traceback"]}
  </record>
  remove_keys traceback
</filter>

#Process errors that don't have tracebacks
<match conductor.err.message>
  @type parser
  key_name message
  format multiline
  format1 /^(?<message>(?<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}),\d*\s*(?<log>[a-zA-Z_]*)\s*(?<severity>[A-Z]*):\s*(?<report>.*))/
  tag conductor.err.message.report
</match>

#For errors without tracebacks we have to stub out some fields that
#error reporting requires, but we don't have
<filter conductor.err.message.report>
  @type record_transformer
  <record>
      serviceContext {
        "service": "${record[\"log\"]}"
      }
      message ${record["report"]}
      reportLocation {
        "filePath": "None",
        "lineNumber": 0,
        "functionName": "None"
      }
  </record>
</filter>

#Send to StackDriver logging!
<match conductor.**>
  @type google_cloud
  buffer_chunk_limit 2M
  flush_interval 5s
  max_retry_wait 300
  disable_retry_limit
  num_threads 8
</match>