google-cloud-platform - 为 GKE 中的容器创建包含 'monitoring_v3' 的自定义指标时出错

Question

我们收到以下错误

"One or more TimeSeries could not be written: One or more points were written more frequently than the maximum sampling period configured for the metric.: timeSeries[0]"

当我们尝试在 GKE 中创建容器级别指标时。我们正在创建自定义指标来每秒更新 app_queue 长度。正常情况下效果很好。但是，当 Pod 规模超过 100 时，我们会收到某些 API 调用的此错误，而且数量非常多。

问题:我们是否缺少任何配额或设置？或者创建自定义指标的一些最佳实践？

堆栈跟踪:

      File "/usr/local/lib/python3.6/dist-packages/Base-1.0-py3.6.egg/base/custom_metrics.py", line 63, in create_custom_metric
        client.create_time_series(project_name, [series])
      File "/usr/local/lib/python3.6/dist-packages/google/cloud/monitoring_v3/gapic/metric_service_client.py", line 775, in create_time_series
        request, retry=retry, timeout=timeout, metadata=metadata)
      File "/usr/local/lib/python3.6/dist-packages/google/api_core/gapic_v1/method.py", line 139, in __call__
        return wrapped_func(*args, **kwargs)
      File "/usr/local/lib/python3.6/dist-packages/google/api_core/retry.py", line 260, in retry_wrapped_func
        on_error=on_error,
      File "/usr/local/lib/python3.6/dist-packages/google/api_core/retry.py", line 177, in retry_target
        return target()
      File "/usr/local/lib/python3.6/dist-packages/google/api_core/timeout.py", line 206, in func_with_timeout
        return func(*args, **kwargs)
      File "/usr/local/lib/python3.6/dist-packages/google/api_core/grpc_helpers.py", line 61, in error_remapped_callable
        six.raise_from(exceptions.from_grpc_error(exc), exc)
      File "<string>", line 3, in raise_from
    google.api_core.exceptions.InvalidArgument: 400 One or more TimeSeries could not be written: One or more points were written more frequently than the maximum sampling period configured for the metric.: timeSeries[0]

Answer 1

如果您达到任何配额限制，则应在 Stackdriver Logging 中记录错误。您还可以访问您的配额页面，查看是否达到任何限制 [菜单>IAM 和管理>配额]。

关于要遵循的规则，请看看这个 documentation详细了解如何将数据写入自定义指标类型。将多个数据点添加到同一时间序列时的最佳实践是添加速度不要超过每分钟一次。