java - Kinesis Firehose 客户端 putRecord 抛出 UnknownOperationException

Question

我正在从某个来源获取数据，现在想要将其提供给 Kinesis Firehose 传输流。我在 amazonKinesisFirehoseClient.putRecord(putRecordRequest); 处收到“状态代码:400；错误代码:UnknownOperationException”；

我的代码

while (data != null) {
      PutRecordRequest putRecordRequest = new PutRecordRequest();    
      putRecordRequest.setDeliveryStreamName("xxxxxxxxxx");
      String line = data + "\n";
      System.out.println("Data ---------> " + line);
      com.amazonaws.services.kinesisfirehose.model.Record recordData = createRecord(line);
      putRecordRequest.setRecord(recordData);
      System.out.println("Inside single row transfer 2222222222 --------------------");

      // Put record into the DeliveryStream
      amazonKinesisFirehoseClient.putRecord(putRecordRequest);
      System.out.println("Endddddd single row transfer --------------------");
}

输出

Inside single row transfer 111111111 --------------------

Data --------->{"name":"xxxxxxxx","id":xxx,"address":xxxxxxxx,"phoneNumber":"xxxxxxxxxx","age":xx}

Inside single row transfer 2222222222 --------------------

com.amazonaws.services.kinesisfirehose.model.AmazonKinesisFirehoseException:null (Service: AmazonKinesisFirehose; Status Code: 400; Error Code: UnknownOperationException; Request ID: a084lc4-a0ed-7f9a-a14a-867sdw23wr68)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.kinesisfirehose.AmazonKinesisFirehoseClient.doInvoke(AmazonKinesisFirehoseClient.java:895)
at com.amazonaws.services.kinesisfirehose.AmazonKinesisFirehoseClient.invoke(AmazonKinesisFirehoseClient.java:871)
at com.amazonaws.services.kinesisfirehose.AmazonKinesisFirehoseClient.executePutRecord(AmazonKinesisFirehoseClient.java:640)
at com.amazonaws.services.kinesisfirehose.AmazonKinesisFirehoseClient.putRecord(AmazonKinesisFirehoseClient.java:616)
at com.cloudwick.capstone.project.PersonDetailsRecordProcessor.processRecord(PersonDetailsRecordProcessor.java:142)
at com.cloudwick.capstone.project.PersonDetailsRecordProcessor.processRecords(PersonDetailsRecordProcessor.java:57)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.V1ToV2RecordProcessorAdapter.processRecords(V1ToV2RecordProcessorAdapter.java:42)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.ProcessTask.callProcessRecords(ProcessTask.java:215)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.ProcessTask.call(ProcessTask.java:170)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:49)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:24)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

我非常确定与 Kinesis Firehose 连接的所有凭据都是正确的。我的 S3 和 RedShift 在 Kinesis Firehose 中得到了很好的映射)

提前致谢:)

Answer 1

当我将端点配置为使用 https://kinesis.us-east-1.amazonaws.com 而不是 https://时，我看到过类似的错误firehose.us-east-1.amazonaws.com。尝试设置它看看是否有帮助。

此外，如果您使用的是支持它的 SDK 版本，则可以使用 AmazonKinesisFirehoseClientBuilder 并设置区域字符串以避免使用错误的终端节点:

AmazonKinesisFirehose firehoseClient = AmazonKinesisFirehoseClientBuilder
  .standard()
  .withRegion("us-west-2")
  .build()