amazon-dynamodb-streams - DynamoDB 本地 : Streams. getRecords 返回 TrimmedDataAccessException

Question

我正在使用 DynamoDB local 从 nodejs (Javascript SDK) 运行集成测试。当我调用 getRecords 时，我收到“TrimmedDataAccessException”。如果我在 AWS(非本地)中针对 DynamoDB 运行相同的代码，它可以正常工作。步骤如下:

• 运行 DynamoDB
• 创建表“EventStore”
• 创建表“音乐”
• describeTable“EventStore”(获取LatestStreamArn”)
• describeStream 返回为“LatestStreamArn”(获取 ShardId)
• ShardIteratorType“最新”的 getShardIterator
• 将记录放入“音乐”表
• 将记录放入“EventStore”表
• 从“EventStore”流获取记录

“put”命令的顺序很重要。如果我首先输入“EventStore”，一切正常，但是当我首先输入“Music”时，它会失败。如果我更改顺序，我的应用程序逻辑将是错误的，因此仅更改顺序对我来说就是一个问题。

运行 DynamoDB

java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb -dbPath /data

创建表

{
    TableName: "EventStore",
    KeySchema: [
        { AttributeName: "EntityId", KeyType: "HASH" },
        { AttributeName: "Version", KeyType: "RANGE" }
    ],
    AttributeDefinitions: [
        { AttributeName: "EntityId", AttributeType: "S" },
        { AttributeName: "Version", AttributeType: "N" }
    ],
    ProvisionedThroughput: {
        ReadCapacityUnits: 10,
        WriteCapacityUnits: 10
    },
    StreamSpecification: {
        StreamEnabled: true,
        StreamViewType: "NEW_IMAGE"
    }
}

创建表(响应)

{
  "TableDescription": {
    "AttributeDefinitions": [
      {
        "AttributeName": "EntityId",
        "AttributeType": "S"
      },
      {
        "AttributeName": "Version",
        "AttributeType": "N"
      }
    ],
    "TableName": "EventStore",
    "KeySchema": [
      {
        "AttributeName": "EntityId",
        "KeyType": "HASH"
      },
      {
        "AttributeName": "Version",
        "KeyType": "RANGE"
      }
    ],
    "TableStatus": "ACTIVE",
    "CreationDateTime": "2016-07-14T15:36:42.895Z",
    "ProvisionedThroughput": {
      "LastIncreaseDateTime": "1970-01-01T00:00:00.000Z",
      "LastDecreaseDateTime": "1970-01-01T00:00:00.000Z",
      "NumberOfDecreasesToday": 0,
      "ReadCapacityUnits": 10,
      "WriteCapacityUnits": 10
    },
    "TableSizeBytes": 0,
    "ItemCount": 0,
    "TableArn": "arn:aws:dynamodb:ddblocal:000000000000:table/EventStore",
    "StreamSpecification": {
      "StreamEnabled": true,
      "StreamViewType": "NEW_IMAGE"
    },
    "LatestStreamLabel": "2016-07-14T15:36:42.895",
    "LatestStreamArn": "arn:aws:dynamodb:ddblocal:000000000000:table/EventStore/stream/2016-07-14T15:36:42.895"
}

描述表(EventStore)

{
    "TableName": "EventStore"
}

describeTable(EventStore) 响应

{
  "Table": {
    "AttributeDefinitions": [
      {
        "AttributeName": "EntityId",
        "AttributeType": "S"
      },
      {
        "AttributeName": "Version",
        "AttributeType": "N"
      }
    ],
    "TableName": "EventStore",
    "KeySchema": [
      {
        "AttributeName": "EntityId",
        "KeyType": "HASH"
      },
      {
        "AttributeName": "Version",
        "KeyType": "RANGE"
      }
    ],
    "TableStatus": "ACTIVE",
    "CreationDateTime": "2016-07-14T15:36:42.895Z",
    "ProvisionedThroughput": {
      "LastIncreaseDateTime": "1970-01-01T00:00:00.000Z",
      "LastDecreaseDateTime": "1970-01-01T00:00:00.000Z",
      "NumberOfDecreasesToday": 0,
      "ReadCapacityUnits": 10,
      "WriteCapacityUnits": 10
    },
    "TableSizeBytes": 0,
    "ItemCount": 0,
    "TableArn": "arn:aws:dynamodb:ddblocal:000000000000:table/EventStore",
    "StreamSpecification": {
      "StreamEnabled": true,
      "StreamViewType": "NEW_IMAGE"
    },
    "LatestStreamLabel": "2016-07-14T15:36:42.895",
    "LatestStreamArn": "arn:aws:dynamodb:ddblocal:000000000000:table/EventStore/stream/2016-07-14T15:36:42.895"
  }
}

描述Steam

{ 
    "StreamArn": "arn:aws:dynamodb:ddblocal:000000000000:table/EventStore/stream/2016-07-14T15:36:42.895" 
}

describeStream 响应

{
  "StreamDescription": {
    "StreamArn": "arn:aws:dynamodb:ddblocal:000000000000:table/EventStore/stream/2016-07-14T15:36:42.895",
    "StreamLabel": "2016-07-14T15:36:42.895",
    "StreamStatus": "ENABLED",
    "StreamViewType": "NEW_IMAGE",
    "CreationRequestDateTime": "2016-07-14T15:36:42.895Z",
    "TableName": "EventStore",
    "KeySchema": [
      {
        "AttributeName": "EntityId",
        "KeyType": "HASH"
      },
      {
        "AttributeName": "Version",
        "KeyType": "RANGE"
      }
    ],
    "Shards": [
      {
        "ShardId": "shardId-00000001468510602897-249806fa",
        "SequenceNumberRange": {
          "StartingSequenceNumber": "000000000000000000052"
        }
      }
    ]
  }
}

获取ShardIterator

{
  "ShardId": "shardId-00000001468510602897-249806fa",
  "ShardIteratorType": "LATEST",
  "StreamArn": "arn:aws:dynamodb:ddblocal:000000000000:table/EventStore/stream/2016-07-14T15:36:42.895"
}

getShardIterator 响应

{
  "ShardIterator": "000|arn:aws:dynamodb:ddblocal:000000000000:table/EventStore/stream/2016-07-14T15:36:42.895|c2hhcmRJZC0wMDAwMDAwMTQ2ODUxMDYwMjg5Ny0yNDk4MDZmYXwwMDAwMDAwMDAwMDAwMDAwMDAwNTJ8MDAwMDAwMDAwMDAwMDAwMDAxNDY4NTEwNjc2NjEy"
}

put(表名:音乐)

{
    TableName: "Music",
    Item: {
            Id: "000-000-000-000-000",
            Artist: "No One You Know",
            SongTitle: "Call Me Today"
        },
    "ConditionExpression": "attribute_not_exists(Artist) and attribute_not_exists(SongTitle)"
}

put(表名:EventStore)

{
  "TableName": "EventStore",
  "Item": {
    "EntityId": "000-000-000-000-000",
    "Version": 1468510676704,
    "Payload": [
      {
        "type": "Buffer",
        "data": [
            38,48,48,48,45,48,48,48,45,48,48,48,45,48,48,48,45,48,48,48,30,78,111,32,79,110,101,32,89,111,117,32,75,110,111,119,26,67,97,108,108,32,77,101,32,84,111,100,97,121
        ]
      }
    ],
    "Fingerprint": "8871e2afc3c31edfa9938e4cbb2b5",
    "Timestamp": 1468510676704
  },
  "ConditionExpression": "attribute_not_exists(EntityId) and attribute_not_exists(Version)"
}

getRecords (EventStore)

{ ShardIterator: "000|arn:aws:dynamodb:ddblocal:000000000000:table/EventStore/stream/2016-07-14T15:36:42.895|c2hhcmRJZC0wMDAwMDAwMTQ2ODUxMDYwMjg5Ny0yNDk4MDZmYXwwMDAwMDAwMDAwMDAwMDAwMDAwNTJ8MDAwMDAwMDAwMDAwMDAwMDAxNDY4NTEwNjc2NjEy" }

getRecords (EventStore) 响应

{
  "message": "The operation attempted to read past the oldest stream record in a shard.",
  "code": "TrimmedDataAccessException",
  "time": "2016-07-14T15:37:56.740Z",
  "requestId": "0e4d43ae-ac7d-419b-a1db-c8e2a955a877",
  "statusCode": 400,
  "retryable": false,
  "retryDelay": 30.214022053405643
}

Answer 1

我发现，如果在 getRecords 调用之后，再次调用 getShardIterator (使用相同的参数)，它就会起作用。

显然是本地实现中的一个错误。