8000 failing to use dynamodb for multiple shard consumption · Issue #13 · patrobinson/gokini · GitHub
[go: up one dir, main page]

Skip to content

failing to use dynamodb for multiple shard consumption #13

Open

Description

@pdavis77

Hi, firstly, thank you for gokini and also for integrating with Benthos!

I'm running in a bit of trouble using benthos with gokini 0.1.0 and running in to this error when using the kinesis_balanced module:
we verified dynamodb access using the same credentials and table name, but this error is not very clear - hoping you can help @patrobinson Thanks in advance!

Failed to start Checkpointer: SerializationError: failed to unmarshal response error\n\tstatus code: 403, request id: fe7b59d5-d0c4-a717-a91e-d159f61c25ac\ncaused by: UnmarshalError: failed decoding error message\n\t00000000  3c 49 6e 76 61 6c 69 64  53 69 67 6e 61 74 75 72  |<InvalidSignatur|\n00000010  65 45 78 63 65 70 74 69  6f 6e 3e 0a 20 20 3c 4d  |eException>.  <M|\n00000020  65 73 73 61 67 65 3e 43  72 65 64 65 6e 74 69 61  |essage>Credentia|\n00000030  6c 20 73 68 6f 75 6c 64  20 62 65 20 73 63 6f 70  |l should be scop|\n00000040  65 64 20 74 6f 20 63 6f  72 72 65 63 74 20 73 65  |ed to correct se|\n00000050  72 76 69 63 65 3a 20 27  6b 69 6e 65 73 69 73 27  |rvice: 'kinesis'|\n00000060  2e 20 3c 2f 4d 65 73 73  61 67 65 3e 0a 3c 2f 49  |. </Message>.</I|\n00000070  6e 76 61 6c 69 64 53 69  67 6e 61 74 75 72 65 45  |nvalidSignatureE|\n00000080  78 63 65 70 74 69 6f 6e  3e 0a                    |xception>.|\n\ncaused by: invalid character '<' looking for beginning of value"}

ref ticket in benthos: redpanda-data/connect#457

Activity

  1. pdavis77 commented on Jun 19, 2020

    @pdavis77
    Author

    benthos config for review

    input:
          kinesis_balanced:
            stream: "ABCDEFGHI"
            dynamodb_table: "X.ABCDEFGHI"
            start_from_oldest: true
            region: "us-east-1"
            endpoint: "https://helloworld"
            credentials:
              id: "ASDF"
              secret: "pqrstuv"
            batching:
              count: 10
              byte_size: 0
              period: "1s"
              condition:
                static: false
                type: static
              processors:
              - decode:
                  scheme: base64
              - decompress:
                  algorithm: gzip
              - json:
                  operator: select
                  path: lmnop
              - unarchive:
                  format: json_array
              - type: split
              - json:
                  operator: set
                  path: xvt
                  value: "zyxwvut"
    
  2. patrobinson commented on Jun 21, 2020

    @patrobinson
    Owner

    @pdavis77 the error message (although poorly encoded) comes out as:

    Credential should be scoped to correct service 'kinesis'

    https://forums.aws.amazon.com/thread.jspa?threadID=222499

    aws/aws-sdk-go-v2#163

    I wonder if this is a regression in the AWS SDK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0