Kinesis

Updated on 30 Jun 2022
1 Minute to read

Print
Share
Dark
Light

Article Summary

Share feedback

Thanks for sharing your feedback!

The Amazon Web Services (AWS) module allows users to integrate with AWS's Kinesis Message Queues. Connection to Kinesis queues is configured entirely in the steps.

Prerequisites

Installation of the AWS module.
Active AWS account and Kinesis queue.
An Access Key ID and Secret Access Key configured in AWS Module settings.

Steps

Put Record Step

The Put Record step allows individual messages to be added to an AWS Kinesis data stream. Below are the inputs listed for the step:

Inputs	Description
Data	List of Byte Data that will be passed as a record to the stream.
Hash Key	Optional string representing the hash key.
Partition Key	A Unicode string representing the partition key. This is used to group data by shard within a stream and can be a maximum of 256 characters.
Region	The region the data stream resides in. This can be found in the ARN for the data stream.
Sequence Number	Optional string used for the sequence number. The sequence number is unique for each partition key within a shard, and will increase each time the partition key is used.
Stream Name	The name of the data stream.

Example

The following example covers how to use the Put Records step to send a record to an AWS Kinesis Data Stream.

Verify that an Access Key ID and Secret Access key have been applied in AWS module settings.
Create a Flow and in the Toolbox, search and select the Convert String to Byte Array step. Define a string to be converted.
In the Toolbox, expand Integration > AWS > Kinesis and select the Put Record step.
With the Put Record step selected define the inputs.
For the Data input, a list of Bytes is needed. For this example, this is represented using the output of the Convert String to Bytes step.
For the Region input, change this input to constant then select the pencil icon and enter the region found in the ARN for the data stream. In this case, it is us-east-1.
Debug the Flow. A shard id should be returned as an output when selecting the step.