- 26 May 2022
- 2 Minutes to read
- Updated on 26 May 2022
- 2 Minutes to read
|Installation Location||Core Module|
|Settings Location||System -> Settings Kafka Message Queue Settings|
The Kafka module provides integration with Apache Kafka to read and write messages to a message queue. The Message Handler for Kafka can only retrieve a single message at a time.
To retrieve more than one message a Flow must be configured using a ForEach step.
Kafka Module Settings
- Navigate to System > Settings, then either select Kafka Message Queue Settings or right-click its name in the Report and select Edit.
- From the Edit Kafka MQSettings window, under Bootstrap Servers, click ADD. From the Add Bootstrap Servers screen, provide the Host and Port for the Server then click OK.
If Apache Kafka is using SSL, check the Use SSL box, then provide the CA File Path, Cert File Path, and Private Key File Path that correspond to the SSL Certificate.
If needed, click ADD under Additional Config Options. From here provide the Name and Value pairs, such as the SSL keys. Then, click OK and SAVE.
Name Value socket.keepalive.enable true
Adding a Kafka Message Queue
To connect to a Kafka Message Queue, navigate to System > Integration > Jobs and Events > Message Queues and select the ADD KAFKA QUEUE from the Action Bar.
Kafka Message Queue Properties
|Display Name||The name which will appear in Decisions.|
|Topic||The name of the Topic in Kafka the messages will be sent to.|
|Show Advanced Settings||Displays advanced settings that can be used to configure how messages will be sent and received.|
|Use Custom Connection Info||Allows users to override the Server settings defined from the Edit KafkaMQ Settings window including the ability to add Bootstrap Servers and denote SSL Settings.|
Allows users to create a list of servers that can be added and connected to. The Host and Port should match what is running for Apache Kafka. This is typically the host's DNS name or IP Address and the default or user-declared Port number.
|Use SSL||If enabled, allows the user to specify the SSL certificate.|
|Consumer Group ID||Consumer group identifier.|
|Get Messages from Specific Partition||If enabled, splits incoming messages from a specific partition.|
|Get from Partition||Specifies which partition to pull the messages from.|
|Message Handling Preference||Specifies how messages will be received and handled. The two options are At Least Once or At Most Once.|
|Message Partition Configuration||Designates where an outgoing message will be sent. Currently, there are three options:|
|Always Retry After Connection Errors(Dangerous)||By design, the Kafka Module will stop and prevent message queues from creating threads if 20 consecutive attempts have failed. If this setting is enabled, messages will continue to create threads even after reaching the attempt count.|
Kafka Message Queue Controls
- Navigate to System > Jobs and Events > Message Queues.
- On the Global Action Bar, select CONTROL > Kafka, then the desired action.
|Resync Queue||Recreates the Message Queues if necessary and re-synchronizes with the Message Server.|
|Start Queue||Starts processing messages on the Message Queues (if Stopped).|
|Stop Queue||Stops processing messages on the Message Queues. The Message Server will hold any unprocessed messages.|
|Test Queues||Reads the number of waiting messages on the Queue and displays a popup with the number of waiting messages. This action helps verify connectivity in a Queue.|