Apache Kafka

Last Updated: 10/18/2018 Introduced in Version:

OVERVIEW

Apache Kafka is a messaging system that can be installed as a module allowing Decisions to integrate application to application communication. Kafka Message Queue settings are available and can be configured after install. For more information on Decisions Messaging Services see attached link: https://documentation.decisions.com/setting-messaging-overview/.

 

Install Module

Install Apache KafkaMQ by navigating to Designer Folders and locate System>Administration>Features to open up Decisions available features. Locate Apache Kafka module and select Install. After install, restart the Service Host Manager to complete the process. For information about restarting a server see the attached link: https://documentation.decisions.com/changing-server-settings-settings-xml/. Once you install the Apache KafkaMQ module the ability to add a Kafka Queue is now available in Decisions when you navigate to Systems>Jobs and Events>Message Queues and right click.

In the designer folder locate System>Settings, choose Kafka Message Queue Settings to display setting options and select Edit from the list. 

 

Edit KafkaMQ Settings

When Edit is selected a display shows with the main default connection settings. This page allows you to Add New Bootstrap servers as well as Add any Additional Configurations Options. 

  • Bootstrap Servers: Allows you to create a list of Bootstrap Servers to which you can add and connect. 
  • Additional config Option: Allows name/value pairs to be passed into the Confluent.Kafka library. 

  

 

Select Add New to set the Name and Value of the configuration options.     

 

Select Add New and add Bootstrap Servers, Host and Port.

 

In the designer folder locate Systems>Job and Events and select Message Queues. Locate Add Kafka Queue in the blue action panel across the bottom of the designer studio, select and begin setting the Display Name and Topic.  

Display name is whatever you want it to be to help allow you to remember.

Topic: Is a topic that exists on the server.

 

Check the box next to Show advance settings: This selection opens up an override for the connection settings.

 

Check the box next to Use Custom Connection info: Has the same settings as the Edit KafkaMQ Settings and it allows you to override the main queue using this queue. Check box: Allows you to add connection settings. 

 

Check the box next to Use SSL: Opens up SSL settings. 

 

Check the box next to Get Messages from Specific Partition: Allows you to split incoming messages, you can set it up to read from one users message queue and this setting allows you to specify a number.

Message Handling Preferences: Allows you to set when you mark a message complete. This can be before or after the message gets processed.  

Message Partition Configuration: This allows you to choose where the outgoing message is going. You can calculate the key based on the outgoing message. It opens up a flow and the flow is run every time a message is sent, (_keyFlow). Takes in a message as Byte Array and outputs the key as another Byte Array. 

 

 

In the designer folder locate Control, on the right side of the Add Kafka Queue setting in the blue action panel, choose Control  to select from the settings. 

For each installed message queue you have can select different actions. 

Resync Queues: Recreates the message queues if necessary and re-synchronizes with the message server.

Start Queues: Starts processing messages on the message queues (if stopped).

Stop Queues: Stops processing messages on the message queues.  The message server will hold any unprocessed messages.

Test Queues: Reads the number of waiting messages on the queue and shows a pop up with the amount of waiting messages.  An easy way to verify connectivity to a queue.

 

 

Additional Resources