Background Processing with Decisions Enterprise
  • 15 Mar 2023
  • 3 Minutes to read
  • Dark
    Light

Background Processing with Decisions Enterprise

  • Dark
    Light

Article summary

Overview

The Enterprise Module incorporates the use of work queues to handle background processes in Decisions. The module introduces Flow steps that can be used to maintain and retrieve information about the work queues that are active.

Prerequisites
Installation of the Enterprise module in Decisions. For more information on how to install Modules in Decisions, see Installing Modules.

Work Queues

Background processing is handled in Decisions by putting work into queues and having Decisions servers work on tasks in those queues. If the Decisions Enterprise module is installed, then there is complete control over how the queues behave in the environment.

There are two main Decisions features that make use of work queues:

  • Scheduled Jobs (This includes systematically scheduled jobs like task reminders)

  • Background Workflows (otherwise known as Async Workflows or “Fire and Forget” in older releases)

Whenever one of these features is leveraged, their work task is stored in a queue. The Enterprise module comes with one queue once installed, the Default Queue. To configure additional queues the user must be an administrator and navigate to System > Jobs and Events > Work Queue Types.


Once multiple queues have been added, the administrator can specify which servers in the Decisions cluster are allowed to work on each of the queues. This can help with deciding what kind of background processing each of the servers may be responsible for. This practice is often used to create a dedicated "background processing" machine that is not connected to a load balancer or used for user interaction but works solely on queued tasks and batch processes. Selecting the queues that a server can pull work from is done via System > Administration > Servers. Select a server in the list and choose Edit Serverfrom the Action Menu for the window to appear.


Worker Profiles

In addition to constraining servers to work on specific queues, different worker profiles can also be designated that govern how each server in the cluster handles work that is stored in the queues. Worker profiles are maintained and edited at System > Jobs and Events > Work Queue Workers. These configurations will determine how and when the server checks for work. The screenshot below shows the configuration options available in the Worker Definition window with explanations of each setting to follow.


WorkQueueWorkerNameThe name of the profile. This should be something meaningful so that you know which servers should be assigned to that profile.
CheckTimespanWhenActiveThis is the amount of time that will pass between checks for new work in the queue when the worker is already active and has been processing work from the queue.
CheckTimespanWhenInactiveThis is the amount of time that will pass between checks for new work in the queue when the application server hasn't had any work to work on lately.
PickupItemsOlderThanThis setting specifies the minimum age of a task in the queue that this worker profile is allowed to pick up and work on. The purpose of this setting is to allow you to have servers that do not normally work on background processes to get involved only when the work queue is getting backed up and the servers that address work more aggressively are taking too long to get to tasks in the queue.
WorkerThreadsThis is the number of threads that the Decisions platform will be allowed to commit to background processing. The right value for this number depends on your hardware and the type of processing that you are doing.



Server to Queue Relationship

Scheduled Jobs: Workflows that are running in the Async mode.
serverAndQueues.png


Work Queues: Worker count and Queue count do not have to match.

serverWorkersQueues.png


In the Cluster, servers must know about one another, which allows the data that is cached to be kept in sync. When data is cached on Server 2, it notifies Server 1 and Server 3 that the data must be fetched again next time it is needed. Background Processing in the Cluster: Queues are defined for all servers in the Cluster. Each Server can identify which queues it will work on.
multipleServersOnTheCluster.png


Load Balanced for Users: A load balancer can direct users to any of the servers in the cluster to interact with the Portal.
loadBalancer.png


Background Processing in the Cluster: Queues are defined for all servers in the Cluster. Each Server can identify which queues it will work on. This means that if Server 1 is the main worker, but very busy on the job. Server B can be configured to check infrequently and when it sees the job over 30 minutes in the queue, it can start to work on it.

Common Example: In this Example, Server 1 and 2 are user portal servers behind a load balancer. Server 3 is the main worker.
exampleMain.png


Server 1Server 2Server 3
Has 3 worker threadsHas 3 worker threadsChecks for work every 5 seconds
Checks for work every 10 minutesChecks for work every 10 minutesRuns 8 worker threads
Only works jobs that are over 30 minutes oldOnly works jobs that are over 30 minutes old

exampleServerOne.pngexampleServerTwo.pngexampleServerThree.png




Was this article helpful?

What's Next