Setting Up A Cluster
  • 21 Sep 2022
  • 6 Minutes to read
  • Dark
    Light

Setting Up A Cluster

  • Dark
    Light

A Cluster is a configuration of one or more Application Servers running on the same Database backend. The Servers work together to appropriately distribute load/work so that one Server is not over or underutilized.

The most common configuration is to have a Load Balancer sit in front of two Decisions Servers to distribute work to those servers. The following document discusses how to set up Cluster using multiple v8 environments.


Self Hosted

Prerequisites
To set up a Self Hosted Cluster, the following is required:
  • Latest .NET Core or .NET 6.0
  • An existing SQL Server
  • Both Windows Servers with the same Time Zone settings
  • A network device or shared storage accessible to both servers
  • A Redis Server setup/Account
  • A fully configured Load Balancer. A sticky session Load Balancer configuration and cookie-based persistence are required.
Note that the Redis Server/Account must be unique for each clustered setup as they cannot be used across multiple setups.
  1. Install Decisions on each instance in the environment and ensure that both servers are pointed to the same Database Server.
     
    Apply the appropriate Licenses on each server.  
  2. After installing the second server, remote into both servers and configure the following settings in tandem; open DecisionsServerInstaller.exe and click EDIT SETTINGS to open the Settings.xml
  3. In the Settings.xml window, provide a matching FileStorageLocation so both servers will have full permission to store files.

    The FileStorageLocation must match since the Keys.dat and PortalPublicKey.key files will autogenerate via FileStorageLocation > Settings. If both servers cannot view these files in this shared folder, they cannot connect.
  4. Set the ClusterAddressableIP to the IP of Server 1 for Server 1 and Server 2 for Server 2.

    If the Node is hosted on a special Port or can connect to the server with just the ClusterAddressableIP, set the ClusterPortalBaseURL to the URL of the respective Node. 
  5. Set the PortalBaseUrl to the URL of the Load Balancer.
    If SSL is terminating, then the ForceBaseURL line will need to be configured with the below code example in the Settings.xml file, located at C:\Program Files\Decisions\Decisions Server\Settings.xml.
    <ForceBaseURI> HTTPSAddress </ForceBaseURI>
  6. Via C:\Windows\System32\drivers\etc\hosts, configure a Host File Entry on both servers to resolve the Load Balance DNS Name to 127.0.0.1.
  7. Log into the Studio for each server. Navigate to and open System > Settings > Cluster Settings.
  8. From the Edit Clustering Settings window, enable Turn On Clustering, and provide the Redis Base URL. The Redis URL supports a complete connection string. Please refer to Stack Exchange's Redis Configuration article for all supported options in the connection string.
  9. Click Save
  10. Lastly, Restart the Decisions Server either via RESTART SERVICE Installer or through Services.msc.
    Use HTTPS Issue
    The Use HTTPS setting should be disabled before proceeding with the Cluster setup. If this setting is required, please contact the Support Team before proceeding. 


IIS Hosted

Prerequisites 

In setting up a clustered environment, two or more Windows Servers will need to be fully configured with all of the following dependencies:

  • IIS (Dynamic Compression must be enabled if IIS Hosted)
  • Latest .NET Core
  • An existing SQL Server
  • Both Windows servers with the same Time Zone settings
  • A network device or shared storage accessible to both servers
  • A Redis Server setup/Account
  • A fully configured Load Balancer. A sticky session Load Balancer configuration and cookie-based persistence are required.
Note that the Redis Server/Account must be unique for each clustered setup as they cannot be used across multiple setups.

  1. Ensure that all prerequisites have been met for an IIS Cluster.
  2. Follow each step outlined in the prior Self Hosted example, being sure to instead install each environment as IIS Hosted.
  3. When configuring the <ClusterPortalBaseURL>, append the virtual directory /decisions/ to it (i.e. [ClusterNodeDomain]/decisions/)

Service Account and Shared Storage Considerations

Decisions will need to be configured (either via DecisionsServerInstaller.exe or in Windows Services) to run as a Service Account.

Note that both servers need to be running the same Service Account; this allows the servers within that Cluster to access a Network Shared Drive.

In some File Storage (IIS) configurations, the same Service Account running Decisions Server will also need to be configured in the Application Pool Identity. This can be done in IIS Manager via Application Pools > Advanced Settings > Identity

Azure File Storage 

If Azure is being used as the location for the Shared File Storage, the UNC File Storage path should bare similarity to "\\ServerName\testfileshare\filestorage\". 

SSO Interference with Load Balancer Health Checks 
Follow the link for more on SSO interference with Load Balancer Health Checks

Before using Azure File Storage for storage on a Cluster, complete the following:

  1. Set up a Local Account
  2. Provide the Account with a Username that matches the one used for the Azure File Storage Account and a Password containing a value that matches the Azure Storage Account's Key. 
  3. Change the Account Settings so that the Password does not expire and that users cannot change the Password.
  4. Add the Account to the IIS_IUSRS Group
  5. Set the App Pool running Decisions to run under this Account.

Steps To Validate (For Both Hosting Types)

It is important to validate the steps followed in the previous section.

  1. Go to each server and verify that the Portal can be logged in from HTTP (unless intentionally removed) and HTTPS in a local browser session.
  2. Log in to each server in the Cluster and verify that each one can write a file to the Shared File Storage
  3. Check Load Balancer Timeout: try adding a document to a Folder over 100MB and see if it trips the LB Timeout threshold.
  4. Go through the Load-Balanced URL, locate the Servers Report, and find the server that has Is Me set to true to affirm which Server the Administrator is currently on.

    Turn this server off and ensure that the redirection goes to the other (still active) server after the timeout duration configured in the Load BalancerSettings has passed.
  5. Ensure the Load BalancerHealth Check settings are up. Turn one server off and verify that it becomes red or negative on the Health Check Report, then repeat these same steps with any additional servers.

    Begin turning back on each server to ensure that they return to a healthy state when online.
  6. Remote into each Server, then open a browser and ensure that the IP Address of the other nodes in the Cluster can be reached.

    Repeat this process for every node in the Cluster to ensure each node can reach the Login Page for every other node in the Cluster from a local browser session.
  7. Add/update an Account on one node. Log in to the other node and make sure the changes appear there too.

       Cluster Validation-14112019-030819.zip  

  8. Go to a node in the Cluster, log in to the Decision Studio, navigate to the MyApps Folder, and create a new Flow.

    Log out and go to each additional node in the Cluster to confirm that this Flow also exists in the MyApps Folder when going straight to them.
  9. Then while in the MyApps Folder, create a new Flow that sends an email. Go straight to each node to ensure that the appropriate party receives the email sent from each local Flow run.

Updating a Decisions Cluster (For Both Hosting types)

Decisions Clustered environments support rolling upgrades; this involves taking one Server offline at a time, preventing service interruption, to perform the following actions:

Upgrading Considerations
For more information regarding upgrading Decisions, consider the following documents:

1. Stop Decisions Server or iisreset /stop.

2. Run the Decisions Installer.

3. Start Decisions Server or iisreset /start (Once all Servers have been updated).


Best Practices

When using the <ClusterPortalBaseUrl>

  • It is not required by default, but if not specified, the cluster will default communication to http://<ClusterAddressableIP> (The ClusterAddressIP iPv4 address can be obtained by running the ipconfig command in CMD/PS on the server.)
  • If specified, Decisions will use it for communication and ignore ClusterAddressableIP.
  • If HTTP is available, it can be left blank with no issues.
  • If each node uses a valid certificate (not self-signed, for example, since the other node can't validate the cert for the first node if the other self-signed cert isn't moved over properly) and the HTTPS traffic is the only exposed method, then the HTTPS DNS name would need to be used instead.
  • Can never be the same as <PortalBaseUrl>

How .connected files affect clusters

  • If there are a lot of .connected files in the shared file storage location, there may be an issue with the cluster configuration. These files are used as Heartbeat data to report if a Platform Server is connected to the cluster or not.
  • A lot of .connected files would also increase the platform_server table. New files are generated when details cannot be matched from the machine with the data in the platform_server table.
  • The naming schema of the files should be [platform.server.id].connected


For further information on Installation, visit the Decisions Forum.

Was this article helpful?