- 05 Jul 2023
- 6 Minutes to read
Setting Up A Cluster
- Updated on 05 Jul 2023
- 6 Minutes to read
A Cluster is a configuration of one or more Application Servers running on the same Database backend. The Servers work together to appropriately distribute load/work so that one server is not over or underutilized.
The most common configuration is to have a Load Balancer sit in front of two Decisions Servers to distribute work to those servers. The following document discusses setting up a cluster using multiple v8 environments.
- Install Decisions on each instance in the environment and ensure both servers are pointed to the same Database Server.
Apply the appropriate Licenses on each server.
- After installing the second server, remote into both servers and configure the following settings in tandem; open DecisionsServerInstaller.exe and click EDIT SETTINGS to open the Settings.xml.
- In the Settings window, provide a matching FileStorageLocation so both servers will have full permission to store files.
The FileStorageLocation must match since the Keys.dat and PortalPublicKey.key files will autogenerate via FileStorageLocation > Settings. If both servers cannot view these files in this shared folder, they cannot connect.
- Set the ClusterAddressableIP to the Machine IP (e.g. 192.168.10.100) on each server, respectively.
If the Platform is hosted on a special Port or cannot connect to the server with just the ClusterAddressableIP, set the ClusterPortalBaseURL to the URL of the respective Node in a format of http://[SystemIP]:[Port] (e.g. http://192.168.10.100:8085).
- Set the PortalBaseUrl to the URL of the Load Balancer. This will be FQDN for the Server.If SSL is terminating, then the "ForceBaseURI" line will need to be configured with the below code example in the Settings.xml file, located at C:\Program Files\Decisions\Decisions Server\Settings.xml.
<ForceBaseURI> HTTPSAddress </ForceBaseURI>
- Via C:\Windows\System32\drivers\etc\hosts, configure a Host File Entry on both servers to resolve the Load Balance DNS Name to 127.0.0.1.
- Log into the Studio for each server. Navigate to and open System > Settings > Cluster Settings.
- From the Edit Clustering Settings window, enable Turn On Clustering, and provide the Redis Base URL. The Redis URL supports a complete connection string. Please refer to Stack Exchange's Redis Configuration article for all supported options in the connection string.
- Click Save.
- Lastly, Restart the Decisions Server either via RESTART SERVICE Installer or through Services.msc.Use HTTPS IssueThe Use HTTPS setting should be disabled before proceeding with the Cluster setup. If this setting is required, please contact the Support Team before proceeding.
In setting up a clustered environment, two or more Windows Servers will need to be fully configured with all of the following dependencies:
- IIS (Dynamic Compression must be enabled if IIS Hosted)
- Latest .NET Core
- An existing SQL Server
- Both Windows servers with the same Time Zone settings
- A network device or shared storage accessible to both servers
- A Redis Server setup/Account
- A fully configured Load Balancer. A sticky session Load Balancer configuration and cookie-based persistence are required.
- Ensure that all prerequisites have been met for an IIS Cluster.
- Follow each step outlined in the prior Self Hosted example, being sure to install each environment as IIS Hosted instead.
- When configuring the <ClusterPortalBaseURL>, append the virtual directory /decisions/ to it (i.e. [ClusterNodeDomain]/decisions/)
Service Account and Shared Storage Considerations
Decisions will need to be configured (either via DecisionsServerInstaller.exe or in Windows Services) to run as a Service Account.
Note that both servers need to be running the same Service Account; this allows the servers within that cluster to access a Network Shared Drive.
In some File Storage (IIS) configurations, the same Service Account running Decisions Server will also need to be configured in the Application Pool Identity. This can be done in IIS Manager via Application Pools > Advanced Settings > Identity.
Azure File Storage
If Azure is being used as the location for the Shared File Storage, the UNC File Storage path should bare similarity to "\\ServerName\testfileshare\filestorage\".
Before using Azure File Storage for storage on a Cluster, complete the following:
- Set up a Local Account
- Provide the Account with a Username that matches the one used for the Azure File Storage Account and a Password containing a value that matches the Azure Storage Account's Key.
- Change the Account Settings so that the Password does not expire and that users cannot change the Password.
- Add the Account to the IIS_IUSRS Group.
- Set the App Pool running Decisions to run under this Account.
Steps To Validate (For Both Hosting Types)
It is important to validate the steps followed in the previous section.
- Go to each server and verify that the Portal can be logged in from HTTP (unless intentionally removed) and HTTPS in a local browser session.
- Log in to each server in the cluster and verify that each one can write a file to the Shared File Storage.
- Check Load Balancer Timeout: try adding a document to a Folder over 100MB and see if it trips the LB Timeout threshold.
- Go through the Load-Balanced URL, locate the Servers Report, and find the server that has Is Me set to true to affirm which Server the Administrator is currently on.
Turn this server off and ensure that the redirection goes to the other (still active) server after the timeout duration configured in the Load BalancerSettings has passed.
- Ensure the Load BalancerHealth Check settings are up. Turn one server off and verify that it becomes red or negative on the Health Check Report, then repeat these same steps with any additional servers.
Begin turning back on each server to ensure that they return to a healthy state when online.
- Remote into each server, then open a browser and ensure that the IP Address of the other nodes in the cluster can be reached.
Repeat this process for every node in the cluster to ensure each node can reach the Login Page for every other node in the cluster from a local browser session.
- Add/update an Account on one node. Log in to the other node and ensure the changes appear there too.
- Go to a node in the cluster, log in to the Decision Studio, navigate to the MyApps Folder, and create a new Flow.
Log out and go to each additional node in the cluster to confirm that this Flow exists in the MyApps Folder when going straight to them.
- Then while in the MyApps Folder, create a new Flow that sends an email. Go straight to each node to ensure that the appropriate party receives the email sent from each local Flow run.
Updating a Decisions Cluster (For Both Hosting types)
Decisions Clustered environments support rolling upgrades in minor version upgrades ONLY and require testing in non-production clusters; this involves taking one server offline at a time, preventing service interruption, to perform the following actions.
1. Stop Decisions Server or iisreset /stop.
2. Run the Decisions Installer.
3. Start Decisions Server or iisreset /start (Once all Servers have been updated).
When using the <ClusterPortalBaseUrl>
- It is not required by default, but if not specified, the cluster will default communication to http://<ClusterAddressableIP> (The ClusterAddressIP iPv4 address can be obtained by running the ipconfig command in CMD/PS on the server.)
- If specified, Decisions will use it for communication and ignore ClusterAddressableIP.
- If HTTP is available, it can be left blank with no issues.
- If each node uses a valid certificate (not self-signed, for example, since the other node can't validate the cert for the first node if the other self-signed cert isn't moved over properly) and the HTTPS traffic is the only exposed method, then the HTTPS DNS name would need to be used instead.
- Can never be the same as <PortalBaseUrl>
How .connected files affect clusters
- If there are a lot of .connected files in the shared file storage location, there may be an issue with the cluster configuration. These files are used as Heartbeat data to report whether a Platform Server is connected to the cluster.
- A lot of .connected files would also increase the platform_server table. New files are generated when details cannot be matched from the machine with the data in the platform_server table.
- The naming schema of the files should be [platform. server.id].connected