Champion Challenger for Rules
- 24 Sep 2020
- 5 Minutes to read
- Print
- DarkLight
This documentation version is deprecated, please click here for the latest version.
Champion Challenger for Rules
- Updated on 24 Sep 2020
- 5 Minutes to read
- Print
- DarkLight
Article summary
Did you find this summary helpful?
Thank you for your feedback
Overview
Champion Challenger is a feature available within the Run Flow step and Rule step. This feature is implemented to test how changing logic affects the results/performance of the Flow and Rules. The idea is to compare two or more strategies to promote the one that performs the best. This document discusses this feature and covers how to implement it on a Rule step.
Champion Challenger
The Champion refers to the original element being used in the process, and the Challenger is an element that has the change that a user is testing against the Champion. The Champion/Challenger Rule will have this same Input and output. However, it will have different logic built-in. Two or more competing Decisions logics may be run to provide a Decision service. This is more advantageous than typical testing or reviewing previous data results, as it runs the implementations and shows how they work in real-time. The Champion Challenger in the decision service layer will distribute requests to each of these implementations randomly. The frequency of these implementations is determined by the Challenger Percentage setting. In the case of an A/B Output challenger, this setting is determined by the user, and in the case of an Output Champion/Challenger, the frequency is an equal amount. Essentially, a champion runs, then some Decisions logic challenges the champion; in the event that a challenger performs better, it eventually becomes the champion.
Decision refers to the resolution and process used to answer a question of logic. In the case of the Champion Challenger, the decision would be the result of a Flow or Rule, and the Champion or Challenger refers to the element used to determine that result.
Rule Champion Challenge Configuration
In a Parent Flow, add Rule Step. Navigate to the Step tab, search the "Rule" step, and drag and drop the step into the workspace. Click the step to configure the step. From the "Selection Type" drop-down select "Champion/Challenger" option. This will require the Input for the "Challenger Type". Please see the table below for more information.
| Input | Description |
|---|---|
| Target | This is the current rule that is standard. |
| Challenger | This is the rule that is being proposed as a possible replacement; Rule with the change that is being tested. |
| Challenger Percentage | This value determines how often the Challenger Rule will run. |
| Challenger Type | This Input decides what to do with the challenger. There are three types:
|
Example
Note: Even though the following example utilizes Forms as a means of Input source for the Champion Challenger, Input may also be sourced directly through the Debugger or with Flow Behaviors.
The following example demonstrates how the Champion Challenge feature works for Flows and Rules. The Parent Flow runs a simple transaction process. First, the user fills out a Form. This Form asks for the following: Item Name, Amount, Total Price. After this Form, a Rule step is used to evaluate the Amount. This step will be used as a Champion Challenge Rule setup. After this is the Champion Challenge Sub Flow. The last element is a Show Form that provides the following information: "Is the Rule Challenger Used", "Rule Challenger Results", "Champion Rule Results", "Total price", "Item type".
Champion Challenger Rules
Champion Rule checks evaluate the Amount value. For this example, it checks to see if the Amount is Greater than 100.
The Challenger Rule also evaluates the Amount Value and checks to see if it is Greater than 100, but the logic is different. It uses the Not operator.
Note
A Regular Rule step will have a False and True Path. Champion Challenge Feature does not provide different paths. It contains out Variables that provide boolean data. Output Data Various based on the Challenge Type selected. Below are the various examples that can occur.
A/B Output
Again, this will run either the Champion or Challenger based on the Challenge Percentage. The Output will be for whichever rule was run. The number of times the Challenger is used depends on the Challenger Percentage set on the step.
Example Challenger is used:
"Is Challenger" value indicates if the Challenger was used. In the example below, it is used, so the value is True. The Result is for the Challenger Rule.
Example Champion is used:
In the example below, the Champion is used, so the value is False for "Is Challenger". The Result is for the Champion Rule.
Output Champion/Challenger
This option will run both. The Output will be provided for both Rules. Challenger Percent is not required for this selection since both Rules will be run. "Result Challenger" is the Challenger's result. "Result" is the Champion Results.
A/B, Output Challenger
If the Champion runs, the Challenger gets returned as well. If Challenger runs, just Challenger is returned.
Example of Challenger Run
Example of Champion Run
Back in the Parent Flow, the total Price and Item type are used as Inputs for the Show Form. This Form will display the Champion and Challenger Rules results as well. For the example below, the Rule step was set to "A/B" Challenger type and the Challenger Percentage set to 50. Therefore, the Show Form will have only one value displayed either for the Champion result or Challenger result.
Note
Please note to test the actual performance of a process using either the Champion or Challenger Flow/Rule, it will take multiple iterations of the process to evaluate this information. The example above is not testing the performance but demonstrating how to use the feature and to display the various Outputs.
Debugging
To test the Flow, enter the Debugger via the Debug button from the top toolbar in the Flow Designer.
Run the Flow a few times to see whether the Champion or Challenger is used for each instance.
Notice that a pop-up appeared indicating that the Challenger was used.
To verify, view the Rule Input/Output in the debugger by right-clicking the step and selecting Input/Output. Notice how it states that the Challenger was not used. Therefore, the results are for the Champion Rule which was shown on the Show Form.
Was this article helpful?
