Leased Entities

Overview

This tutorial demonstrates how to create and use Leased Entities. A Leased Entity is a simple Database Structure (User Defined) that can be used instead of a regular Database Structure (User Defined) in cases where large data sets need to be processed. Entity Leasing technology allows multiple Threads and Servers to process Data Items in parallel. This optimizes and speeds up the process. By Leasing Entities, memory is protected from overflowing compared to the process of Fetching all Entities and trying to process them all at once.

Example

In this example, create a User Defined Database Structure [Leased], create a Set of Entities of this type, and then use and explain Entity Leasing technology.

1. Begin in the Designer Folder by clicking Datatypes/Database > User-Defined Types > Database Structure [Leased] on the Folder Actions Panel.

   

2. In the Add Leased Data Structure dialog, define the Structure similarly to one would define a Database Structure. 
3. Name the Structure. 
4. Using the Add button, create Fields for the Structure. 
5. To create Actions for this structure, check the Enable Actions check box.
6. Select OK to save and continue.
   
7. Make a Flow that will create Entities of the Lease Data Structure type. In the Designer Folder, select CREATE FLOW on the Folder Actions Panel. 
8. In the resulting window, name the Flow and click CREATE to proceed to the Flow Designer.

   

9. To create Entities in this Flow, use a CSV file with 1000 records of randomly generated data.

   

10. In the Flow Designer, add a Create Data step from the Favorite Steps category.
    
11. In the Edit step, select the Create Data dialog, click Show Editor under Data Definition, name the data, and define its type as FileData. 
12. Select PICK to save and continue.
13. Click CLOSE to continue building the Flow.

    

14. Select the Create Data step on the workspace to define a Constant CSV File for the File Input.
    
15. After the Create Data step in the Flow, add a For Each Excel or CSV Row step from the All Steps [Catalog] > Data > List > Excel and CSV category.
    
16. Select the For Each Excel or CSV Row step on the workspace to set it up. 
17. In the Input Data section, set Input File Type - CSV, Ignore First Row – true, CSV Delimiter – Comma. 
18. In the Inputs section for File Input, choose Select Value of the file defined in the Create Data step. 
19. Connect the Done outcome from the For Each Excel or CSV Row step to the End step in the Flow.
    
20. On the Next Row outcome from the For Each Excel or CSV Row step, add Create NameLeasedStructure (generated by the System for the Leased Database Structure) from the Integrations > My Integrations > User Defined Types > Database Structures [Leased] > NameLeasedStructure category.
    
21. Select Show Mapping Editor for the Create NameLeasedStructure step.
22. In the Mapping Editor, choose Build Data for Item to Create.
    
23. Map the data of Fields.Text Value from the CSV Row to the properties of the NameLeasedStructure object. 
24. Save and close the Mapping Editor.
    
25. Back in the Flow Designer, connect the Done outcome from the Create NameLeasedStructure step back to the For Each Excel or CSV Row step. 
26. This completes the Flow. Save and close the Flow Designer.

Leased Entities in the Workflow Process

This example will demonstrate how to use Leased Entities in the Workflow process.

1. In the Designer Folder, create another Flow. 
2. Name the Flow and click CREATE to proceed to the Flow Designer.

   

3. In the Flow Designer, add a Go Async [Queue] step from the All Steps [Catalog] > Flow Management category. 
4. Use this step to run the Flow Asynchronously on the Work Queue.

   

5. Add a Lease Entity step from the Integrations > All Integrations > Entity Framework > Leasing category.

   

6. Select the Lease Entity step in the workspace to configure it. 
7. In the Entity Fetch Definition section, select the Leased Entity Structure as the Type Name.

   

8. Pick a Field for the Fetch Criteria and a condition. The Field being Fetched is not recommended to be an Entity ID.
   
   Lease Entities

   Lease Entities look up the chosen Type, get the first Entity and Lease it for the configured amount of time. Then, the Entity is either released or deleted. 

   
9. On the Leased Entity outcome from the Lease Entity step, add the [Start Linked Flow Async] step from the Flows, Rules, Forms, and Reports > Flows category.
10. Process the Leased Entity in this Flow. Use the Start Linked Flow Async step to be able to Lease several Entities at the same time on different threads.
    
11. Select Create or Pick Flow on the [Start Linked Flow Async] step.
    
12. Name the Flow and select CREATE to proceed to the Flow Designer on the resulting window.
    
13. In the Flow Designer, select the Start step in the workspace. 
14. Then Add the New Flow Input of the Leased Entity Structure Type.
    
    
    
15. Add String Flow Input data to be the LeaseID for the Leased Entity.
16. Add a Save Entities step from Integration > All Integrations > Entity Framework category after the Start step in the Flow.
    
17. Select the Save Entities step and pick the Leased Entity Structure as a Type for the Entity Save Definition.
    
18. Select Show Mapping Editor for the Save Entities step. In the Mapping Editor, Build Data for the Entity to Save.
19. Map all Fields from the Entity Flow Input to the Entity to Save except ID. 
20. For the ID Field, use the Merge Plain Text Mapping Type and click Show Editor.
    
21. In the Merge Text Editor, use the Entity ID Field along with Flow Data Start Time to modify the Entity ID. 
22. Click OK to save and continue. Then save and close the Mapping Editor.
23. Back in the Flow Designer, add a Release Lease step from the Integration > All Integrations > Entity Framework > Leasing category.
    
24. Select Show Mapping Editor for the Release Lease step.
    
25. In the Mapping Editor, map Entity Flow Input Data to the Entity Field, LeasedID Flow Input Data to the Leased ID Field, and set up a Constant time for the How Long Field. 
26. Save and close the Mapping Editor.
    
27. After the Release Lease step in the Flow, add a Log (Fast) step from the All Steps [Catalog] > Flow Management category.
    
28. Select Show Mapping Editor for the Log (Fast) step.
29. In the Mapping Editor, select Constant Warn level for Log and Ignore all Logs except log01. 
30. For log01, pick the Merge Plain Text Mapping Type and click Show Editor.
    
31. Use the Entity's Fields in the Merge Text Editor to write to the Log. 
32. When finished, click OK to save and close Merge Text Editor. Save and close the Mapping Editor.
33. Back in the Flow Designer, connect the Done outcome from the Log (Fast) step to the End step. 
34. This completes the Linked Flow. Save and close the Flow Designer.
    
35. In the Main Flow on the No Entity to Lease outcome from the Lease Entity step, add a Log (Fast) step from the All Steps [Catalog] > Flow Management category.
    
36. Connect the Done outcome from the Log (Fast) step to the End Step in the Flow and the Show Mapping Editor for the Log (Fast) Step.
37. In the Mapping Editor, pick Constant Warn level for Log; ignore all Logs except log01. 
38. For log01, define a Constant String saying, ' There are No Entities to Lease.' 
39. Save and close the Mapping Editor.

    

40. This completes the Main Flow. Save and close the Flow Designer.
41. In the Designer Folder, Run the Create Leased Entities Flow. Then, Run the Lease Entity Flow.
42. The Entities are being Leased and processed on a Workflow queue. Check this in the System > Administration > Systems > Jobs and Events on the Workflow Queues Page.