How to integrate buffer data from one organization to another organization

[navessa]

Linking buffers when linking organizations does link the buffer schema but it does not integrate the buffer data/ values. In order to integrate the buffer data/ values, please follow the below instructions:

NB: It consists of creating 2 flows in the organization that has the buffer data and 1 flow in the organization that you want to integrate the buffer data to.

1. Create a flow with the steps illustrated in the below image in the organization that has the buffer data (organization 1). A blank Record Generator or a Start Flow can be used as the source step. Add the Sync Flow after the 2nd flow is created.

2. Add code in the calculator step to add a batch limit in order to send the buffer data/ records/ values in batches as shown in the below page. The batch limit in this example is 1000 and can be changed according to the number of records that is in the buffer. It works the same way as sending data in batches to a buffer or another destination step.

You can copy and paste the below code in your Calculator step:

let inputRecord=input.record;
let limit=1000;
return {
    limit:limit,
    skip:(input.rowNumber-1)*limit
};

3. Illustrated in the below image is the 2nd flow that needs to be created. Click on the box circled next to the Buffer source step to add input (flow) parameters.

4. The below page will appear. Add the 2 parameters and set the datatype as an integer. Click on the highlighted plus icon to add a new parameter and fill in the parameter details. Then click on the highlighted green tick on the far right of the field to confirm the parameter and then click on the Save button to save all your input/ flow parameters.

5. The Buffer source step will reference the buffer that you want to integrate the values from by selecting the required buffer. Reference the 2 parameters at the bottom of the Buffer source step by either clicking on the link icons next to the fields or using the @ symbol as highlighted in the below image.

6. Add a Sync Flow at the end of the 1st flow that was created. Illustrated in the below image is the configuration of the 2nd flow in the Sync Flow step. Input the parameters in the From Path fields by clicking on the highlighted pen icon.

7. In the organization that you want to integrate the buffer data to (organization 2), create the flow in the below image and select the buffer that you want the data to integrate to (it will be an insert operation). A JSON reader is used to pass the data from an API.

8. Create a Rest API using the Gateway API Builder. The below page is how the API configuration will look. It will reference the organization’s name (organization 2) in the API paths. The Default Security Scheme will need to be selected and saved after the security scheme is created by clicking on the plus icon. First create the required path and then add the security scheme.

9. Add a path by clicking on the highlighted plus icon next to Path as highlighted in the below image. Select the method POST and input the path extension (eg. /UploadToBuffer). Click on the Source tab, then the Flow tab and select the flow that was created which will import the buffer data. The Processing Mode will be Sync. The Return Source will be Auto, the Input Content Type will be JSON and the Output Content Type will be None. Please click on the Save button after these changes.

10. Click on the highlighted plus icon next to Security Schemes as per the below image, input a name and select an authentication type as a security scheme. We are using the API Key as the security scheme. Click on the Regenerate Key button to retrieve an API Key for the path. This API Key is different from the organization’s API Key. Please copy the API Key and paste it in a safe place as you will reference this API Key in organization 1.

11. Click on the Info page in the API page and select the Default Security Scheme that was created in the above instruction as shown in the below image. Click on the Save Info button to ensure that this change is saved.

12. In the HTTP Service destination step in flow 2 which is in organization 1, the inputted URL path will consist of the Service path from the API path in organization 2 with the path name (API path) and the API Key that was copied from the Security Scheme. The method will be a POST and the header will be a content-type to pass the data as JSON. The below image is an example of how the configuration in the HTTP Service destination step will look.

13. To integrate the buffer data, execute flow 1 in organization 1 and it will transfer the data by sending one record at a time to the selected buffer in organization 2.

Creating an organization link for linking organizations