How to delay the number of requests when pulling records from an API

When transferring data from a source to a destination in our tool, records will go through one at a time.

If there are a lot of requests from an API, there is a chance that the records could be overlapping and not pulling through fully due to the request limit of the maximum number of records that the API can pull in a minute. This can be viewed in the Run Events of an executed run by looking at the time each record is taking to go through or by researching the maximum number of requests that the API can pull in a minute.

NB: Please note that the Delay step does slow down the processing time of a run by the number you input in the step.

Below is an image where the Run Events is situated in an executed run:

image

To resolve this issue, a Delay step can be used to delay the request for each record that the API is pulling according to the number of milliseconds that is inputted in the Delay step.

Illustrated in the below image is an example of a flow where the API is populated in the HTTP Service source step in a subflow:

NB: A reader step has to be added before you insert the HTTP Service source step.

To delay the number of requests from the API, the Delay step is inserted before the subflow of the HTTP Service source step as shown in the below image.

The below image is the Delay step where you can changed the number of milliseconds and save the change. You can increase or decrease the number of milliseconds by typing out a number or by using the up and down arrows which appears on the right corner of the field.

image

Additional information:

1 second = 1000 milliseconds
1 minute = 60000 milliseconds