In this guide, we'll walk you through the process of creating a Query in the App Connector Builder.
Let's first learn about Queries and their usage in an Integration Flow.
What is a Query?
In most of the integrations, a big chunk of data from one app needs to be imported to the other app once at the time of the integration execution. For example, if you are creating an integration to import and sync your contacts from Mailchimp, you will need to import all the existing contacts and their details from the Mailchimp account once.
Integry has created the Query activity type to serve this purpose of fetching records from an external app.
Query is a special type of activity and differs from a trigger or an action. A trigger can run at any time after the integration is created. An action is always executed in response to a trigger. A query usually runs only once per integration, when the initial importing of data takes place.
Query is a step that usually takes longer to complete than the other steps as it fetches a large number of records from another app. Queries are usually paginated and run as a loop in our Flows.
Creating a Query
In this article, we're going to create a Query for Github's App Connector as an example.
Let's get started.
Step 1: Go to the Activities tab in the App Connector Builder and select Query as the Activity type.
Step 2: Enter an appropriate name and description of the query, and click on 'Continue.'
Step 3: Configure Fields for your Query. Visit this article to learn how to configure different fields for your Activities. Go through your application's API documentation to learn which fields to add to your Query. For example, from Github's API documentation, we learned that we'd need to configure a username field for our Query to get repository data from Github.
After configuring your field(s), click on 'Continue.'
Step 4: Next, select your authorization type and connect your account to test your Query Field(s). Click on 'Continue.'
Step 5: Next, enter values for the field(s) you added for your Query. For example, we'll enter the username of the end-user whose repositories we wish to fetch using this Query. Then, click on 'Continue' to proceed.
Step 6: Next, you have to configure an API Call to fetch the data for your Query. For example, in this case, we need to fetch repositories from a Github account, so we'll configure a call to Github's API.
You have to configure an API call using GET method to the third-party API. Integry will make this API call on the behalf of an end-user to fetch data from their account.
Configure the Request URL (HTTP method, base URL, resource URI) and the Headers for your call. Go through the third-party application's documentation to learn how to configure this call. Next, we'll discuss Pagination.
Pagination
Any app where you want to get records from can have a lot of data. If all this data is accessed in a single call, it can overwhelm our systems. The backend engine’s memory can be exhausted as it maintains all that information. As a solution to this problem, many APIs provide a method to get a specific amount of data on a page-by-page basis. This process of pagination is generally used in actions such as bulk import.
Implementing Pagination in your Query
A paginated query fetches a page of records and runs all its child steps (in the Flow) one by one. It keeps fetching the data until all the pages are exhausted.
Here's how you can configure pagination for your Query.
Step 7: Turn the 'Pagination' toggle on when configuring your API call.
Following are the fields that you have to configure to enable pagination for your Query.
- Limit Parameter - This is the parameter that will pass the count/page size value to the URL. The name of this variable that exists in the API goes in the field parameter name. In the running example of Github, it is "per_page". The actual value of count i.e. how many repositories should be returned per page in one request goes in the field Limit value. If it is left blank, the count is set to the default value of the API.
- Offset Parameter - This is the key that will pass the number of records to skip from the overall dataset so we can get the subsequent records. In the running example of Github, it is "page". In Integry, the backend engine is configured to automatically determine its value. For example, if the Limit Parameter is set to 2 and it is the first call, the offset will be zero. For the next request, it will skip the first two repositories, so its value will be 2. Next value will be 4 and so on.
- Total Items Variable - An API usually tells us the total number of items or records in the response of every request. The name of that parameter can be given here. This is read by our execution engine. We use this to figure out the number of iterations to run. This is an optional field.
You can click on the 'Advanced Settings' to get more optional parameters.
- Page Size - APIs sometimes return the page size in the response body. We use this as the value of the Limit Parameter.
- Offset - APIs may also return the next offset value. We use this to specify the Offset Parameter.
- Next Link Parameter - Some APIs don’t have a Total Items Variable parameter and they return the URL for the next page instead. We can specify the parameter of that URL here.
- More Items Parameter - APIs can have a Boolean parameter that specifies that there are more records. We can give that parameter here.
- Increment Offset - Typically we increment the offset by the page size that is mentioned in the Limit Parameter field. You can set this to a constant such as 1 if your API uses page-based pagination.
Setting the Location of the Parameters
While setting the pagination fields, you can specify where to read the variables from, by selecting the Location of every field.
A variable can be sent via the Json body, URL, or Header.
The default Location selected for every parameter is Body. If you wish to change it, click on the default value and choose from the given options.
What's Next?
Next, you have to test your Query and check the response from the API of the third-party app.