Rate Limits

Let’s brush up on some basic information before getting into how Integry responds to API rate limits by request throttling.

What are API Rate Limits?

APIs have rate limits that determine the number of API calls that can be sent to them in a certain amount of time. If we ignore these limits, the server might ignore our requests or our API calls may altogether fail. So in order to follow the rate limit, we need to slow down the rate at which we send requests. This is called throttling.

Benefits of API Rate Limiting

API rate limits ensure the safety of the API. They can protect you against slow performance and denial of service attacks. Rate limiting also helps make your API scalable, otherwise, there can be unexpected spikes in traffic, causing severe lag time.

How do Rate Limits Work?

Rate limits control the amount of incoming or outgoing traffic to or from a network. For example, if an API rate limit is set to be 100 requests per minute and if that number is ever exceeded, it will generate an error response that the API limit for a time period is reached and ask the requester to retry after a specific time period. Usually, this error is a response with a 429 status code which is for “Too Many Requests”.

What is API Throttling?

Throttling is a process that is used to control the usage of APIs by consumers during a given time period. You can define throttling at the application level and API level. Throttling offers an extra layer of protection for your backend resources to which APIs are a gateway. It also helps in enhancing performance and improving the end-user experience by making sure that a single user is not suffocating your applications. 

How we Throttle Requests in Response to API Rate Limits

Integry implements request throttling in response to API rate limits when they are reached. It is important to know that each integration on Integry gets a separate queue which is used to execute all of its jobs so that it does not negatively affect the processing rate of any other integration on the system. So that when one job is being executed, the rest of them wait in the queue. 

If the rate limit is reached and the server responds with a 429 (Too Many Requests) status code, we check whether the Retry-After header is also present on the response. If it is present, we throttle the queue of that specific integration by the value of the header e.g. 1 min.  This means that the requests waiting on the queue of this specific integration are not sent out for execution for 1 min. If the Retry-After header doesn’t appear, the queue is still throttled for 1 min. If the 429 error still appears after 1 min, we keep throttling the queue based on powers of 2 i.e. 2 min, 4 min, 8 min, 16 min up until 30 min.

If the error continues to appear after 30 min, we keep retrying after every 30 mins. If the queue remains throttled for 36 hrs continuously, we flush all the messages that are older than 36 hrs.

When a message from the throttled queue is processed, and it doesn’t face a 429 error, the queue for that particular integration is resumed at a normal processing rate immediately. 

This implementation is done across the board for all users and integrations no matter what pricing package you are on. If a task fails due to a 429 error i.e. “Too Many Requests”, it appears in the integration view with a 429 status code and a tooltip saying “Too Many Requests” upon hovering as shown in the image below.