API rate limiting and Streaming API endpoints - clarification

API Question
#1

Hi!

I want to get some clarification regarding the Public Data API rate limitatitons. Initially I found Rate Limiting Clarification, and it is clear to me that the 600 rate limit applies per API key. However, I can’t seem to find the number of API keys that can be (allowed to be) generated to an application (or, being bold - how many applications can be created by a registrated user). If there are existing docs on this question, please, link them.

Also, I am trying to get a similar picture to what I read on the Streaming API - How many streaming api keys do you need for simultaneous connection to all six streaming end points?. Here, while the documentations states that I can create 2 connections per account, it conflicts with the answer that 3 keys (all in one account) can connect to all the 6 endpoints.

Thanks in advance!

Shared Usage Query
Rate Limit for multi-tenant solution
#2

I don’t think there is any specific documentation about this. Multiple keys and applications are allowed for legitimate reasons (such as a test and live key etc), but using them to circumvent the rate limit is (based on previous posts by CH staff) frowned on. It has been stated that abusing keys to get around the rate limit can lead to being IP banned without warning. As well as the documented rate limit, there is an unpublished IP limit which I assume is used to police this. I remember them being rather alarmed by someone who had 50 keys or so at some point in the past!

When people ask similar questions they’re often referred to the following:

Acceptable use of Companies House Searchable Databases
The Companies House searchable databases are designed to meet the needs of a wide range of users wishing to interrogate our information on-line.
Due to limitations of equipment and bandwidth, they are not intended to be a source for bulk downloads.
Individuals, companies, IP addresses or blocks of IP addresses who deny or degrade service to other users by generating unusually high numbers of daily database accesses, whether generated manually or in an automated fashion, may be denied access to these services without notice.

I don’t use the streaming API so don’t know about that.

2 Likes
#3

In response to part one of of your query:-
The API rate limit of 600 requests in a 5 minute period is indeed linked to the API key. A user is allowed to establish multiple API keys but that is on the understanding that these are for separate applications - though, as mentioned above, use of a separate key for a test version of a service is reasonable. Use of multiple keys for the same, live application is not allowed and would be viewed as an attempt to circumvent rate limiting and make unreasonable use of the service, which could result in a service ban - unless this had been specifically agreed with Companies House. If you feel you have a legitimate need to exceed the rate limit then you should contact us to discuss, so that we can advise on the best approach, which may include the use of alternate Companies House services.

In order to protect the service, we also apply rate restrictions at an IP level - currently 2,000 in 5 mins for API traffic but we reserve the right to change this.

In both cases, if you breach the limit, you will get errors until a new, 5 minute period starts - 429 errors for an API Key rate breach and 403 errors for an IP limit breach. You should code your applications to not breach the rate limits at all and to back off gracefully if you do ie to avoid retrying and generating errors until the new period starts. If we see repeated breach of the limits from a particular application or IP, we may institute a service ban.

We want people to use our services and are keen to help you get the best out of them but we also have to ensure that all users get reasonable access and performance without some users dominating resources. Again, if you have legitimate issues with the rate limits in terms of using our services, please contact us to discuss so we can help identify the best approach to take.

Response to part 2 :-
The streaming API has a different type of limit on its keys, it uses a connection limit to ensure no more than 2 active connections are allowed per stream key at any time.
This 2 connection limit is currently being reviewed and will most likely be increased very soon to match the current number of streaming endpoints.

Hope this helps.

#4

Quite rightly as this limit doe not make sense on any level, at least from a consumer point of view. Shall we get an announcement here in the form of a sticky when the limit is removed?

#5

Limit has been removed as a trial. Yes, there will be a sticky notice issued after a short period of monitoring.

1 Like
#6

Hi.

Are there any updates regarding the Streaming API limits? It looks like more than 2 connections are possible now but the documentation doesn’t seem to have been updated yet.

Thank you very much in advance.

Streaming connection suddenly stops pushing events