API usage and website beta 403 Forbidden

[chrisW]

Hi,

I work for a large organisation that requires to undertake a significant number of checks on companies house each day (I believe it is thousands per day).

From time to time we have issues on the website where a 403 Forbidden HTML response is received. Out IT teams are having dialogue with companies house about this - I believe there is a restriction on our IP address making calls to the website.

I have created a simple application to check companies status which due to volumes I am utilising the companies house API. I have set up the call so it stops if it breaches the 600 limit per 5 minutes of API calls and can then be ran again a few minutes later to continue from where it stopped. When running this I found that the beta search website was returning a 403 restricted response to all users in our organisation (single IP address I assume).

What I wanted to get confirmation on is would putting volumes through the API impact 403 restrictions on the website? The calls will be originating from our same API but obviously calling slightly different address for the companies house service.

Thanks

Chris

[voracityemail]

In your “simple application” (and your main one) you should be handling this using the rate limit headers CH send (on the http responses) rather than just counting up to 600. You may well be doing this already of course! Some info in and linked from this thread:

View your request count for rate limiting

Are you referring to the Streaming API or the main/document REST API? Main API Although not listed in the documentation: …I thought this was available via the http headers? I thought the X-Ratelimit-... fields were what we should use for this purpose - or will things change? Roughly as in the thread: For what it’s worth I just checked and this still seems functional: curl -uMY_API_KEY: -I "https://api.companieshouse.gov.uk/company/NF004299" HTTP/1.1 200 OK Date: Wed, 19 Feb 2020 23:3…

Not from Companies House myself but if I recall they certainly do take IP address into consideration (including requests to both the API and website access). I can’t find the specific item on this that I remember but from a couple of years ago this thread has a response:

New IP Address Rate Limiting Query

With the introduction of the new IP address rate limiting, I was hoping someone could clarify the following points - Does each IP now get the 600 calls in 300 second limit or have these numbers changed? Is there any impact on how many API keys are used? e.g. if we use separate keys for different end points in the API does the combined number of calls still have to meet the 600/300 seconds or does each key have any form of independent limiting? Previously, the document API had its own rate lim…

Slightly more recently there’s a note about the API and more general (AWS) limits:

API Limit Reached?

I’ve been using the API to check SIC codes for our new customers. I use it sporadically but recently it only seems to complete a few requests before giving me errors. I thought it was something wrong with the code I was using but upon testing it on our own business company reg number I found it was sporadic when it would actually provide data even if I waited 5 minutes for any rate limit to reset. Have I triggered some other rate limit that requires an increase or could it be my side where the i…

Good luck!

[chrisW]

Thanks @voracityemail
I am not counting I am using the header file rather than counting to 600 - I did initially put in a counter then very quickly realised it was a really bad approach!!!

Hopefully someone from companies house can confirm the limits point, to be explicit about the 2 different things I was talking about
My “simple application” calls https://api.company-information.service.gov.uk
The website used by most in my company is https://find-and-update.company-information.service.gov.uk
So what I am trying to understand is that if we already have high volumes via the website could calling the api from the same IP impact users in our organisation using the website. I appreciate the 600 / 5 min limit would be specific to the API key, but unclear about any high level limits which were referred to in the link voracityemail provided “API Limit Reached?”.

Thank you.

[MArkWilliams]

The API limit is a separate limit to the web site.

[chrisW]

Thank you @MArkWilliams