403 error/IP block - Fetching dissolved company information

I’ve been trying to gather profiles of companies founded since 2016 that have since been dissolved. In the absence of these being available as a bulk download, I have been making Company Profile endpoint requests for each company number that doesn’t appear in the list of (mostly) active companies provided as a bulk download. This obviously means making a lot of requests (around 890k), but I’ve been using the API a lot for most of this week for other requests without any issues with the rate limit.

However I made a mistake in my code’s handling of cases where a response’s headers don’t include the rate-limit counters and wait time (I was getting a few 402 errors, from an awselb server, which didn’t have any of the expected headers), but the same endpoint would work if I tried again after 10 seconds or so. I also added what I thought was sensible logic to handle a 404 or 403 status code, which was to move on to try requesting the next company profile. Unfortunately including 403s turned out to be a very bad idea, and I woke up this morning to find that I’d made around 179k invalid requests because for some reason I’d triggered a 403 response at around 4am, so my code was just carrying on through the list of numbers it wanted to get information for despite those responses. Unsurprisingly it means my IP has been blocked (it was blocked on both the API and the website, but now the site is back, but the API is not)!

I’m not sure what triggered the first 404/403 response in the first place, was there a problem with the way I had been making the requests up until that point? Please would it be possible for my IP address (which I’ve now added to my app’s IP list) / API key to be reinstated? I have already written an alternative way of handling a 403 for future cases to make sure that doesn’t happen again.

Many thanks and apologies for the poor error handling on my part!

I would very much like to add my name to the list of people keen for a bulk download of dissolved company information…

1 Like

Are you now able to access the API ?

Hi Mark, thanks so much for getting back to me - I was still having troubles on Monday when you first replied, but I just tried the API again and it seems to be working now. Thank you very much for your help with it, very much appreciated!

If there is a way to get a bulk download of company information for dissolved companies founded after Jan 1st 2010 in order to save making lots of requests, that would be very useful. Would it be possible to arrange this?

Sorry but there is no bulk dissolved company product.

We specialise in putting Companies House data at your fingertips and can probably help you here.

We hold data for 1.75 million Dissolved companies that is available for download as csv or API ingestion This data increases daily and the updates are also available for download on ingestion.

Please contact admin@statbooks.co.uk

to find out how to use our services.