I have made use of the Free Company Data Product, which is very useful, but I was wondering if I could access the data after the last snapshot was taken.
I understand that the API is not supposed to provide bulk data, so I would like to query /search/companies for a subset of results, e.g. companies that were incorporated or dissolved from 1 May 2020, or companies with a specific SICCode.
I tried to use different keywords to achieve this but did not work and I am not sure which database fields the search is looking into.
There are a number of other bulk products that might be of use to you.
Also we have a streaming API that is constantly publishing changes to companies and officers.
documentation for that service can be found https://developer-specs.companieshouse.gov.uk/99
see The new Streaming API for more details
Hi @MArkWilliams, I am using the /companies endpoint of the Streaming API to get all the update events since 1 May, when the latest bulk product was released.
According to the documentation, βThe streaming API endpoints are each backed by a queue, so that clients can disconnect periodically and reconnect some time later. This queue backlog contains as many events as needed to bring the most recently produced data snapshot up to date.β
However, I receive a 416 error when my timepoint is older than 11-12 days ago. I read that the 416 means the timepoint is too far in the past. It seems that the limit of how back in the past I can go does not reach the data snapshot cutoff point.
Also, I noticed that if I go back in time gradually, 1000-2000 timepoints a time, I may avoid the 416. Thus I managed to scrape events back to timepoint = 9692100.
Thank you for your time, your help is greatly appreciated.
With respect to Data Snapshots: It is still our intention to develop data snapshots for the Streaming API service and the current thinking is that we will form a project to do this next financial year, with Discovery potentially starting in April.
With respect to the Officers data stream: We have developed this (and one for PSCs - Persons of Significant Control) and hope to release them soon.