Streaming API missing events

API Issues
#1

Evening all. We are using the streaming API in our software. 99% of the time it’s working exactly as documented, and we’re very pleased with it.

However, we’ve been advised by some of our clients that certain events aren’t triggering updates on the streaming API. Initially I assumed the error must have been with our software, but closer investigations show that clearly the error isn’t with us, and indeed events are missing from the streaming API.

In one case an extension request that was granted by Companies House and resulted in a due date being changed, clearly changed the content of the Company object when requested direct from Companies House, via the REST API. However, the extension being granted did not lead to the streaming API endpoint supplying the new information.

We’re now left in a tricky position, we’ve spent over a year and significant resources building the streaming API into our software, relying on it to update our information which we then present to our users, we’ve promoted it significantly and talk of it as being a ‘huge leap forward’.

But if we can’t trust it to actually provide the events, I fear we may have to abandon it. Are other users experiencing the same problems? It’s 99% great, but then there are some events that the documentation suggests should be included, but aren’t.

When talking to my colleagues I’m left unsure as to what to say, I believed it would be reliable, but I’m now not so sure. We’re grateful for the fact it exists, and appreciate that it’s a ‘free’ service, but we also assumed that being backed by an organisation such as Companies House, it would be reliable, right now I don’t believe it is.

Is the Stream API reliable
#2

The stream is reliable.
We have seen a couple of incidents where items like you mentioned (a due date change) do not appear to have been reflected out of the stream.
This is currently being investigated.
If you can supply some examples of events that are missing from the streaming API, we will add that to the investigation.

#3

The filing of accounts and the extension of a filing deadline do not appear to result in the company data appearing in the company updates stream.

These events affects values in the Company Profile data object, so you would assume that them changing would mean the company appearing in the stream, given that the Company Profile data object has changed.

We have been keeping a separate log of all the updates from the stream during the last couple of months. That’s well over a million updates. So plenty of updates are coming through all the time, but there are situations when data in a Company Profile will change without that company then appearing in the stream.

The documentation suggests that any changes to a Company Profile would result in that company profile appearing on the stream. So when I say reliable, I can’t rely on this assumption unfortunately as we have evidence to demonstrate it being untrue. I can provide company numbers and the dates that data should have appeared.

#4

If you can supply us with evidence/examples of these events that are missing from the streaming API, we can investigate.

#5

We have been able to resolve our issue with missing data. The error was at our end. Sorry for suggesting there was an error with the streaming API!

One suggestion I would make, it should be made more clear that the timepoint is sequential with no gaps. Initially we assumed it was some kind of timestamp (given the wording) and that the odd gap here and there represented quiet periods in the data stream.

Our issue was related to long JSON objects that spanned multiple chunks of data. This was happening once every couple of minutes and thus wasn’t easy to spot, given we assumed gaps in the timepoint value were to be expected.

Anyway, please feel free to delete my thread. Thanks and sorry for wasting your time.

#6

Thanks for letting us know, much appreciated.