Company & PSC Stream

Hello!

I have couple of questions about 2 streams in streaming API.

Company Stream

So far I have managed to bulk import .csv of basic company data into our database did a transformation to the data format so it plays nice with the streaming updates. With this I can keep company data up-to-date by doing lookup in our DB with company number and play the incoming changes.

I have 2 questions:

I have realised the updates from this stream are not just updates to existing companies and also the new creations. I don’t see a clear identifier in the data differentiating those.

  • Do I just do my lookup in our bulk imported data and if I can’t find it treat it as a creation or just check incorporation date is the same as event production date maybe?
  • Given I can real-time update/add companies to our db with the stream is there any point re-running the bulk import every month with the .csv file?

PSC-Stream

I don’t understand how can I keep the PSC data up-to-date with the streaming API at all.

  • Do I just look at the events and create a new entry in our end like Company number to PSC pairs.
  • Do I need to delete old records or update them somehow. If I want to these operations what is the unique identifier in PSC stream that lets me do that (What do I use to do my lookup is it etag)?

I understand it might be a basic question but I am lacking the domain knowledge to understand how to keep this information up-to-date. Any information is greatly appreciated.

Thanks in advance.