Missing data (dissolved company records) - Request for Improvement

[voracityemail]

Continuing the discussion from Removal of Dissolved Company Records:

In a very late response to the policy of data removal:

@mfairhurst

Companies House will shortly be reviewing the timescale for retaining dissolved company records, which currently stands at 20 years. We will be consulting with our customers and depending on the outcome of those discussions, may well seek to amend the current retention policy in line with any decisions reached.

As an interim measure, we have taken the decision to not make information on companies dissolved more than 6 years ago available on the new CH beta service. We apologise for any inconvenience this may cause.

Could be some indication of “known unknowns” in the API in the following cases?

• When requesting data on a company which is not in the API data set.
• When querying company officer appointments where an appointment was made to a company not in the API data set.
• Similarly to above for PSCs - I presume if a corporate PSC was appointed and then dissolved, after 6 years this PSC would presumably be removed from the list of previous PSCs (hypothetical).

Rationale
Company search / profile
If I wish to provide information to a 3rd party on the basis of the information in the API about a certain company currently I can only state the positive. If a user searches for a company and no information is returned, I can’t say “isn’t registered with CH” - I have to go check the company number under the URI because the company could have closed / dissolved more than 6 years ago. And if a user has entered a name rather than the number all I can do is say “go look on webCHeck”.

CH already does this ("partial-data-available" field in the companyProfile resource) for companies where it doesn’t hold data e.g. a Royal Charter company etc. In this case I can report to the user “is this the company you meant, this is the name” (and usually address) and “further information isn’t held by CH, try here”.

Officer appointments (and PSCs?)
If a user asks about an officer and positions they’ve held I can’t return the list and report “only these ones” because the company might have closed/dissolved (6 year limit again) and been removed from this list (as explained here). If the company data itself was gone from CH it would still be useful if the record of directorship remained.

(If the “item” in the appointments list was actually represented by a relationship in the company’s record [thus logically disappearing if you deleted the company] could that record remain but in stub form e.g. name / company number / status and “incomplete information held” flag as per e.g. companies whose info is held elsewhere?).

Since (presumably) the data in the CH API / Beta dataset is imported (filtered / validated / processed) from another store it would be possible to indicate where information exists but is incomplete / discarded / excluded? I’m not chasing issues in the underlying dataset (examples here and here), rather when there seems to be a choice at the level of the CH API.

I’ve given this no thought but this could possibly follow the idea of the “partial-data-available” field in the companyProfile resource.

I accept an answer to this could be “no, pay up and use the XML Gateway where this is available”. We used to, but we’ve been lured here as a) it’s free b) the API seems neater and works very well in general c) not complaining about Gateway support but support here so far does seem to be good.