I don't find the command for the page number with "charges"!

1

I am looking for the list of all charges from all the companies available in the UK.
However there is no command for looping over the charges for each company, which makes this task impossible.
If someone knows a way to collect such a dataset, I will be happy to hear from you!
I need some help!

2

Welcome. As far as I know there is no such bulk dataset exactly.

However the Company data product (Companies House) contains the following information on charges (Mortgages) - according to the specification:

NumMortCharges - the total number of charges (in the categories below)
NumMortOutstanding - the outstanding charges
NumMortPartSatisfied - partially satisfied charges
NumMortSatisfied - satisfied charges

…so you could - for example - work through that data set (large), extract the company numbers of companies with charges that you’re interested in and then use the List Charges endpoint of the API to get the full details for each.

Of course this would take time to do and would be out of date by the time you’d assembled it. To keep it up to date you might be able to use the Charges Stream from the streaming API.

A similar question was asked here:

https://forum.aws.chdev.org/t/bulk-charges-data/1558

… and as far as I know this is not something that Companies House directly provides. I’m not part of Companies House though so you could try adding your request to the end of that thread.

Good luck.

3

Thank you very much for your answer.
The issue I tried what you have proposed,
however we have an issue if for instance a company have more than 25 charges, we only have information about the first page of items (the first 25 items), I am trying to get the full list of charges for the companies I want…
Thank you again for your interest,
I am looking forward to hearing from you !

4

however we have an issue if for instance a company have more than 25 charges, we only have information about the first page of items (the first 25 items),

I wasn’t aware of that - I thought the Charge List endpoint would return all charges.

I’ve just tried this for 01777777 and it appears you’re quite correct - I get 25 charges back but the numbers suggested here are much larger:

curl -u MY_API_KEY_HERE: “https://api.company-information.service.gov.uk/company/01777777/charges”

{
"satisfied_count" : 139,
"unfiltered_count" : 855,
"part_satisfied_count" : 0,
"total_count" : 855,
"items" : [ ... ]
}
5

Moreover there is no command for looping through the pages of the charges…
I was struggling to get this information for the whole week end!
Thank you for your interest !

6

Ah - yes, I remember now. Charges have the same paging feature as some of the other end points - but this is not listed in the documentation! (I thought this must be the case because the web search allows you to page through these e.g. for my example company).

— EDIT 2021-09-20 —
So the original documentation for the endpoint still doesn’t mention the additional parameters (start_index and items_per_page).

The Swagger / OpenAPI specification for the main API *
…does document these parameters for the ChargeList endpoint:
https://developer-specs.company-information.service.gov.uk/api.ch.gov.uk-specifications/swagger-2.0/spec/charges.json#/chargeList"

The remaining bug / inconsistency is that this endpoint should return start_index and items_per_page in the charge list resource but it doesn’t. The documentation is correct here e.g. it doesn’t list these as being returned.

— END EDIT —

…so what you need to do is use the same parameters as the other pageable endpoints (e.g. Search, Officers etc.):

items_per_page - the number of items to return (keep this a smallish number e.g. 20, 50 - I think Companies House limit this anyway e.g. I don’t think asking for 1000 will work).
start_index - what position in the list to start at. Should be a multiple of items_per_page e.g. the first index is 0.

Another “feature” is that the other pageable endpoints return the items_per_page and the start_index but the Charges endpoint doesn’t, even if you provide these. So you’ll need to count how many items you got back because if e.g. you requested 200 items but Companies House only allowed you 100 then you’ll obviously need to request the next 100.

So for example to page through our example company:

curl -u MY_API_KEY: "https://api.company-information.service.gov.uk/company/01777777/charges?items_per_page=5&start_index=0"

… should give you the first 5 items (but count them to check!), then:

curl -u MY_API_KEY: "https://api.company-information.service.gov.uk/company/01777777/charges?items_per_page=5&start_index=5"

…should give you the next 5 etc.

7

Thank you so much ! That helps me a lot!
However are we limited by the number of API requests?

8

Correct. Rate limiting applies. As Companies House always say the API is not for bulk data mining, more for targetted searches / lookup. The bulk products are there if you really want everything. As I’ve suggested you can then fill in some detail using the Companies API - without exceeding your rate quota - and then keep that up to date using the streaming API.

It’s a free service - so there are some rules so that everyone can benefit!

9

Thank you again for your answer ! Where can we request the bulk products? The database I am looking for is not available in their products website, is there someone I can contact?

10

The database I am looking for is not available in their products website

Which database are you referring to? We’ve already established there isn’t a direct dataset with details of all the Charges for all companies. There is a downloadable dataset for all companies which gives (among other things) the numbers of charges in different categories.

As you’ve said - “dataset I am trying to construct” - you will need to construct this (e.g. in the manner I’ve suggested). It’s not something that I’m aware of that already exists in a form you can just download, or I would have told you.

11

I meant the dataset I am trying to construct containing all the charges of all companies…
Is there a way to increase the API request allowance for a period of time, that will allow me to construct it

12

You can request an increase in your rate limit from Companies House - search the forum and you’ll find a couple of threads where people ask this. Just post to one of them requesting an increase, they will get someone to contact you to discuss this.

13

Thank you for your help !

14

Not sure if this is of any use to you (as it’s not complete), but I have a list of all the charges broadcast on the charges streaming API in the last 3 months (about 45,000 of them).
Let me know if this would help you, and I can provide you with the list.

15

Hi Ebrian,
Thank you so much for your help!
I think that will help me a lot, here attached my email: yassinemarrakchi28@gmail.com
I also would like to know how did you manage to get this.
Looking forward to hearing from you,
Yassine