View your request count for rate limiting

[annamackenzie2]

Is there a way to see how many requests the api thinks you have made or when your 5 minute window started? I have written a limiter into my code to prevent it making more than 600 requests in 5 minutes however it looks like my request count is not in line with CH or perhaps the problem is that when I am recording the start of the window is not actually when it began, is there a way to extract this information?

[voracityemail]

Are you referring to the Streaming API or the main/document REST API?

Main API
Although not listed in the documentation:

Rate limiting

…I thought this was available via the http headers? I thought the X-Ratelimit-... fields were what we should use for this purpose - or will things change?

Roughly as in the thread:

How is rate limiting implemented?

I would like to implement a multithreaded client which may scale up or down. The simplest way to handle the rate limiting is to rely on the API to tell me when I've hit a limit; then each client can exponentially back off until the rate limit is restored. But this depends on how rate limiting is implemented. When I hit my limit, do I get a specific error code? Am I limited for a certain amount of time? Is the 600 requests counter reset every 5 minutes, or averaged over a different period? Th…

For what it’s worth I just checked and this still seems functional:

curl -uMY_API_KEY: -I "https://api.companieshouse.gov.uk/company/NF004299"

HTTP/1.1 200 OK
Date: Wed, 19 Feb 2020 23:36:41 GMT
Content-Type: application/json
Content-Length: 1302
Connection: keep-alive
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: X-RateLimit-Query, origin, content-type, content-length, user-agent, host, accept, authorization
Access-Control-Expose-Headers: X-RateLimit-Window, X-RateLimit-Limit, X-RateLimit-Remain, X-RateLimit-Reset, Location, www-authenticate, cache-control, pragma, content-type, expires, last-modified
Access-Control-Expose-Headers: Location,www-authenticate,cache-control,pragma,content-type,expires,last-modified
Access-Control-Max-Age: 3600
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Ratelimit-Limit: 600
X-Ratelimit-Remain: 599
X-Ratelimit-Reset: 1582155701
X-Ratelimit-Window: 5m
Server: CompaniesHouse

I also checked the documents as if I recall correctly they had separate rate limits some time ago - but I also recall reading something contradicting that more recently.

I believe there are some minor differences between the documents API and main API e.g. per the following thread:

Rate limit header naming issue

The rate limit header names returned by https://api.companieshouse.gov.uk/... URI's are: X-Ratelimit-Limit X-Ratelimit-Remain X-Ratelimit-Reset Those returned by https://document-api.companieshouse.gov.uk are: X-Ratelimit-Limit X-Ratelimit-Remaining X-Ratelimit-Reset For consistency please use either X-Ratelimit-Remain or X-Ratelimit-Remaining and stick with it as this particular header is important for keeping track of usage. Thanks.

Also see:

Not a big issue - format of X-Ratelimit-Window "Rate Limit Window" value

Hi all, Quite happy with the rate limit imposed on the API, just asking about this for completeness. The current response for rate limits can include the following values: X-Ratelimit-Limit: 600 // The rate limit. X-Ratelimit-Remain: 599 // Requests remaining. X-Ratelimit-Reset: 1534248540 // Time of next reset (e.g. 14 August 2018 12:09:00 GMT). X-Ratelimit-Window: 5m // How long the rate limit reset window is. Just for completeness, is there documentation for the…

My test (some info snipped):

curl -v -uMY_API_KEY: "https://frontend-doc-api.companieshouse.gov.uk/document/VuqbfmUhDPN8O3uo0FB8EZIQyCi9Fw_9YK3zwdrzWqs"

...

< HTTP/1.1 200 OK
< Access-Control-Allow-Origin: *
< Content-Type: application/json
< Date: Wed, 19 Feb 2020 23:50:35 GMT
< Server: nginx/1.14.1
< X-Ratelimit-Limit: 600
< X-Ratelimit-Remaining: 597
< X-Ratelimit-Reset: 1582156298
< Content-Length: 485
< Connection: keep-alive

I believe the Python chwrapper library (https://github.com/JamesGardiner/chwrapper) uses this also (if anyone uses that).

Streaming API
There doesn’t seem to be any information suggesting that you can check the rate limiting for the streaming API however - if you get a “too many attempts” you just have to back off.

https://developer-specs.companieshouse.gov.uk/streaming-api/guides/overview

[voracityemail]

Found the reference for my comment above:

@voracityemail

I also checked the documents as if I recall correctly they had separate rate limits some time ago - but I also recall reading something contradicting that more recently.

Found that post - it was:

New IP Address Rate Limiting Query

With the introduction of the new IP address rate limiting, I was hoping someone could clarify the following points - Does each IP now get the 600 calls in 300 second limit or have these numbers changed? Is there any impact on how many API keys are used? e.g. if we use separate keys for different end points in the API does the combined number of calls still have to meet the 600/300 seconds or does each key have any form of independent limiting? Previously, the document API had its own rate li…

This seemed to say “there is only one limit” (presumably 600 / 5 min per API key - although not per IP address…) but the main and documents API certainly seem to have separate rate limits - or at least the X-Ratelimit http headers report as if they do. Quick test using one API key to access each in turn:

curl -uMY_API_KEY: -I "https://api.companieshouse.gov.uk/company/NF004299"
HTTP/1.1 200 OK
Date: Thu, 20 Feb 2020 16:07:55 GMT
...
X-Ratelimit-Limit: 600
X-Ratelimit-Remain: 599
X-Ratelimit-Reset: 1582215175
X-Ratelimit-Window: 5m
Server: CompaniesHouse

curl -v -uMY_API_KEY: "https://frontend-doc-api.companieshouse.gov.uk/document/VuqbfmUhDPN8O3uo0FB8EZIQyCi9Fw_9YK3zwdrzWqs"
...
< HTTP/1.1 200 OK
< Access-Control-Allow-Origin: *
< Content-Type: application/json
< Date: Thu, 20 Feb 2020 16:07:56 GMT
< Server: nginx/1.14.1
< X-Ratelimit-Limit: 600
< X-Ratelimit-Remaining: 599
< X-Ratelimit-Reset: 1582215176
...


curl -uMY_API_KEY: -I "https://api.companieshouse.gov.uk/company/NF004299"

HTTP/1.1 200 OK
Date: Thu, 20 Feb 2020 16:07:57 GMT
...
X-Ratelimit-Limit: 600
X-Ratelimit-Remain: 598
X-Ratelimit-Reset: 1582215175
X-Ratelimit-Window: 5m
Server: CompaniesHouse

curl -v -uMY_API_KEY: "https://frontend-doc-api.companieshouse.gov.uk/document/VuqbfmUhDPN8O3uo0FB8EZIQyCi9Fw_9YK3zwdrzWqs"
...
< HTTP/1.1 200 OK
< Access-Control-Allow-Origin: *
< Content-Type: application/json
< Date: Thu, 20 Feb 2020 16:07:57 GMT
< Server: nginx/1.14.1
< X-Ratelimit-Limit: 600
< X-Ratelimit-Remaining: 598
< X-Ratelimit-Reset: 1582215176
...

curl -uMY_API_KEY: -I "https://api.companieshouse.gov.uk/company/NF004299"

HTTP/1.1 200 OK
Date: Thu, 20 Feb 2020 16:07:58 GMT
...
X-Ratelimit-Limit: 600
X-Ratelimit-Remain: 597
X-Ratelimit-Reset: 1582215175
X-Ratelimit-Window: 5m
Server: CompaniesHouse

curl -v -uMY_API_KEY: "https://frontend-doc-api.companieshouse.gov.uk/document/VuqbfmUhDPN8O3uo0FB8EZIQyCi9Fw_9YK3zwdrzWqs"

...
< HTTP/1.1 200 OK
< Access-Control-Allow-Origin: *
< Content-Type: application/json
< Date: Thu, 20 Feb 2020 16:07:59 GMT
< Server: nginx/1.14.1
< X-Ratelimit-Limit: 600
< X-Ratelimit-Remaining: 597
< X-Ratelimit-Reset: 1582215176
...