Hi - I have a function that goes as below to query the API which has worked perfectly few weeks ago to extract 1000+ appointments from self_appointments links.
Today I am trying to get ~300,000 profiles (as I need dissolved data too - absent from the bulk data) but I keep getting a 429 error.
raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: ...
Is there anything wrong with the API, or am I missing something here?
import requests
from requests import ConnectionError
from ratelimit import limits, sleep_and_retry
from backoff import on_exception, expo
KEY = get_key("API_key")
FIVE_MINUTES = 300 # Number of seconds in five minutes.
API_BASE_URL = "https://api.companieshouse.gov.uk"
@sleep_and_retry # if we exceed the ratelimit imposed by @limits forces sleep until we can start again.
@on_exception(expo, ConnectionError, max_tries=5)
@limits(calls=600, period=FIVE_MINUTES)
def call_api(url, api_key):
r = requests.get(url, auth=(api_key, ""))
if not (r.status_code == 200 or r.status_code == 404):
r.raise_for_status()
elif r.status_code == 404:
return dict({"error": "not found"})
else:
return r.json()