code still results in a 401 error using Python 3 request package. I’ve tried all end points, base64 encoded and decoded api keys.
This has been asked several times for Python (and javascript etc.) and mostly resolved. The best answer seems to be by @csmith above.
Other threads with this question (fixed):
If you still have problems:
- Is the issue also occurring when using the developer console (the “Explore this API” form at the bottom of documentation pages e.g. see https://developer.companieshouse.gov.uk/api/docs/search/search.html)?
If so maybe re-post on the following thread:
Persistant 401 errors - If you’re using a local host you may have problems but can work around this, see thread:
Allow localhost JavaScript domain? - Can you get another method to work from your server e.g. curl? An example would be:
curl -uYOUR_APIKEY: "https://api.companieshouse.gov.uk/company/00048839"
- If still stuck post a minimal example (omitting your API key details) with any other relevant info (e.g. “I’ve tested the API key using ‘Explore this API’ and it’s correct”, “I’m using a virtual server / IP address” etc.)
Most problems with authentication seem to be:
a) The username and password for http Basic Authentication. The username is actually the Companies House API key and there is no password (so this should be blank). You don’t need to base64 encode the password when using most frameworks / libraries / tools.
b) Making an http header (if needed). Just follow instructions from @csmith above.
But to spell it out another way:
You’d send the header
Authorization: Basic base-64-encode( http_username : http_password )
…and since the http_username is the API key and http_password is blank, this means you concatenate a colon ‘:’ to the end of your API key, base-64 encode the result , and then send the header:
Authorization: Basic result of above process
Hi,
For python 3 you will only need requests
The API uses Basic Authentication where the ‘user’ is your API key and no password is passed (you input '"")
http://docs.python-requests.org/en/master/user/authentication/
You can take the API key as produced by the site when you register the application
import requests
r = requests.get('https://api.companieshouse.gov.uk/search/[your query]', auth = ([YourApiKey],'')
Then to test it works,
print (r.status_code)“> authorization: Basic YTViM***************************2Og==”
< www-authenticate: Invalid or no Authorisation header has been provided
< server: CompaniesHouse
From what you’ve posted I doubt anyone can help. If you’re posting to a forum asking for advice or a question it is a good idea to:
a) Ask a question! Say what you are trying to achieve overall (the big picture)
b) Say what you’ve done - in outline. e.g. what you tried, from where and any error / response / what the issue is. You’ve done the second part of that but the first part would be useful.
b) Give supporting details. You’ve posted an error which is useful. (It seems to show that the authorization header is correctly encoded e.g. you’ve used the API key as the username and not used a password - which would be correct). So in this case supporting information would be what API and endpoint are you calling (the URL)? What tool / language are you using? Where are you trying to do this from (e.g. “I’ve set up a server on localhost …”). Presumably you’ve registered for the service - did you register which IP / domain(s) you were using? (If it’s localhost please search the forum for information on getting that to work). Are you using the “sandbox” or “live” environment? (Search the forum but I think “use the live one” is good advice).
See my post further up the thread e.g. I’ve given an example:
-
using curl since this is very simple and most people will be able to use / follow that. (I always recommend using that for ease of following what’s going on). Obviously report what you’re using though e.g. Python etc. with code fragment as appropriate.
-
Giving the form of the API key (user / password) e.g. in curl for Basic authetication you just put in the username and password (in plain text, separated by a colon) and the first is the API key and the password is blank (which it looks like you’ve done correctly)
-
Giving the api and endpoint
curl -u YOUR_APIKEY: "https://api.companieshouse.gov.uk/company/00048839"
The tool / environment is of interest as sometimes that can be an issue e.g. maybe you’re doing this from the command line and there’s an issue with quote characters? Perhaps the tool only provides a certain size of input for the username etc.?
Good luck.
First of all… I genuinely very-much appreciate you taking the time to interact with me today and your clearly well-intentioned advice. I have tried all manner of workarounds to get my API access working today to no avail. I’ve wasted at least four hours. I work with many APIs and this is not my usual experience. I didn’t post supporting details because I was extremely frustrated and trying to make a point. I think, given my experiences, that’s not unreasonable. I appreciate that much of the work is probably a donation of valuable free time from voluntary contributors but one of the key tenants of exposing an API is that it should be easy to use / integrate with… Based on the comments I’ve read, I am not an exception in experiencing problems caused by a lack of sufficient explanation around authentication. When I want to use an API, the last thing I want to do is to try and guess / reverse engineer the correct usage based on the supporting forum? Again, I am wary of criticising something that you have told me is basically an underfunded community endeavour… I finally managed to get access but it was not using my preferred method…
Glad you sorted this. What was the issue and how did you resolve it?
All the following is just editorial so if you’re here for information go back to the search results or use the search button (top right)! I’m just another punter, not part of Companies House, not employed by the UK government so I’m just opining.
TL/DR: You get what you pay for and this is a free service. Expect issues from very large / old / public datasets. Just because something’s a part of a government doesn’t mean it is particularly important to the government.
I agree that this is not a perfect system. There are issues with everything from the overall role of Companies House and how it’s resourced down through the API to the documentation. (I find the API works well once you’ve mastered providing authorization / rate limiting and you understand the limitations of the data.)
Based on my own experience with several APIs both government and corporate (REST, SOAP etc.) I have found that this almost never “works” straight away. “Standards” mean something rather loose in the REST world, certainly. In several cases this has involved “reverse engineering” or hacking about as you’ve described. In a number of those cases a “specification” was provided and that also proved to have issues / not match the API.
So maybe I have lower expectations than you? In my experience large systems will generally have inconsistencies, missing information, undocumented “features” - or (in my limited experience with Microsoft / Google) will overwhelm you with complexity, jargon / concepts and tutorials and examples. Many of the latter turn out not to work as given e.g. possibly are out of date. So now we expect and budget for time spend “just making it work”. This goes double for very large datasets, triple for old datasets and through the roof if - as in Companies House - you effectively have the public inputting data!
As someone who has spent far too much time trying to get access to other APIs I do understand that’s extremely frustrating. I think the way Companies House provide your login and some of the settings around that are confusing. However as I wrote in reply to you elsewhere the majority of issues seem to be around people’s unfamiliarity with http Basic and / or how to make that work with their tool / language. Yes, Companies House could have picked a different authorisation / authentication method (e.g. OAuth with bearer token, or http Basic but using a username and password rather than empty password). However as long as it’s implemented correctly - which I believe it is - getting that working is down to the users.
I suppose they could maintain a series of articles on access from different software and systems but they clearly don’t have time to keep the rest of the documents up-to-date and that’s an endless treadmill anyway. Aside from taxes if you live in the UK, no-one’s paying to use this. (If you do want a more “professional” service you can pay - there is the XML Gateway however (a) it’s SOAP / XML which many people find more complex and (b) I believe that’s due to be phased out in favour of the REST API at some point).
I would like to see:
-
An example with walk through - using some very general tool, possibly with http header fragments shown - maybe from searching for a company, obtaining the company profile, listing the filings and then downloading a document. I’d guess that covers both some of the most commonly used endpoints and that illustrates all the features e.g. looking up text in the “enum constants”, “paging” (search, filing history) and (the most troublesome part) the change from their http basic to Amazon authentication (downloading).
-
Fixing some simple issues with documentation and the specification. Some of these have been known for years and are well-reported here.
1 Like
Apologise for short reply as it’s late and may thanks once again for your ongoing help. I used the postman script in this thread provided by Eoin… Connect via Postman says invalid authorisation header - #11 by 7rajnesh7
1 Like
Hello @mfairhurst_gmail
Thank you very much for the detailed steps.
I don’t know how many hours I have spent on this API to make it work but still no luck for me. Changed the API_Key followed by ‘:’ . Tried different ways to provide api_key as well including base64. However, I am still unlucky.
My deadline for API integration is approaching and I am worrying to implement it in Python. I have a list of company names. All I want is their company number that is provided by this API in the response.
No matter what I try in python or postman… I still get the same error.
{
"error": "Invalid Authorization",
"type": "ch:service"
}
Could you please help in any manner…
Thanks in advance.
My Goodness… It’s working now… The whole PURPOSE IS NOT TO USE TEST KEY. To make it working it needs to be a LIVE key.
Why the hell is test key there if it won’t work.
Thanks for all the support in the forum.
5 Likes
Just as an update, as of Jan 2023 this still seems to be an issue. (i.e. Test API keys don’t work, but Live API keys do work). As a simple mitigation, could the authentication documentation be updated to advise users to use the Live API key until this issue is resolved. I don’t know how many collective hours/days this will save.
Again thanks for maintaining this API, it really is very useful.
3 Likes
thank you so much. They really need to update the instructions for using keys can’t believe the documentation is so incomplete. I almost had a panic attack when I couldn’t figure out for the life in me why it wasn’t working and raising my BP.
The best comment ever, made my evening
Now test environment works. I realized that I should use the REST type of the application key and not Stream as stated in the docs.
Example:
curl -X GET 'https://api-sandbox.company-information.service.gov.uk/search/companies?q=capital' -H 'Authorization: Basic OGQ3YmM4Z.....ZmOg=='
if web client credentials are used, what is the correct snippet to retrieve access token?
How did u retrieve the access token? Could u please share the python snippet?
Thank you and thank @siddhant.kaushal
As of 28 Oct, TEST key still doesn’t work.
Spent more than 1 hour trying to see why the code was not working (the basic of Python code).
It turns out, you need to use the LIVE key!!!
1 Like