Streaming API Key - Connection Establishment

API Question
#1

Hi,

I am currently working towards fetching data from company house using its Streaming API.
For this, I am using Azure Synapse Pipeline (Web activity).

I have created account on Company House, created an application, created a Streaming API key as well which are needed to fetch streamed data.

I needed assistance to establish a connection between this Streaming API and azure synapse.

Below tries I have done till now which has given me “401: Invalid authorization” error:
-Created Stream Key
-Appended “:” to the key and converted it to Base64
-Passed converted key as Header i.e., Basic {base64 code} as GET method.
-Even tried to pass username and password along with API key

None of the above seems to work.

please provide your assistance regarding this, about what am I doing wrong OR what things I am missing etc.

Thanks,
Neeraj

#2

The method you described to produce the authorization header sounds correct, the only detail missing is the header name (I’m assuming you put it in Authorization).
I suggest trying to connect to the stream from a locally running script or command line, eg cURL.
If you’re familiar with javascript, this nodejs example may be of use: GitHub - mrbrianevans/companies-house-stream: Stream events from companies house in real time (or the scripts in the examples folder).
See if you get this to work: curl --user APIKEY: -s https://stream.companieshouse.gov.uk/filings.
If these both work for you, then its just a case of getting it to work with Azure Synapse. If these aren’t working for you, maybe there’s a problem with your key (make sure its created for a “live” application and not in “test” mode).
You can search in this forum for examples of people connecting with other programming langauges such as C#. I can’t offer any advice specific to Azure Synapse.

#3

Thanks, Ebrian

I was actually trying to access live env with test key, hence the issue (slight miss by me)
It seems to work with curl query given by you.

Failing with Synapse though, will try to work it out.