You do not need to delete an application to create a new API key.
Simply login and go to 'Your Applications' (https://developer.companieshouse.gov.uk/developer/applications)
It will list you current API key(s), click 'Register an application' and name it,
Start using this new API key and stop using the old key. When you are sure you are not using the old API key, select the old key in 'Your Applications' and click 'Remove Application'.
Hope this helps answer your question.