Repository navigation
Clearer errors in case of incorrect authentication #137
Description
Is your feature request related to a problem? Please describe.
When using an old url or an incorrect apikey, the errors/warnings that cdsapi gives are not clear. This happens to several users, for instance in https://forum.ecmwf.int/t/cdsapi-key-and-404-issues/10905.
Describe the solution you'd like
To avoid users getting confused, I generated authentication functions for cdsapi in my own toolbox (including tests). It would be great if these could be (at least partially) transferred to cdsapi itself. The code is in here:
https://github.com/Deltares/dfm_tools/blob/af2ea58f8ebfc937274c92dc1a42854d50d6dc22/dfm_tools/download.py#L105-L195
The corresponding tests are here:
https://github.com/Deltares/dfm_tools/blob/af2ea58f8ebfc937274c92dc1a42854d50d6dc22/tests/test_download.py#L26-L213
Describe alternatives you've considered
Any alternative method that generates clear errors would suit. For instance, the check_authentication() method sounds promising:
import cdsapi
c = cdsapi.Client()
c.client.check_authentication()However, it raises "AttributeError: 'Client' object has no attribute 'client'" for old apikeys including a :. Currently, apikeys with : result in a cdsapi.api.Client() instance and new apikeys result in a ecmwf.datastores.legacy_client.LegacyClient() instance because of this code:
Lines 281 to 287 in 72d7cec
Would be good to drop support for keys containing
: and always use the same client class, so a decent error is thrown.
Additional context
Authentication check with import cdsapi; c = cdsapi.Client(); c.client.check_authentication() often fails inconveniently, useful errors would be helpful. These usecases follow the same order as the test_cds_credentials_* tests in test_download.py of dfm_tools:
- in case of authentication via
CDSAPI_KEYenvvar instead of rcfile:Exception: Missing/incomplete configuration file: C:\Users\veenstra/.cdsapirc>> would be useful if default url would be assumed instead. However, this case does succeed if the envvarCDSAPI_URLis also supplied. - in case of new url and incorrect apikey (via rcfile or envvars):
HTTPError: 401 Client Error: Unauthorized for url: https://cds.climate.copernicus.eu/api/profiles/v1/account/verification/pat Authentication failed>> acceptable error/message - in case of old v2 url and incorrect apikey (via rcfile or env vars):
HTTPError: 404 Client Error: Not Found for url: https://cds.climate.copernicus.eu/api/v2/profiles/v1/account/verification/pat>> acceptable error message - in case of old cds-beta url and incorrect apikey (via rcfile or envvars):
SSLError: HTTPSConnectionPool(host='cds-beta.climate.copernicus.eu', port=443): Max retries exceeded with url: /api/profiles/v1/account/verification/pat (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1006)')))>> this is not super useful (HTTPError 404 would be way better), discontinue the url or fix the certificate - in case of new url and apikey containing
:(this was normal with old CDS) (via rcfile or envvars):AttributeError: 'Client' object has no attribute 'client'>> this is not helpful, why is the client not correctly set up, would be great ifcheck_authentication()would be available but raise an error in this case
In short:
- the failing certificate (SSLError) makes error handling more cumbersome. Only catching HTTPError (401/404/etc) is easier
- with old apikeys containing
:thecheck_authentication()function cannot be reached, this is inconvenient. - optional: in case of envvars, it would be useful if the default cds url is assumed if no
CDSAPI_URLenvvar is supplied
Organisation
Deltares
Activity
[-]Better error handling for incorrect authentication[/-][+]Clearer errors in case of incorrect authentication[/+]veenstrajelmer commentedon Jan 16, 2025
@EddyCMWF