[openssl-dev] [openssl.org #4577] X509_LU_RETRY and X509_LU_FAIL are slightly confused

[David Benjamin via RT]

Hey folks,

It seems the X509_LU_* values (now an X509_LOOKUP_TYPE enum in master) are
slightly confused. See this commit message and diff for details:
https://boringssl.googlesource.com/boringssl/+/da7f0c65efb72556f8fc92e460e6c90cd1b1add7%5E%21/

The relevant point is that X509_LU_RETRY doesn't work and has never worked
since SSLeay. The free of an uninitialized pointer seems to have gotten
fixed as a consequence of how X509_OBJECT was opaquified, but the
current_method = j bug is still there. It's possible that's all that's
needed to fix it, but I doubt it. (Come to think of it, I bet
aae41f8c54257d9fa6904d3a9aa09c5db6cefd0d
destroyed any hope of X509_LU_RETRY ever working again...)

I would propose that you all do something similar to the BoringSSL change
above, especially since 1.1.0 is allowed to break API. Remove X509_LU_RETRY
support and both error enums completely. X509_LOOKUP_TYPE is now just a
type enum (then you can remove the default cases added in
bca3f06b84de3c0b428724ac535995064c54aee3). The functions in question just
return 0/1 (or -1/0/1, but I think the less error-prone 0/1 is generally
better).

David

PS: It appears bca3f06b84de3c0b428724ac535995064c54aee3
removed X509_LU_PKEY without removing X509_OBJECT.data.pkey. That too can
probably go.

-- 
Ticket here: http://rt.openssl.org/Ticket/Display.html?id=4577
Please log in as guest with password guest if prompted