[openssl-dev] Fwd: latest OpenSSL causes OpenSMTPD to segv

[Richard Levitte]

In message <20160202.003940.2270696010208807774.levitte at openssl.org> on Tue, 02 Feb 2016 00:39:40 +0100 (CET), Richard Levitte <levitte at openssl.org> said:

levitte> In message <20160201231650.GF4987 at mournblade.imrryr.org> on Mon, 1 Feb 2016 23:16:50 +0000, Viktor Dukhovni <openssl-users at dukhovni.org> said:
levitte> 
levitte> openssl-users> On Mon, Feb 01, 2016 at 10:52:56PM +0000, Viktor Dukhovni wrote:
levitte> openssl-users> 
levitte> openssl-users> > The only thing I see that's plausibly pertinent is:
levitte> openssl-users> > 
levitte> openssl-users> > commit 6656ba7152dfe4bba865e327dd362ea08544aa80
levitte> openssl-users> > Author: Dr. Stephen Henson <steve at openssl.org>
levitte> openssl-users> > Date:   Sun Dec 20 18:18:43 2015 +0000
levitte> openssl-users> > 
levitte> openssl-users> >     Don't check RSA_FLAG_SIGN_VER.
levitte> openssl-users> > 
levitte> openssl-users> >     Reviewed-by: Richard Levitte <levitte at openssl.org>
levitte> openssl-users> > 
levitte> openssl-users> 
levitte> openssl-users> This is related to:
levitte> openssl-users> 
levitte> openssl-users>     commit 1c80019a2c8f59410552197723829fd72ab45a5e
levitte> openssl-users>     Author: Dr. Stephen Henson <steve at openssl.org>
levitte> openssl-users>     Date:   Sat Sep 18 22:37:44 1999 +0000
levitte> openssl-users> 
levitte> openssl-users> 	Add new sign and verify members to RSA_METHOD and change SSL code to use sign
levitte> openssl-users> 	and verify rather than direct encrypt/decrypt.
levitte> openssl-users> 
levitte> openssl-users> Which was already present in 0.9.7.  Thus, presumably engines have
levitte> openssl-users> been expected to implement the "new" methods, if they were ported
levitte> openssl-users> to OpenSSL 0.9.7 or later.
levitte> openssl-users> 
levitte> openssl-users> It seems that perhaps the need to implemnt sign/verify and not just
levitte> openssl-users> encrypt/decrypt has not been communicated to the engine maintainers.
levitte> openssl-users> 
levitte> openssl-users> The master branch has:
levitte> openssl-users> 
levitte> openssl-users>     commit 19c6d3ea2d3b4e0ad3e978e42cc7cbdf0c09891f
levitte> openssl-users>     Author: Dr. Stephen Henson <steve at openssl.org>
levitte> openssl-users>     Date:   Wed Dec 2 14:30:39 2015 +0000
levitte> openssl-users> 
levitte> openssl-users> 	Remove RSA_FLAG_SIGN_VER flag.
levitte> openssl-users> 
levitte> openssl-users> 	Remove RSA_FLAG_SIGN_VER: this was origininally used to retain binary
levitte> openssl-users> 	compatibility after RSA_METHOD was extended to include rsa_sign and
levitte> openssl-users> 	rsa_verify fields. It is no longer needed.
levitte> openssl-users> 
levitte> openssl-users> 	Reviewed-by: Richard Levitte <levitte at openssl.org>
levitte> openssl-users> 
levitte> openssl-users> And while indeed the structure has been stable with sign/verify
levitte> openssl-users> methods for ages, engines that don't implement sign/verify may well
levitte> openssl-users> exist, so dropping the flag check can break some engines.
levitte> 
levitte> Hold on a minute...  there is a test that the function pointer is
levitte> assigned:
levitte> 
levitte>     if (rsa->meth->rsa_sign) {
levitte>         return rsa->meth->rsa_sign(type, m, m_len, sigret, siglen, rsa);
levitte>     }
levitte> 
levitte> So what I can conclude without looking is that one of two things have
levitte> happened:
levitte> 
levitte> 1. the RSA_METHOD hasn't been fully initialised, so the rsa_sign
levitte>    pointer is garbage.
levitte> 
levitte> 2. the function that rsa_sign points as is faulty in some way, but has
levitte>    never been called before now because there was no RSA_FLAG_SIGN_VER
levitte>    bit present.
levitte> 
levitte> I just downloaded the latest portable OpenSMTPD and am noticing that
levitte> rsa_sign, rsa_verify and rsa_keygen are filled in (with rsae_sign,
levitte> rsae_verify and rsae_keygen), but that there are no bits at all
levitte> assigned to the flags field.  As far as I can see, this means that
levitte> these functions have never been called...  before now.
levitte> 
levitte> Ref: opensmtpd-5.7.3p1.tar.gz, smtpd/ca.c

Further exploration shows that rsae_sign flatly calls
rsa_default->rsa_sign.  So where does rsa_default come from?  Quick
look shows RSA_get_default_method(), which defaults to returning a
pointer to rsa_pkcs1_ossl_meth, found in crypto/rsa/rsa_ossl.c, and
that structure...  does.  not.  assign.  rsa_sign, rsa_verify and
rsa_keygen.

I would say that the issue here lies with rsae_sign, rsae_verify and
rsae_keygen for not checking that those pointers are non-NULL before
using them, regardless of if flags is checked for RSA_FLAG_SIGN_VER is
checked or not.

Cheers,
Richard

-- 
Richard Levitte         levitte at openssl.org
OpenSSL Project         http://www.openssl.org/~levitte/