From kurt at roeckx.be Mon Jan 2 16:38:40 2017 From: kurt at roeckx.be (Kurt Roeckx) Date: Mon, 2 Jan 2017 17:38:40 +0100 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <1483224763.2518.24.camel@HansenPartnership.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> Message-ID: <20170102163840.ykvtfbc444ges3gu@roeckx.be> On Sat, Dec 31, 2016 at 02:52:43PM -0800, James Bottomley wrote: > This patch adds RSA signing for TPM2 keys. There's a limitation to the > way TPM2 does signing: it must recognise the OID for the signature. > That fails for the MD5-SHA1 signatures of the TLS/SSL certificate > verification protocol, so I'm using RSA_Decrypt for both signing > (encryption) and decryption ... meaning that this only works with TPM > decryption keys. It is possible to use the prior code, which preserved > the distinction of signing and decryption keys, but only at the expense > of not being able to support SSL or TLS lower than 1.2 Please submit patches via github. Kurt From James.Bottomley at HansenPartnership.com Mon Jan 2 16:50:24 2017 From: James.Bottomley at HansenPartnership.com (James Bottomley) Date: Mon, 02 Jan 2017 08:50:24 -0800 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <20170102163840.ykvtfbc444ges3gu@roeckx.be> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> Message-ID: <1483375824.2458.15.camel@HansenPartnership.com> On Mon, 2017-01-02 at 17:38 +0100, Kurt Roeckx wrote: > On Sat, Dec 31, 2016 at 02:52:43PM -0800, James Bottomley wrote: > > This patch adds RSA signing for TPM2 keys. There's a limitation to > > the way TPM2 does signing: it must recognise the OID for the > > signature. That fails for the MD5-SHA1 signatures of the TLS/SSL > > certificate verification protocol, so I'm using RSA_Decrypt for > > both signing (encryption) and decryption ... meaning that this only > > works with TPM decryption keys. It is possible to use the prior > > code, which preserved the distinction of signing and decryption > > keys, but only at the expense of not being able to support SSL or > > TLS lower than 1.2 > > Please submit patches via github. Um, that's not really possible given that openssl_tpm_engine is a sourceforge project. James From rsalz at akamai.com Mon Jan 2 17:53:15 2017 From: rsalz at akamai.com (Salz, Rich) Date: Mon, 2 Jan 2017 17:53:15 +0000 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <1483375824.2458.15.camel@HansenPartnership.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> <1483375824.2458.15.camel@HansenPartnership.com> Message-ID: > Um, that's not really possible given that openssl_tpm_engine is a > sourceforge project. Sure it is. You just find it easier to email patches. This is now the second time you?ve been asked. And also, you had concerns about the CLA before. Have they been resolved? If not you should probably stop. From James.Bottomley at HansenPartnership.com Mon Jan 2 18:12:06 2017 From: James.Bottomley at HansenPartnership.com (James Bottomley) Date: Mon, 02 Jan 2017 10:12:06 -0800 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> <1483375824.2458.15.camel@HansenPartnership.com> Message-ID: <1483380726.2458.20.camel@HansenPartnership.com> On Mon, 2017-01-02 at 17:53 +0000, Salz, Rich wrote: > > Um, that's not really possible given that openssl_tpm_engine is a > > sourceforge project. > > Sure it is. Really, how? By pull request, you mean one against the openssl github account so people subscribing to that account see it, I presume? For that to happen, the tree the patch is against must actually exist within the account, which this one doesn't. > You just find it easier to email patches. This patch is mostly FYI, so yes, I do given that multiple mailing lists have some interest. > This is now the second time you?ve been asked. > > And also, you had concerns about the CLA before. Have they been > resolved? If not you should probably stop. I'm still waiting on a reply ... I assume holidays are contributing to the delay. However, openssl_tpm_engine is a DCO project, so that concern is irrelevant here. James From rsalz at akamai.com Mon Jan 2 18:22:21 2017 From: rsalz at akamai.com (Salz, Rich) Date: Mon, 2 Jan 2017 18:22:21 +0000 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <1483380726.2458.20.camel@HansenPartnership.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> <1483375824.2458.15.camel@HansenPartnership.com> <1483380726.2458.20.camel@HansenPartnership.com> Message-ID: <568f4b6ee03c47fb882cec1f4743f4bd@usma1ex-dag1mb1.msg.corp.akamai.com> > Really, how? By pull request, you mean one against the openssl github > account so people subscribing to that account see it, I presume? For that to > happen, the tree the patch is against must actually exist within the account, > which this one doesn't. You clone the openssl git repo, create your own branch off master, apply the diffs you are mailing to the list, and commit/push and then make a PR. Yes it's a bit of work for you. But it then becomes near-zero work for anyone on openssl to look at it. > This patch is mostly FYI, so yes, I do given that multiple mailing lists have > some interest. It's all about trade-offs. Multiple people have said multiple times that PR's are the best way to work with OpenSSL. If those other groups, individually or collectively, are higher on your priority list, that's fine. But do understand what's going on. > I'm still waiting on a reply ... I assume holidays are contributing to the delay. > However, openssl_tpm_engine is a DCO project, so that concern is irrelevant > here. Sorry, I'll push to get the bylaws made public, is that what you need? And no, it's not irrelevant. If this is ever going to appear in OpenSSL, a CLA must be signed. From kurt at roeckx.be Tue Jan 3 00:00:45 2017 From: kurt at roeckx.be (Kurt Roeckx) Date: Tue, 3 Jan 2017 01:00:45 +0100 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <1483375824.2458.15.camel@HansenPartnership.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> <1483375824.2458.15.camel@HansenPartnership.com> Message-ID: <20170103000044.avsnl2ogflxuwfsm@roeckx.be> On Mon, Jan 02, 2017 at 08:50:24AM -0800, James Bottomley wrote: > On Mon, 2017-01-02 at 17:38 +0100, Kurt Roeckx wrote: > > On Sat, Dec 31, 2016 at 02:52:43PM -0800, James Bottomley wrote: > > > This patch adds RSA signing for TPM2 keys. There's a limitation to > > > the way TPM2 does signing: it must recognise the OID for the > > > signature. That fails for the MD5-SHA1 signatures of the TLS/SSL > > > certificate verification protocol, so I'm using RSA_Decrypt for > > > both signing (encryption) and decryption ... meaning that this only > > > works with TPM decryption keys. It is possible to use the prior > > > code, which preserved the distinction of signing and decryption > > > keys, but only at the expense of not being able to support SSL or > > > TLS lower than 1.2 > > > > Please submit patches via github. > > Um, that's not really possible given that openssl_tpm_engine is a > sourceforge project. I obviously didn't look at it and assumed it was for openssl, not some other project. Kurt From levitte at openssl.org Tue Jan 3 11:19:54 2017 From: levitte at openssl.org (Richard Levitte) Date: Tue, 03 Jan 2017 12:19:54 +0100 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <568f4b6ee03c47fb882cec1f4743f4bd@usma1ex-dag1mb1.msg.corp.akamai.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> <1483375824.2458.15.camel@HansenPartnership.com> <1483380726.2458.20.camel@HansenPartnership.com> <568f4b6ee03c47fb882cec1f4743f4bd@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: <1e5962ae-62e4-4bb5-bb4e-1573c164661e@typeapp.com> ?There seems to be some confusion here. James, I understand the tpm engine as an external project, not part of the OpenSSL source proper and not intended to be. However, openssl-dev at openssl.org is a list focused on the development of OpenSSL proper. That makes it a bit odd to discuss the tpm engine here. Largely off topic. Cheers Richard Skickat fr?n BlueMail ? Den 2 jan. 2017 19:22, kI 19:22, "Salz, Rich" skrev: >> Really, how? By pull request, you mean one against the openssl >github >> account so people subscribing to that account see it, I presume? For >that to >> happen, the tree the patch is against must actually exist within the >account, >> which this one doesn't. > >You clone the openssl git repo, create your own branch off master, >apply the diffs you are mailing to the list, and commit/push and then >make a PR. Yes it's a bit of work for you. But it then becomes >near-zero work for anyone on openssl to look at it. > >> This patch is mostly FYI, so yes, I do given that multiple mailing >lists have >> some interest. > >It's all about trade-offs. Multiple people have said multiple times >that PR's are the best way to work with OpenSSL. If those other >groups, individually or collectively, are higher on your priority list, >that's fine. But do understand what's going on. > >> I'm still waiting on a reply ... I assume holidays are contributing >to the delay. >> However, openssl_tpm_engine is a DCO project, so that concern is >irrelevant >> here. > >Sorry, I'll push to get the bylaws made public, is that what you need? > >And no, it's not irrelevant. If this is ever going to appear in >OpenSSL, a CLA must be signed. > >-- >openssl-dev mailing list >To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Tue Jan 3 12:44:17 2017 From: rsalz at akamai.com (Salz, Rich) Date: Tue, 3 Jan 2017 12:44:17 +0000 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> <1483375824.2458.15.camel@HansenPartnership.com> <1483380726.2458.20.camel@HansenPartnership.com> Message-ID: <19887d0c894f4c62a3765d5085cc0e9e@usma1ex-dag1mb1.msg.corp.akamai.com> > > I'm still waiting on a reply ... I assume holidays are contributing to the delay. > > However, openssl_tpm_engine is a DCO project, so that concern is > > irrelevant here. > > Sorry, I'll push to get the bylaws made public, is that what you need? The OSF bylaws are now linked to from https://www.openssl.org/policies/ or available directly at https://www.openssl.org/policies/osf-bylaws.pdf From James.Bottomley at HansenPartnership.com Tue Jan 3 23:22:56 2017 From: James.Bottomley at HansenPartnership.com (James Bottomley) Date: Tue, 03 Jan 2017 15:22:56 -0800 Subject: [openssl-dev] [TrouSerS-tech] [tpmdd-devel] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <20170103231126.GE29656@obsidianresearch.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170103231126.GE29656@obsidianresearch.com> Message-ID: <1483485776.2464.50.camel@HansenPartnership.com> On Tue, 2017-01-03 at 16:11 -0700, Jason Gunthorpe wrote: > On Sat, Dec 31, 2016 at 02:52:43PM -0800, James Bottomley wrote: > > This patch adds RSA signing for TPM2 keys. There's a limitation to > > the way TPM2 does signing: it must recognise the OID for the > > signature. That fails for the MD5-SHA1 signatures of the TLS/SSL > > certificate verification protocol, so I'm using RSA_Decrypt for > > both signing (encryption) and decryption ... meaning that this only > > works with TPM decryption keys. It is possible to use the prior > > code, which preserved the distinction of signing and decryption > > keys, but only at the expense of not being able to support SSL or > > TLS lower than 1.2 > > [Note, I haven't looked closely at TPM2, but TPM1.2 has a concept of > key usage, and I assume that is carried over in the below comments] The TPM1.2 all uses the correct signing functions, the problem is only with 2.0. > I think it is very important to natively support the sign-only key > usage restriction. TPM1.2 goes so far as to declare keys that can be > used for arbitary decrypt as 'legacy do not use'. > > IMHO the best way to do this is to look at the sign operation openssl > is trying to do and see if it can be sent down the sign path to the > TPM. Only if that fails should the decrypt path be used. The problem is the MD5-SHA1 signature of SSL and TLS < v1.2. This cannot be performed by the TPM because it's not listed as a supported signature, so the choice is either to deprecate these protocols (not really viable since they're in wide use in old websites) or use decrypt to do the signatures. Once we get to the point of having to use decrypt, there's no reason to preserve the signing distinction since we never know when a key will be used to decrypt or sign. Note that google took an alternative approach and modified their TSS to work with a MD5-SHA1 signature: https://chromium-review.googlesource.com/#/c/420811/ But this requires a modification to the TPM as well, which we can't do. > For TPM1.2 you could create a sign-only key with the > TPM_SS_RSASSAPKCS1v15_DER mode and feed it arbitary NIDs - the TPM > did not check the data to sign, AFAIK. > > Ideally the user should be able to setup a sign-only key and the > correct SSL negotiation parameters and have a working system, eg a > sign-only key used with TLS 1.3 and ephemeral keyx should work. > > > + /* this is slightly paradoxical that we're doing a Decrypt > > + * operation: the only material difference between decrypt > > and > > + * encrypt is where the padding is applied or checked, so > > if > > + * you apply your own padding up to the RSA block size and > > use > > + * TPM_ALG_NULL, which means no padding check, a decrypt > > + * operation effectively becomes an encrypt */ > > IIRC this duality is exactly why key usage exists, and why good > crypto practice has been to forbid decrypt/encrypt on the same key. Given the signature restrictions, TPM 2.0 just can't be made to work with the older SSL/TLS protocols, so I'm opting to keep compatibility and not benefit from the distinction between signing and decryption keys. James > Jason > > --------------------------------------------------------------------- > --------- > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > TrouSerS-tech mailing list > TrouSerS-tech at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/trousers-tech > From simon.piche at tc.gc.ca Tue Jan 3 23:28:46 2017 From: simon.piche at tc.gc.ca (Piche, Simon) Date: Tue, 3 Jan 2017 18:28:46 -0500 Subject: [openssl-dev] [TrouSerS-tech] [tpmdd-devel] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <1483485776.2464.50.camel@HansenPartnership.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170103231126.GE29656@obsidianresearch.com> <1483485776.2464.50.camel@HansenPartnership.com> Message-ID: -----Original Message----- From: openssl-dev [mailto:openssl-dev-bounces at openssl.org] On Behalf Of James Bottomley Sent: Tuesday, January 03, 2017 6:23 PM To: Jason Gunthorpe Cc: trousers-tech at lists.sourceforge.net; tpmdd-devel at lists.sourceforge.net; ibmtpm20tss-users at lists.sourceforge.net; openssl-dev at openssl.org Subject: Re: [openssl-dev] [TrouSerS-tech] [tpmdd-devel] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine On Tue, 2017-01-03 at 16:11 -0700, Jason Gunthorpe wrote: > On Sat, Dec 31, 2016 at 02:52:43PM -0800, James Bottomley wrote: > > This patch adds RSA signing for TPM2 keys. There's a limitation to > > the way TPM2 does signing: it must recognise the OID for the > > signature. That fails for the MD5-SHA1 signatures of the TLS/SSL > > certificate verification protocol, so I'm using RSA_Decrypt for both > > signing (encryption) and decryption ... meaning that this only works > > with TPM decryption keys. It is possible to use the prior code, > > which preserved the distinction of signing and decryption keys, but > > only at the expense of not being able to support SSL or TLS lower > > than 1.2 > > [Note, I haven't looked closely at TPM2, but TPM1.2 has a concept of > key usage, and I assume that is carried over in the below comments] The TPM1.2 all uses the correct signing functions, the problem is only with 2.0. > I think it is very important to natively support the sign-only key > usage restriction. TPM1.2 goes so far as to declare keys that can be > used for arbitary decrypt as 'legacy do not use'. > > IMHO the best way to do this is to look at the sign operation openssl > is trying to do and see if it can be sent down the sign path to the > TPM. Only if that fails should the decrypt path be used. The problem is the MD5-SHA1 signature of SSL and TLS < v1.2. This cannot be performed by the TPM because it's not listed as a supported signature, so the choice is either to deprecate these protocols (not really viable since they're in wide use in old websites) or use decrypt to do the signatures. Once we get to the point of having to use decrypt, there's no reason to preserve the signing distinction since we never know when a key will be used to decrypt or sign. Note that google took an alternative approach and modified their TSS to work with a MD5-SHA1 signature: https://chromium-review.googlesource.com/#/c/420811/ But this requires a modification to the TPM as well, which we can't do. > For TPM1.2 you could create a sign-only key with the > TPM_SS_RSASSAPKCS1v15_DER mode and feed it arbitary NIDs - the TPM did > not check the data to sign, AFAIK. > > Ideally the user should be able to setup a sign-only key and the > correct SSL negotiation parameters and have a working system, eg a > sign-only key used with TLS 1.3 and ephemeral keyx should work. > > > + /* this is slightly paradoxical that we're doing a Decrypt > > + * operation: the only material difference between decrypt > > and > > + * encrypt is where the padding is applied or checked, so > > if > > + * you apply your own padding up to the RSA block size and > > use > > + * TPM_ALG_NULL, which means no padding check, a decrypt > > + * operation effectively becomes an encrypt */ > > IIRC this duality is exactly why key usage exists, and why good crypto > practice has been to forbid decrypt/encrypt on the same key. Given the signature restrictions, TPM 2.0 just can't be made to work with the older SSL/TLS protocols, so I'm opting to keep compatibility and not benefit from the distinction between signing and decryption keys. James > Jason > > --------------------------------------------------------------------- > --------- > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > TrouSerS-tech mailing list > TrouSerS-tech at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/trousers-tech > -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev From James.Bottomley at HansenPartnership.com Tue Jan 3 23:42:51 2017 From: James.Bottomley at HansenPartnership.com (James Bottomley) Date: Tue, 03 Jan 2017 15:42:51 -0800 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <568f4b6ee03c47fb882cec1f4743f4bd@usma1ex-dag1mb1.msg.corp.akamai.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> <1483375824.2458.15.camel@HansenPartnership.com> <1483380726.2458.20.camel@HansenPartnership.com> <568f4b6ee03c47fb882cec1f4743f4bd@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: <1483486971.2464.57.camel@HansenPartnership.com> On Mon, 2017-01-02 at 18:22 +0000, Salz, Rich wrote: > > I'm still waiting on a reply ... I assume holidays are contributing > > to the delay. However, openssl_tpm_engine is a DCO project, so that > > concern is irrelevant here. > > Sorry, I'll push to get the bylaws made public, is that what you > need? There were two requests: the bylaws and whether modified grant would be acceptable. If, instead of an unrestricted grant in the CLA it were restricted to relicensing to an OSI approved licence, the need to do due diligence on the foundation goes away. > And no, it's not irrelevant. If this is ever going to appear in > OpenSSL, a CLA must be signed. It's not actually my code: I'm just updating it, so I'm unable to say what the long term plan actually is. I would think, though, that hardware engines, since they're highly OS support dependent, would be difficult to keep within openssl itself given that you want to compile on multiple platforms. James From James.Bottomley at HansenPartnership.com Tue Jan 3 23:44:35 2017 From: James.Bottomley at HansenPartnership.com (James Bottomley) Date: Tue, 03 Jan 2017 15:44:35 -0800 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <1e5962ae-62e4-4bb5-bb4e-1573c164661e@typeapp.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> <1483375824.2458.15.camel@HansenPartnership.com> <1483380726.2458.20.camel@HansenPartnership.com> <568f4b6ee03c47fb882cec1f4743f4bd@usma1ex-dag1mb1.msg.corp.akamai.com> <1e5962ae-62e4-4bb5-bb4e-1573c164661e@typeapp.com> Message-ID: <1483487075.2464.59.camel@HansenPartnership.com> On Tue, 2017-01-03 at 12:19 +0100, Richard Levitte wrote: > ?There seems to be some confusion here. > > James, I understand the tpm engine as an external project, not part > of the OpenSSL source proper and not intended to be. > > However, openssl-dev at openssl.org is a list focused on the development > of OpenSSL proper. That makes it a bit odd to discuss the tpm engine > here. Largely off topic. Fair enough. You were cc'd since it's a modification of code used by openSSL, in case there was interest. James > Cheers > Richard > > Skickat fr?n BlueMail > > Den 2 jan. 2017 19:22, kI 19:22, "Salz, Rich" > skrev: > > > Really, how? By pull request, you mean one against the openssl > > github > > > account so people subscribing to that account see it, I presume? > > > For > > that to > > > happen, the tree the patch is against must actually exist within > > > the > > account, > > > which this one doesn't. > > > > You clone the openssl git repo, create your own branch off master, > > apply the diffs you are mailing to the list, and commit/push and > > then > > make a PR. Yes it's a bit of work for you. But it then becomes > > near-zero work for anyone on openssl to look at it. > > > > > This patch is mostly FYI, so yes, I do given that multiple > > > mailing > > lists have > > > some interest. > > > > It's all about trade-offs. Multiple people have said multiple > > times > > that PR's are the best way to work with OpenSSL. If those other > > groups, individually or collectively, are higher on your priority > > list, > > that's fine. But do understand what's going on. > > > > > I'm still waiting on a reply ... I assume holidays are > > > contributing > > to the delay. > > > However, openssl_tpm_engine is a DCO project, so that concern is > > irrelevant > > > here. > > > > Sorry, I'll push to get the bylaws made public, is that what you > > need? > > > > And no, it's not irrelevant. If this is ever going to appear in > > OpenSSL, a CLA must be signed. > > > > -- > > openssl-dev mailing list > > To unsubscribe: > > https://mta.openssl.org/mailman/listinfo/openssl-dev From matt at openssl.org Wed Jan 4 00:04:39 2017 From: matt at openssl.org (Matt Caswell) Date: Wed, 4 Jan 2017 00:04:39 +0000 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <19887d0c894f4c62a3765d5085cc0e9e@usma1ex-dag1mb1.msg.corp.akamai.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> <1483375824.2458.15.camel@HansenPartnership.com> <1483380726.2458.20.camel@HansenPartnership.com> <19887d0c894f4c62a3765d5085cc0e9e@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: <546e0ed1-d19a-26ec-7c29-dc5ad1dbbf17@openssl.org> On 03/01/17 12:44, Salz, Rich wrote: >>> I'm still waiting on a reply ... I assume holidays are contributing to the delay. >>> However, openssl_tpm_engine is a DCO project, so that concern is >>> irrelevant here. >> >> Sorry, I'll push to get the bylaws made public, is that what you need? > > The OSF bylaws are now linked to from https://www.openssl.org/policies/ I can't actually see this link...am I just missing it, or did you not push? Matt From James.Bottomley at HansenPartnership.com Wed Jan 4 00:11:56 2017 From: James.Bottomley at HansenPartnership.com (James Bottomley) Date: Tue, 03 Jan 2017 16:11:56 -0800 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <546e0ed1-d19a-26ec-7c29-dc5ad1dbbf17@openssl.org> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> <1483375824.2458.15.camel@HansenPartnership.com> <1483380726.2458.20.camel@HansenPartnership.com> <19887d0c894f4c62a3765d5085cc0e9e@usma1ex-dag1mb1.msg.corp.akamai.com> <546e0ed1-d19a-26ec-7c29-dc5ad1dbbf17@openssl.org> Message-ID: <1483488716.2464.74.camel@HansenPartnership.com> On Wed, 2017-01-04 at 00:04 +0000, Matt Caswell wrote: > > On 03/01/17 12:44, Salz, Rich wrote: > > > > I'm still waiting on a reply ... I assume holidays are > > > > contributing to the delay. > > > > However, openssl_tpm_engine is a DCO project, so that concern > > > > is > > > > irrelevant here. > > > > > > Sorry, I'll push to get the bylaws made public, is that what you > > > need? > > > > The OSF bylaws are now linked to from > > https://www.openssl.org/policies/ > > I can't actually see this link...am I just missing it, or did you not > push? https://www.openssl.org/policies/osf-bylaws.pdf Thanks for doing this! James From levitte at openssl.org Wed Jan 4 00:13:37 2017 From: levitte at openssl.org (Richard Levitte) Date: Wed, 04 Jan 2017 01:13:37 +0100 (CET) Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <1483487075.2464.59.camel@HansenPartnership.com> References: <568f4b6ee03c47fb882cec1f4743f4bd@usma1ex-dag1mb1.msg.corp.akamai.com> <1e5962ae-62e4-4bb5-bb4e-1573c164661e@typeapp.com> <1483487075.2464.59.camel@HansenPartnership.com> Message-ID: <20170104.011337.971580973587379465.levitte@openssl.org> In message <1483487075.2464.59.camel at HansenPartnership.com> on Tue, 03 Jan 2017 15:44:35 -0800, James Bottomley said: James.Bottomley> On Tue, 2017-01-03 at 12:19 +0100, Richard Levitte wrote: James.Bottomley> > ?There seems to be some confusion here. James.Bottomley> > James.Bottomley> > James, I understand the tpm engine as an external project, not part James.Bottomley> > of the OpenSSL source proper and not intended to be. James.Bottomley> > James.Bottomley> > However, openssl-dev at openssl.org is a list focused on the development James.Bottomley> > of OpenSSL proper. That makes it a bit odd to discuss the tpm engine James.Bottomley> > here. Largely off topic. James.Bottomley> James.Bottomley> Fair enough. You were cc'd since it's a modification of code used by James.Bottomley> openSSL, in case there was interest. Strictly speaking, that belongs in openssl-users at openssl.org. The reason I point this out is that for code that isn't meant to be part of OpenSSL proper, the whole discussion about CLAs, licenses and whatnot is a red herring that belongs neither here not there. As long as you do stuff as a separate project, YOU (collective you) decide what license to use, let alone your contribution policy. Of course, I do recall that there was an attempt of patches to be applied to OpenSSL proper. That alone is subject to our license and our policies, if that's still interesting (I don't know if it is). If it is, that should be contributed as a separate patch, preferably as a github PR (sourceforge is entirely uninteresting to us). Me, I haven't really minded the discussion here, as long as it didn't become confusing. After all, it did spark some discussion around my STORE project ;-) Did I leave something out or is the situation clear? Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From James.Bottomley at HansenPartnership.com Wed Jan 4 00:17:06 2017 From: James.Bottomley at HansenPartnership.com (James Bottomley) Date: Tue, 03 Jan 2017 16:17:06 -0800 Subject: [openssl-dev] [TrouSerS-tech] [tpmdd-devel] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <20170103234053.GA32185@obsidianresearch.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170103231126.GE29656@obsidianresearch.com> <1483485776.2464.50.camel@HansenPartnership.com> <20170103234053.GA32185@obsidianresearch.com> Message-ID: <1483489026.2464.76.camel@HansenPartnership.com> On Tue, 2017-01-03 at 16:40 -0700, Jason Gunthorpe wrote: > On Tue, Jan 03, 2017 at 03:22:56PM -0800, James Bottomley wrote: > > > I think it is very important to natively support the sign-only > > > key usage restriction. TPM1.2 goes so far as to declare keys that > > > can be used for arbitary decrypt as 'legacy do not use'. > > > > > > IMHO the best way to do this is to look at the sign operation > > > openssl is trying to do and see if it can be sent down the sign > > > path to the TPM. Only if that fails should the decrypt path be > > > used. > > > > The problem is the MD5-SHA1 signature of SSL and TLS < v1.2. This > > cannot be performed by the TPM because it's not listed as a > > supported signature, so the choice is either to deprecate these > > protocols (not really viable since they're in wide use in old > > websites) or use decrypt to do the signatures. Once we get to the > > point of having to use decrypt, there's no reason to preserve the > > signing distinction since we never know when a key will be used to > > decrypt or sign. > > I'm not disputing your analysis, just remarking that it seem very > undesirable to ban *all* sign-only keys just to support a single > legacy SSL configuration. It's not just a single situation. MD5-SHA1 is where it will fall apart on backwards compatibility but my current TPM doesn't understand anything other than sha1 or sha256, so it wouldn't allow more state of the art algorithms like sha224, sha384 or sha512 either. > This is why I suggest looking at the sign being requested and using > the sign path if it is *possible*, otherwise requiring a key with the > broader key usage. It's a trade off: the user configuration nightmare: how does an ordinary user know what type of TPM key they need ... they'll need to know what their TPM is capable of and what signatures they're likely to need. VS the benefits of having single use keys. I'm just not sure I see enough benefits to trying to preserve the decrypt vs sign distinction, whereas I do see the floods of complaints from users who got it wrong or think it should work as advertised. > Not everything is SSL - openssh uses these routines too and it should > be able to use the sign only key type without a limitation? Agreed that openssh only has forward compatibility problems, but it still has them thanks to the newer algorithm problem. Actually, the next problem would be gnome-keyring. It does openssl signatures and fishing the algorithm type out by the time James From rsalz at akamai.com Wed Jan 4 13:34:24 2017 From: rsalz at akamai.com (Salz, Rich) Date: Wed, 4 Jan 2017 13:34:24 +0000 Subject: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <1483486971.2464.57.camel@HansenPartnership.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170102163840.ykvtfbc444ges3gu@roeckx.be> <1483375824.2458.15.camel@HansenPartnership.com> <1483380726.2458.20.camel@HansenPartnership.com> <568f4b6ee03c47fb882cec1f4743f4bd@usma1ex-dag1mb1.msg.corp.akamai.com> <1483486971.2464.57.camel@HansenPartnership.com> Message-ID: > There were two requests: the bylaws and whether modified grant would be > acceptable. If, instead of an unrestricted grant in the CLA it were restricted > to relicensing to an OSI approved licence, the need to do due diligence on > the foundation goes away. We're not interested in changing the CLA at this time. From stiju.easo at gmail.com Fri Jan 6 01:11:22 2017 From: stiju.easo at gmail.com (Stiju Easo) Date: Fri, 6 Jan 2017 06:41:22 +0530 Subject: [openssl-dev] Info Needed - Extended Master Secret Message-ID: Hi, I could see from release notes that extended master secret support is there in 1.1.x branch Does this support also exist in 1.0.2 branch too? or is there any plan for this in near future? -- Stiju The unexamined life is not worth living for man. Socrates, in Plato, Dialogues, Apology Greek philosopher in Athens (469 BC - 399 BC) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Fri Jan 6 02:44:05 2017 From: rsalz at akamai.com (Salz, Rich) Date: Fri, 6 Jan 2017 02:44:05 +0000 Subject: [openssl-dev] Info Needed - Extended Master Secret In-Reply-To: References: Message-ID: It is a new feature, and features do not go into older releases, just bugfixes. From leonard-lists at den.ottolander.nl Sun Jan 8 15:39:35 2017 From: leonard-lists at den.ottolander.nl (Leonard den Ottolander) Date: Sun, 08 Jan 2017 16:39:35 +0100 Subject: [openssl-dev] [PATCH] Missing NULLs in ssl_cipher_methods init Message-ID: <1483889975.5666.17.camel@quad> Hello, The number of NULLS in the initialization of ssl_cipher_methods in ssl/ssl_ciph.c does not match the count SSL_ENC_NUM_IDX in 1.1.0c. Attached patch for a fix. Regards, Leonard. -- mount -t life -o ro /dev/dna /genetic/research -------------- next part -------------- A non-text attachment was scrubbed... Name: ssl_ciph.c.diff Type: text/x-patch Size: 459 bytes Desc: not available URL: From rsalz at akamai.com Sun Jan 8 17:16:56 2017 From: rsalz at akamai.com (Salz, Rich) Date: Sun, 8 Jan 2017 17:16:56 +0000 Subject: [openssl-dev] [PATCH] Missing NULLs in ssl_cipher_methods init In-Reply-To: <1483889975.5666.17.camel@quad> References: <1483889975.5666.17.camel@quad> Message-ID: <5243e51eb014461a88ada1b171523357@usma1ex-dag1mb1.msg.corp.akamai.com> > The number of NULLS in the initialization of ssl_cipher_methods in > ssl/ssl_ciph.c does not match the count SSL_ENC_NUM_IDX in 1.1.0c. A better fix is to remove all the initializers and just count on C's "init to NULL" guarantee. From freemonj at gmail.com Mon Jan 9 16:57:21 2017 From: freemonj at gmail.com (Freemon Johnson) Date: Mon, 9 Jan 2017 11:57:21 -0500 Subject: [openssl-dev] x509 extension support Message-ID: Hello, Can anyone help me in discerning which version of openssl supports sbgp-autonomousSysNum and sbgp-ipAddrBlock? If it has been deprecated then providing the alternative would be greatly appreciated. A sample openssl.cnf is provided below. When I perform a request for req it fails because of the objects described above. The version of openssl I am using when attempting this req generation is version OpenSSL 1.0.2g 1 Mar 2016 [req]default_bits = 2048default_md = sha256distinguished_name = req_dnprompt = noencrypt_key = no [req_dn]CN = Testbed RPKI root certificate [x509v3_extensions]basicConstraints = critical,CA:truesubjectKeyIdentifier = hashkeyUsage = critical,keyCertSign,cRLSignsubjectInfoAccess = @siacertificatePolicies = critical,1.3.6.1.5.5.7.14.2sbgp-autonomousSysNum = critical, at rfc3779_asnssbgp-ipAddrBlock = critical, at rfc3997_addrs [sia]1.3.6.1.5.5.7.48.5;URI = rsync://example.org/rpki/root/1.3.6.1.5.5.7.48.10;URI = rsync://example.org/rpki/root/root.mft [rfc3779_asns]AS.0 = 64496-64511AS.1 = 65536-65551 [rfc3997_addrs]IPv4.0 = 192.0.2.0/24IPv4.1 = 198.51.100.0/24IPv4.2 = 203.0.113.0/24 IPv6.0 = 2001:0DB8::/32 Cheers, Freemon -------------- next part -------------- An HTML attachment was scrubbed... URL: From leonard-lists at den.ottolander.nl Mon Jan 9 17:46:37 2017 From: leonard-lists at den.ottolander.nl (Leonard den Ottolander) Date: Mon, 09 Jan 2017 18:46:37 +0100 Subject: [openssl-dev] Revert commit 10621ef white space nightmare Message-ID: <1483983997.5112.1.camel@quad> Hello, https://github.com/openssl/openssl/commit/10621efd3296a92f489f6ab26a88e88d9790930e#diff-4b59eddb1c722b1dc3d17b5f64149e12 is a white space nightmare. The replacement of "#define"s by "# define"s etc. is just silly and makes it unnecessarily hard to port patches between different releases (working with RHEL 7.3 "hobbled 1.0.1e" vs. 1.0.1u). Sadly patch -l chokes on this kind of white space nonsense and who can blame it? If one wants to indent directives space is normally inserted before the hash sign. I don't remember ever seeing directives being indented by adding white space between the hash sign and the directive. Please revert this commit (in all branches), even though it has been a while. Thank you. Regards, Leonard. From rsalz at akamai.com Mon Jan 9 18:03:00 2017 From: rsalz at akamai.com (Salz, Rich) Date: Mon, 9 Jan 2017 18:03:00 +0000 Subject: [openssl-dev] Revert commit 10621ef white space nightmare In-Reply-To: <1483983997.5112.1.camel@quad> References: <1483983997.5112.1.camel@quad> Message-ID: Sorry you feel this way, but the patch is not being reverted. First of all, 1.0.1 is now end of life and gets no updates :) As for the specific pre-processor, there are systems out there that only recognized the poundsign if it was in the first column (silly but true). Also, we prefer the whitespace on CPP lines the way we have it. You can find some details about the code reformat at two blog entries: https://www.openssl.org/blog/blog/2015/01/05/source-code-reformat/ and https://www.openssl.org/blog/blog/2015/02/11/code-reformat-finished/ The best way to address your issue is to start using 1.0.2 or later. If you cannot do that, then a perl script like this can help: perl -pi.bak -e 's/^\s*#\s*/#/' files... From matt at openssl.org Mon Jan 9 18:06:54 2017 From: matt at openssl.org (Matt Caswell) Date: Mon, 9 Jan 2017 18:06:54 +0000 Subject: [openssl-dev] Revert commit 10621ef white space nightmare In-Reply-To: <1483983997.5112.1.camel@quad> References: <1483983997.5112.1.camel@quad> Message-ID: <4a7e4586-f94f-0f84-9aab-7643bc82ff89@openssl.org> On 09/01/17 17:46, Leonard den Ottolander wrote: > Hello, > > https://github.com/openssl/openssl/commit/10621efd3296a92f489f6ab26a88e88d9790930e#diff-4b59eddb1c722b1dc3d17b5f64149e12 > > is a white space nightmare. The replacement of "#define"s by "# define"s > etc. is just silly and makes it unnecessarily hard to port patches > between different releases (working with RHEL 7.3 "hobbled 1.0.1e" vs. > 1.0.1u). Sadly patch -l chokes on this kind of white space nonsense and > who can blame it? > > If one wants to indent directives space is normally inserted before the > hash sign. I don't remember ever seeing directives being indented by > adding white space between the hash sign and the directive. > > Please revert this commit (in all branches), even though it has been a > while. Thank you. That particular commit was the result of a lot work and discussion on this list and in other places. This is the code reformat commit and changes the format of the source to be consistent with the OpenSSL coding style: https://www.openssl.org/policies/codingstyle.html Like I said there was a lot of discussion at the time with arguments in favour and against the reformat - although the on the whole most people were in favour. The coding style as a whole was based heavily on the Linux Kernel coding style. I don't recall the provenance of the pre-processor indentation style. Needless to say though everyone has their particular thoughts about what makes "good" style - and no matter what you come up with no one is going to be pleased with all of it. Personally I believe the reformat has done nothing but good for us. It has made maintaining the source code *far* easier. Whatever your thoughts on it a lot of water has gone under the bridge since then and there have been a lot of commits to all the branches. Reverting this patch is just not an option at this stage even if we wanted to (which we don't) - you would disappear into conflict hell never to return. Of course 1.0.1 is now out of support anyway so we won't be making any more commits to that particular branch! Matt From ca+ssl-dev at esmtp.org Mon Jan 9 17:57:06 2017 From: ca+ssl-dev at esmtp.org (Claus Assmann) Date: Mon, 9 Jan 2017 09:57:06 -0800 Subject: [openssl-dev] Revert commit 10621ef white space nightmare In-Reply-To: <1483983997.5112.1.camel@quad> References: <1483983997.5112.1.camel@quad> Message-ID: <20170109175706.GA16324@x2.esmtp.org> On Mon, Jan 09, 2017, Leonard den Ottolander wrote: > If one wants to indent directives space is normally inserted before the > hash sign. I don't remember ever seeing directives being indented by > adding white space between the hash sign and the directive. Then you didn't look at source code for sendmail, libesmtp, deliver, mutt, nail, afl, flex, bison, make, (just some stuff I have available for grep '^# *[dei]') and many others... BTW: the reason the spaces are after '#' is because some (old?) pre-preprocessors handle '#' only in the first column. From ssx at av8n.com Mon Jan 9 18:01:57 2017 From: ssx at av8n.com (John Denker) Date: Mon, 9 Jan 2017 11:01:57 -0700 Subject: [openssl-dev] Revert commit 10621ef white space nightmare In-Reply-To: <1483983997.5112.1.camel@quad> References: <1483983997.5112.1.camel@quad> Message-ID: <723313d1-aa02-ae12-741f-0ecbea5e928d@av8n.com> On 01/09/2017 10:46 AM, Leonard den Ottolander wrote: > I don't remember ever seeing directives being indented by adding > white space between the hash sign and the directive. In my world, that is quite common. > If one wants to indent directives space is normally inserted before > the hash sign. No, that is not normal. It is not even permitted by traditional versions of the preprocessor. I quote from https://gcc.gnu.org/onlinedocs/gcc-3.1/cpp/Traditional-Mode.html >> Preprocessing directives are recognized in traditional C only when >> their leading # appears in the first column. There can be no >> whitespace between the beginning of the line and the #. From leonard-lists at den.ottolander.nl Mon Jan 9 18:45:19 2017 From: leonard-lists at den.ottolander.nl (Leonard den Ottolander) Date: Mon, 09 Jan 2017 19:45:19 +0100 Subject: [openssl-dev] Revert commit 10621ef white space nightmare In-Reply-To: <4a7e4586-f94f-0f84-9aab-7643bc82ff89@openssl.org> References: <1483983997.5112.1.camel@quad> <4a7e4586-f94f-0f84-9aab-7643bc82ff89@openssl.org> Message-ID: <1483987519.5112.34.camel@quad> Hello Matt, On Mon, 2017-01-09 at 18:06 +0000, Matt Caswell wrote: > That particular commit was the result of a lot work and discussion on > this list and in other places. This is the code reformat commit and > changes the format of the source to be consistent with the OpenSSL > coding style: > > https://www.openssl.org/policies/codingstyle.html Yeah I sort of guessed that but wasn't around when that discussion took place. I'll have a look at that guide and will learn to live with it ;) . With some hand editing I got my patch fixed as you can see from my previous post. Regards, Leonard. From leonard-lists at den.ottolander.nl Mon Jan 9 18:57:43 2017 From: leonard-lists at den.ottolander.nl (Leonard den Ottolander) Date: Mon, 09 Jan 2017 19:57:43 +0100 Subject: [openssl-dev] Enabling AES-192 ciphers, how to expose DHE-RSA-AES192-GCM-SHA384 Message-ID: <1483988263.5112.35.camel@quad> Hello, Considering that AES-192 seems to be very resistant against related key attacks (http://eprint.iacr.org/2009/317) and the algorithm is already available in the openssl code I am trying to expose the AES-192 ciphers. Attached is a patch against 1.0.1u (adapted from the version I created against RHEL "1.0.1e hobbled") that tries to accomplish this for plain and EDH ciphers. Once I get this to work I will continue by adding the EECDH ciphers. The patch seems to work for most parts, except from exposing the AES192-GCM ciphers. When the self test is run (make -C test apps tests) it chokes with a client error: ERROR in CLIENT 140069906728640:error:140740B5:SSL routines:SSL23_CLIENT_HELLO:no ciphers available:s23_clnt.c:502: TLSv1.2, cipher (NONE) (NONE) 1 handshakes of 256 bytes done Failed DHE-RSA-AES192-GCM-SHA384 make: *** [test_ssl] Error 1 The error occurs in ssl23_client_hello(); Note that the last hunk disables the testing of AES-192-GCM ciphers. This is a hack to get the adapted RHEL srpm to build that should eventually be removed. Time stamps on the files are also garbled as I have not normalized the patch against a fresh tree yet. This is no problem when applying it. So my question is, could someone point me in the right direction on how to expose the AES-192-GCM ciphers, i.e. what am I doing wrong that is causing the client error? Would the development team consider adding a patch exposing AES-192 ciphers in openssl once it's complete? Thanks for your help. Regards, Leonard. -- mount -t life -o ro /dev/dna /genetic/research -------------- next part -------------- A non-text attachment was scrubbed... Name: openssl-1.0.1u.aes192.patch Type: text/x-patch Size: 25070 bytes Desc: not available URL: From openssl-users at dukhovni.org Mon Jan 9 19:25:50 2017 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Mon, 9 Jan 2017 19:25:50 +0000 Subject: [openssl-dev] Enabling AES-192 ciphers, how to expose DHE-RSA-AES192-GCM-SHA384 In-Reply-To: <1483988263.5112.35.camel@quad> References: <1483988263.5112.35.camel@quad> Message-ID: <20170109192550.GL13486@mournblade.imrryr.org> On Mon, Jan 09, 2017 at 07:57:43PM +0100, Leonard den Ottolander wrote: > Considering that AES-192 seems to be very resistant against related key > attacks (http://eprint.iacr.org/2009/317) and the algorithm is already > available in the openssl code I am trying to expose the AES-192 > ciphers. There are no AES-192 ciphersuites in the IANA TLS registry: http://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-4 so these cannot (interoperably) be used with TLS. > +/* AES-192 */ > + /* Cipher A8 */ > + { > + 0, /* not implemented (non-ephemeral DH) */ > + TLS1_TXT_DH_DSS_WITH_AES_192_SHA256, > + TLS1_CK_DH_DSS_WITH_AES_192_SHA256, That codepoint is: TLS_PSK_WITH_AES_128_GCM_SHA256 > + /* Cipher A9 */ > + { > + 0, /* not implemented (non-ephemeral DH) */ > + TLS1_TXT_DH_RSA_WITH_AES_192_SHA256, > + TLS1_CK_DH_RSA_WITH_AES_192_SHA256, That codepoint is: TLS_PSK_WITH_AES_256_GCM_SHA384 > + /* Cipher AA */ > + { > + 1, > + TLS1_TXT_DHE_DSS_WITH_AES_192_SHA256, > + TLS1_CK_DHE_DSS_WITH_AES_192_SHA256, Another conflict. > + /* Cipher AB */ > + { > + 1, > + TLS1_TXT_DHE_RSA_WITH_AES_192_SHA256, > + TLS1_CK_DHE_RSA_WITH_AES_192_SHA256, Another conflict... -- Viktor. From leonard-lists at den.ottolander.nl Mon Jan 9 19:47:01 2017 From: leonard-lists at den.ottolander.nl (Leonard den Ottolander) Date: Mon, 09 Jan 2017 20:47:01 +0100 Subject: [openssl-dev] Enabling AES-192 ciphers, how to expose DHE-RSA-AES192-GCM-SHA384 In-Reply-To: <20170109192550.GL13486@mournblade.imrryr.org> References: <1483988263.5112.35.camel@quad> <20170109192550.GL13486@mournblade.imrryr.org> Message-ID: <1483991221.5112.41.camel@quad> Hello Viktor, On Mon, 2017-01-09 at 19:25 +0000, Viktor Dukhovni wrote: > There are no AES-192 ciphersuites in the IANA TLS registry: > > http://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-4 > > so these cannot (interoperably) be used with TLS. Right. I thought these reference IDs were only for internal use, but of course that is not the case... Thanks for clearing that up. Has anyone ever attempted to get such ciphers included in that IANA list? It seems AES-192 is being treated rather stepmotherly in the standards. Regards, Leonard. -- mount -t life -o ro /dev/dna /genetic/research From rsalz at akamai.com Mon Jan 9 19:52:35 2017 From: rsalz at akamai.com (Salz, Rich) Date: Mon, 9 Jan 2017 19:52:35 +0000 Subject: [openssl-dev] Enabling AES-192 ciphers, how to expose DHE-RSA-AES192-GCM-SHA384 In-Reply-To: <1483991221.5112.41.camel@quad> References: <1483988263.5112.35.camel@quad> <20170109192550.GL13486@mournblade.imrryr.org> <1483991221.5112.41.camel@quad> Message-ID: > Has anyone ever attempted to get such ciphers included in that IANA list? It > seems AES-192 is being treated rather stepmotherly in the standards. AES 192 has been discussed at various times in the IETF mailing lists (see CFRG and TLS for most likely places). Here's one posting: https://www.ietf.org/mail-archive/web/cfrg/current/msg04820.html My short summary is that if 128 isn't good enough for you, use 256. 192 is a midpoint that only makes things more complicated by adding more options (and potentially increases the size of the clienthello message, which has had deployment problems with some platforms). From leonard-lists at den.ottolander.nl Mon Jan 9 20:25:45 2017 From: leonard-lists at den.ottolander.nl (Leonard den Ottolander) Date: Mon, 09 Jan 2017 21:25:45 +0100 Subject: [openssl-dev] Enabling AES-192 ciphers, how to expose DHE-RSA-AES192-GCM-SHA384 In-Reply-To: References: <1483988263.5112.35.camel@quad> <20170109192550.GL13486@mournblade.imrryr.org> <1483991221.5112.41.camel@quad> Message-ID: <1483993545.5112.48.camel@quad> Hello Rich, On Mon, 2017-01-09 at 19:52 +0000, Salz, Rich wrote: > AES 192 has been discussed at various times in the IETF mailing lists > (see CFRG and TLS for most likely places). Here's one posting: > https://www.ietf.org/mail-archive/web/cfrg/current/msg04820.html > > My short summary is that if 128 isn't good enough for you, use 256. > 192 is a midpoint that only makes things more complicated by adding > more options (and potentially increases the size of the clienthello > message, which has had deployment problems with some platforms). Doesn't the fact that AES-192 seems to be more resistant against related key attacks than AES-256 "in a world of 2^50 keys" count as an argument for inclusion? A related question, is the fact that AES-192 is more resistant to related key attacks caused by the fact that it uses a key size that is not an exponent of 2? Regards, Leonard. -- mount -t life -o ro /dev/dna /genetic/research From rsalz at akamai.com Mon Jan 9 20:30:08 2017 From: rsalz at akamai.com (Salz, Rich) Date: Mon, 9 Jan 2017 20:30:08 +0000 Subject: [openssl-dev] Enabling AES-192 ciphers, how to expose DHE-RSA-AES192-GCM-SHA384 In-Reply-To: <1483993545.5112.48.camel@quad> References: <1483988263.5112.35.camel@quad> <20170109192550.GL13486@mournblade.imrryr.org> <1483991221.5112.41.camel@quad> <1483993545.5112.48.camel@quad> Message-ID: <1e56a47c96534fb2aa19a94589f36636@usma1ex-dag1mb1.msg.corp.akamai.com> > Doesn't the fact that AES-192 seems to be more resistant against related key > attacks than AES-256 "in a world of 2^50 keys" count as an argument for > inclusion? > > A related question, is the fact that AES-192 is more resistant to related key > attacks caused by the fact that it uses a key size that is not an exponent of 2? Take it up with the IETF folks; I was just summarizing why. From rsalz at akamai.com Tue Jan 10 04:05:22 2017 From: rsalz at akamai.com (Salz, Rich) Date: Tue, 10 Jan 2017 04:05:22 +0000 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? Message-ID: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> Should we move to using SIPHash for the default string hashing function in OpenSSL? It's now in the kernel https://lkml.org/lkml/2017/1/9/619 Overview at https://131002.net/siphash/ -- Senior Architect, Akamai Technologies Member, OpenSSL Dev Team IM: richsalz at jabber.at Twitter: RichSalz -------------- next part -------------- An HTML attachment was scrubbed... URL: From freemonj at gmail.com Tue Jan 10 14:42:02 2017 From: freemonj at gmail.com (Freemon Johnson) Date: Tue, 10 Jan 2017 09:42:02 -0500 Subject: [openssl-dev] x509 extension support Message-ID: Hello, Can anyone help me in discerning which version of openssl supports sbgp-autonomousSysNum and sbgp-ipAddrBlock? If it has been deprecated then providing the alternative would be greatly appreciated. A sample openssl.cnf is provided below. When I perform a request for req it fails because of the objects described above. The version of openssl I am using when attempting this req generation is version OpenSSL 1.0.2g 1 Mar 2016 [req]default_bits = 2048default_md = sha256distinguished_name = req_dnprompt = noencrypt_key = no [req_dn]CN = Testbed RPKI root certificate [x509v3_extensions]basicConstraints = critical,CA:truesubjectKeyIdentifier = hashkeyUsage = critical,keyCertSign,cRLSignsubjectInfoAccess = @siacertificatePolicies = critical,1.3.6.1.5.5.7.14.2sbgp-autonomousSysNum = critical, at rfc3779_asnssbgp-ipAddrBlock = critical, at rfc3997_addrs [sia]1.3.6.1.5.5.7.48.5;URI = rsync://example.org/rpki/root/1.3.6.1.5.5.7.48.10;URI = rsync://example.org/rpki/root/root.mft [rfc3779_asns]AS.0 = 64496-64511AS.1 = 65536-65551 [rfc3997_addrs]IPv4.0 = 192.0.2.0/24IPv4.1 = 198.51.100.0/24IPv4.2 = 203.0.113.0/24 IPv6.0 = 2001:0DB8::/32 Cheers, Freemon -------------- next part -------------- An HTML attachment was scrubbed... URL: From sra at hactrn.net Tue Jan 10 14:59:39 2017 From: sra at hactrn.net (Rob Austein) Date: Tue, 10 Jan 2017 09:59:39 -0500 Subject: [openssl-dev] x509 extension support In-Reply-To: References: Message-ID: <20170110145939.5DCD1461ED3A@minas-ithil.hactrn.net> At Tue, 10 Jan 2017 09:42:02 -0500, Freemon Johnson wrote: > > Can anyone help me in discerning which version of openssl supports > sbgp-autonomousSysNum and sbgp-ipAddrBlock? If it has been > deprecated then providing the alternative would be greatly > appreciated. RFC 3779 support has been in the code base for going on ten years now, and as far as I know is still available in all supported versions. Most likely the problem you're seeing is that the RFC 3779 code isn't enabled at compile time in the binaries you're using. Some platforms enable it, some don't. You could always try lobbying the maintainer of whatever packaging you're using (if any), but, failing that, you may have to build your own binaries, in which case you'll need to enable RFC 3779 support when you run ./Configure. From rschm2 at unh.newhaven.edu Tue Jan 10 16:15:22 2017 From: rschm2 at unh.newhaven.edu (Schmicker, Robert) Date: Tue, 10 Jan 2017 16:15:22 +0000 Subject: [openssl-dev] Linker error when adding new cipher in crypto folder In-Reply-To: References: Message-ID: rschm2> Hello, rschm2> rschm2> I am attempting to add a new cipher into the crypto library. I have rschm2> done the following so far? rschm2> rschm2> 1. Added my code to the openssl/crypto folder rschm2> 2. Created a build.info for make to compile my code (created this rschm2> based off of openssl/crypto/dh?s build.info) rschm2> 3. Added my cipher name to the list of ciphers to compile in Configure rschm2> 4. Compiled and installed all code without errors and object files for rschm2> my new cipher are created in openssl/crypto/mycipher/ rschm2> 5. Created a test.c file that verifies that the library is installed rschm2> and working properly by generating a MD5 hash of a string rschm2> Compiled as: rschm2> gcc test.c -I/usr/local/openssl/include/openssl/ -o test -lcrypto - rschm2> lssl -Wall rschm2> 6. I am able to properly include in my test.c rschm2> file rschm2> rschm2> However, as soon as I make a call to my cipher in test.c I get a rschm2> linker error and gcc is unable to find any of my functions. rschm2> rschm2> It seems that the header file I have in the openssl/include folder rschm2> isn?t being linked somehow to my code in the openssl/cypto folder. rschm2> rschm2> I feel like I?m missing a step here? rschm2> rschm2> Any help is much appreciated! levitte>On way is to, as already mentioned, edit util/libcrypto.num. The levitte>other is to edit util/mkdef.pl and then run 'make update'. In levitte>mkdef.pl, you'll find a bunch of lines like this: levitte> $crypto.="include/openssl/whatever.h" levitte>Simply add a line like that for mycipher.h. levitte>(note: mkdef.pl might be a bit picky sometimes) levitte>Cheers, levitte>Richard levitte>-- levitte>Richard Levitte levitte at openssl.org levitte>OpenSSL Project http://www.openssl.org/~levitte/ Hello, I would just like to thank Richard Levitte and Viktor for their help as I was finally able to figure it out. In case anyone else would like this same process I have included some general documentation below: 1) git clone openssl project 2) edit util/mkdef.pl for header file (header file must be extern and have (void) in function declaration if no arguments are being passed) 3) edit Configure (line 313ish) and add in crypto function amongst others 4) create build.info under /crypto/mycipher/ (follow other crypto build.info?s to create) 5) ./config 6) make update 7) make -j Rob On Dec 31, 2016, at 5:52 PM, openssl-dev-request at openssl.org wrote: Re: Linker error when adding new cipher in crypto folder -------------- next part -------------- An HTML attachment was scrubbed... URL: From rschm2 at unh.newhaven.edu Tue Jan 10 16:16:31 2017 From: rschm2 at unh.newhaven.edu (Schmicker, Robert) Date: Tue, 10 Jan 2017 16:16:31 +0000 Subject: [openssl-dev] build.info documentation Message-ID: <2AFF5D6A-5EFF-4527-AA55-CF55A1270E18@unh.newhaven.edu> Hello, Can anyone here point me in the direction to some documentation on build.info files? For the most part I?m creating mine using examples from other crypto ciphers but could use some more in depth explanation of what is going on when it is being parsed. More specifically, I?m attempting to create a build.info file that will trigger a Makefile in a subdirectory for my new encryption cipher to later include using the INCLUDE[?]. Thank you in advance to any advice or help! Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Tue Jan 10 16:46:21 2017 From: levitte at openssl.org (Richard Levitte) Date: Tue, 10 Jan 2017 17:46:21 +0100 Subject: [openssl-dev] build.info documentation In-Reply-To: <2AFF5D6A-5EFF-4527-AA55-CF55A1270E18@unh.newhaven.edu> References: <2AFF5D6A-5EFF-4527-AA55-CF55A1270E18@unh.newhaven.edu> Message-ID: <86213985-AAC9-492B-8A89-DF5703B752DA@openssl.org> The READMEs in Configurations/ have pretty in depth information. It sounds like you want a build.info with a BEGINRAW..ENDRAW section that contains what you need to do a sub-make in your subdir. Quite frankly, though, if this is something that you intend for inclusion in OpenSSL, you're better off making a proper build.info. INCLUDE only indicates to the compiler where it can find header files, i.e. translates to -I options on Unix. Cheers Richard "Schmicker, Robert" skrev: (10 januari 2017 17:16:31 CET) >Hello, > >Can anyone here point me in the direction to some documentation on >build.info files? > >For the most part I?m creating mine using examples from other crypto >ciphers but could use some more in depth explanation of what is going >on when it is being parsed. > >More specifically, I?m attempting to create a >build.info file that will trigger a Makefile in a >subdirectory for my new encryption cipher to later include using the >INCLUDE[?]. > >Thank you in advance to any advice or help! >Rob -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From bkaduk at akamai.com Tue Jan 10 17:48:32 2017 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Tue, 10 Jan 2017 11:48:32 -0600 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: On 01/09/2017 10:05 PM, Salz, Rich wrote: > > Should we move to using SIPHash for the default string hashing > function in OpenSSL? It?s now in the kernel > https://lkml.org/lkml/2017/1/9/619 > > > Overview at https://131002.net/siphash/ > > > Instead of this? %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% /*- unsigned char b[16]; MD5(c,strlen(c),b); return(b[0]|(b[1]<<8)|(b[2]<<16)|(b[3]<<24)); */ n = 0x100; while (*c) { v = n | (*c); n += 0x100; r = (int)((v >> 2) ^ v) & 0x0f; ret = (ret << r) | (ret >> (32 - r)); ret &= 0xFFFFFFFFL; ret ^= v * v; c++; } return ((ret >> 16) ^ ret); %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Heck, yes! -Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Tue Jan 10 18:31:22 2017 From: levitte at openssl.org (Richard Levitte) Date: Tue, 10 Jan 2017 19:31:22 +0100 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> Benjamin Kaduk skrev: (10 januari 2017 18:48:32 CET) >On 01/09/2017 10:05 PM, Salz, Rich wrote: >> >> Should we move to using SIPHash for the default string hashing >> function in OpenSSL? It?s now in the kernel >> https://lkml.org/lkml/2017/1/9/619 >> > >> >> Overview at https://131002.net/siphash/ >> > >> >> > >Instead of this? > >%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > >/*- > unsigned char b[16]; > MD5(c,strlen(c),b); > return(b[0]|(b[1]<<8)|(b[2]<<16)|(b[3]<<24)); >*/ > > n = 0x100; > while (*c) { > v = n | (*c); > n += 0x100; > r = (int)((v >> 2) ^ v) & 0x0f; > ret = (ret << r) | (ret >> (32 - r)); > ret &= 0xFFFFFFFFL; > ret ^= v * v; > c++; > } > return ((ret >> 16) ^ ret); > >%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > >Heck, yes! > >-Ben I fail to see what that would give us. OPENSSL_LH_strhash() is used to get a reasonable index for LHASH entries. Also SIPhash gives at least 64 bits results, do we really expect to see large enough hash tables to warrant that? Cheers Richard -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From bkaduk at akamai.com Tue Jan 10 19:19:21 2017 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Tue, 10 Jan 2017 13:19:21 -0600 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> Message-ID: On 01/10/2017 12:31 PM, Richard Levitte wrote: > > Benjamin Kaduk skrev: (10 januari 2017 18:48:32 CET) >> On 01/09/2017 10:05 PM, Salz, Rich wrote: >>> Should we move to using SIPHash for the default string hashing >>> function in OpenSSL? It?s now in the kernel >>> https://lkml.org/lkml/2017/1/9/619 >>> >> Heck, yes! >> -Ben > I fail to see what that would give us. OPENSSL_LH_strhash() is used to get a reasonable index for LHASH entries. Also SIPhash gives at least 64 bits results, do we really expect to see large enough hash tables to warrant that? > We don't need to use the full output width of a good hash function. My main point is, "why would we want to ignore the last 20 years of advancement in hash function research?" Section 7 of the siphash paper (https://131002.net/siphash/siphash.pdf) explicitly talks about using it for hash tables, including using hash table indices H(m) mod l. -Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From kgoldman at us.ibm.com Tue Jan 10 19:38:58 2017 From: kgoldman at us.ibm.com (Ken Goldman) Date: Tue, 10 Jan 2017 14:38:58 -0500 Subject: [openssl-dev] [TrouSerS-tech] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine In-Reply-To: <1483485776.2464.50.camel@HansenPartnership.com> References: <1483224485.2518.20.camel@HansenPartnership.com> <1483224763.2518.24.camel@HansenPartnership.com> <20170103231126.GE29656@obsidianresearch.com> <1483485776.2464.50.camel@HansenPartnership.com> Message-ID: On 1/3/2017 6:22 PM, James Bottomley wrote: > > Note that google took an alternative approach and modified their TSS to > work with a MD5-SHA1 signature: > > https://chromium-review.googlesource.com/#/c/420811/ > > But this requires a modification to the TPM as well, which we can't do. Right. It's not a TSS issue. Modification is futile. It just passes parameters on to the TPM. The place for this change is the TCG's TPM work group. From uri at ll.mit.edu Tue Jan 10 21:04:53 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Tue, 10 Jan 2017 21:04:53 +0000 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> Message-ID: <9ED2BF3D-9524-416B-B3F3-7EB65DA89E85@ll.mit.edu> We don?t need the full output width of a good hash function, but for _this_ purpose (as far as I understand) we don?t need the strength of a good hash function either ? and we surely don?t need the unnecessary performance hit of a good hash where we don?t need a good hash. Or am I missing something? ? Regards, Uri On 1/10/17, 2:19 PM, "openssl-dev on behalf of Benjamin Kaduk" wrote: On 01/10/2017 12:31 PM, Richard Levitte wrote: Benjamin Kaduk skrev: (10 januari 2017 18:48:32 CET) On 01/09/2017 10:05 PM, Salz, Rich wrote: Should we move to using SIPHash for the default string hashing function in OpenSSL?? It?s now in the kernel https://lkml.org/lkml/2017/1/9/619 Heck, yes! -Ben I fail to see what that would give us. OPENSSL_LH_strhash() is used to get a reasonable index for LHASH entries. Also SIPhash gives at least 64 bits results, do we really expect to see large enough hash tables to warrant that? We don't need to use the full output width of a good hash function. My main point is, "why would we want to ignore the last 20 years of advancement in hash function research?" Section 7 of the siphash paper (https://131002.net/siphash/siphash.pdf) explicitly talks about using it for hash tables, including using hash table indices H(m) mod l. -Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5211 bytes Desc: not available URL: From levitte at openssl.org Tue Jan 10 21:55:18 2017 From: levitte at openssl.org (Richard Levitte) Date: Tue, 10 Jan 2017 22:55:18 +0100 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> Message-ID: <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> Benjamin Kaduk skrev: (10 januari 2017 20:19:21 CET) >On 01/10/2017 12:31 PM, Richard Levitte wrote: >> >> Benjamin Kaduk skrev: (10 januari 2017 18:48:32 >CET) >>> On 01/09/2017 10:05 PM, Salz, Rich wrote: >>>> Should we move to using SIPHash for the default string hashing >>>> function in OpenSSL? It?s now in the kernel >>>> https://lkml.org/lkml/2017/1/9/619 >>>> >>> Heck, yes! >>> -Ben >> I fail to see what that would give us. OPENSSL_LH_strhash() is used >to get a reasonable index for LHASH entries. Also SIPhash gives at >least 64 bits results, do we really expect to see large enough hash >tables to warrant that? >> > >We don't need to use the full output width of a good hash function. > >My main point is, "why would we want to ignore the last 20 years of >advancement in hash function research?" Section 7 of the siphash paper >(https://131002.net/siphash/siphash.pdf) explicitly talks about using >it >for hash tables, including using hash table indices H(m) mod l. I agree with the advice when one can expect huge tables. The tables we handle are pretty small (I think, please correct me if I'm wrong) and would in all likelihood not benefit very much if at all from SIPhash's relative safety. Of course, one can ask the question if someone uses LHASH as a general purpose hash table implementation rather than just for the stuff OpenSSL. Frankly, I would probably look at a dedicated hash table library first... Cheers Richard -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From tshort at akamai.com Tue Jan 10 22:42:17 2017 From: tshort at akamai.com (Short, Todd) Date: Tue, 10 Jan 2017 22:42:17 +0000 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> Message-ID: <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> I think I might have an init/update/final version of siphash24 lying around somewhere that would be compatible with OpenSSL?s EVP_PKEY mechanism (similar to Poly1305, in that it needs a key). -- -Todd Short // tshort at akamai.com // "One if by land, two if by sea, three if by the Internet." On Jan 10, 2017, at 4:55 PM, Richard Levitte > wrote: Benjamin Kaduk > skrev: (10 januari 2017 20:19:21 CET) On 01/10/2017 12:31 PM, Richard Levitte wrote: Benjamin Kaduk > skrev: (10 januari 2017 18:48:32 CET) On 01/09/2017 10:05 PM, Salz, Rich wrote: Should we move to using SIPHash for the default string hashing function in OpenSSL? It?s now in the kernel https://lkml.org/lkml/2017/1/9/619 Heck, yes! -Ben I fail to see what that would give us. OPENSSL_LH_strhash() is used to get a reasonable index for LHASH entries. Also SIPhash gives at least 64 bits results, do we really expect to see large enough hash tables to warrant that? We don't need to use the full output width of a good hash function. My main point is, "why would we want to ignore the last 20 years of advancement in hash function research?" Section 7 of the siphash paper (https://131002.net/siphash/siphash.pdf) explicitly talks about using it for hash tables, including using hash table indices H(m) mod l. I agree with the advice when one can expect huge tables. The tables we handle are pretty small (I think, please correct me if I'm wrong) and would in all likelihood not benefit very much if at all from SIPhash's relative safety. Of course, one can ask the question if someone uses LHASH as a general purpose hash table implementation rather than just for the stuff OpenSSL. Frankly, I would probably look at a dedicated hash table library first... Cheers Richard -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From pwalten at au1.ibm.com Wed Jan 11 01:28:36 2017 From: pwalten at au1.ibm.com (Peter Waltenberg) Date: Wed, 11 Jan 2017 11:28:36 +1000 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> Message-ID: Reality check Others have pointed this out but I don't think it's making it through. LHash doesn't need a cryptographic hash and it doesn't have security implications. It certainly doesn't need a keyed hash. LHash does need to be something that's good at distinguishing short text strings, that's not necessarilly the same thing as a good cryptographic hash, and possibly it's exactly the opposite thing due to the limitted incoming symbol space (ascii text). About the only thing LHash needs is high performance in it's use area. I'd suspect that switching MD5 to SHA-1 in the existing algorithm would get you that simply because SHA-1 is asm optimized on most platforms now and MD5 typically isn't. I'd suggest that anyone wishing to change this should at least have to demonstrate improved performance in the OpenSSL use case before it's accepted. Peter From: "Short, Todd" To: "openssl-dev at openssl.org" Date: 11/01/2017 08:42 Subject: Re: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? Sent by: "openssl-dev" I think I might have an init/update/final version of siphash24 lying around somewhere that would be compatible with OpenSSL?s EVP_PKEY mechanism (similar to Poly1305, in that it needs a key). -- -Todd Short // tshort at akamai.com // "One if by land, two if by sea, three if by the Internet." On Jan 10, 2017, at 4:55 PM, Richard Levitte wrote: Benjamin Kaduk skrev: (10 januari 2017 20:19:21 CET) On 01/10/2017 12:31 PM, Richard Levitte wrote: Benjamin Kaduk skrev: (10 januari 2017 18:48:32 CET) On 01/09/2017 10:05 PM, Salz, Rich wrote: Should we move to using SIPHash for the default string hashing function in OpenSSL? It?s now in the kernel https://lkml.org/lkml/2017/1/9/619 Heck, yes! -Ben I fail to see what that would give us. OPENSSL_LH_strhash() is used to get a reasonable index for LHASH entries. Also SIPhash gives at least 64 bits results, do we really expect to see large enough hash tables to warrant that? We don't need to use the full output width of a good hash function. My main point is, "why would we want to ignore the last 20 years of advancement in hash function research?" Section 7 of the siphash paper (https://131002.net/siphash/siphash.pdf) explicitly talks about using it for hash tables, including using hash table indices H(m) mod l. I agree with the advice when one can expect huge tables. The tables we handle are pretty small (I think, please correct me if I'm wrong) and would in all likelihood not benefit very much if at all from SIPhash's relative safety. Of course, one can ask the question if someone uses LHASH as a general purpose hash table implementation rather than just for the stuff OpenSSL. Frankly, I would probably look at a dedicated hash table library first... Cheers Richard -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Wed Jan 11 03:13:39 2017 From: rsalz at akamai.com (Salz, Rich) Date: Wed, 11 Jan 2017 03:13:39 +0000 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> Message-ID: <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> The needs for OpenSSL's LHASH are exactly what SipHash was designed for: fast on short strings. OpenSSL's hash currently *does not* call MD5 or SHA1; the MD5 code is commented out. Yes, performance tests would greatly inform the decision. From michel.sales at free.fr Wed Jan 11 08:58:09 2017 From: michel.sales at free.fr (Michel) Date: Wed, 11 Jan 2017 09:58:09 +0100 Subject: [openssl-dev] Build fail when configured using no-nextprotoneg Message-ID: <000901d26be8$d5fc6b80$81f54280$@sales@free.fr> Can we assume it is temporary due to "the process of transitioning from NPN to ALPN" mentioned in ssl_locl.h ? Regards, Michel. cl /I "." /I "include" -DDSO_WIN32 -DOPENSSL_THREADS -DOPENSSL_NO_DYNAM IC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSS L_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_A SM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DEC P_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSL_API_COMPAT=0x10100000L "-D ENGINESDIR=\"C:\\OpenSSL_dbg\\lib\\engines-1_1\"" "-DOPENSSLDIR=\"c:\\OpenSSL_db g\\ssl\"" -W3 -wd4090 -Gs0 -GF -Gy -nologo -DOPENSSL_SYS_WIN32 -DWIN32_LEAN_AND_ MEAN -DL_ENDIAN -D_CRT_SECURE_NO_DEPRECATE -DUNICODE -D_UNICODE /Od -DDEBUG -D_ DEBUG /Zi /Fdossl_static /MT /Zl -c /Fossl\ssl_lib.obj "ssl\ssl_lib.c" ssl_lib.c ssl\ssl_lib.c(626): error C2039: 'alpn': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(627): error C2039: 'alpn_len': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(627): warning C4047: 'function': 'std::size_t' differs in levels o f indirection from 'char [14]' ssl\ssl_lib.c(627): warning C4024: 'CRYPTO_malloc': different types for formal a nd actual parameter 1 ssl\ssl_lib.c(627): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int' ssl\ssl_lib.c(627): warning C4024: 'CRYPTO_malloc': different types for formal a nd actual parameter 2 ssl\ssl_lib.c(627): error C2198: 'CRYPTO_malloc': too few arguments for call ssl\ssl_lib.c(630): error C2039: 'alpn': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(630): error C2039: 'alpn_len': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(630): error C2198: 'memcpy': too few arguments for call ssl\ssl_lib.c(631): error C2039: 'alpn_len': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(2304): error C2039: 'alpn': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(2304): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int' ssl\ssl_lib.c(2304): warning C4024: 'CRYPTO_free': different types for formal an d actual parameter 2 ssl\ssl_lib.c(2304): error C2198: 'CRYPTO_free': too few arguments for call ssl\ssl_lib.c(2305): error C2039: 'alpn': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(2306): error C2039: 'alpn': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(2310): error C2039: 'alpn_len': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(2343): error C2039: 'alpn_select_cb': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(2344): error C2039: 'alpn_select_cb_arg': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(2622): error C2039: 'alpn': is not a member of '' e:\devsrc\openssl.git\ssl\ssl_locl.h(782): note: see declaration of '' ssl\ssl_lib.c(2622): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int' ssl\ssl_lib.c(2622): warning C4024: 'CRYPTO_free': different types for formal an d actual parameter 2 ssl\ssl_lib.c(2622): error C2198: 'CRYPTO_free': too few arguments for call NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 14.0 \VC\BIN\cl.EXE"' : return code '0x2' Stop. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tmraz at redhat.com Wed Jan 11 09:02:41 2017 From: tmraz at redhat.com (Tomas Mraz) Date: Wed, 11 Jan 2017 10:02:41 +0100 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: <1484125361.12353.5.camel@redhat.com> On Wed, 2017-01-11 at 03:13 +0000, Salz, Rich wrote: > The needs for OpenSSL's LHASH are exactly what SipHash was designed > for: fast on short strings. > OpenSSL's hash currently *does not* call MD5 or SHA1; the MD5 code is > commented out. > Yes, performance tests would greatly inform the decision. +1 Is there really no use of LHASH tables in OpenSSL where an attacker attempting a DoS attack can control the contents of the tables? If you are reasonably sure that there is no such occurrence or that the number of entries attacker can insert into such table is severally limited by other means then perhaps it really makes no sense to replace the existing algorithm. But we need to know this first. -- Tomas Mraz No matter how far down the wrong road you've gone, turn back. Turkish proverb (You'll never know whether the road is wrong though.) From michel.sales at free.fr Wed Jan 11 09:33:53 2017 From: michel.sales at free.fr (Michel) Date: Wed, 11 Jan 2017 10:33:53 +0100 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: <001901d26bed$d3746ed0$7a5d4c70$@sales@free.fr> And what about using FNV or CityHash ? https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo_hash_function https://en.wikipedia.org/wiki/CityHash From levitte at openssl.org Wed Jan 11 10:22:10 2017 From: levitte at openssl.org (Richard Levitte) Date: Wed, 11 Jan 2017 11:22:10 +0100 (CET) Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <001901d26bed$d3746ed0$7a5d4c70$@sales@free.fr> References: <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> <001901d26bed$d3746ed0$7a5d4c70$@sales@free.fr> Message-ID: <20170111.112210.591121838732185295.levitte@openssl.org> In message <001901d26bed$d3746ed0$7a5d4c70$@sales at free.fr> on Wed, 11 Jan 2017 10:33:53 +0100, "Michel" said: michel.sales> And what about using FNV or CityHash ? michel.sales> michel.sales> https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo_hash_function I'm a little worried about the zero hash value issue mentioned here: https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo_hash_function#Non-cryptographic_hash michel.sales> https://en.wikipedia.org/wiki/CityHash Google has replaced that with FarmHash according to that page... -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Wed Jan 11 10:37:06 2017 From: matt at openssl.org (Matt Caswell) Date: Wed, 11 Jan 2017 10:37:06 +0000 Subject: [openssl-dev] Build fail when configured using no-nextprotoneg In-Reply-To: <000901d26be8$d5fc6b80$81f54280$@sales@free.fr> References: <000901d26be8$d5fc6b80$81f54280$@sales@free.fr> Message-ID: On 11/01/17 08:58, Michel wrote: > Can we assume it is temporary due to ?the process of transitioning from > NPN to ALPN? mentioned in ssl_locl.h ? Its a bug: https://github.com/openssl/openssl/pull/2212 Matt From michel.sales at free.fr Wed Jan 11 14:18:18 2017 From: michel.sales at free.fr (Michel) Date: Wed, 11 Jan 2017 15:18:18 +0100 Subject: [openssl-dev] wiki update for enc command Message-ID: Hi, Looks like one of my previous mail (see below) was lost in the ?cloud? ;-) Might be helpfull to send it again here ? Regards, Michel De : Michel [mailto:michel.sales at free.fr] Envoy? : samedi 19 novembre 2016 14:16 ? : 'wiki-support at openssl.org' Objet : wiki update HI, FYI, the page at https://wiki.openssl.org/index.php/Manual:Enc(1) read options ?salt and ?nosalt twice. I believe it would be better to remove the first ?salt section and the second ?nosalt section. Also the statement in the NOTES : ?When the salt is being used the first eight bytes of the encrypted data are reserved for the salt? is wrong. It?s a 16 bytes header including the ?magic? ?Salted__?. (the same applies to : https://www.openssl.org/docs/man1.1.0/apps/enc.html) [ ] Regards, Michel. https://github.com/EasySec -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Wed Jan 11 14:29:26 2017 From: rsalz at akamai.com (Salz, Rich) Date: Wed, 11 Jan 2017 14:29:26 +0000 Subject: [openssl-dev] wiki update for enc command In-Reply-To: References: Message-ID: <12a5511caf85432ca32091aaf44a0546@usma1ex-dag1mb1.msg.corp.akamai.com> That whole ?Manual:? thing on the wiki should probably go away and just refer to the website which gets updated every time we change things. Can you open an issue for the manpage bug you found? Thanks! -- Senior Architect, Akamai Technologies Member, OpenSSL Dev Team IM: richsalz at jabber.at Twitter: RichSalz From: Michel [mailto:michel.sales at free.fr] Sent: Wednesday, January 11, 2017 9:18 AM To: openssl-dev at openssl.org Subject: [openssl-dev] wiki update for enc command Hi, Looks like one of my previous mail (see below) was lost in the ?cloud? ;-) Might be helpfull to send it again here ? Regards, Michel De : Michel [mailto:michel.sales at free.fr] Envoy? : samedi 19 novembre 2016 14:16 ? : 'wiki-support at openssl.org' Objet : wiki update HI, FYI, the page at https://wiki.openssl.org/index.php/Manual:Enc(1) read options ?salt and ?nosalt twice. I believe it would be better to remove the first ?salt section and the second ?nosalt section. Also the statement in the NOTES : ?When the salt is being used the first eight bytes of the encrypted data are reserved for the salt? is wrong. It?s a 16 bytes header including the ?magic? ?Salted__?. (the same applies to : https://www.openssl.org/docs/man1.1.0/apps/enc.html) [?] Regards, Michel. https://github.com/EasySec -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Wed Jan 11 14:34:58 2017 From: levitte at openssl.org (Richard Levitte) Date: Wed, 11 Jan 2017 15:34:58 +0100 (CET) Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> References: <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: <20170111.153458.1623912899597806811.levitte@openssl.org> In message <1e19cdfea8224717b3eee11e2d8acda5 at usma1ex-dag1mb1.msg.corp.akamai.com> on Wed, 11 Jan 2017 03:13:39 +0000, "Salz, Rich" said: rsalz> The needs for OpenSSL's LHASH are exactly what SipHash was designed for: fast on short strings. rsalz> OpenSSL's hash currently *does not* call MD5 or SHA1; the MD5 code is commented out. rsalz> Yes, performance tests would greatly inform the decision. Done, using the reference siphash implementation. https://github.com/levitte/openssl/tree/test-string-hashes A run on my laptop gave these results: : ; ./util/shlib_wrap.sh apps/openssl speed siphash lhash Doing lhash for 3s on 16 size blocks: 27635188 lhash's in 3.00s Doing lhash for 3s on 64 size blocks: 6934726 lhash's in 3.00s Doing lhash for 3s on 256 size blocks: 1698489 lhash's in 3.00s Doing lhash for 3s on 1024 size blocks: 431185 lhash's in 3.00s Doing lhash for 3s on 8192 size blocks: 53868 lhash's in 3.00s Doing lhash for 3s on 16384 size blocks: 27041 lhash's in 3.00s Doing siphash for 3s on 16 size blocks: 22488748 siphash's in 3.00s Doing siphash for 3s on 64 size blocks: 10485674 siphash's in 3.00s Doing siphash for 3s on 256 size blocks: 3320898 siphash's in 3.00s Doing siphash for 3s on 1024 size blocks: 894647 siphash's in 3.00s Doing siphash for 3s on 8192 size blocks: 114170 siphash's in 3.00s Doing siphash for 3s on 16384 size blocks: 57151 siphash's in 3.00s OpenSSL 1.1.1-dev xx XXX xxxx built on: reproducible build, date unspecified options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) blowfish(ptr) compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\"" -DDEBUG_UNUSED -Wswitch -DPEDANTIC -pedantic -Wno-long-long -Wall -Wsign-compare -Wmissing-prototypes -Wshadow -Wformat -Wtype-limits -Werror -Wa,--noexecstack The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes lhash 147387.67k 147940.82k 144937.73k 147177.81k 147095.55k 147679.91k siphash 119939.99k 223694.38k 283383.30k 305372.84k 311760.21k 312120.66k So it seems that for short strings, OPENSSL_LH_strhash (*) wins some, while siphash wins big for larger strings. I have no idea how they compare with regard to distribution (which, considering I ask for the same size output from both, should be the main factor that affects the sensitivity to hash flooding)... Our use of OPENSSL_LH_strhash() is with configuration sections and names, ASN.1 object names and the function names in the openssl app. All our other uses of lhash use their own hashing functions, and are usually very short still (definitely less than 16 bytes). My conclusion is that performance-wise, siphash doesn't give us any advantage over OpenSSL_LH_strhash for our uses. Cheers, Richard (*) Strictly speaking, it's a modified version that takes a length and tolerates all 8-bit bytes, including 0x00. -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From michel.sales at free.fr Wed Jan 11 14:36:29 2017 From: michel.sales at free.fr (Michel) Date: Wed, 11 Jan 2017 15:36:29 +0100 Subject: [openssl-dev] wiki update for enc command In-Reply-To: <12a5511caf85432ca32091aaf44a0546@usma1ex-dag1mb1.msg.corp.akamai.com> References: <12a5511caf85432ca32091aaf44a0546@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: <004401d26c18$19bdf4e0$4d39dea0$@sales@free.fr> > Can you open an issue for the manpage bug you found? Yes, I will. -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Wed Jan 11 14:43:20 2017 From: levitte at openssl.org (Richard Levitte) Date: Wed, 11 Jan 2017 15:43:20 +0100 (CET) Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <20170111.153458.1623912899597806811.levitte@openssl.org> References: <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> <20170111.153458.1623912899597806811.levitte@openssl.org> Message-ID: <20170111.154320.1733542398711982655.levitte@openssl.org> A note: I have absolutely nothing against the addition of SIPhash in our collection of hash algos. My scepticism was only in regards to using it as a string hasher for our hash tables indexes. Cheers, Richard In message <20170111.153458.1623912899597806811.levitte at openssl.org> on Wed, 11 Jan 2017 15:34:58 +0100 (CET), Richard Levitte said: levitte> In message <1e19cdfea8224717b3eee11e2d8acda5 at usma1ex-dag1mb1.msg.corp.akamai.com> on Wed, 11 Jan 2017 03:13:39 +0000, "Salz, Rich" said: levitte> levitte> rsalz> The needs for OpenSSL's LHASH are exactly what SipHash was designed for: fast on short strings. levitte> rsalz> OpenSSL's hash currently *does not* call MD5 or SHA1; the MD5 code is commented out. levitte> rsalz> Yes, performance tests would greatly inform the decision. levitte> levitte> Done, using the reference siphash implementation. levitte> levitte> https://github.com/levitte/openssl/tree/test-string-hashes levitte> levitte> A run on my laptop gave these results: levitte> levitte> : ; ./util/shlib_wrap.sh apps/openssl speed siphash lhash levitte> Doing lhash for 3s on 16 size blocks: 27635188 lhash's in 3.00s levitte> Doing lhash for 3s on 64 size blocks: 6934726 lhash's in 3.00s levitte> Doing lhash for 3s on 256 size blocks: 1698489 lhash's in 3.00s levitte> Doing lhash for 3s on 1024 size blocks: 431185 lhash's in 3.00s levitte> Doing lhash for 3s on 8192 size blocks: 53868 lhash's in 3.00s levitte> Doing lhash for 3s on 16384 size blocks: 27041 lhash's in 3.00s levitte> Doing siphash for 3s on 16 size blocks: 22488748 siphash's in 3.00s levitte> Doing siphash for 3s on 64 size blocks: 10485674 siphash's in 3.00s levitte> Doing siphash for 3s on 256 size blocks: 3320898 siphash's in 3.00s levitte> Doing siphash for 3s on 1024 size blocks: 894647 siphash's in 3.00s levitte> Doing siphash for 3s on 8192 size blocks: 114170 siphash's in 3.00s levitte> Doing siphash for 3s on 16384 size blocks: 57151 siphash's in 3.00s levitte> OpenSSL 1.1.1-dev xx XXX xxxx levitte> built on: reproducible build, date unspecified levitte> options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) blowfish(ptr) levitte> compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\"" -DDEBUG_UNUSED -Wswitch -DPEDANTIC -pedantic -Wno-long-long -Wall -Wsign-compare -Wmissing-prototypes -Wshadow -Wformat -Wtype-limits -Werror -Wa,--noexecstack levitte> The 'numbers' are in 1000s of bytes per second processed. levitte> type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes levitte> lhash 147387.67k 147940.82k 144937.73k 147177.81k 147095.55k 147679.91k levitte> siphash 119939.99k 223694.38k 283383.30k 305372.84k 311760.21k 312120.66k levitte> levitte> So it seems that for short strings, OPENSSL_LH_strhash (*) wins some, levitte> while siphash wins big for larger strings. levitte> levitte> I have no idea how they compare with regard to distribution (which, levitte> considering I ask for the same size output from both, should be the levitte> main factor that affects the sensitivity to hash flooding)... levitte> levitte> Our use of OPENSSL_LH_strhash() is with configuration sections and levitte> names, ASN.1 object names and the function names in the openssl app. levitte> All our other uses of lhash use their own hashing functions, and are levitte> usually very short still (definitely less than 16 bytes). levitte> levitte> My conclusion is that performance-wise, siphash doesn't give us any levitte> advantage over OpenSSL_LH_strhash for our uses. levitte> levitte> Cheers, levitte> Richard levitte> levitte> (*) Strictly speaking, it's a modified version that takes a length and levitte> tolerates all 8-bit bytes, including 0x00. levitte> levitte> -- levitte> Richard Levitte levitte at openssl.org levitte> OpenSSL Project http://www.openssl.org/~levitte/ levitte> -- levitte> openssl-dev mailing list levitte> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev levitte> From levitte at openssl.org Wed Jan 11 14:44:17 2017 From: levitte at openssl.org (Richard Levitte) Date: Wed, 11 Jan 2017 15:44:17 +0100 (CET) Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> References: <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> Message-ID: <20170111.154417.2302923138478061243.levitte@openssl.org> Can we look forward to a github PR? In message <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127 at akamai.com> on Tue, 10 Jan 2017 22:42:17 +0000, "Short, Todd" said: tshort> I think I might have an init/update/final version of siphash24 lying tshort> around somewhere that would be compatible with OpenSSL?s EVP_PKEY tshort> mechanism (similar to Poly1305, in that it needs a key). tshort> -- tshort> -Todd Short tshort> // tshort at akamai.com tshort> // "One if by land, two if by sea, three if by the Internet." tshort> tshort> On Jan 10, 2017, at 4:55 PM, Richard Levitte tshort> wrote: tshort> tshort> tshort> tshort> tshort> Benjamin Kaduk skrev: (10 januari 2017 tshort> 20:19:21 CET) tshort> tshort> On 01/10/2017 12:31 PM, Richard Levitte wrote: tshort> tshort> tshort> Benjamin Kaduk skrev: (10 januari 2017 tshort> 18:48:32 tshort> tshort> tshort> CET) tshort> tshort> On 01/09/2017 10:05 PM, Salz, Rich wrote: tshort> tshort> Should we move to using SIPHash for the default string tshort> hashing tshort> function in OpenSSL? It?s now in the kernel tshort> https://lkml.org/lkml/2017/1/9/619 tshort> tshort> tshort> tshort> Heck, yes! tshort> -Ben tshort> tshort> tshort> I fail to see what that would give us. OPENSSL_LH_strhash tshort> () is used tshort> tshort> tshort> to get a reasonable index for LHASH entries. Also SIPhash tshort> gives at tshort> least 64 bits results, do we really expect to see large enough tshort> hash tshort> tables to warrant that? tshort> tshort> tshort> tshort> tshort> We don't need to use the full output width of a good hash tshort> function. tshort> tshort> My main point is, "why would we want to ignore the last 20 tshort> years of tshort> advancement in hash function research?" Section 7 of the tshort> siphash paper tshort> (https://131002.net/siphash/siphash.pdf) explicitly talks tshort> about using tshort> it tshort> for hash tables, including using hash table indices H(m) mod tshort> l. tshort> tshort> tshort> I agree with the advice when one can expect huge tables. The tshort> tables we handle are pretty small (I think, please correct me if tshort> I'm wrong) and would in all likelihood not benefit very much if at tshort> all from SIPhash's relative safety. tshort> tshort> Of course, one can ask the question if someone uses LHASH as a tshort> general purpose hash table implementation rather than just for the tshort> stuff OpenSSL. Frankly, I would probably look at a dedicated hash tshort> table library first... tshort> tshort> Cheers tshort> Richard tshort> -- tshort> Sent from my Android device with K-9 Mail. Please excuse my tshort> brevity. tshort> -- tshort> openssl-dev mailing list tshort> To unsubscribe: tshort> https://mta.openssl.org/mailman/listinfo/openssl-dev tshort> From rsalz at akamai.com Wed Jan 11 15:36:38 2017 From: rsalz at akamai.com (Salz, Rich) Date: Wed, 11 Jan 2017 15:36:38 +0000 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <1484125361.12353.5.camel@redhat.com> References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> <1484125361.12353.5.camel@redhat.com> Message-ID: <300d0cd433314c3b80d2c1272ba55de4@usma1ex-dag1mb1.msg.corp.akamai.com> > Is there really no use of LHASH tables in OpenSSL where an attacker > attempting a DoS attack can control the contents of the tables? The only use of LHASH is in SSL_SESSION and X509_NAME, which use their own hashing functions, and are only used after the session and/or certs have been cryptographically verified. From tshort at akamai.com Wed Jan 11 16:04:14 2017 From: tshort at akamai.com (Short, Todd) Date: Wed, 11 Jan 2017 16:04:14 +0000 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <20170111.154417.2302923138478061243.levitte@openssl.org> References: <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <20170111.154417.2302923138478061243.levitte@openssl.org> Message-ID: <2C383C34-DAE1-4AF4-A2ED-8CB427D13D3B@akamai.com> I?d be doing it in a manner similar to Poly1305, since that?s a fresh memory? it shouldn?t take long. -- -Todd Short // tshort at akamai.com // "One if by land, two if by sea, three if by the Internet." On Jan 11, 2017, at 9:44 AM, Richard Levitte > wrote: Can we look forward to a github PR? In message <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127 at akamai.com> on Tue, 10 Jan 2017 22:42:17 +0000, "Short, Todd" > said: tshort> I think I might have an init/update/final version of siphash24 lying tshort> around somewhere that would be compatible with OpenSSL?s EVP_PKEY tshort> mechanism (similar to Poly1305, in that it needs a key). tshort> -- tshort> -Todd Short tshort> // tshort at akamai.com tshort> // "One if by land, two if by sea, three if by the Internet." tshort> tshort> On Jan 10, 2017, at 4:55 PM, Richard Levitte > tshort> wrote: tshort> tshort> tshort> tshort> tshort> Benjamin Kaduk > skrev: (10 januari 2017 tshort> 20:19:21 CET) tshort> tshort> On 01/10/2017 12:31 PM, Richard Levitte wrote: tshort> tshort> tshort> Benjamin Kaduk > skrev: (10 januari 2017 tshort> 18:48:32 tshort> tshort> tshort> CET) tshort> tshort> On 01/09/2017 10:05 PM, Salz, Rich wrote: tshort> tshort> Should we move to using SIPHash for the default string tshort> hashing tshort> function in OpenSSL? It?s now in the kernel tshort> https://lkml.org/lkml/2017/1/9/619 tshort> tshort> tshort> tshort> Heck, yes! tshort> -Ben tshort> tshort> tshort> I fail to see what that would give us. OPENSSL_LH_strhash tshort> () is used tshort> tshort> tshort> to get a reasonable index for LHASH entries. Also SIPhash tshort> gives at tshort> least 64 bits results, do we really expect to see large enough tshort> hash tshort> tables to warrant that? tshort> tshort> tshort> tshort> tshort> We don't need to use the full output width of a good hash tshort> function. tshort> tshort> My main point is, "why would we want to ignore the last 20 tshort> years of tshort> advancement in hash function research?" Section 7 of the tshort> siphash paper tshort> (https://131002.net/siphash/siphash.pdf) explicitly talks tshort> about using tshort> it tshort> for hash tables, including using hash table indices H(m) mod tshort> l. tshort> tshort> tshort> I agree with the advice when one can expect huge tables. The tshort> tables we handle are pretty small (I think, please correct me if tshort> I'm wrong) and would in all likelihood not benefit very much if at tshort> all from SIPhash's relative safety. tshort> tshort> Of course, one can ask the question if someone uses LHASH as a tshort> general purpose hash table implementation rather than just for the tshort> stuff OpenSSL. Frankly, I would probably look at a dedicated hash tshort> table library first... tshort> tshort> Cheers tshort> Richard tshort> -- tshort> Sent from my Android device with K-9 Mail. Please excuse my tshort> brevity. tshort> -- tshort> openssl-dev mailing list tshort> To unsubscribe: tshort> https://mta.openssl.org/mailman/listinfo/openssl-dev tshort> -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From dnsands at sandia.gov Wed Jan 11 18:09:04 2017 From: dnsands at sandia.gov (Sands, Daniel) Date: Wed, 11 Jan 2017 18:09:04 +0000 Subject: [openssl-dev] [EXTERNAL] Re: use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <20170111.154320.1733542398711982655.levitte@openssl.org> References: <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> <20170111.153458.1623912899597806811.levitte@openssl.org> <20170111.154320.1733542398711982655.levitte@openssl.org> Message-ID: <1484158144.3968.123.camel@sandia.gov> Just a note from my own experience way back when: I tried hashing using various algos and measuring bucket use as the main comparison criteria. I found that the crypto hashes left a fair number of unused buckets. Of course, CRCs were far worse. What gave the most normal distribution was to simply take the bytes 4 at a time as 32-bit integers and simply add them. Not to say that this is really the holy grail, just pointing out that compute speed shouldn't be the only criteria you use for comparison. On Wed, 2017-01-11 at 15:43 +0100, Richard Levitte wrote: > A note: I have absolutely nothing against the addition of SIPhash in > our collection of hash algos. My scepticism was only in regards to > using it as a string hasher for our hash tables indexes. > > Cheers, > Richard > > In message <20170111.153458.1623912899597806811.levitte at openssl.org> on Wed, 11 Jan 2017 15:34:58 +0100 (CET), Richard Levitte said: > > levitte> In message <1e19cdfea8224717b3eee11e2d8acda5 at usma1ex-dag1mb1.msg.corp.akamai.com> on Wed, 11 Jan 2017 03:13:39 +0000, "Salz, Rich" said: > levitte> > levitte> rsalz> The needs for OpenSSL's LHASH are exactly what SipHash was designed for: fast on short strings. > levitte> rsalz> OpenSSL's hash currently *does not* call MD5 or SHA1; the MD5 code is commented out. > levitte> rsalz> Yes, performance tests would greatly inform the decision. > levitte> > levitte> Done, using the reference siphash implementation. > levitte> > levitte> https://github.com/levitte/openssl/tree/test-string-hashes > levitte> > levitte> A run on my laptop gave these results: > levitte> > levitte> : ; ./util/shlib_wrap.sh apps/openssl speed siphash lhash > levitte> Doing lhash for 3s on 16 size blocks: 27635188 lhash's in 3.00s > levitte> Doing lhash for 3s on 64 size blocks: 6934726 lhash's in 3.00s > levitte> Doing lhash for 3s on 256 size blocks: 1698489 lhash's in 3.00s > levitte> Doing lhash for 3s on 1024 size blocks: 431185 lhash's in 3.00s > levitte> Doing lhash for 3s on 8192 size blocks: 53868 lhash's in 3.00s > levitte> Doing lhash for 3s on 16384 size blocks: 27041 lhash's in 3.00s > levitte> Doing siphash for 3s on 16 size blocks: 22488748 siphash's in 3.00s > levitte> Doing siphash for 3s on 64 size blocks: 10485674 siphash's in 3.00s > levitte> Doing siphash for 3s on 256 size blocks: 3320898 siphash's in 3.00s > levitte> Doing siphash for 3s on 1024 size blocks: 894647 siphash's in 3.00s > levitte> Doing siphash for 3s on 8192 size blocks: 114170 siphash's in 3.00s > levitte> Doing siphash for 3s on 16384 size blocks: 57151 siphash's in 3.00s > levitte> OpenSSL 1.1.1-dev xx XXX xxxx > levitte> built on: reproducible build, date unspecified > levitte> options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) blowfish(ptr) > levitte> compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\"" -DDEBUG_UNUSED -Wswitch -DPEDANTIC -pedantic -Wno-long-long -Wall -Wsign-compare -Wmissing-prototypes -Wshadow -Wformat -Wtype-limits -Werror -Wa,--noexecstack > levitte> The 'numbers' are in 1000s of bytes per second processed. > levitte> type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes > levitte> lhash 147387.67k 147940.82k 144937.73k 147177.81k 147095.55k 147679.91k > levitte> siphash 119939.99k 223694.38k 283383.30k 305372.84k 311760.21k 312120.66k > levitte> > levitte> So it seems that for short strings, OPENSSL_LH_strhash (*) wins some, > levitte> while siphash wins big for larger strings. > levitte> > levitte> I have no idea how they compare with regard to distribution (which, > levitte> considering I ask for the same size output from both, should be the > levitte> main factor that affects the sensitivity to hash flooding)... > levitte> > levitte> Our use of OPENSSL_LH_strhash() is with configuration sections and > levitte> names, ASN.1 object names and the function names in the openssl app. > levitte> All our other uses of lhash use their own hashing functions, and are > levitte> usually very short still (definitely less than 16 bytes). > levitte> > levitte> My conclusion is that performance-wise, siphash doesn't give us any > levitte> advantage over OpenSSL_LH_strhash for our uses. > levitte> > levitte> Cheers, > levitte> Richard > levitte> > levitte> (*) Strictly speaking, it's a modified version that takes a length and > levitte> tolerates all 8-bit bytes, including 0x00. > levitte> > levitte> -- > levitte> Richard Levitte levitte at openssl.org > levitte> OpenSSL Project http://www.openssl.org/~levitte/ > levitte> -- > levitte> openssl-dev mailing list > levitte> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev > levitte> From pwalten at au1.ibm.com Wed Jan 11 22:25:43 2017 From: pwalten at au1.ibm.com (Peter Waltenberg) Date: Thu, 12 Jan 2017 08:25:43 +1000 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: And the reason I said you certainly don't need a keyed hash ? Behaviour of the hash function will change with key and in some cases performance would degenerate to that of a linked list. (Ouch). And since the obvious thing to do is use a random key, OpenSSL's performance would get *very* erratic. Simpler functions than cryptographic hashes will almost certainly yield better results here. I note someone further up the thread someone else has pointed that out. Peter From: "Salz, Rich" To: "openssl-dev at openssl.org" Date: 11/01/2017 13:14 Subject: Re: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? Sent by: "openssl-dev" The needs for OpenSSL's LHASH are exactly what SipHash was designed for: fast on short strings. OpenSSL's hash currently *does not* call MD5 or SHA1; the MD5 code is commented out. Yes, performance tests would greatly inform the decision. -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy.farrell at oracle.com Wed Jan 11 23:29:04 2017 From: jeremy.farrell at oracle.com (J. J. Farrell) Date: Wed, 11 Jan 2017 23:29:04 +0000 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: <7b58d7ec-c6ba-ceee-ef12-fd4b133c74b3@oracle.com> Are the issues you raise true of SipHash, given that a prime motivator for its design was generating hash tables for short inputs while being secure against hash flooding attacks? It achieves this with the performance of a portable C implementation the order of four times faster than MD5, and not much slower than other modern hash algorithms. I'd have thought the main thing to consider is whether or not there is any practical way a hash flooding attack could be used against OpenSSL's hash tables, and it sounds like there isn't. In that case, the fastest algorithm for the usage patterns would be best. Regards, jjf On 11/01/2017 22:25, Peter Waltenberg wrote: > And the reason I said you certainly don't need a keyed hash ? > > Behaviour of the hash function will change with key and in some cases > performance would degenerate to that of a linked list. (Ouch). And > since the obvious thing to do is use a random key, OpenSSL's > performance would get *very* erratic. > > Simpler functions than cryptographic hashes will almost certainly > yield better results here. I note someone further up the thread > someone else has pointed that out. > > Peter > > From: "Salz, Rich" > To: "openssl-dev at openssl.org" > Date: 11/01/2017 13:14 > Subject: Re: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? > Sent by: "openssl-dev" > ------------------------------------------------------------------------ > > The needs for OpenSSL's LHASH are exactly what SipHash was designed > for: fast on short strings. > OpenSSL's hash currently *does not* call MD5 or SHA1; the MD5 code is > commented out. > Yes, performance tests would greatly inform the decision. -- J. J. Farrell Not speaking for Oracle -------------- next part -------------- An HTML attachment was scrubbed... URL: From dnsands at sandia.gov Thu Jan 12 00:39:32 2017 From: dnsands at sandia.gov (Sands, Daniel) Date: Thu, 12 Jan 2017 00:39:32 +0000 Subject: [openssl-dev] [EXTERNAL] Re: use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <7b58d7ec-c6ba-ceee-ef12-fd4b133c74b3@oracle.com> References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> <7b58d7ec-c6ba-ceee-ef12-fd4b133c74b3@oracle.com> Message-ID: <1484181572.3968.135.camel@sandia.gov> With a small number of buckets, it seems to me that no hash algo will make you safe from a flooding attack. You can simply generate your hashes locally using whichever algo the server uses, and only send those that fit into your attack scheme. The data could even be pre-generated. The only way to guard against a flood that makes sense to me is to limit the number of items that can be accepted before deciding you're being trolled. On Wed, 2017-01-11 at 23:29 +0000, J. J. Farrell wrote: > Are the issues you raise true of SipHash, given that a prime motivator > for its design was generating hash tables for short inputs while being > secure against hash flooding attacks? It achieves this with the > performance of a portable C implementation the order of four times > faster than MD5, and not much slower than other modern hash > algorithms. > > I'd have thought the main thing to consider is whether or not there is > any practical way a hash flooding attack could be used against > OpenSSL's hash tables, and it sounds like there isn't. In that case, > the fastest algorithm for the usage patterns would be best. > > Regards, > jjf > > On 11/01/2017 22:25, Peter Waltenberg wrote: > > > And the reason I said you certainly don't need a keyed hash ? > > > > Behaviour of the hash function will change with key and in some > > cases performance would degenerate to that of a linked list. (Ouch). > > And since the obvious thing to do is use a random key, OpenSSL's > > performance would get *very* erratic. > > > > Simpler functions than cryptographic hashes will almost certainly > > yield better results here. I note someone further up the thread > > someone else has pointed that out. > > > > Peter > > > > From: "Salz, Rich" > > To: "openssl-dev at openssl.org" > > Date: 11/01/2017 13:14 > > Subject: Re: [openssl-dev] use SIPhash for > > OPENSSL_LH_strhash? > > Sent by: "openssl-dev" > > > > ____________________________________________________________________ > > > > The needs for OpenSSL's LHASH are exactly what SipHash was designed > > for: fast on short strings. > > OpenSSL's hash currently *does not* call MD5 or SHA1; the MD5 code > > is commented out. > > Yes, performance tests would greatly inform the decision. > > -- > J. J. Farrell > Not speaking for Oracle > -- > openssl-dev mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev From jeremy.farrell at oracle.com Thu Jan 12 01:16:49 2017 From: jeremy.farrell at oracle.com (Jeremy Farrell) Date: Thu, 12 Jan 2017 01:16:49 +0000 Subject: [openssl-dev] [EXTERNAL] Re: use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <1484181572.3968.135.camel@sandia.gov> References: <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> <7b58d7ec-c6ba-ceee-ef12-fd4b133c74b3@oracle.com> <1484181572.3968.135.camel@sandia.gov> Message-ID: <5bedab81-a094-003e-94c6-d316e74b5fcf@oracle.com> For something like SipHash, knowing "whichever algo the server uses" effectively implies knowing the 128-bit random key currently being used for the hash table in question. Regards, jjf On 12/01/2017 00:39, Sands, Daniel wrote: > With a small number of buckets, it seems to me that no hash algo will > make you safe from a flooding attack. You can simply generate your > hashes locally using whichever algo the server uses, and only send those > that fit into your attack scheme. The data could even be pre-generated. > > The only way to guard against a flood that makes sense to me is to limit > the number of items that can be accepted before deciding you're being > trolled. > > On Wed, 2017-01-11 at 23:29 +0000, J. J. Farrell wrote: >> Are the issues you raise true of SipHash, given that a prime motivator >> for its design was generating hash tables for short inputs while being >> secure against hash flooding attacks? It achieves this with the >> performance of a portable C implementation the order of four times >> faster than MD5, and not much slower than other modern hash >> algorithms. >> >> I'd have thought the main thing to consider is whether or not there is >> any practical way a hash flooding attack could be used against >> OpenSSL's hash tables, and it sounds like there isn't. In that case, >> the fastest algorithm for the usage patterns would be best. >> >> Regards, >> jjf >> >> On 11/01/2017 22:25, Peter Waltenberg wrote: >> >>> And the reason I said you certainly don't need a keyed hash ? >>> >>> Behaviour of the hash function will change with key and in some >>> cases performance would degenerate to that of a linked list. (Ouch). >>> And since the obvious thing to do is use a random key, OpenSSL's >>> performance would get *very* erratic. >>> >>> Simpler functions than cryptographic hashes will almost certainly >>> yield better results here. I note someone further up the thread >>> someone else has pointed that out. >>> >>> Peter >>> >>> From: "Salz, Rich" >>> To: "openssl-dev at openssl.org" >>> Date: 11/01/2017 13:14 >>> Subject: Re: [openssl-dev] use SIPhash for >>> OPENSSL_LH_strhash? >>> Sent by: "openssl-dev" >>> >>> ____________________________________________________________________ >>> >>> The needs for OpenSSL's LHASH are exactly what SipHash was designed >>> for: fast on short strings. >>> OpenSSL's hash currently *does not* call MD5 or SHA1; the MD5 code >>> is commented out. >>> Yes, performance tests would greatly inform the decision. >> -- >> J. J. Farrell >> Not speaking for Oracle >> -- >> openssl-dev mailing list >> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -- J. J. Farrell w: +44 161 493 4838 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pwalten at au1.ibm.com Thu Jan 12 06:39:28 2017 From: pwalten at au1.ibm.com (Peter Waltenberg) Date: Thu, 12 Jan 2017 06:39:28 +0000 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <7b58d7ec-c6ba-ceee-ef12-fd4b133c74b3@oracle.com> References: <7b58d7ec-c6ba-ceee-ef12-fd4b133c74b3@oracle.com>, <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> Message-ID: An HTML attachment was scrubbed... URL: From pwalten at au1.ibm.com Thu Jan 12 06:41:11 2017 From: pwalten at au1.ibm.com (Peter Waltenberg) Date: Thu, 12 Jan 2017 06:41:11 +0000 Subject: [openssl-dev] [EXTERNAL] Re: use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <5bedab81-a094-003e-94c6-d316e74b5fcf@oracle.com> References: <5bedab81-a094-003e-94c6-d316e74b5fcf@oracle.com>, <93ef7352d3fd4237b28e3d7ea6095a2a@usma1ex-dag1mb1.msg.corp.akamai.com> <06666C1D-0F2C-44D5-91F2-DA2145F67AAD@openssl.org> <9F0185AB-2074-4837-9ED5-41B2CC0F41F5@openssl.org> <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> <7b58d7ec-c6ba-ceee-ef12-fd4b133c74b3@oracle.com> <1484181572.3968.135.camel@sandia.gov> Message-ID: An HTML attachment was scrubbed... URL: From hkario at redhat.com Thu Jan 12 12:23:24 2017 From: hkario at redhat.com (Hubert Kario) Date: Thu, 12 Jan 2017 13:23:24 +0100 Subject: [openssl-dev] X25519 is the default curve for ECDHE in OpenSSL 1.1.0 In-Reply-To: <1653994.qcj3bjmN45@pintsize.usersys.redhat.com> References: <1653994.qcj3bjmN45@pintsize.usersys.redhat.com> Message-ID: <1947098.MorAexmU5K@pintsize.usersys.redhat.com> On Friday, 16 September 2016 17:26:03 CET Hubert Kario wrote: > I've been running tests on the openssl 1.1.0 release recently and I've > noticed that if the client doesn't include the supported_groups extension, > OpenSSL will pick curve with id 0x001d, that is ecdh_x25519, as the curve > to do ECDHE over. > > While this is not incorrect behaviour according to the standard (it is quite > explicit that if client doesn't provide this extension, server can pick any > curve it wants), I'm afraid that this will cause interoperability problems. > > The majority of servers (71%) support *only* prime256v1 curve and of the > ones that default to ECDHE key exchange nearly 83% will also default to > this curve. OpenSSL 1.0.2h also defaults to this curve if there are no > curves advertised by client. > > So it is very likely that any client that doesn't advertise curves will > expect the server to select prime256v1. At the same time it is very > unlikely that it will support x25519 (given how new it is). I've filed a bug on github so that it doesn't fall off the radar... https://github.com/openssl/openssl/issues/2219 -- Regards, Hubert Kario Senior Quality Engineer, QE BaseOS Security team Web: www.cz.redhat.com Red Hat Czech s.r.o., Purky?ova 99/71, 612 45, Brno, Czech Republic -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part. URL: From hkario at redhat.com Thu Jan 12 14:57:39 2017 From: hkario at redhat.com (Hubert Kario) Date: Thu, 12 Jan 2017 15:57:39 +0100 Subject: [openssl-dev] [EXTERNAL] Re: use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <1484158144.3968.123.camel@sandia.gov> References: <20170111.154320.1733542398711982655.levitte@openssl.org> <1484158144.3968.123.camel@sandia.gov> Message-ID: <6037864.WzqnMTiZox@pintsize.usersys.redhat.com> On Wednesday, 11 January 2017 18:09:04 CET Sands, Daniel wrote: > Just a note from my own experience way back when: I tried hashing using > various algos and measuring bucket use as the main comparison criteria. > I found that the crypto hashes left a fair number of unused buckets. I can explain that only in two ways: 1. "crypto hashes" were biased 2. the test used too many buckets compared to number of entries to put it mildly, option 1 is "unlikely" so since I don't want to point fingers, I'd say we need more data before we can discredit use of cryptographically secure hashes for this use case... > Of > course, CRCs were far worse. What gave the most normal distribution was > to simply take the bytes 4 at a time as 32-bit integers and simply add > them. > > Not to say that this is really the holy grail, just pointing out that > compute speed shouldn't be the only criteria you use for comparison. > > On Wed, 2017-01-11 at 15:43 +0100, Richard Levitte wrote: > > A note: I have absolutely nothing against the addition of SIPhash in > > our collection of hash algos. My scepticism was only in regards to > > using it as a string hasher for our hash tables indexes. > > > > Cheers, > > Richard > > > > In message <20170111.153458.1623912899597806811.levitte at openssl.org> on > > Wed, 11 Jan 2017 15:34:58 +0100 (CET), Richard Levitte > > said: > > > > levitte> In message > > <1e19cdfea8224717b3eee11e2d8acda5 at usma1ex-dag1mb1.msg.corp.akamai.com> on > > Wed, 11 Jan 2017 03:13:39 +0000, "Salz, Rich" said: > > levitte> > > levitte> rsalz> The needs for OpenSSL's LHASH are exactly what SipHash was > > designed for: fast on short strings. levitte> rsalz> OpenSSL's hash > > currently *does not* call MD5 or SHA1; the MD5 code is commented out. > > levitte> rsalz> Yes, performance tests would greatly inform the decision. > > levitte> > > levitte> Done, using the reference siphash implementation. > > levitte> > > levitte> https://github.com/levitte/openssl/tree/test-string-hashes > > levitte> > > levitte> A run on my laptop gave these results: > > levitte> > > levitte> : ; ./util/shlib_wrap.sh apps/openssl speed siphash lhash > > levitte> Doing lhash for 3s on 16 size blocks: 27635188 lhash's in > > 3.00s levitte> Doing lhash for 3s on 64 size blocks: 6934726 lhash's > > in 3.00s levitte> Doing lhash for 3s on 256 size blocks: 1698489 > > lhash's in 3.00s levitte> Doing lhash for 3s on 1024 size blocks: > > 431185 lhash's in 3.00s levitte> Doing lhash for 3s on 8192 size > > blocks: 53868 lhash's in 3.00s levitte> Doing lhash for 3s on 16384 > > size blocks: 27041 lhash's in 3.00s levitte> Doing siphash for 3s on > > 16 size blocks: 22488748 siphash's in 3.00s levitte> Doing siphash > > for 3s on 64 size blocks: 10485674 siphash's in 3.00s levitte> Doing > > siphash for 3s on 256 size blocks: 3320898 siphash's in 3.00s levitte> > > Doing siphash for 3s on 1024 size blocks: 894647 siphash's in 3.00s > > levitte> Doing siphash for 3s on 8192 size blocks: 114170 siphash's > > in 3.00s levitte> Doing siphash for 3s on 16384 size blocks: 57151 > > siphash's in 3.00s levitte> OpenSSL 1.1.1-dev xx XXX xxxx > > levitte> built on: reproducible build, date unspecified > > levitte> options:bn(64,64) rc4(16x,int) des(int) aes(partial) > > idea(int) blowfish(ptr) levitte> compiler: gcc -DDSO_DLFCN > > -DHAVE_DLFCN_H -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC > > -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 > > -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM > > -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM > > -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM > > -DOPENSSLDIR="\"/usr/local/ssl\"" > > -DENGINESDIR="\"/usr/local/lib/engines-1.1\"" -DDEBUG_UNUSED -Wswitch > > -DPEDANTIC -pedantic -Wno-long-long -Wall -Wsign-compare > > -Wmissing-prototypes -Wshadow -Wformat -Wtype-limits -Werror > > -Wa,--noexecstack levitte> The 'numbers' are in 1000s of bytes per > > second processed. levitte> type 16 bytes 64 bytes > > 256 bytes 1024 bytes 8192 bytes 16384 bytes levitte> lhash > > 147387.67k 147940.82k 144937.73k 147177.81k 147095.55k > > 147679.91k levitte> siphash 119939.99k 223694.38k > > 283383.30k 305372.84k 311760.21k 312120.66k levitte> > > levitte> So it seems that for short strings, OPENSSL_LH_strhash (*) wins > > some, levitte> while siphash wins big for larger strings. > > levitte> > > levitte> I have no idea how they compare with regard to distribution > > (which, levitte> considering I ask for the same size output from both, > > should be the levitte> main factor that affects the sensitivity to hash > > flooding)... levitte> > > levitte> Our use of OPENSSL_LH_strhash() is with configuration sections > > and > > levitte> names, ASN.1 object names and the function names in the openssl > > app. levitte> All our other uses of lhash use their own hashing > > functions, and are levitte> usually very short still (definitely less > > than 16 bytes). levitte> > > levitte> My conclusion is that performance-wise, siphash doesn't give us > > any levitte> advantage over OpenSSL_LH_strhash for our uses. > > levitte> > > levitte> Cheers, > > levitte> Richard > > levitte> > > levitte> (*) Strictly speaking, it's a modified version that takes a > > length and levitte> tolerates all 8-bit bytes, including 0x00. > > levitte> > > levitte> -- > > levitte> Richard Levitte levitte at openssl.org > > levitte> OpenSSL Project http://www.openssl.org/~levitte/ > > levitte> -- > > levitte> openssl-dev mailing list > > levitte> To unsubscribe: > > https://mta.openssl.org/mailman/listinfo/openssl-dev levitte> -- Regards, Hubert Kario Senior Quality Engineer, QE BaseOS Security team Web: www.cz.redhat.com Red Hat Czech s.r.o., Purky?ova 99/71, 612 45, Brno, Czech Republic -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part. URL: From bkaduk at akamai.com Thu Jan 12 17:17:10 2017 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Thu, 12 Jan 2017 11:17:10 -0600 Subject: [openssl-dev] [openssl-commits] UI_METHOD In-Reply-To: <1484155657.149323.23405.nullmailer@dev.openssl.org> References: <1484155657.149323.23405.nullmailer@dev.openssl.org> Message-ID: <9da4cbdc-7437-b942-0d2e-e05808cd1e28@akamai.com> On 01/11/2017 11:27 AM, Richard Levitte wrote: > The branch master has been updated > via 66ed24b1624606593a23c9fe78d459718d26409c (commit) > via 78b19e90b4aade1ffdf8d918277910b01dac1d76 (commit) > via cc10f2275594eac2bcdaa218c1d6f672c2b891b7 (commit) > via 3ab3c8cb274aca1b5c0fbf83af96e49baf422708 (commit) > via 0fe1fc858a0519c3866c0d2e88513e677b674926 (commit) > via 18cfc668eae2c296e9bc90ffc989d9bbe61cc82f (commit) > via a223ffe6d35209c59ed498ee89d1bac6e82e2ac2 (commit) > via 264b2d92511572a247ecb673d61ff385deb9eb8d (commit) > from 5eeb6c6e562937dcfdd4b79619a699a118deadba (commit) > +static int ui_method_data_index() > +{ > + static int idx = -1; > + > + if (idx == -1) > + idx = CRYPTO_get_ex_new_index(CRYPTO_EX_INDEX_UI_METHOD, > + 0, NULL, > + ui_new_method_data, > + ui_dup_method_data, > + ui_free_method_data); Should this have some synchronization around it to prevent concurrent checks in multiple threads? It looks like we use a lock for our internal stuff, but a run_once ought to be fine, too. -Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Thu Jan 12 19:09:28 2017 From: levitte at openssl.org (Richard Levitte) Date: Thu, 12 Jan 2017 20:09:28 +0100 (CET) Subject: [openssl-dev] [openssl-commits] UI_METHOD In-Reply-To: <9da4cbdc-7437-b942-0d2e-e05808cd1e28@akamai.com> References: <1484155657.149323.23405.nullmailer@dev.openssl.org> <9da4cbdc-7437-b942-0d2e-e05808cd1e28@akamai.com> Message-ID: <20170112.200928.37785830485742648.levitte@openssl.org> In message <9da4cbdc-7437-b942-0d2e-e05808cd1e28 at akamai.com> on Thu, 12 Jan 2017 11:17:10 -0600, Benjamin Kaduk said: bkaduk> On 01/11/2017 11:27 AM, Richard Levitte wrote: bkaduk> bkaduk> The branch master has been updated bkaduk> via 66ed24b1624606593a23c9fe78d459718d26409c (commit) bkaduk> via 78b19e90b4aade1ffdf8d918277910b01dac1d76 (commit) bkaduk> via cc10f2275594eac2bcdaa218c1d6f672c2b891b7 (commit) bkaduk> via 3ab3c8cb274aca1b5c0fbf83af96e49baf422708 (commit) bkaduk> via 0fe1fc858a0519c3866c0d2e88513e677b674926 (commit) bkaduk> via 18cfc668eae2c296e9bc90ffc989d9bbe61cc82f (commit) bkaduk> via a223ffe6d35209c59ed498ee89d1bac6e82e2ac2 (commit) bkaduk> via 264b2d92511572a247ecb673d61ff385deb9eb8d (commit) bkaduk> from 5eeb6c6e562937dcfdd4b79619a699a118deadba (commit) bkaduk> bkaduk> +static int ui_method_data_index() bkaduk> +{ bkaduk> + static int idx = -1; bkaduk> + bkaduk> + if (idx == -1) bkaduk> + idx = CRYPTO_get_ex_new_index(CRYPTO_EX_INDEX_UI_METHOD, bkaduk> + 0, NULL, bkaduk> + ui_new_method_data, bkaduk> + ui_dup_method_data, bkaduk> + ui_free_method_data); bkaduk> bkaduk> Should this have some synchronization around it to prevent concurrent bkaduk> checks in multiple threads? It looks like we use a lock for our bkaduk> internal stuff, but a run_once ought to be fine, too. Good point... and quite easily done. Incidently, include/internal/thread_once.h isn't telling the truth about the return value of RUN_ONCE... I'll take advantage of that! -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From bkaduk at akamai.com Thu Jan 12 20:07:54 2017 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Thu, 12 Jan 2017 14:07:54 -0600 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <20170111.154320.1733542398711982655.levitte@openssl.org> References: <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> <20170111.153458.1623912899597806811.levitte@openssl.org> <20170111.154320.1733542398711982655.levitte@openssl.org> Message-ID: On 01/11/2017 08:43 AM, Richard Levitte wrote: > A note: I have absolutely nothing against the addition of SIPhash in > our collection of hash algos. My scepticism was only in regards to > using it as a string hasher for our hash tables indexes. > Understood. Can you further clarify whether you would like to maintain the existing 20-year-old hand-rolled hash function for that purpose or are open to using a more modern hash (not necessarily SIPhash; there are also things like the Jenkins hash to consider)? -Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Thu Jan 12 20:19:46 2017 From: levitte at openssl.org (Richard Levitte) Date: Thu, 12 Jan 2017 21:19:46 +0100 (CET) Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: References: <20170111.153458.1623912899597806811.levitte@openssl.org> <20170111.154320.1733542398711982655.levitte@openssl.org> Message-ID: <20170112.211946.1731629422103762583.levitte@openssl.org> In message on Thu, 12 Jan 2017 14:07:54 -0600, Benjamin Kaduk said: bkaduk> On 01/11/2017 08:43 AM, Richard Levitte wrote: bkaduk> bkaduk> A note: I have absolutely nothing against the addition of SIPhash in bkaduk> our collection of hash algos. My scepticism was only in regards to bkaduk> using it as a string hasher for our hash tables indexes. bkaduk> bkaduk> Understood. Can you further clarify whether you would like to maintain bkaduk> the existing 20-year-old hand-rolled hash function for that purpose or bkaduk> are open to using a more modern hash (not necessarily SIPhash; there bkaduk> are also things like the Jenkins hash to consider)? If it makes sense, yes. I found SMhasher (*) and started playing with it for this very purpose. Just for kicks, I added OPENSSL_LH_strhash_ex to it and am currently doing a huge run of all hashes it knows. I did a test with Google's FarmHash, and that one shows promise, speed wise (it performs twice as fast as OPENSSL_LH_strhash_ex). Cheers, Richard (*) https://github.com/rurban/smhasher -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From bkaduk at akamai.com Thu Jan 12 20:28:29 2017 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Thu, 12 Jan 2017 14:28:29 -0600 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <20170112.211946.1731629422103762583.levitte@openssl.org> References: <20170111.153458.1623912899597806811.levitte@openssl.org> <20170111.154320.1733542398711982655.levitte@openssl.org> <20170112.211946.1731629422103762583.levitte@openssl.org> Message-ID: On 01/12/2017 02:19 PM, Richard Levitte wrote: > In message on Thu, 12 Jan 2017 14:07:54 -0600, Benjamin Kaduk said: > > bkaduk> On 01/11/2017 08:43 AM, Richard Levitte wrote: > bkaduk> > bkaduk> A note: I have absolutely nothing against the addition of SIPhash in > bkaduk> our collection of hash algos. My scepticism was only in regards to > bkaduk> using it as a string hasher for our hash tables indexes. > bkaduk> > bkaduk> Understood. Can you further clarify whether you would like to maintain > bkaduk> the existing 20-year-old hand-rolled hash function for that purpose or > bkaduk> are open to using a more modern hash (not necessarily SIPhash; there > bkaduk> are also things like the Jenkins hash to consider)? > > If it makes sense, yes. I found SMhasher (*) and started playing with > it for this very purpose. Just for kicks, I added OPENSSL_LH_strhash_ex > to it and am currently doing a huge run of all hashes it knows. > > I did a test with Google's FarmHash, and that one shows promise, speed > wise (it performs twice as fast as OPENSSL_LH_strhash_ex). > Cool! I look forward to seeing the results :) -Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Thu Jan 12 20:51:35 2017 From: rsalz at akamai.com (Salz, Rich) Date: Thu, 12 Jan 2017 20:51:35 +0000 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: References: <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> <20170111.153458.1623912899597806811.levitte@openssl.org> <20170111.154320.1733542398711982655.levitte@openssl.org> Message-ID: > Understood.? Can you further clarify whether you would like to maintain the existing 20-year-old hand-rolled hash function for that purpose or are open to using a more modern hash (not necessarily SIPhash; there are also things like the Jenkins hash to consider)? Because it works, because nobody has looked at our touched that code, because there are no security concerns and it's unclear if the speed gain(s) make a difference. From michel.sales at free.fr Fri Jan 13 23:34:53 2017 From: michel.sales at free.fr (Michel) Date: Sat, 14 Jan 2017 00:34:53 +0100 Subject: [openssl-dev] about enc 'magic' data and salt handling Message-ID: <003f01d26df5$a4fd1430$eef73c90$@sales@free.fr> Hi, FWIW, just sharing my opinion : Thanks to the team, OpenSSL comes with lots of powerful tools. They are not always easy to use, but some have no equivalent and are very helpful to test, debug, experiment, learn . all things that looks to me as their primary interest. Among them is the 'enc' command. I would like it to be as usefull as other tools but was a little disappointed at first, not only because it lacks some important feature (iteration count, standard KDF, ...) but above all because of the way the salt is handled. I believe it is better to be consistent about the use of the command : if a salt was given to encrypt, it should be given to decrypt (as would be the case for the iv if PBKDF2 is supported). If we agree on that, the salt would be stored with the encrypted data, *ONLY* when it was generated on the fly. It will allow us not only to decrypt the data with the right key and iv (not currently possible), but also use OpenSSL to decrypt raw data encrypted by other software(s) using the original password. I was able to work around all that but I now feel that next step will probably be to add some more 'magic' (unfortunately not from Pixar/Disney :-). I have the conviction it would be better NOT to enforce the proprietary nature of the file. For example a companion file (same name, different extension) can be use instead. As I understand there is lot of implications beyond my scope and that allowing to work with external raw data, is not necessarily the main concern of other people, it could be easier, depending what will be implemented, to just have a new parameter (or another command tool ?) able to separate raw encrypted data from all the new 'magic' (kind of import/export). Regards, Michel. -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.francis.jr at pobox.com Sat Jan 14 00:50:24 2017 From: thomas.francis.jr at pobox.com (Tom Francis) Date: Fri, 13 Jan 2017 19:50:24 -0500 Subject: [openssl-dev] about enc 'magic' data and salt handling In-Reply-To: <003f01d26df5$a4fd1430$eef73c90$@sales@free.fr> References: <003f01d26df5$a4fd1430$eef73c90$@sales@free.fr> Message-ID: <7560560D-B6E9-4182-8EA2-394C6D689815@pobox.com> > On Jan 13, 2017, at 6:34 PM, Michel wrote: > > Hi, > > FWIW, just sharing my opinion : > > Thanks to the team, OpenSSL comes with lots of powerful tools. > They are not always easy to use, but some have no equivalent and are very helpful to test, debug, experiment, learn ? all things that looks to me as their primary interest. > Among them is the 'enc' command. > I would like it to be as usefull as other tools but was a little disappointed at first, not only because it lacks some important feature (iteration count, standard KDF, ...) but above all because of the way the salt is handled. > > I believe it is better to be consistent about the use of the command : if a salt was given to encrypt, it should be given to decrypt (as would be the case for the iv if PBKDF2 is supported). > If we agree on that, the salt would be stored with the encrypted data, *ONLY* when it was generated on the fly. > It will allow us not only to decrypt the data with the right key and iv (not currently possible), but also use OpenSSL to decrypt raw data encrypted by other software(s) using the original password. > > I was able to work around all that but I now feel that next step will probably be to add some more 'magic' (unfortunately not from Pixar/Disney :-). > I have the conviction it would be better NOT to enforce the proprietary nature of the file. > For example a companion file (same name, different extension) can be use instead. > As I understand there is lot of implications beyond my scope and that allowing to work with external raw data, is not necessarily the main concern of other people, it could be easier, > depending what will be implemented, to just have a new parameter (or another command tool ?) able to separate raw encrypted data from all the new 'magic' (kind of import/export). > > Regards, > > Michel. The enc command is really just an example, IMO. If you want something that's useful for production purposes (and even follows standards!), I recommend looking at the cms command. It'll encrypt, decrypt, sign (and verify signatures) data in a standards-based format. It's not the easiest thing to use, but it's better to focus on something like that, rather than a proprietary format that was never really intended for real data exchange. TOM -------------- next part -------------- An HTML attachment was scrubbed... URL: From michel.sales at free.fr Sat Jan 14 10:25:39 2017 From: michel.sales at free.fr (Michel) Date: Sat, 14 Jan 2017 11:25:39 +0100 Subject: [openssl-dev] about enc 'magic' data and salt handling In-Reply-To: <7560560D-B6E9-4182-8EA2-394C6D689815@pobox.com> References: <003f01d26df5$a4fd1430$eef73c90$@sales@free.fr> <7560560D-B6E9-4182-8EA2-394C6D689815@pobox.com> Message-ID: <000901d26e50$8e264f00$aa72ed00$@sales@free.fr> > If you want something that's useful for production purposes (and even follows standards!), > I recommend looking at the cms command. Yes, thanks for pointing out that to me. However I believe that for production purposes I would even prefer using gpg (thanks to M. Zimmermann). Regards, Michel. From openssl-users at dukhovni.org Sat Jan 14 19:36:24 2017 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Sat, 14 Jan 2017 14:36:24 -0500 Subject: [openssl-dev] about enc 'magic' data and salt handling In-Reply-To: <7560560D-B6E9-4182-8EA2-394C6D689815@pobox.com> References: <003f01d26df5$a4fd1430$eef73c90$@sales@free.fr> <7560560D-B6E9-4182-8EA2-394C6D689815@pobox.com> Message-ID: > On Jan 13, 2017, at 7:50 PM, Tom Francis wrote: > > > The enc command is really just an example, IMO. If you want something that's useful for production purposes (and even follows standards!), I recommend looking at the cms command. It'll encrypt, decrypt, sign (and verify signatures) data in a standards-based format. It's not the easiest thing to use, but it's better to focus on something like that, rather than a proprietary format that was never really intended for real data exchange. While CMS is indeed often the more appropriate tool, it has a drawback for streaming data. Only the write side can stream. CMS readers must hold the entire decrypted object in memory. For this reason, I've sometimes used enc(1) with the symmetric random encryption key protected to a public key, and a separate signed MAC computed over the whole stream. Otherwise, indeed enc(1) is often more useful for raw symmetric operations when doing troubleshoots. -- Viktor. From appro at openssl.org Sun Jan 15 10:45:41 2017 From: appro at openssl.org (Andy Polyakov) Date: Sun, 15 Jan 2017 11:45:41 +0100 Subject: [openssl-dev] use SIPhash for OPENSSL_LH_strhash? In-Reply-To: <20170111.153458.1623912899597806811.levitte@openssl.org> References: <97D0BE2D-11C6-4D01-9A5D-FACCC5B27127@akamai.com> <1e19cdfea8224717b3eee11e2d8acda5@usma1ex-dag1mb1.msg.corp.akamai.com> <20170111.153458.1623912899597806811.levitte@openssl.org> Message-ID: <5bb4dc35-93bb-cc45-ef24-d7522f62769f@openssl.org> > A run on my laptop gave these results: > > : ; ./util/shlib_wrap.sh apps/openssl speed siphash lhash > type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes > lhash 147387.67k 147940.82k 144937.73k 147177.81k 147095.55k 147679.91k > siphash 119939.99k 223694.38k 283383.30k 305372.84k 311760.21k 312120.66k > > So it seems that for short strings, OPENSSL_LH_strhash (*) wins some, > while siphash wins big for larger strings. This is just *one* data point. Most notably what about 32-bit systems? Another factor is code size, or rather time it takes to bring it into cache, as well as what does it invalidate. Conventional benchmarks don't tell you that, but it's only sensible to consider code size in comparison to "typical" input size. From tshort at akamai.com Tue Jan 17 22:38:19 2017 From: tshort at akamai.com (Short, Todd) Date: Tue, 17 Jan 2017 22:38:19 +0000 Subject: [openssl-dev] RSA_METHOD_FLAG_NO_CHECK and RSA_FLAG_EXT_PKEY? Message-ID: <3FE2D8BE-76E0-4107-BF8B-14BC3EA2B0B2@akamai.com> Hi, The RSA_METHOD_FLAG_NO_CHECK and RSA_FLAG_EXT_PKEY seem to have similar meanings. These are the definitions in header files: # define RSA_METHOD_FLAG_NO_CHECK 0x0001/* don't check pub/private * match */ /* * This flag means the private key operations will be handled by rsa_mod_exp * and that they do not depend on the private key components being present: * for example a key stored in external hardware. Without this flag * bn_mod_exp gets called when private key components are absent. */ # define RSA_FLAG_EXT_PKEY 0x0020 In both cases, it implies that the private key may not be present, and the code should not be checked against the public key. The RSA_METHOD_FLAG_NO_CHECK is checked when setting certificates and private keys. The RSA_FLAG_EXT_PKEY is checked when doing RSA private key operations and determines whether rsa_mod_exp() or bn_mod_exp() is called. So, my question is, should RSA_FLAG_EXT_PKEY (implying the external storage of the private key) also be used when setting certificates/private keys? Does it matter? I?m really looking to start a discussion as to whether these flags have identical or very-close-to-each-other meanings. Also, should there be an ECC_FLAG_EXT_PKEY? This is all in reference to https://github.com/openssl/openssl/pull/2243 Thanks, -- -Todd Short // tshort at akamai.com // "One if by land, two if by sea, three if by the Internet." -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Wed Jan 25 08:59:22 2017 From: matt at openssl.org (Matt Caswell) Date: Wed, 25 Jan 2017 08:59:22 +0000 Subject: [openssl-dev] Fwd: [openssl-announce] Forthcoming OpenSSL releases In-Reply-To: References: Message-ID: In case anyone on these lists missed this on the openssl-announce list: -------- Forwarded Message -------- Subject: [openssl-announce] Forthcoming OpenSSL releases Date: Mon, 23 Jan 2017 21:08:50 +0000 (GMT) From: OpenSSL Reply-To: openssl-users at openssl.org To: openssl-announce at openssl.org Forthcoming OpenSSL releases ============================ The OpenSSL project team would like to announce the forthcoming release of OpenSSL versions 1.0.2k, 1.1.0d. These releases will be made available on 26th January 2017 between approximately 1300-1700 UTC. They will fix several security defects with maximum severity "moderate". Please see the following page for further details of severity levels: https://www.openssl.org/policies/secpolicy.html Please also note that, as per our previous announcements, support for 1.0.1 ended on 31st December 2016. Yours The OpenSSL Project Team -- openssl-announce mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-announce -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 480 bytes Desc: OpenPGP digital signature URL: From openssl at openssl.org Mon Jan 23 21:08:50 2017 From: openssl at openssl.org (OpenSSL) Date: Mon, 23 Jan 2017 21:08:50 +0000 (GMT) Subject: [openssl-dev] [openssl-announce] Forthcoming OpenSSL releases Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Forthcoming OpenSSL releases ============================ The OpenSSL project team would like to announce the forthcoming release of OpenSSL versions 1.0.2k, 1.1.0d. These releases will be made available on 26th January 2017 between approximately 1300-1700 UTC. They will fix several security defects with maximum severity "moderate". Please see the following page for further details of severity levels: https://www.openssl.org/policies/secpolicy.html Please also note that, as per our previous announcements, support for 1.0.1 ended on 31st December 2016. Yours The OpenSSL Project Team -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBCAAGBQJYhk6JAAoJEAEKUEB8TIy9LZIH/0nJUa7i/rUZTG2laUZmtcVa K7kmYAG7EQd836eoSLvbC4DAbZL/fpQwGMSdvbNNmsgaLvyLS4fO5VaiT7gWptBz 2QKnwCr32Ixn59USbpHth+EqpzwKl3nOJcNZlGJ5Pid2Q0SKY8Bw8gMx3hVzEoAi d6X4cD71wkigBV41yb7iQ7upRzBvGFWN8qnvSBf9hLPetOq4Dw0imcZT165wYrIC vFiLV5WexU4J5iGdUeWVF6CMw7duExeuY/Ii0mBQ/d/DsWz0S9KnOyN5MANYGkbZ 2y4i6zgPtNrprHoYXPDkBpjRaEuW/WtVEEMCmd49mkYNZkDnuF6ViaKP3UMfWDs= =GksO -----END PGP SIGNATURE----- -- openssl-announce mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-announce From openssl at openssl.org Thu Jan 26 14:01:36 2017 From: openssl at openssl.org (OpenSSL) Date: Thu, 26 Jan 2017 14:01:36 +0000 Subject: [openssl-dev] OpenSSL version 1.0.2k published Message-ID: <20170126140136.GA22894@openssl.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 OpenSSL version 1.0.2k released =============================== OpenSSL - The Open Source toolkit for SSL/TLS https://www.openssl.org/ The OpenSSL project team is pleased to announce the release of version 1.0.2k of our open source toolkit for SSL/TLS. For details of changes and known issues see the release notes at: https://www.openssl.org/news/openssl-1.0.2-notes.html OpenSSL 1.0.2k is available for download via HTTP and FTP from the following master locations (you can find the various FTP mirrors under https://www.openssl.org/source/mirror.html): * https://www.openssl.org/source/ * ftp://ftp.openssl.org/source/ The distribution file name is: o openssl-1.0.2k.tar.gz Size: 5309236 SHA1 checksum: 5f26a624479c51847ebd2f22bb9f84b3b44dcb44 SHA256 checksum: 6b3977c61f2aedf0f96367dcfb5c6e578cf37e7b8d913b4ecb6643c3cb88d8c0 The checksums were calculated using the following commands: openssl sha1 openssl-1.0.2k.tar.gz openssl sha256 openssl-1.0.2k.tar.gz Yours, The OpenSSL Project Team. -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJYifggAAoJENnE0m0OYESRHWUIAJXlSdrySe40QGFuOfSUQya0 iWTa0PcIPCPjSaM1Rige1BMSwTVhWwNmdimAyG94I1/y6uAoJzFsfNs4xSPlsGGO 9pdAiqeu9GAnJ3z4H/mkasUAmzXkemeWt5dN9KdANbT7HAg+ROKmthrmWiffftoU x+sGZ/lThIfKIYf+Ch5EmM4VFuCU9gG6YTwlUSKQAcnmcPt1AVrGlSxyVTx1Ahku Pl3iYbQors6vfBlh4BZZnxh7NJW4WRpJ77+1OH8xaOobJQMDZ5VQqnPMIIbbMLoM MAJz7T0nfzTaQ8Yzwv078qNPuwzeumI92NGybETbOI8zThHdEffsOegBaHqV+Jk= =KDh3 -----END PGP SIGNATURE----- From openssl at openssl.org Thu Jan 26 14:02:01 2017 From: openssl at openssl.org (OpenSSL) Date: Thu, 26 Jan 2017 14:02:01 +0000 Subject: [openssl-dev] OpenSSL version 1.1.0d published Message-ID: <20170126140201.GA22932@openssl.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 OpenSSL version 1.1.0d released =============================== OpenSSL - The Open Source toolkit for SSL/TLS https://www.openssl.org/ The OpenSSL project team is pleased to announce the release of version 1.1.0d of our open source toolkit for SSL/TLS. For details of changes and known issues see the release notes at: https://www.openssl.org/news/openssl-1.1.0-notes.html OpenSSL 1.1.0d is available for download via HTTP and FTP from the following master locations (you can find the various FTP mirrors under https://www.openssl.org/source/mirror.html): * https://www.openssl.org/source/ * ftp://ftp.openssl.org/source/ The distribution file name is: o openssl-1.1.0d.tar.gz Size: 5201626 SHA1 checksum: f423e9253ad8a8617fc9fb3562788a9fa066d46f SHA256 checksum: 7d5ebb9e89756545c156ff9c13cf2aa6214193b010a468a3bc789c3c28fe60df The checksums were calculated using the following commands: openssl sha1 openssl-1.1.0d.tar.gz openssl sha256 openssl-1.1.0d.tar.gz Yours, The OpenSSL Project Team. -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJYifVEAAoJENnE0m0OYESRcPYH/AqSYRJDURNli0/PhHa45ynT eXGk5v44l0q/TBqHOiUjPEAE6ctMe+Dg2Bw+2UI5msGZDyD+alksUrf94OzFMIll UC3FPAyAAIxu/DNyL/tRyGwBqOyHrzvKzwl+yGnOtzQ/a9/la79KCSTUMheDDBgk 2LiPoGi35wfNpJ+jBr/ucPto15+dWQfTMxX8aeYWkdVjUGGcO4XmAuqpIo5Jl7gw szSzpgvLB55sRZ8o6edU9knoIGRad+TH9JjfKAIzAIUH5BWaALmswQUMWhKTQkpX jHosHsLUPBLqqPJtmlLr1T7bHXGqz3fvQF0/lHYKA9eQDm0Owd0izeJlsiqaCac= =WZYB -----END PGP SIGNATURE----- From openssl at openssl.org Thu Jan 26 14:04:46 2017 From: openssl at openssl.org (OpenSSL) Date: Thu, 26 Jan 2017 14:04:46 +0000 Subject: [openssl-dev] OpenSSL Security Advisory Message-ID: <20170126140446.GA24451@openssl.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 OpenSSL Security Advisory [26 Jan 2017] ======================================== Truncated packet could crash via OOB read (CVE-2017-3731) ========================================================= Severity: Moderate If an SSL/TLS server or client is running on a 32-bit host, and a specific cipher is being used, then a truncated packet can cause that server or client to perform an out-of-bounds read, usually resulting in a crash. For OpenSSL 1.1.0, the crash can be triggered when using CHACHA20/POLY1305; users should upgrade to 1.1.0d For Openssl 1.0.2, the crash can be triggered when using RC4-MD5; users who have not disabled that algorithm should update to 1.0.2k This issue was reported to OpenSSL on 13th November 2016 by Robert ?wi?cki of Google. The fix was developed by Andy Polyakov of the OpenSSL development team. Bad (EC)DHE parameters cause a client crash (CVE-2017-3730) =========================================================== Severity: Moderate If a malicious server supplies bad parameters for a DHE or ECDHE key exchange then this can result in the client attempting to dereference a NULL pointer leading to a client crash. This could be exploited in a Denial of Service attack. OpenSSL 1.1.0 users should upgrade to 1.1.0d This issue does not affect OpenSSL version 1.0.2. Note that this issue was fixed prior to it being recognised as a security concern. This means the git commit with the fix does not contain the CVE identifier. The relevant fix commit can be identified by commit hash efbe126e3. This issue was reported to OpenSSL on 14th January 2017 by Guido Vranken. The fix was developed by Matt Caswell of the OpenSSL development team. BN_mod_exp may produce incorrect results on x86_64 (CVE-2017-3732) ================================================================== Severity: Moderate There is a carry propagating bug in the x86_64 Montgomery squaring procedure. No EC algorithms are affected. Analysis suggests that attacks against RSA and DSA as a result of this defect would be very difficult to perform and are not believed likely. Attacks against DH are considered just feasible (although very difficult) because most of the work necessary to deduce information about a private key may be performed offline. The amount of resources required for such an attack would be very significant and likely only accessible to a limited number of attackers. An attacker would additionally need online access to an unpatched system using the target private key in a scenario with persistent DH parameters and a private key that is shared between multiple clients. For example this can occur by default in OpenSSL DHE based SSL/TLS ciphersuites. Note: This issue is very similar to CVE-2015-3193 but must be treated as a separate problem. OpenSSL 1.1.0 users should upgrade to 1.1.0d OpenSSL 1.0.2 users should upgrade to 1.0.2k This issue was reported to OpenSSL on 15th January 2017 by the OSS-Fuzz project. The fix was developed by Andy Polyakov of the OpenSSL development team. Montgomery multiplication may produce incorrect results (CVE-2016-7055) ======================================================================= Severity: Low This issue was previously fixed in 1.1.0c and covered in security advisory https://www.openssl.org/news/secadv/20161110.txt OpenSSL 1.0.2k users should upgrade to 1.0.2k Note ==== Support for version 1.0.1 ended on 31st December 2016. Support for versions 0.9.8 and 1.0.0 ended on 31st December 2015. Those versions are no longer receiving security updates. References ========== URL for this Security Advisory: https://www.openssl.org/news/secadv/20170126.txt Note: the online version of the advisory may be updated with additional details over time. For details of OpenSSL severity classifications please see: https://www.openssl.org/policies/secpolicy.html -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJYifonAAoJENnE0m0OYESRnhYH/1ldFYDEZ894DleZfjRrZulX OQkEH7w6v+D6YFp8i2v6rJaDq8caOPEhzupQCxPcqYitBUnww9UzUvYJ77aBV0CG DQ3UvE9XeEn5D7MGAGq/ut5Z5WpvlYL7n7PaciX751vpTsWTBKfGecQ8YV0aT6y+ 7V7vHz6NVFnuTQDMUYs9C9aTsCDTNy3Bl84d7gYyoDWXUXds5k008g9LFRI4YQ8l +4z+GXRVcvAFr6fKH94Yq1RMAp6cJi0RDkyuwcGhSOUwVfSLTN8+i2v4xqzKgsx1 q2qPo3+7uederE5ZaNZScl0xAzEilotxLQyy9XSVx/DDXHz0in1500qxgxNFELU= =12E/ -----END PGP SIGNATURE----- From marquess at opensslfoundation.org Thu Jan 19 17:25:06 2017 From: marquess at opensslfoundation.org (Steve Marquess) Date: Thu, 19 Jan 2017 12:25:06 -0500 Subject: [openssl-dev] [openssl-announce] Akamai sponsors TLS 1.3 In-Reply-To: <1a7e4a62-465b-5af4-dfd8-c4481d65c11c@opensslfoundation.org> References: <1a7e4a62-465b-5af4-dfd8-c4481d65c11c@opensslfoundation.org> Message-ID: Many companies use OpenSSL, relatively few of them support it. I'm pleased to announce that Akamai (https://www.akamai.com/) has expanded its already substantial presence in the latter category. Akamai has contractually committed to funding implementation of TLS 1.3 in OpenSSL. That is something we were going to do anyway, an initiative that has been delayed by the major overhaul leading up to the 1.1 release. But, by contracting us to perform that implementation Akamai accomplishes two things: 1) A known schedule (with a key deadline of 2017-04-05). Left to our own devices and the usual resource contentions we would have taken longer; by funding us for a specific deliverable and schedule Akamai ensures a result that meets their own needs. 2) Significant financial support for OpenSSL, funding we would not otherwise have received which will be used for long term support of OpenSSL. While not technically a donation (because payment is contingent on specific deliverables) the end effect is the same, as we would have eventually done a comparable TLS 1.3 implementation anyway. Thank you Akamai for your well-considered initiative to both address your own business requirements and support OpenSSL and the OpenSSL user community at the same time; a win-win situation all around. -Steve M. -- Steve Marquess OpenSSL Software Foundation 20-22 Wenlock Road London N1 7GU United Kingdom +44 1785508015 +1 301 874 2571 direct marquess at opensslfoundation.org stevem at openssl.org -- openssl-announce mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-announce From R.Falck at comforte.com Fri Jan 27 05:13:24 2017 From: R.Falck at comforte.com (Rod Falck) Date: Fri, 27 Jan 2017 05:13:24 +0000 Subject: [openssl-dev] Netscape Comment Tag Value Message-ID: <2E4E24F6E6E47843B01089BF75379228B0AFA0BC@cf-mail.comforte.local> Hi, I have an OpenSSL based client which fails when validating a certificate generated by IBM RACF. It fails because the ASN.1 tag for the X509v3 extension Netscape Comment is 19 (V_ASN1_PRINTABLESTRING) and OpenSSL is expecting 22 (V_ASN1_IA5STRING). Is this a bug in OpenSSL or RACF? Can anyone point me to the RFC, or any document, that specifies the expected tag value? Rod. -- Rod Falck. Software Architect. comForte Pty Ltd. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kurt at x64architecture.com Fri Jan 27 12:14:00 2017 From: kurt at x64architecture.com (Kurt Cancemi) Date: Fri, 27 Jan 2017 07:14:00 -0500 Subject: [openssl-dev] Netscape Comment Tag Value In-Reply-To: <2E4E24F6E6E47843B01089BF75379228B0AFA0BC@cf-mail.comforte.local> References: <2E4E24F6E6E47843B01089BF75379228B0AFA0BC@cf-mail.comforte.local> Message-ID: OpenSSL is correct to expect the extension as an IA5STRING. The netscape-comment extension is defined with the OID 2.16.840.1.113730.1.13 and should be an IA5STRING. Some references (It's not in any RFC afaik): https://docs.oracle.com/cd/E19957-01/816-5533-10/ext.htm#1043093 https://msdn.microsoft.com/en-us/library/windows/desktop/aa378149(v=vs.85).aspx -- Kurt Cancemi https://www.x64architecture.com On Fri, Jan 27, 2017 at 12:13 AM, Rod Falck wrote: > Hi, > > > > I have an OpenSSL based client which fails when validating a certificate > generated by IBM RACF. It fails because the ASN.1 tag for the X509v3 > extension Netscape Comment is 19 (V_ASN1_PRINTABLESTRING) and OpenSSL is > expecting 22 (V_ASN1_IA5STRING). Is this a bug in OpenSSL or RACF? Can > anyone point me to the RFC, or any document, that specifies the expected tag > value? > > > > Rod. > > -- > > Rod Falck. Software Architect. comForte Pty Ltd. > From bkaduk at akamai.com Fri Jan 27 16:54:35 2017 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Fri, 27 Jan 2017 10:54:35 -0600 Subject: [openssl-dev] [openssl/openssl] ABI compatibility 1.0.0-->1.0.1-->1.0.2 In-Reply-To: References: Message-ID: [moving from github to -dev] On 01/27/2017 07:36 AM, mattcaswell wrote: > > 1.0.2 is the software version. > The numbers on the end of lbssl.so.1.0.0 refer to the ABI version - > which is different. Software version 1.0.2 is a drop in replacement > for 1.0.1, which is a drop in replacement for 1.0.0 - hence they all > have the same ABI version. > > There was some discussion about 1.0.1 being EoL on a FreeBSD list [0], and whether it would make sense to move to 1.0.2 on their stable branch, which led to someone making the claim that 1.0.2 has removed 4 symbols compared to 1.0.1, and thus is not strictly ABI compatible, linking to https://abi-laboratory.pro/tracker/timeline/openssl/ . If I start semi-randomly clicking around, I can find a page [1] that seems to claim the missing symbols are: ASN1_STRING_clear_free() ENGINE_load_rsax() SRP_user_pwd_free() SRP_VBASE_get1_by_user() It may be too late to get the 1.0.x series fully compatible, but it's probably worth thinking about how we can use automation to ensure that the 1.1.x series remains ABI compatible going forward. I just learned about abi-laboratory.pro from the FreeBSD posting, so I don't know if it is appropriate or we would want to use some other tool. One (naive?) idea for a home-grown solution would be to come up with a scheme to serialize the public ABI to a file in the repo, maybe regenerated as part of 'make test', and ensure that that file is append-only, at least between releases. But I don't know if the state of the art is more advanced than that -- are there better options? -Ben [0] https://lists.freebsd.org/pipermail/freebsd-security/2017-January/009211.html [1] https://abi-laboratory.pro/tracker/compat_report/openssl/1.0.1u/1.0.2/c63bf/abi_compat_report.html#Removed -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Fri Jan 27 17:01:04 2017 From: matt at openssl.org (Matt Caswell) Date: Fri, 27 Jan 2017 17:01:04 +0000 Subject: [openssl-dev] [openssl/openssl] ABI compatibility 1.0.0-->1.0.1-->1.0.2 In-Reply-To: References: Message-ID: On 27/01/17 16:54, Benjamin Kaduk via openssl-dev wrote: > [moving from github to -dev] > > On 01/27/2017 07:36 AM, mattcaswell wrote: >> >> 1.0.2 is the software version. >> The numbers on the end of lbssl.so.1.0.0 refer to the ABI version - >> which is different. Software version 1.0.2 is a drop in replacement >> for 1.0.1, which is a drop in replacement for 1.0.0 - hence they all >> have the same ABI version. >> >> > > There was some discussion about 1.0.1 being EoL on a FreeBSD list [0], > and whether it would make sense to move to 1.0.2 on their stable branch, > which led to someone making the claim that 1.0.2 has removed 4 symbols > compared to 1.0.1, and thus is not strictly ABI compatible, linking to > https://abi-laboratory.pro/tracker/timeline/openssl/ . If I start > semi-randomly clicking around, I can find a page [1] that seems to claim > the missing symbols are: > ASN1_STRING_clear_free() > ENGINE_load_rsax() > SRP_user_pwd_free() > SRP_VBASE_get1_by_user() > > It may be too late to get the 1.0.x series fully compatible, but it's Why can't we just add those symbols back in? I'm not sure why they were removed...we should look at that, but if nothing else make them no-ops or something. I'd look on it as a bug if there are missing symbols. > probably worth thinking about how we can use automation to ensure that > the 1.1.x series remains ABI compatible going forward. I just learned > about abi-laboratory.pro from the FreeBSD posting, so I don't know if it > is appropriate or we would want to use some other tool. > There is an abi build for 1.0.2 here already: https://openssl-sanity.cisco.com:8443/job/1_0_2_abi/ Unfortunately John Foley left cisco so AFAIK this farm is no longer being maintained...although it is still running. Matt > One (naive?) idea for a home-grown solution would be to come up with a > scheme to serialize the public ABI to a file in the repo, maybe > regenerated as part of 'make test', and ensure that that file is > append-only, at least between releases. But I don't know if the state > of the art is more advanced than that -- are there better options? > > -Ben > > > [0] > https://lists.freebsd.org/pipermail/freebsd-security/2017-January/009211.html > [1] > https://abi-laboratory.pro/tracker/compat_report/openssl/1.0.1u/1.0.2/c63bf/abi_compat_report.html#Removed > > From rsalz at akamai.com Fri Jan 27 18:48:55 2017 From: rsalz at akamai.com (Salz, Rich) Date: Fri, 27 Jan 2017 18:48:55 +0000 Subject: [openssl-dev] [openssl/openssl] ABI compatibility 1.0.0-->1.0.1-->1.0.2 In-Reply-To: References: Message-ID: The tool looks good, but either you didn't find the right link, or it's got bugs. Of the four symbols you found, ASN1_STRING_clear_free(), SRP_user_pwd_free(), and SRP_VBASE_get1_by_user() all exist; only ENGINE_load_rsax() was removed. -- Senior Architect, Akamai Technologies Member, OpenSSL Dev Team IM: richsalz at jabber.at Twitter: RichSalz From michel.sales at free.fr Fri Jan 27 20:43:18 2017 From: michel.sales at free.fr (Michel) Date: Fri, 27 Jan 2017 21:43:18 +0100 Subject: [openssl-dev] [openssl/openssl] ABI compatibility 1.0.0-->1.0.1-->1.0.2 In-Reply-To: References: Message-ID: <001a01d278de$00e0a1c0$02a1e540$@sales@free.fr> Hi, SRP_VBASE_get1_by_user() was ADDED to 1.0.2g 1 march 2016 [CVE-2016-0798]. I remember it very well ! ;-) Michel -----Message d'origine----- De?: openssl-dev [mailto:openssl-dev-bounces at openssl.org] De la part de Salz, Rich via openssl-dev Envoy??: vendredi 27 janvier 2017 19:49 ??: Kaduk, Ben; openssl-dev at openssl.org Objet?: Re: [openssl-dev] [openssl/openssl] ABI compatibility 1.0.0-->1.0.1-->1.0.2 The tool looks good, but either you didn't find the right link, or it's got bugs. Of the four symbols you found, ASN1_STRING_clear_free(), SRP_user_pwd_free(), and SRP_VBASE_get1_by_user() all exist; only ENGINE_load_rsax() was removed. From bkaduk at akamai.com Fri Jan 27 21:17:48 2017 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Fri, 27 Jan 2017 15:17:48 -0600 Subject: [openssl-dev] [openssl/openssl] ABI compatibility 1.0.0-->1.0.1-->1.0.2 In-Reply-To: <001a01d278de$00e0a1c0$02a1e540$@sales@free.fr> References: <001a01d278de$00e0a1c0$02a1e540$@sales@free.fr> Message-ID: <556953ab-2f6e-6675-5082-a729ab109bad@akamai.com> I guess the dashboard is only picking up incremental differences, then, so the four missing symbols is just for 1.0.1u to 1.0.2 (no letter); any symbols that were added to both 1.0.1 and 1.0.2 letter releases (e.g., for CVE fixes) would show up as "removed" since they weren't in the initial 1.0.2 release. I guess the tool needs more investigation than the quickest look... -Ben On 01/27/2017 02:43 PM, Michel wrote: > Hi, > SRP_VBASE_get1_by_user() was ADDED to 1.0.2g 1 march 2016 [CVE-2016-0798]. > I remember it very well ! > ;-) > > Michel > > -----Message d'origine----- > De : openssl-dev [mailto:openssl-dev-bounces at openssl.org] De la part de > Salz, Rich via openssl-dev > Envoy? : vendredi 27 janvier 2017 19:49 > ? : Kaduk, Ben; openssl-dev at openssl.org > Objet : Re: [openssl-dev] [openssl/openssl] ABI compatibility > 1.0.0-->1.0.1-->1.0.2 > > The tool looks good, but either you didn't find the right link, or it's got > bugs. Of the four symbols you found, ASN1_STRING_clear_free(), > SRP_user_pwd_free(), and SRP_VBASE_get1_by_user() all exist; only > ENGINE_load_rsax() was removed. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kurt at roeckx.be Sun Jan 29 22:34:43 2017 From: kurt at roeckx.be (Kurt Roeckx) Date: Sun, 29 Jan 2017 23:34:43 +0100 Subject: [openssl-dev] MD5 speed Message-ID: <20170129223443.y6megql3vq3gfg6a@roeckx.be> I had some surprising results of the speed command when testing the md5 speed on the 1.1.0-stable branch (for both a shared and a static build): openssl speed md5 returns: type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes md5 115869.46k 268237.29k 473617.41k 589905.92k 636772.35k 639429.29k openssl speed -evp md5 returns: type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes md5 53991.08k 160454.36k 364985.86k 537559.38k 624238.59k 633066.84k On the other hand, with 1.0.1 and 1.0.2-stable using a static build I get: md5 38045.25k 123423.76k 310729.30k 505120.09k 620333.74k md5 43182.80k 135651.38k 331369.48k 518042.97k 622193.32k Using a shared build I get: md5 57529.01k 169850.56k 376685.74k 545938.09k 626952.87k md5 65634.19k 186133.65k 397608.96k 558070.78k 629697.19k So was surprised me is that speed for small packets seems to be a lot better in 1.1.0 when not using the EVP interface, but worse when using it compared to 1.0.2. It this expected behaviour? Sha1 doesn't seem to have this difference for instance. Kurt From pwalten at au1.ibm.com Mon Jan 30 06:53:23 2017 From: pwalten at au1.ibm.com (Peter Waltenberg) Date: Mon, 30 Jan 2017 06:53:23 +0000 Subject: [openssl-dev] MD5 speed In-Reply-To: <20170129223443.y6megql3vq3gfg6a@roeckx.be> References: <20170129223443.y6megql3vq3gfg6a@roeckx.be> Message-ID: An HTML attachment was scrubbed... URL: From kudzu at tenebras.com Mon Jan 30 07:17:32 2017 From: kudzu at tenebras.com (Michael Sierchio) Date: Sun, 29 Jan 2017 23:17:32 -0800 Subject: [openssl-dev] MD5 speed In-Reply-To: References: <20170129223443.y6megql3vq3gfg6a@roeckx.be> Message-ID: On Sun, Jan 29, 2017 at 10:53 PM, Peter Waltenberg wrote: > > No one cares ?. I was rather thinking the same thing. Pretty much the same deprecated status for SHA1, too. Want to talk about poly1305? - M -------------- next part -------------- An HTML attachment was scrubbed... URL: From appro at openssl.org Mon Jan 30 09:52:26 2017 From: appro at openssl.org (Andy Polyakov) Date: Mon, 30 Jan 2017 10:52:26 +0100 Subject: [openssl-dev] MD5 speed In-Reply-To: <20170129223443.y6megql3vq3gfg6a@roeckx.be> References: <20170129223443.y6megql3vq3gfg6a@roeckx.be> Message-ID: <8698661d-50b4-ed55-bd14-cce53f1b79df@openssl.org> > I had some surprising results of the speed command when testing the > md5 speed on the 1.1.0-stable branch (for both a shared and a static > build): > openssl speed md5 returns: > type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes > md5 115869.46k 268237.29k 473617.41k 589905.92k 636772.35k 639429.29k > > openssl speed -evp md5 returns: > type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes > md5 53991.08k 160454.36k 364985.86k 537559.38k 624238.59k 633066.84k > > On the other hand, with 1.0.1 and 1.0.2-stable using a static build I get: > md5 38045.25k 123423.76k 310729.30k 505120.09k 620333.74k > md5 43182.80k 135651.38k 331369.48k 518042.97k 622193.32k > > Using a shared build I get: > md5 57529.01k 169850.56k 376685.74k 545938.09k 626952.87k > md5 65634.19k 186133.65k 397608.96k 558070.78k 629697.19k > > So was surprised me is that speed for small packets seems to be a > lot better in 1.1.0 when not using the EVP interface, but worse when > using it compared to 1.0.2. It this expected behaviour? Yes. In 1.1.0 speed md5 calls MD5() directly, as result 1.1.0 speed md5 is always faster than speed -evp md5 [for smaller inputs]. And in 1.0.x speed md5 invokes EVP_Digest(...,EVP_get_digestbyname("md5"),...) in inner loop, while speed -evp md5 effectively takes EVP_get_digestbyname out of inner loop, so that -evp md5 is faster. As for variations between static vs. shared. Is your operating frequency variable? What I'm hinting at is that bringing a shared library into address space means some additional computations, and adaptive frequency increase is likely to kick in earlier, which would [positively] affect first column. But second column should be same, and it's not... Another possibility is relative position of pieces of code modulo cache size. I mean differences might be side effect of cache contention, which are relatively diminishing with input length. > Sha1 doesn't seem to have this difference for instance. Well, compare sha256 vs. -evp sha256 in 1.0.2 then :-) From mischa.salle at gmail.com Mon Jan 30 10:13:52 2017 From: mischa.salle at gmail.com (Mischa Salle) Date: Mon, 30 Jan 2017 11:13:52 +0100 Subject: [openssl-dev] SSL_set_bio(ssl, bio, bio) and BIO_up_ref(bio) Message-ID: Hi all, I noticed a doublefree when calling SSL_set_bio(ssl, bio, bio) followed by either SSL_set_bio(ssl, NULL, NULL) or SSL_set_io_SSL_free(ssl). Valgrind shows the double free, and I see the assert in https://github.com/openssl/openssl/blob/master/crypto/bio/bio_lib.c#L122 fail. This is all due to the same bio being using for read and write. I found that in https://github.com/openssl/openssl/blob/master/ssl/bio_ssl.c#L331-L332 the ref-count is manually adjusted, which indeed also fixes my doublefree. However, it seems that in a number of other places where SSL_set_bio is called with equal rbio and wbio, this is not the case, e.g. in apps/s_server.c (L2157, L2735, L3099) and also in https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L1161 itself. So the question is, when exactly is it necessary to manually adjust the ref count, and couldn't this be done automatically in e.g. the SSL_set_bio(ssl, bio, bio) ? Best wishes, Mischa Salle -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Mon Jan 30 11:13:44 2017 From: matt at openssl.org (Matt Caswell) Date: Mon, 30 Jan 2017 11:13:44 +0000 Subject: [openssl-dev] SSL_set_bio(ssl, bio, bio) and BIO_up_ref(bio) In-Reply-To: References: Message-ID: <6e860a08-f8a7-c34a-60fa-668fd6c657ee@openssl.org> On 30/01/17 10:13, Mischa Salle wrote: > Hi all, > > I noticed a doublefree when calling SSL_set_bio(ssl, bio, bio) followed > by either SSL_set_bio(ssl, NULL, NULL) or SSL_set_io_SSL_free(ssl). > Valgrind shows the double free, and I see the assert in > https://github.com/openssl/openssl/blob/master/crypto/bio/bio_lib.c#L122 > fail. This is all due to the same bio being using for read and write. > I found that in > https://github.com/openssl/openssl/blob/master/ssl/bio_ssl.c#L331-L332 > the ref-count is manually adjusted, which indeed also fixes my > doublefree. However, it seems that in a number of other places where > SSL_set_bio is called with equal rbio and wbio, this is not the case, > e.g. in apps/s_server.c (L2157, L2735, L3099) and also in > https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L1161 itself. > So the question is, when exactly is it necessary to manually adjust the > ref count, and couldn't this be done automatically in e.g. the > SSL_set_bio(ssl, bio, bio) ? SSL_set_bio() is a curious beast and its memory management semantics are confusing at best. It's behaviour is retained for historical consistency. The man page now recommends using SSL_set0_rbio() and SSL_set0_wbio() in preference because of this. However they only exist in OpenSSL 1.1.0, so if you need to support 1.0.2 then you are stuck with it. The memory management rules are documented on the latest version of the man page, here: https://www.openssl.org/docs/man1.1.0/ssl/SSL_set_bio.html SSL_set_bio() passes ownership of the BIO's to the SSL object. They will get freed when the SSL object gets freed. Once called you should not then attempt to free them yourself directly, *unless* you have called BIO_up_ref(). If the rbio and wbio are different then ownership of both objects is transferred. If the rbio and the wbio are the same object then ownership is still transferred - but only one reference is consumed, i.e. you are not transferring ownership of two references even though you have passed the BIO to the function "twice" (once for the rbio and once for the wbio). You references a few places in the code: https://github.com/openssl/openssl/blob/master/ssl/bio_ssl.c#L331-L332 Here we up ref before passing the same bio in both arguments in a call to SSL_set_bio(). This is processing a BIO_ctrl call with a BIO_CTRL_PUSH operation. This operation is typically only used internally. It's semantics does *not* transfer ownership of its argument to the BIO_CTRL_PUSH code. However, we want to call SSL_set_bio() with it which will transfer an ownership that we don't currently hold! Therefore we need to up ref first. You also mention apps/s_server.c (L2157, L2735, L3099. In this case we just created the BIO and therefore own a reference to it. We then transfer that ownership to the SSL object in the SSL_set_bio() call. You will notice that after that call we never then attempt to free the BIO again...we no longer own it, so we don't need to. It will get freed when we free the SSL object. Finally you mention this code: https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L1161 Again, in this case, we just created the BIO object and therefore own a reference to it. We then transfer that ownership to the SSL object in the SSL_set_bio() call. You will note that, again, we never explicitly free the BIO object we just created. It will get freed when we free the SSL object. I hope that helps, Matt From mischa.salle at gmail.com Mon Jan 30 17:19:22 2017 From: mischa.salle at gmail.com (Mischa Salle) Date: Mon, 30 Jan 2017 18:19:22 +0100 Subject: [openssl-dev] SSL_set_bio(ssl, bio, bio) and BIO_up_ref(bio) In-Reply-To: <6e860a08-f8a7-c34a-60fa-668fd6c657ee@openssl.org> References: <6e860a08-f8a7-c34a-60fa-668fd6c657ee@openssl.org> Message-ID: Hi Matt, thanks for the quick and extensive answer! I've tried by replacing all SSL_set_bio(ssl, bio, bio) with a separate SSL_set0_rbio(ssl, bio) and SSL_set0_wbio(ssl, bio). I've also removed all BIO_free statements and if I understand you correctly, I should then *not* need to call BIO_up_ref() manually, or did I misunderstand? However, I still seem to need to do need it (as also indicated in the man-page) otherwise I get a double free and a ref-counter assertion failure from: https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L977-L978 The only other thing could be that the code (which I inherited) is calling a SSL_shutdown() beforehand which does something I have missed...? Best wishes, Mischa On Mon, Jan 30, 2017 at 12:13 PM, Matt Caswell wrote: > > > > On 30/01/17 10:13, Mischa Salle wrote: > > Hi all, > > > > I noticed a doublefree when calling SSL_set_bio(ssl, bio, bio) followed > > by either SSL_set_bio(ssl, NULL, NULL) or SSL_set_io_SSL_free(ssl). > > Valgrind shows the double free, and I see the assert in > > https://github.com/openssl/openssl/blob/master/crypto/bio/bio_lib.c#L122 > > fail. This is all due to the same bio being using for read and write. > > I found that in > > https://github.com/openssl/openssl/blob/master/ssl/bio_ssl.c#L331-L332 > > the ref-count is manually adjusted, which indeed also fixes my > > doublefree. However, it seems that in a number of other places where > > SSL_set_bio is called with equal rbio and wbio, this is not the case, > > e.g. in apps/s_server.c (L2157, L2735, L3099) and also in > > https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L1161 itself. > > So the question is, when exactly is it necessary to manually adjust the > > ref count, and couldn't this be done automatically in e.g. the > > SSL_set_bio(ssl, bio, bio) ? > > > SSL_set_bio() is a curious beast and its memory management semantics are > confusing at best. It's behaviour is retained for historical > consistency. The man page now recommends using SSL_set0_rbio() and > SSL_set0_wbio() in preference because of this. However they only exist > in OpenSSL 1.1.0, so if you need to support 1.0.2 then you are stuck > with it. > > The memory management rules are documented on the latest version of the > man page, here: > > https://www.openssl.org/docs/man1.1.0/ssl/SSL_set_bio.html > > > SSL_set_bio() passes ownership of the BIO's to the SSL object. They will > get freed when the SSL object gets freed. Once called you should not > then attempt to free them yourself directly, *unless* you have called > BIO_up_ref(). > > If the rbio and wbio are different then ownership of both objects is > transferred. If the rbio and the wbio are the same object then ownership > is still transferred - but only one reference is consumed, i.e. you are > not transferring ownership of two references even though you have passed > the BIO to the function "twice" (once for the rbio and once for the wbio). > > You references a few places in the code: > > https://github.com/openssl/openssl/blob/master/ssl/bio_ssl.c#L331-L332 > > Here we up ref before passing the same bio in both arguments in a call > to SSL_set_bio(). This is processing a BIO_ctrl call with a > BIO_CTRL_PUSH operation. This operation is typically only used > internally. It's semantics does *not* transfer ownership of its argument > to the BIO_CTRL_PUSH code. However, we want to call SSL_set_bio() with > it which will transfer an ownership that we don't currently hold! > Therefore we need to up ref first. > > > You also mention apps/s_server.c (L2157, L2735, L3099. > > In this case we just created the BIO and therefore own a reference to > it. We then transfer that ownership to the SSL object in the > SSL_set_bio() call. You will notice that after that call we never then > attempt to free the BIO again...we no longer own it, so we don't need > to. It will get freed when we free the SSL object. > > > Finally you mention this code: > https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L1161 > > Again, in this case, we just created the BIO object and therefore own a > reference to it. We then transfer that ownership to the SSL object in > the SSL_set_bio() call. You will note that, again, we never explicitly > free the BIO object we just created. It will get freed when we free the > SSL object. > > I hope that helps, > > Matt > -- > openssl-dev mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev From matt at openssl.org Mon Jan 30 19:51:08 2017 From: matt at openssl.org (Matt Caswell) Date: Mon, 30 Jan 2017 19:51:08 +0000 Subject: [openssl-dev] SSL_set_bio(ssl, bio, bio) and BIO_up_ref(bio) In-Reply-To: References: <6e860a08-f8a7-c34a-60fa-668fd6c657ee@openssl.org> Message-ID: <39a461a3-5bad-83ef-1b54-4443ef3de732@openssl.org> On 30/01/17 17:19, Mischa Salle wrote: > Hi Matt, > > thanks for the quick and extensive answer! > > I've tried by replacing all SSL_set_bio(ssl, bio, bio) with a separate > SSL_set0_rbio(ssl, bio) and SSL_set0_wbio(ssl, bio). > I've also removed all BIO_free statements and if I understand you correctly, > I should then *not* need to call BIO_up_ref() manually, or did I misunderstand? Well, SSL_set0_rbio() and SSL_set0_wbio() still transfer the ownership of memory - but their semantics are a lot simpler. There is no special-casing for instances where the rbio and wbio are the same. SSL_set0_rbio() *always* transfers ownership of the rbio, and SSL_set0_wbio() *always* transfers ownership of the wbio. If you call SSL_set0_rbio() followed immediately by SSL_set0_wbio() with the *same* BIO for both then that means you are transferring *2* references (once for the rbio and once for the wbio) - so you better make sure you own 2 refs before you start. That will most likely require a BIO_up_ref() call. Matt > > However, I still seem to need to do need it (as also indicated in the man-page) > otherwise I get a double free and a ref-counter assertion failure from: > https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L977-L978 > The only other thing could be that the code (which I inherited) is calling a > SSL_shutdown() beforehand which does something I have missed...? > > Best wishes, > Mischa > > > On Mon, Jan 30, 2017 at 12:13 PM, Matt Caswell wrote: >> >> >> >> On 30/01/17 10:13, Mischa Salle wrote: >>> Hi all, >>> >>> I noticed a doublefree when calling SSL_set_bio(ssl, bio, bio) followed >>> by either SSL_set_bio(ssl, NULL, NULL) or SSL_set_io_SSL_free(ssl). >>> Valgrind shows the double free, and I see the assert in >>> https://github.com/openssl/openssl/blob/master/crypto/bio/bio_lib.c#L122 >>> fail. This is all due to the same bio being using for read and write. >>> I found that in >>> https://github.com/openssl/openssl/blob/master/ssl/bio_ssl.c#L331-L332 >>> the ref-count is manually adjusted, which indeed also fixes my >>> doublefree. However, it seems that in a number of other places where >>> SSL_set_bio is called with equal rbio and wbio, this is not the case, >>> e.g. in apps/s_server.c (L2157, L2735, L3099) and also in >>> https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L1161 itself. >>> So the question is, when exactly is it necessary to manually adjust the >>> ref count, and couldn't this be done automatically in e.g. the >>> SSL_set_bio(ssl, bio, bio) ? >> >> >> SSL_set_bio() is a curious beast and its memory management semantics are >> confusing at best. It's behaviour is retained for historical >> consistency. The man page now recommends using SSL_set0_rbio() and >> SSL_set0_wbio() in preference because of this. However they only exist >> in OpenSSL 1.1.0, so if you need to support 1.0.2 then you are stuck >> with it. >> >> The memory management rules are documented on the latest version of the >> man page, here: >> >> https://www.openssl.org/docs/man1.1.0/ssl/SSL_set_bio.html >> >> >> SSL_set_bio() passes ownership of the BIO's to the SSL object. They will >> get freed when the SSL object gets freed. Once called you should not >> then attempt to free them yourself directly, *unless* you have called >> BIO_up_ref(). >> >> If the rbio and wbio are different then ownership of both objects is >> transferred. If the rbio and the wbio are the same object then ownership >> is still transferred - but only one reference is consumed, i.e. you are >> not transferring ownership of two references even though you have passed >> the BIO to the function "twice" (once for the rbio and once for the wbio). >> >> You references a few places in the code: >> >> https://github.com/openssl/openssl/blob/master/ssl/bio_ssl.c#L331-L332 >> >> Here we up ref before passing the same bio in both arguments in a call >> to SSL_set_bio(). This is processing a BIO_ctrl call with a >> BIO_CTRL_PUSH operation. This operation is typically only used >> internally. It's semantics does *not* transfer ownership of its argument >> to the BIO_CTRL_PUSH code. However, we want to call SSL_set_bio() with >> it which will transfer an ownership that we don't currently hold! >> Therefore we need to up ref first. >> >> >> You also mention apps/s_server.c (L2157, L2735, L3099. >> >> In this case we just created the BIO and therefore own a reference to >> it. We then transfer that ownership to the SSL object in the >> SSL_set_bio() call. You will notice that after that call we never then >> attempt to free the BIO again...we no longer own it, so we don't need >> to. It will get freed when we free the SSL object. >> >> >> Finally you mention this code: >> https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L1161 >> >> Again, in this case, we just created the BIO object and therefore own a >> reference to it. We then transfer that ownership to the SSL object in >> the SSL_set_bio() call. You will note that, again, we never explicitly >> free the BIO object we just created. It will get freed when we free the >> SSL object. >> >> I hope that helps, >> >> Matt >> -- >> openssl-dev mailing list >> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev From mischa.salle at gmail.com Mon Jan 30 21:07:54 2017 From: mischa.salle at gmail.com (Mischa Salle) Date: Mon, 30 Jan 2017 22:07:54 +0100 Subject: [openssl-dev] SSL_set_bio(ssl, bio, bio) and BIO_up_ref(bio) In-Reply-To: <39a461a3-5bad-83ef-1b54-4443ef3de732@openssl.org> References: <6e860a08-f8a7-c34a-60fa-668fd6c657ee@openssl.org> <39a461a3-5bad-83ef-1b54-4443ef3de732@openssl.org> Message-ID: Ah, right, so indeed I did misunderstand. So if I use SSL_set_bio without calling BIO_free myself, the refcount works out automatically and I don't need to up myself. That indeed works, and explains why the s_server and other places don't suffer from this. In that case, I would say that for having rbio == wbio, SSL_set_bio() would be the preferred function... In any case, thanks again for the clarifications! Best wishes, Mischa On Mon, Jan 30, 2017 at 8:51 PM, Matt Caswell wrote: > > > On 30/01/17 17:19, Mischa Salle wrote: >> Hi Matt, >> >> thanks for the quick and extensive answer! >> >> I've tried by replacing all SSL_set_bio(ssl, bio, bio) with a separate >> SSL_set0_rbio(ssl, bio) and SSL_set0_wbio(ssl, bio). >> I've also removed all BIO_free statements and if I understand you correctly, >> I should then *not* need to call BIO_up_ref() manually, or did I misunderstand? > > Well, SSL_set0_rbio() and SSL_set0_wbio() still transfer the ownership > of memory - but their semantics are a lot simpler. There is no > special-casing for instances where the rbio and wbio are the same. > SSL_set0_rbio() *always* transfers ownership of the rbio, and > SSL_set0_wbio() *always* transfers ownership of the wbio. > > If you call SSL_set0_rbio() followed immediately by SSL_set0_wbio() with > the *same* BIO for both then that means you are transferring *2* > references (once for the rbio and once for the wbio) - so you better > make sure you own 2 refs before you start. That will most likely require > a BIO_up_ref() call. > > Matt > > > >> >> However, I still seem to need to do need it (as also indicated in the man-page) >> otherwise I get a double free and a ref-counter assertion failure from: >> https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L977-L978 >> The only other thing could be that the code (which I inherited) is calling a >> SSL_shutdown() beforehand which does something I have missed...? >> >> Best wishes, >> Mischa >> >> >> On Mon, Jan 30, 2017 at 12:13 PM, Matt Caswell wrote: >>> >>> >>> >>> On 30/01/17 10:13, Mischa Salle wrote: >>>> Hi all, >>>> >>>> I noticed a doublefree when calling SSL_set_bio(ssl, bio, bio) followed >>>> by either SSL_set_bio(ssl, NULL, NULL) or SSL_set_io_SSL_free(ssl). >>>> Valgrind shows the double free, and I see the assert in >>>> https://github.com/openssl/openssl/blob/master/crypto/bio/bio_lib.c#L122 >>>> fail. This is all due to the same bio being using for read and write. >>>> I found that in >>>> https://github.com/openssl/openssl/blob/master/ssl/bio_ssl.c#L331-L332 >>>> the ref-count is manually adjusted, which indeed also fixes my >>>> doublefree. However, it seems that in a number of other places where >>>> SSL_set_bio is called with equal rbio and wbio, this is not the case, >>>> e.g. in apps/s_server.c (L2157, L2735, L3099) and also in >>>> https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L1161 itself. >>>> So the question is, when exactly is it necessary to manually adjust the >>>> ref count, and couldn't this be done automatically in e.g. the >>>> SSL_set_bio(ssl, bio, bio) ? >>> >>> >>> SSL_set_bio() is a curious beast and its memory management semantics are >>> confusing at best. It's behaviour is retained for historical >>> consistency. The man page now recommends using SSL_set0_rbio() and >>> SSL_set0_wbio() in preference because of this. However they only exist >>> in OpenSSL 1.1.0, so if you need to support 1.0.2 then you are stuck >>> with it. >>> >>> The memory management rules are documented on the latest version of the >>> man page, here: >>> >>> https://www.openssl.org/docs/man1.1.0/ssl/SSL_set_bio.html >>> >>> >>> SSL_set_bio() passes ownership of the BIO's to the SSL object. They will >>> get freed when the SSL object gets freed. Once called you should not >>> then attempt to free them yourself directly, *unless* you have called >>> BIO_up_ref(). >>> >>> If the rbio and wbio are different then ownership of both objects is >>> transferred. If the rbio and the wbio are the same object then ownership >>> is still transferred - but only one reference is consumed, i.e. you are >>> not transferring ownership of two references even though you have passed >>> the BIO to the function "twice" (once for the rbio and once for the wbio). >>> >>> You references a few places in the code: >>> >>> https://github.com/openssl/openssl/blob/master/ssl/bio_ssl.c#L331-L332 >>> >>> Here we up ref before passing the same bio in both arguments in a call >>> to SSL_set_bio(). This is processing a BIO_ctrl call with a >>> BIO_CTRL_PUSH operation. This operation is typically only used >>> internally. It's semantics does *not* transfer ownership of its argument >>> to the BIO_CTRL_PUSH code. However, we want to call SSL_set_bio() with >>> it which will transfer an ownership that we don't currently hold! >>> Therefore we need to up ref first. >>> >>> >>> You also mention apps/s_server.c (L2157, L2735, L3099. >>> >>> In this case we just created the BIO and therefore own a reference to >>> it. We then transfer that ownership to the SSL object in the >>> SSL_set_bio() call. You will notice that after that call we never then >>> attempt to free the BIO again...we no longer own it, so we don't need >>> to. It will get freed when we free the SSL object. >>> >>> >>> Finally you mention this code: >>> https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L1161 >>> >>> Again, in this case, we just created the BIO object and therefore own a >>> reference to it. We then transfer that ownership to the SSL object in >>> the SSL_set_bio() call. You will note that, again, we never explicitly >>> free the BIO object we just created. It will get freed when we free the >>> SSL object. >>> >>> I hope that helps, >>> >>> Matt >>> -- >>> openssl-dev mailing list >>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev > -- > openssl-dev mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev From nmav at redhat.com Tue Jan 31 07:20:32 2017 From: nmav at redhat.com (Nikos Mavrogiannopoulos) Date: Tue, 31 Jan 2017 08:20:32 +0100 Subject: [openssl-dev] [openssl/openssl] ABI compatibility 1.0.0-->1.0.1-->1.0.2 In-Reply-To: References: Message-ID: <1485847232.6050.11.camel@redhat.com> On Fri, 2017-01-27 at 10:54 -0600, Benjamin Kaduk via openssl-dev wrote: > [moving from github to -dev] > > On 01/27/2017 07:36 AM, mattcaswell wrote: > > 1.0.2 is the software version. > > The numbers on the end of lbssl.so.1.0.0 refer to the ABI version - > > which is different. Software version 1.0.2 is a drop in replacement > > for 1.0.1, which is a drop in replacement for 1.0.0 - hence they > > all have the same ABI version. > > > ? > There was some discussion about 1.0.1 being EoL on a FreeBSD list > [0], and whether it would make sense to move to 1.0.2 on their stable > branch, which led to someone making the claim that 1.0.2 has removed > 4 symbols compared to 1.0.1, and thus is not strictly ABI compatible, > linking to https://abi-laboratory.pro/tracker/timeline/openssl/ .? If > I start semi-randomly clicking around, I can find a page [1] that > seems to claim the missing symbols are: > ASN1_STRING_clear_free() > ENGINE_load_rsax() > SRP_user_pwd_free() > SRP_VBASE_get1_by_user() > > ... > One (naive?) idea for a home-grown solution would be to come up with > a scheme to serialize the public ABI to a file in the repo, maybe > regenerated as part of 'make test', and ensure that that file is > append-only, at least between releases.? But I don't know if the > state of the art is more advanced than that -- are there better > options? The abi-dumper and abi-compliance-checker tools can be used for that purpose. In fact they are the back-end tools used by the abi-laboratory you quote above. regards, Nikos