From matt at openssl.org Tue Nov 5 17:06:10 2019 From: matt at openssl.org (Matt Caswell) Date: Tue, 5 Nov 2019 17:06:10 +0000 Subject: Monthly Status Report (October) Message-ID: As well as normal reviews, responding to user queries, wiki user requests, OMC business, handling security reports, etc., key activities this month: - Attended the OMC f2f, and committers f2f meetings in Nuremburg - Fixed no-err - Committed the provider support for Evp_Digest(Sign|Verify) - Fixed an issue in EVP_DigestInit_ex where we were incorrectly following the legacy path in certain circumstances - Fixed EVP_CIPHER_CTX_block_size() for RC4 - Fixed no-autoerrinit - Created an ongoing PR to correct some algorithm naming inconsistencies - Fixed an issue to ensure we send bad_record_mac instead of decryption_failed - Reviewed more of the CMP contribution (chunks 5 and 6) - Added support for calling EVP_DigestUpdate instead of EVP_Digest(Sign|Verify)Update in 3.0 - Created PR to implement asymmetric cipher support in the providers - Did some work looking at how to prevent the recurring SSL_SESSION immutability issue - Fixed an s_server arbitrary read on Windows - Fixed error in the TLSv1.3 PSK documentation - Fixed an in i2v_GENERAL_NAME and GENERAL_NAME_print which was causing a fuzzing failure Matt From matt at openssl.org Thu Nov 7 22:37:05 2019 From: matt at openssl.org (Matt Caswell) Date: Thu, 7 Nov 2019 22:37:05 +0000 Subject: OpenSSL Blog Post Message-ID: <25f6721c-5e60-b392-2a73-0a4d682746de@openssl.org> Please take a look at my blog post that gives an update on OpenSSL 3.0 development, FIPS and 1.0.2 EOL: https://www.openssl.org/blog/blog/2019/11/07/3.0-update/ Matt From kurt at roeckx.be Sun Nov 10 22:59:27 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Sun, 10 Nov 2019 23:59:27 +0100 Subject: New audit before 3.0 release Message-ID: <20191110225927.GA25034@roeckx.be> Should we let someone do a new audit before the 3.0 release? Kurt From matt at openssl.org Mon Nov 11 09:42:50 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 11 Nov 2019 09:42:50 +0000 Subject: New audit before 3.0 release In-Reply-To: <20191110225927.GA25034@roeckx.be> References: <20191110225927.GA25034@roeckx.be> Message-ID: <52f62885-891b-a486-4877-416cb5c9c4a0@openssl.org> On 10/11/2019 22:59, Kurt Roeckx wrote: > Should we let someone do a new audit before the 3.0 release? If someone came along with some money to fund it and offering to do it, then that would be a good idea. Matt From paul.dale at oracle.com Thu Nov 14 06:39:31 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Thu, 14 Nov 2019 16:39:31 +1000 Subject: #10388 Message-ID: <1630AF0A-18D3-481A-97C5-2551E534385A@oracle.com> With reference to #10388: https://github.com/openssl/openssl/pull/10388 This PR intends to add missing APIs to 1.1.1. I?m fine with this being considered a bug and adding them. I?m more concerned about adding these to 3.0. It means we?ll have to support the new API for a long time and it is one which we are currently trying to move away from. Thoughts or comments anyone? Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Thu Nov 14 08:41:57 2019 From: matt at openssl.org (Matt Caswell) Date: Thu, 14 Nov 2019 08:41:57 +0000 Subject: #10388 In-Reply-To: <1630AF0A-18D3-481A-97C5-2551E534385A@oracle.com> References: <1630AF0A-18D3-481A-97C5-2551E534385A@oracle.com> Message-ID: <49584a83-1b43-6502-869f-155a2bcc377b@openssl.org> On 14/11/2019 06:39, Dr Paul Dale wrote: > With reference to #10388: https://github.com/openssl/openssl/pull/10388 > > This PR intends to add missing APIs to 1.1.1. ?I?m fine with this being > considered a bug and adding them. > > > I?m more concerned about adding these to 3.0. ?It means we?ll have to > support the new API for a long time and it is one which we are currently > trying to move away from. > > > Thoughts or comments anyone? I think that we should not add them to 1.1.1 without also adding them to 3.0. I'm in two minds about it. I'm not particularly keen on adding new functions only to deprecate them again in the next release. Also, I wonder how widely used this would be? These things have not been available since 1.1.0 came out 3 years ago and its taken until now for someone to notice. OTOH if you have a 1.0.2 application that uses these things then not having them would represent a barrier to moving off of 1.0.2. And it doesn't add that much burden to us from the perspective of moving these things to the legacy bridge, because we've got to do all the other "meth" calls anyway and these are just two more. Matt From openssl-users at dukhovni.org Thu Nov 14 09:22:20 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Thu, 14 Nov 2019 04:22:20 -0500 Subject: #10388 In-Reply-To: <49584a83-1b43-6502-869f-155a2bcc377b@openssl.org> References: <1630AF0A-18D3-481A-97C5-2551E534385A@oracle.com> <49584a83-1b43-6502-869f-155a2bcc377b@openssl.org> Message-ID: <20191114092220.GG34850@straasha.imrryr.org> On Thu, Nov 14, 2019 at 08:41:57AM +0000, Matt Caswell wrote: > I think that we should not add them to 1.1.1 without also adding them to 3.0. Yes. > OTOH if you have a 1.0.2 application that uses these things then not having > them would represent a barrier to moving off of 1.0.2. And it doesn't add > that much burden to us from the perspective of moving these things to the > legacy bridge, because we've got to do all the other "meth" calls anyway and > these are just two more. My take is that we should just add them, and then deprecate them in 3.0.0 when it comes to deprecate all the existing related functions. Having a few more members of the same family to deprecate is not a burden. The deprecation won't be any harder for having an extra handful of code-points. This is based on reports that these are basically missing accessors, not a substantial new feature, and there is some plausible need for them, now that the relevant structures are opaque. -- Viktor. From rsalz at akamai.com Thu Nov 14 13:43:14 2019 From: rsalz at akamai.com (Salz, Rich) Date: Thu, 14 Nov 2019 13:43:14 +0000 Subject: #10388 In-Reply-To: <1630AF0A-18D3-481A-97C5-2551E534385A@oracle.com> References: <1630AF0A-18D3-481A-97C5-2551E534385A@oracle.com> Message-ID: >I?m more concerned about adding these to 3.0. It means we?ll have to support the new API for a long time and it is one which we are currently trying to move away from. Will you? Have you already decided that 3.0 is an LTS release? Otherwise, you have the LTS burden and when the rest of ENGINE goes away, this does as well. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Thu Nov 14 14:15:53 2019 From: matt at openssl.org (Matt Caswell) Date: Thu, 14 Nov 2019 14:15:53 +0000 Subject: #10388 In-Reply-To: References: <1630AF0A-18D3-481A-97C5-2551E534385A@oracle.com> Message-ID: <3652b8f3-891d-ea43-9bbe-7520b6b7633b@openssl.org> On 14/11/2019 13:43, Salz, Rich wrote: > *>*I?m more concerned about adding these to 3.0. ?It means we?ll have to > support the new API for a long time and it is one which we are currently > trying to move away from. > > ? > > Will you?? Have you already decided that 3.0 is an LTS release?? > Otherwise, you have the LTS burden and when the rest of ENGINE goes > away, this does as well. > Quoting the release strategy: "No existing public interface can be removed until its replacement has been in place in an LTS stable release. The original interface must also have been documented as deprecated for at least 5 years. A public interface is any function, structure or macro declared in a public header file." So the functions we're talking about don't yet have a replacement - there will be one in 3.0. So presumably they will be documented as deprecated in 3.0. So we have to support them for at least 5 years from the release of 3.0. Matt From openssl-users at dukhovni.org Thu Nov 14 19:40:31 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Thu, 14 Nov 2019 14:40:31 -0500 Subject: #10388 In-Reply-To: <3652b8f3-891d-ea43-9bbe-7520b6b7633b@openssl.org> References: <1630AF0A-18D3-481A-97C5-2551E534385A@oracle.com> <3652b8f3-891d-ea43-9bbe-7520b6b7633b@openssl.org> Message-ID: <6F96081E-04C3-4B6D-9102-F4087F2A7A00@dukhovni.org> > On Nov 14, 2019, at 9:15 AM, Matt Caswell wrote: > > "No existing public interface can be removed until its replacement has > been in place in an LTS stable release. The original interface must also > have been documented as deprecated for at least 5 years. A public > interface is any function, structure or macro declared in a public > header file." > > So the functions we're talking about don't yet have a replacement - > there will be one in 3.0. So presumably they will be documented as > deprecated in 3.0. So we have to support them for at least 5 years from > the release of 3.0. I think that when we're deprecating an entire family of *related* accessors (for the same structure) that have been in place for some time, the addition of a missing member of that family is reasonably interpreted as not resetting the support clock on the entire family. We can still remove them all as though the missing members of the family had been in place all along. That is, we should (and if that requires a policy update and vote so be it) be able to interpret the rules based on the "spirit" (intent) without getting unduly burdened by the "letter", where common sense would suggest that we're getting the intended outcome. -- Viktor. From Matthias.St.Pierre at ncp-e.com Fri Nov 15 12:40:04 2019 From: Matthias.St.Pierre at ncp-e.com (Matthias St. Pierre) Date: Fri, 15 Nov 2019 13:40:04 +0100 Subject: #10388 In-Reply-To: <6F96081E-04C3-4B6D-9102-F4087F2A7A00@dukhovni.org> References: <1630AF0A-18D3-481A-97C5-2551E534385A@oracle.com> <3652b8f3-891d-ea43-9bbe-7520b6b7633b@openssl.org> <6F96081E-04C3-4B6D-9102-F4087F2A7A00@dukhovni.org> Message-ID: <84aebbab-e430-b8c9-8889-b20d283bdb98@ncp-e.com> On 14.11.19 20:40, Viktor Dukhovni wrote: >> On Nov 14, 2019, at 9:15 AM, Matt Caswell wrote: >> >> "No existing public interface can be removed until its replacement has >> been in place in an LTS stable release. The original interface must also >> have been documented as deprecated for at least 5 years. A public >> interface is any function, structure or macro declared in a public >> header file." >> >> So the functions we're talking about don't yet have a replacement - >> there will be one in 3.0. So presumably they will be documented as >> deprecated in 3.0. So we have to support them for at least 5 years from >> the release of 3.0. > I think that when we're deprecating an entire family of *related* accessors > (for the same structure) that have been in place for some time, the addition > of a missing member of that family is reasonably interpreted as not resetting > the support clock on the entire family. We can still remove them all as though > the missing members of the family had been in place all along. > > That is, we should (and if that requires a policy update and vote so be it) be > able to interpret the rules based on the "spirit" (intent) without getting > unduly burdened by the "letter", where common sense would suggest that we're > getting the intended outcome. I don't think we need a reinterpretation, or even a change, of the existing policy for this specific case,? since already now the *entire* engine api can only be deprecated in version 3.0 and removed afterwards according to the policy cited by Matt. We can simply add the missing accessor as bugfix to 1.1.1 and 3.0, and immediately deprecate it on the latter. The only difference will be that we end up with n+1 deprecated functions instead of n? (for some natural number n) for version 3.0. Even after 3.0 is released and as long as 1.1.1 LTS is still supported, we can proceed in the same way without violating existing policies. Matthias From paul.dale at oracle.com Fri Nov 15 22:34:25 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Sat, 16 Nov 2019 08:34:25 +1000 Subject: #10388 In-Reply-To: <84aebbab-e430-b8c9-8889-b20d283bdb98@ncp-e.com> References: <1630AF0A-18D3-481A-97C5-2551E534385A@oracle.com> <3652b8f3-891d-ea43-9bbe-7520b6b7633b@openssl.org> <6F96081E-04C3-4B6D-9102-F4087F2A7A00@dukhovni.org> <84aebbab-e430-b8c9-8889-b20d283bdb98@ncp-e.com> Message-ID: <9A785473-5FF1-46C7-84E0-D91932897C27@oracle.com> The consensus seems to be to add the deprecated API to 3.0. I?ve removed the hold. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 15 Nov 2019, at 10:40 pm, Matthias St. Pierre wrote: > > > > On 14.11.19 20:40, Viktor Dukhovni wrote: >>> On Nov 14, 2019, at 9:15 AM, Matt Caswell wrote: >>> >>> "No existing public interface can be removed until its replacement has >>> been in place in an LTS stable release. The original interface must also >>> have been documented as deprecated for at least 5 years. A public >>> interface is any function, structure or macro declared in a public >>> header file." >>> >>> So the functions we're talking about don't yet have a replacement - >>> there will be one in 3.0. So presumably they will be documented as >>> deprecated in 3.0. So we have to support them for at least 5 years from >>> the release of 3.0. >> I think that when we're deprecating an entire family of *related* accessors >> (for the same structure) that have been in place for some time, the addition >> of a missing member of that family is reasonably interpreted as not resetting >> the support clock on the entire family. We can still remove them all as though >> the missing members of the family had been in place all along. >> >> That is, we should (and if that requires a policy update and vote so be it) be >> able to interpret the rules based on the "spirit" (intent) without getting >> unduly burdened by the "letter", where common sense would suggest that we're >> getting the intended outcome. > > I don't think we need a reinterpretation, or even a change, of the existing policy for > this specific case, since already now the *entire* engine api can only be deprecated > in version 3.0 and removed afterwards according to the policy cited by Matt. > > We can simply add the missing accessor as bugfix to 1.1.1 and 3.0, and immediately > deprecate it on the latter. The only difference will be that we end up with n+1 deprecated > functions instead of n (for some natural number n) for version 3.0. > > Even after 3.0 is released and as long as 1.1.1 LTS is still supported, we can proceed > in the same way without violating existing policies. > > > Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From Matthias.St.Pierre at ncp-e.com Mon Nov 18 21:48:38 2019 From: Matthias.St.Pierre at ncp-e.com (Dr. Matthias St. Pierre) Date: Mon, 18 Nov 2019 21:48:38 +0000 Subject: Build failures on master?! Message-ID: <74ffec7814564f6cb54afd69005fac2c@Ex13.ncp.local> The last 19 commits on https://github.com/openssl/openssl/commits/master, starting from Nov 14 have a red cross from the CIs. What's going on again? My personal impression is that builds on master are failing 50% of the time. It is really tedious to check pull requests for build errors just to find that those errors are well known failures from master. In particular, the krb5 an pyca tests are notoriously failing. Are there any plans to fix them or disable them? 95-test_external_krb5.t (Wstat: 256 Tests: 1 Failed: 1) 95-test_external_pyca.t (Wstat: 256 Tests: 1 Failed: 1) Clean builds on master should have highest priority. Because if there are too many red crosses, nobody cares about them anymore. Matthias From kurt at roeckx.be Mon Nov 18 22:07:53 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Mon, 18 Nov 2019 23:07:53 +0100 Subject: Build failures on master?! In-Reply-To: <74ffec7814564f6cb54afd69005fac2c@Ex13.ncp.local> References: <74ffec7814564f6cb54afd69005fac2c@Ex13.ncp.local> Message-ID: <20191118220753.GA1392@roeckx.be> On Mon, Nov 18, 2019 at 09:48:38PM +0000, Dr. Matthias St. Pierre wrote: > The last 19 commits on https://github.com/openssl/openssl/commits/master, > starting from Nov 14 have a red cross from the CIs. What's going on again? I have filed 2 issues on Nov 9 that that caused the CIs to fail, that that haven't been closed yet. > My personal impression is that builds on master are failing 50% of the time. > > It is really tedious to check pull requests for build errors just to find that those errors > are well known failures from master. In particular, the krb5 an pyca tests are notoriously > failing. Are there any plans to fix them or disable them? > > 95-test_external_krb5.t (Wstat: 256 Tests: 1 Failed: 1) > 95-test_external_pyca.t (Wstat: 256 Tests: 1 Failed: 1) > > Clean builds on master should have highest priority. Because if there are too many red > crosses, nobody cares about them anymore. Note that they are also broken in the 1.1.1-stable branch. At least pyca is something you should be able to fix with a new upstream version. Kurt From matt at openssl.org Mon Nov 18 22:12:52 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 18 Nov 2019 22:12:52 +0000 Subject: Build failures on master?! In-Reply-To: <74ffec7814564f6cb54afd69005fac2c@Ex13.ncp.local> References: <74ffec7814564f6cb54afd69005fac2c@Ex13.ncp.local> Message-ID: <4e5c6cef-9328-e2b3-d5f6-bf1afa4ce3bb@openssl.org> On 18/11/2019 21:48, Dr. Matthias St. Pierre wrote: > The last 19 commits on https://github.com/openssl/openssl/commits/master, > starting from Nov 14 have a red cross from the CIs. What's going on again? > My personal impression is that builds on master are failing 50% of the time. The builds on master have been failing for *much* longer than that. They should be succeeded on the PRs, but its the extended tests that fail. > > It is really tedious to check pull requests for build errors just to find that those errors > are well known failures from master. In particular, the krb5 an pyca tests are notoriously > failing. Are there any plans to fix them or disable them? The build failures mostly come from the external tests (or that was the case I haven't checked recently). The problem with these is that, on occasion we need the maintainers of these packages to help us track down a problem. Or in some cases we actually need them to make a change (for example because something changed on master which revealed that they were relying on some behaviour in master which they really shouldn't be relying on). Once one of them is failing then multiple errors can build up and they get masked by the earlier failure. I'll try and find some time to take another pass at the current failures. I suspect the answer might be to disable some of the external tests. Matt > > 95-test_external_krb5.t (Wstat: 256 Tests: 1 Failed: 1) > 95-test_external_pyca.t (Wstat: 256 Tests: 1 Failed: 1) > > Clean builds on master should have highest priority. Because if there are too many red > crosses, nobody cares about them anymore. > > Matthias > > From Matthias.St.Pierre at ncp-e.com Mon Nov 18 22:29:10 2019 From: Matthias.St.Pierre at ncp-e.com (Dr. Matthias St. Pierre) Date: Mon, 18 Nov 2019 22:29:10 +0000 Subject: AW: Build failures on master?! In-Reply-To: <4e5c6cef-9328-e2b3-d5f6-bf1afa4ce3bb@openssl.org> References: <74ffec7814564f6cb54afd69005fac2c@Ex13.ncp.local> <4e5c6cef-9328-e2b3-d5f6-bf1afa4ce3bb@openssl.org> Message-ID: > > The last 19 commits on https://github.com/openssl/openssl/commits/master, > > starting from Nov 14 have a red cross from the CIs. What's going on again? > > My personal impression is that builds on master are failing 50% of the time. > > The builds on master have been failing for *much* longer than that. They > should be succeeded on the PRs, but its the extended tests that fail. Yes I know. I was just counting the last consecutive failures. What actually wonders me is the fact that the tests do not fail throughout. In between there are always short periods where the tests succced. Why is this? The reason why I issued this cry for help is because after watching it for a long time I got the impression that due to the ongoing failures everybody has become dull and indifferent agains the red crosses. Matthias From matt at openssl.org Mon Nov 18 22:32:23 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 18 Nov 2019 22:32:23 +0000 Subject: AW: Build failures on master?! In-Reply-To: References: <74ffec7814564f6cb54afd69005fac2c@Ex13.ncp.local> <4e5c6cef-9328-e2b3-d5f6-bf1afa4ce3bb@openssl.org> Message-ID: <51371288-59f4-1aab-ccb3-ffa51be4e710@openssl.org> On 18/11/2019 22:29, Dr. Matthias St. Pierre wrote: >>> The last 19 commits on https://github.com/openssl/openssl/commits/master, >>> starting from Nov 14 have a red cross from the CIs. What's going on again? >>> My personal impression is that builds on master are failing 50% of the time. >> >> The builds on master have been failing for *much* longer than that. They >> should be succeeded on the PRs, but its the extended tests that fail. > > Yes I know. I was just counting the last consecutive failures. What actually wonders > me is the fact that the tests do not fail throughout. In between there are always short > periods where the tests succced. Why is this? Really? I don't see that. The travis dashboard shows red for as far back as I checked: https://travis-ci.org/openssl/openssl/builds Matt > > The reason why I issued this cry for help is because after watching it for a long time I > got the impression that due to the ongoing failures everybody has become dull and > indifferent agains the red crosses. > > Matthias > From Matthias.St.Pierre at ncp-e.com Mon Nov 18 22:39:47 2019 From: Matthias.St.Pierre at ncp-e.com (Dr. Matthias St. Pierre) Date: Mon, 18 Nov 2019 22:39:47 +0000 Subject: AW: AW: Build failures on master?! In-Reply-To: <51371288-59f4-1aab-ccb3-ffa51be4e710@openssl.org> References: <74ffec7814564f6cb54afd69005fac2c@Ex13.ncp.local> <4e5c6cef-9328-e2b3-d5f6-bf1afa4ce3bb@openssl.org> <51371288-59f4-1aab-ccb3-ffa51be4e710@openssl.org> Message-ID: > > Yes I know. I was just counting the last consecutive failures. What actually wonders > > me is the fact that the tests do not fail throughout. In between there are always short > > periods where the tests succced. Why is this? > > Really? I don't see that. The travis dashboard shows red for as far back > as I checked: > > https://travis-ci.org/openssl/openssl/builds > Interesting. I was just scrolling down the commits in the timeline of master: https://github.com/openssl/openssl/commits/master there are a lot of (clickable) red crosses next to the commits. The last commits without red crosses are some of yours: https://github.com/openssl/openssl/commits/11d876df50d90ec4b26364c4b7d317ea19139e13 Note that they don't show any sign of success, only the red cross is missing. I have no idea what the difference between those commits with and without red crosses is, if travis is failing all the time. Natthias From matt at openssl.org Mon Nov 18 22:58:00 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 18 Nov 2019 22:58:00 +0000 Subject: AW: AW: Build failures on master?! In-Reply-To: References: <74ffec7814564f6cb54afd69005fac2c@Ex13.ncp.local> <4e5c6cef-9328-e2b3-d5f6-bf1afa4ce3bb@openssl.org> <51371288-59f4-1aab-ccb3-ffa51be4e710@openssl.org> Message-ID: <70b6f15f-4497-0e56-1255-3c297884bd5a@openssl.org> On 18/11/2019 22:39, Dr. Matthias St. Pierre wrote: >>> Yes I know. I was just counting the last consecutive failures. What actually wonders >>> me is the fact that the tests do not fail throughout. In between there are always short >>> periods where the tests succced. Why is this? >> >> Really? I don't see that. The travis dashboard shows red for as far back >> as I checked: >> >> https://travis-ci.org/openssl/openssl/builds >> > > Interesting. I was just scrolling down the commits in the timeline of master: > > https://github.com/openssl/openssl/commits/master > > there are a lot of (clickable) red crosses next to the commits. > > The last commits without red crosses are some of yours: > > https://github.com/openssl/openssl/commits/11d876df50d90ec4b26364c4b7d317ea19139e13 > > Note that they don't show any sign of success, only the red cross is missing. > > I have no idea what the difference between those commits with and without red crosses is, > if travis is failing all the time. If you push a whole bunch of commits all at the same time (e.g. because a PR contains more than one commit), then I think travis only runs for the most recent HEAD and skips the intervening commits. Matt From beldmit at gmail.com Thu Nov 21 03:26:14 2019 From: beldmit at gmail.com (Dmitry Belyavsky) Date: Thu, 21 Nov 2019 06:26:14 +0300 Subject: Malloc failures check Message-ID: Hello, Observing a series of similar bugs related to a lack of checks of the malloc return values, I wonder if we could automate the search of these errors on the compile level (e.g. similar to the __owur macro)? -- SY, Dmitry Belyavsky -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.dale at oracle.com Thu Nov 21 05:47:32 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Thu, 21 Nov 2019 15:47:32 +1000 Subject: Malloc failures check In-Reply-To: References: Message-ID: <94F98E4B-2D79-45E2-AC38-608DFF109435@oracle.com> Adding a compile time check would be good. I?m not sure how. It would be possible to implement a malloc failure feature in the test suite that systematically runs a test many times, failing successive malloc calls. I?m kind of surprised that the various static analysers hadn't found some of these. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 21 Nov 2019, at 1:26 pm, Dmitry Belyavsky wrote: > > Hello, > > Observing a series of similar bugs related to a lack of checks of the malloc return values, I wonder if we could automate the search of these errors on the compile level (e.g. similar to the __owur macro)? > > -- > SY, Dmitry Belyavsky -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Thu Nov 21 05:55:27 2019 From: rsalz at akamai.com (Salz, Rich) Date: Thu, 21 Nov 2019 05:55:27 +0000 Subject: Malloc failures check In-Reply-To: <94F98E4B-2D79-45E2-AC38-608DFF109435@oracle.com> References: <94F98E4B-2D79-45E2-AC38-608DFF109435@oracle.com> Message-ID: * It would be possible to implement a malloc failure feature in the test suite that systematically runs a test many times, failing successive malloc calls. It?s there; look crypto/mem.c, shouldfail() and FAILTEST. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Thu Nov 21 10:30:31 2019 From: rsalz at akamai.com (Salz, Rich) Date: Thu, 21 Nov 2019 10:30:31 +0000 Subject: Malloc failures check In-Reply-To: References: <94F98E4B-2D79-45E2-AC38-608DFF109435@oracle.com> Message-ID: * It would be possible to implement a malloc failure feature in the test suite that systematically runs a test many times, failing successive malloc calls. It?s there; look crypto/mem.c, shouldfail() and FAILTEST. More detail, from off-list discusson: i=0 while : ; do ((i++)) export MALLOC_FAILURE_CHECKS=${i}@100 openssl foo etc? test -f core && echo crashed && exit 1 done -------------- next part -------------- An HTML attachment was scrubbed... URL: From beldmit at gmail.com Thu Nov 21 13:29:45 2019 From: beldmit at gmail.com (Dmitry Belyavsky) Date: Thu, 21 Nov 2019 16:29:45 +0300 Subject: Malloc failures check In-Reply-To: References: <94F98E4B-2D79-45E2-AC38-608DFF109435@oracle.com> Message-ID: Unfortunately, it is not a compile-time check... I mean smth like https://github.com/openssl/openssl/blob/ab5c77b4766e0992751d86560193ca42b49cf316/include/openssl/e_os2.h#L198-L202 but not sure it is applicable to external functions... On Thu, Nov 21, 2019 at 1:42 PM Salz, Rich wrote: > > - It would be possible to implement a malloc failure feature in the > test suite that systematically runs a test many times, failing successive > malloc calls. > > > > It?s there; look crypto/mem.c, shouldfail() and FAILTEST. > > > > More detail, from off-list discusson: > > > > i=0 > > while : ; do > > ((i++)) > > export MALLOC_FAILURE_CHECKS=${i}@100 openssl foo etc? > > test -f core && echo crashed && exit 1 > > done > > > > > -- SY, Dmitry Belyavsky -------------- next part -------------- An HTML attachment was scrubbed... URL: From kaduk at mit.edu Fri Nov 22 03:43:40 2019 From: kaduk at mit.edu (Benjamin Kaduk) Date: Thu, 21 Nov 2019 19:43:40 -0800 Subject: Build failures on master?! In-Reply-To: <4e5c6cef-9328-e2b3-d5f6-bf1afa4ce3bb@openssl.org> References: <74ffec7814564f6cb54afd69005fac2c@Ex13.ncp.local> <4e5c6cef-9328-e2b3-d5f6-bf1afa4ce3bb@openssl.org> Message-ID: <20191122034340.GX32847@mit.edu> On Mon, Nov 18, 2019 at 10:12:52PM +0000, Matt Caswell wrote: > > > On 18/11/2019 21:48, Dr. Matthias St. Pierre wrote: > > The last 19 commits on https://github.com/openssl/openssl/commits/master, > > starting from Nov 14 have a red cross from the CIs. What's going on again? > > My personal impression is that builds on master are failing 50% of the time. > > The builds on master have been failing for *much* longer than that. They > should be succeeded on the PRs, but its the extended tests that fail. > > > > > It is really tedious to check pull requests for build errors just to find that those errors > > are well known failures from master. In particular, the krb5 an pyca tests are notoriously > > failing. Are there any plans to fix them or disable them? > > The build failures mostly come from the external tests (or that was the > case I haven't checked recently). The problem with these is that, on > occasion we need the maintainers of these packages to help us track down > a problem. Or in some cases we actually need them to make a change (for > example because something changed on master which revealed that they > were relying on some behaviour in master which they really shouldn't be > relying on). > > Once one of them is failing then multiple errors can build up and they > get masked by the earlier failure. > > I'll try and find some time to take another pass at the current > failures. I suspect the answer might be to disable some of the external > tests. Please let me know if you want me to look at the krb5 tests (or pester folks at upstream there to do so). I'm in Singapore for the IETF this week and have been behind on, well, everything. -Ben From matt at openssl.org Fri Nov 22 09:38:17 2019 From: matt at openssl.org (Matt Caswell) Date: Fri, 22 Nov 2019 09:38:17 +0000 Subject: OpenSSL Technical Committee Message-ID: <3e948df0-ef07-9a90-3635-4424e1283aa2@openssl.org> I'd like to draw your attention to this PR that I just created: https://github.com/openssl/web/pull/143 This represents a significant update to the project by-laws so we'd like to gather feedback from the community on the proposal. This was discussed at some length at the recent OMC and commmitter face-2-face meetings in Nuremburg, but has not yet gone to an OMC vote. The idea is to split the current responsibilities of the OMC into two. We would retain the OMC for overall management of the project. The OMC would own the requirements, and make decisions around what features and platforms we are going to support, as well as decisions around release timing, support and EOL etc. A newly formed group, the OpenSSL Technical Committee (OTC), would have responsibilities for making technical decisions around design, architecture, testing, documentation, etc. Responsibility for PR approvals would move from the OMC to the OTC. You can think of it like the OMC owning the answers to WHAT questions, with the OTC owning HOW questions. Please take a look. Matt From levitte at openssl.org Wed Nov 27 09:38:41 2019 From: levitte at openssl.org (Richard Levitte) Date: Wed, 27 Nov 2019 10:38:41 +0100 Subject: Check NULL pointers or not... Message-ID: <87r21ty7z2.wl-levitte@openssl.org> Depending on who you're asking, you get different responses. The topic is: should we check pointers coming from applications before using them or not? There are two positions on this: 1. Yes we should so we don't crash processes unnecessarily, and we should return an error if pointers are NULL (unless that's valid input, such as for the freeing functions). There is a generic error reason code for it, ERR_R_PASSED_NULL_PARAMETER. 2. No we should not, as we don't cater for user programming errors. Also, it allows us to catch our own errors early. For a while, our pendulum was going with option 2, but lately, we seem to go back to option 1. Both options have their pros and cons: 1. cons: there's the very real possibility that users don't check for errors as they should, and things go wrong further on, sometimes in ways where the actual cause is very hard to figure out. pros: programs will not spuriously crash because we didn't catch an internal corner case in our tests. 2. cons: programs may crash, sometimes due to our own mistakes, sometimes due to user errors. pros: some very subtle bugs are found, either quicker or at all. An actual case is PR 7630 [1], which was uncovered because there was a crash due to a NULL being used (this was ultimately due to users not checking errors). As a middle ground, and perhaps a way to satify both camps, we could use ossl_assert() in our checks for input NULL, as follows: if (!ossl_assert(ptr != NULL)) { ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); return 0; } This means that in a debug build, there may be crashes due to pointers being NULL, while in a release build, the caller gets an error. Thoughts? Cheers, Richard [1] https://github.com/openssl/openssl/pull/7630 -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From Matthias.St.Pierre at ncp-e.com Wed Nov 27 10:53:01 2019 From: Matthias.St.Pierre at ncp-e.com (Dr. Matthias St. Pierre) Date: Wed, 27 Nov 2019 10:53:01 +0000 Subject: AW: Check NULL pointers or not... In-Reply-To: <87r21ty7z2.wl-levitte@openssl.org> References: <87r21ty7z2.wl-levitte@openssl.org> Message-ID: <2639514f673643a8b3c327ce63123bb6@Ex13.ncp.local> FYI: there was a related discussion a year ago on GitHub about an ossl_is_null() macro, https://github.com/openssl/openssl/pull/7218 which ended up undecided. Matthias From levitte at openssl.org Thu Nov 28 11:39:34 2019 From: levitte at openssl.org (Richard Levitte) Date: Thu, 28 Nov 2019 12:39:34 +0100 Subject: AW: Check NULL pointers or not... In-Reply-To: <2639514f673643a8b3c327ce63123bb6@Ex13.ncp.local> References: <87r21ty7z2.wl-levitte@openssl.org> <2639514f673643a8b3c327ce63123bb6@Ex13.ncp.local> Message-ID: <87lfs0xma1.wl-levitte@openssl.org> On Wed, 27 Nov 2019 11:53:01 +0100, Dr. Matthias St. Pierre wrote: > > FYI: there was a related discussion a year ago on GitHub about an ossl_is_null() macro, > > https://github.com/openssl/openssl/pull/7218 > > which ended up undecided. Thank you. I had totally forgotten that one, so reminder appreciated. Looking at the diff shows me that there's a precedent for what I proposed (using ossl_assert). Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From kaduk at mit.edu Thu Nov 28 19:19:59 2019 From: kaduk at mit.edu (Benjamin Kaduk) Date: Thu, 28 Nov 2019 11:19:59 -0800 Subject: Check NULL pointers or not... In-Reply-To: <87r21ty7z2.wl-levitte@openssl.org> References: <87r21ty7z2.wl-levitte@openssl.org> Message-ID: <20191128191932.GT32847@mit.edu> On Wed, Nov 27, 2019 at 10:38:41AM +0100, Richard Levitte wrote: > Depending on who you're asking, you get different responses. > > The topic is: should we check pointers coming from applications before > using them or not? > > There are two positions on this: > > 1. Yes we should so we don't crash processes unnecessarily, and we > should return an error if pointers are NULL (unless that's valid > input, such as for the freeing functions). There is a generic > error reason code for it, ERR_R_PASSED_NULL_PARAMETER. > > 2. No we should not, as we don't cater for user programming errors. > Also, it allows us to catch our own errors early. > > For a while, our pendulum was going with option 2, but lately, we seem > to go back to option 1. > > Both options have their pros and cons: > > 1. cons: there's the very real possibility that users don't check for > errors as they should, and things go wrong further on, > sometimes in ways where the actual cause is very hard to > figure out. > pros: programs will not spuriously crash because we didn't catch an > internal corner case in our tests. > > 2. cons: programs may crash, sometimes due to our own mistakes, > sometimes due to user errors. > pros: some very subtle bugs are found, either quicker or at all. > An actual case is PR 7630 [1], which was uncovered because > there was a crash due to a NULL being used (this was > ultimately due to users not checking errors). > > As a middle ground, and perhaps a way to satify both camps, we could > use ossl_assert() in our checks for input NULL, as follows: > > if (!ossl_assert(ptr != NULL)) { > ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); > return 0; > } > > This means that in a debug build, there may be crashes due to pointers > being NULL, while in a release build, the caller gets an error. > > Thoughts? My recollection was that we had some reasonable support for this "middle ground" approach. I don't personally have strong enough feelings to advocate a specific position, and will try to adhere to the group consensus going forward. -Ben From tmraz at redhat.com Fri Nov 29 07:43:17 2019 From: tmraz at redhat.com (Tomas Mraz) Date: Fri, 29 Nov 2019 08:43:17 +0100 Subject: Check NULL pointers or not... In-Reply-To: <20191128191932.GT32847@mit.edu> References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> Message-ID: On Thu, 2019-11-28 at 11:19 -0800, Benjamin Kaduk wrote: > On Wed, Nov 27, 2019 at 10:38:41AM +0100, Richard Levitte wrote: > > Depending on who you're asking, you get different responses. > > > > The topic is: should we check pointers coming from applications > > before > > using them or not? > > > > There are two positions on this: > > > > 1. Yes we should so we don't crash processes unnecessarily, and we > > should return an error if pointers are NULL (unless that's valid > > input, such as for the freeing functions). There is a generic > > error reason code for it, ERR_R_PASSED_NULL_PARAMETER. > > > > 2. No we should not, as we don't cater for user programming errors. > > Also, it allows us to catch our own errors early. > > > > For a while, our pendulum was going with option 2, but lately, we > > seem > > to go back to option 1. > > > > Both options have their pros and cons: > > > > 1. cons: there's the very real possibility that users don't check > > for > > errors as they should, and things go wrong further on, > > sometimes in ways where the actual cause is very hard to > > figure out. > > pros: programs will not spuriously crash because we didn't catch > > an > > internal corner case in our tests. > > > > 2. cons: programs may crash, sometimes due to our own mistakes, > > sometimes due to user errors. > > pros: some very subtle bugs are found, either quicker or at all. > > An actual case is PR 7630 [1], which was uncovered because > > there was a crash due to a NULL being used (this was > > ultimately due to users not checking errors). > > > > As a middle ground, and perhaps a way to satify both camps, we > > could > > use ossl_assert() in our checks for input NULL, as follows: > > > > if (!ossl_assert(ptr != NULL)) { > > ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); > > return 0; > > } > > > > This means that in a debug build, there may be crashes due to > > pointers > > being NULL, while in a release build, the caller gets an error. > > > > Thoughts? > > My recollection was that we had some reasonable support for this > "middle > ground" approach. I don't personally have strong enough feelings to > advocate a specific position, and will try to adhere to the group > consensus > going forward. I'd the option 2 or the middle-ground option. However if we choose the middle-ground will we try to apply it to old code or will we just add it to the new API calls or when the code of an existing API call is being significantly reworked anyway? Also there might be exceptional cases where taking the option 1 is actually a better choice, but I do not think this should be the norm. Also in some cases where the API call is called very frequently in performance sensitive code it might be good idea to choose option 2 even in case we would decide to use the middle-ground otherwise. -- Tom?? Mr?z No matter how far down the wrong road you've gone, turn back. Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.] From tjh at cryptsoft.com Fri Nov 29 08:34:49 2019 From: tjh at cryptsoft.com (Tim Hudson) Date: Fri, 29 Nov 2019 18:34:49 +1000 Subject: Check NULL pointers or not... In-Reply-To: References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> Message-ID: The way I view the issue is to look at what happens when things go wrong - what is the impact - and evaluate the difference in behaviour between the approaches. You have to start from the premise that (in general) software is not tested for all possible usage models - i.e. test coverage isn't at 100% - so there are always code paths and circumstances that can occur that simply haven't been hit during testing. That means that having a special assert enabled build doesn't solve the problem - in that a gap in testing always remains - so the "middle ground" doesn't alter the fundamentals as such and does also lead to basically testing something other than what is running in a default (non-debug) build. Testing precisely what you will actually run is a fairly normal principle to work from - but many overlook those sorts of differences. In a production environment, it is almost never appropriate to simply crash in an uncontrolled manner (i.e. dereferencing a NULL pointer). There are many contexts that generate real issues where a more graceful failure mode (i.e. anything other than crashing) results in a much better user outcome. Denial-of-service attacks often rely on this sort of issue - basically recognising that testing doesn't cover everything and finding unusual circumstances that create some form of load on a system that impacts other things. You are effectively adding in a whole pile of abort() references in the code by not checking - that is the actual effect - and abort isn't something that should ever be called in production code. The other rather practical thing is that when you do check for incoming NULL pointers, you end up with a lot less reports of issues in a library - as you aren't sitting in the call chain of a crash - and that in itself saves a lot of time when dealing with users. Many people will report issues like this - but if they get an error return rather than a crash they do (generally) keep looking. And developer time for OpenSSL (like many projects) is the most scarce resource that we have. Anything that reduces the impact on looking at crashes enables people to perform more useful work rather than debugging a coding error of an OpenSSL user. Another simple argument that helps to make a choice is that whatever we do we need to be consistent in our handling of things - and at the moment many functions check and many functions do not. And if we make things consistent we certainly don't really have the option of undoing any of the null checks because that will break code - it is effectively part of the API contract - that it is safe to call certain functions with NULL arguments. Adding extra checks from an API contract point of view is harmless, removing them isn't an option. And if we want consistency then we pretty much have only one choice - to check everywhere. There are *very few places* where such checks will have a substantial performance impact in real-world contexts. Arguments about the C runtime library not checking simply aren't relevant. The C runtime library doesn't set the API contracts and usage model that we use. And there are C runtime functions that do check. What we should be looking at in my view is the impact when things go wrong - and a position that basically says the caller isn't allowed to make mistakes ever for simple things like this which are easy to check isn't appropriate. That it does also reduce the burden on the project (and it also increases the users perception of the quality because they are always looking at their own bugs and not thinking that OpenSSL has a major issue with an uncontrolled crash - even if they don't report it publicly). Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.dale at oracle.com Fri Nov 29 08:38:28 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Fri, 29 Nov 2019 18:38:28 +1000 Subject: Check NULL pointers or not... In-Reply-To: <20191128191932.GT32847@mit.edu> References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> Message-ID: <5E496DB2-90CA-4490-85E1-6267D20FA4E8@oracle.com> I?d prefer option 1 or the middle ground. I?ve lost count of the number of times I?ve seen programs crashing in the crypto library which required mammoth debugging efforts to irrefutably demonstrate that the caller is at fault rather than the crypto library :( Option 1 would be preferable from this point of view but it can cause a performance hit ? most of the time it wouldn?t matter but when it does it would be a big deal. The middle ground doesn?t entail any performance loss in production code (it does in debug but that shouldn?t be relevant). Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On Wed, Nov 27, 2019 at 10:38:41AM +0100, Richard Levitte wrote: >> Depending on who you're asking, you get different responses. >> >> The topic is: should we check pointers coming from applications before >> using them or not? >> >> There are two positions on this: >> >> 1. Yes we should so we don't crash processes unnecessarily, and we >> should return an error if pointers are NULL (unless that's valid >> input, such as for the freeing functions). There is a generic >> error reason code for it, ERR_R_PASSED_NULL_PARAMETER. >> >> 2. No we should not, as we don't cater for user programming errors. >> Also, it allows us to catch our own errors early. >> >> For a while, our pendulum was going with option 2, but lately, we seem >> to go back to option 1. >> >> Both options have their pros and cons: >> >> 1. cons: there's the very real possibility that users don't check for >> errors as they should, and things go wrong further on, >> sometimes in ways where the actual cause is very hard to >> figure out. >> pros: programs will not spuriously crash because we didn't catch an >> internal corner case in our tests. >> >> 2. cons: programs may crash, sometimes due to our own mistakes, >> sometimes due to user errors. >> pros: some very subtle bugs are found, either quicker or at all. >> An actual case is PR 7630 [1], which was uncovered because >> there was a crash due to a NULL being used (this was >> ultimately due to users not checking errors). >> >> As a middle ground, and perhaps a way to satify both camps, we could >> use ossl_assert() in our checks for input NULL, as follows: >> >> if (!ossl_assert(ptr != NULL)) { >> ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); >> return 0; >> } >> >> This means that in a debug build, there may be crashes due to pointers >> being NULL, while in a release build, the caller gets an error. >> >> Thoughts? From tmraz at redhat.com Fri Nov 29 09:07:54 2019 From: tmraz at redhat.com (Tomas Mraz) Date: Fri, 29 Nov 2019 10:07:54 +0100 Subject: Check NULL pointers or not... In-Reply-To: References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> Message-ID: The "always check for NULL pointers" approach does not avoid catastrophical errors in applications. For example let's say an application code encrypts some plaintext in-place and sends it out as encrypted. Let's say we check for the NULL EVP_CIPHER_CTX in EVP_CipherUpdate() but the app does not bother checking for the error return as it did not bother for the same on EVP_CIPHER_CTX_new(). The application will then happily (and silently) send out a plaintext instead of ciphertext. -- Tom?? Mr?z No matter how far down the wrong road you've gone, turn back. Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.] From matt at openssl.org Fri Nov 29 09:22:01 2019 From: matt at openssl.org (Matt Caswell) Date: Fri, 29 Nov 2019 09:22:01 +0000 Subject: Check NULL pointers or not... In-Reply-To: <5E496DB2-90CA-4490-85E1-6267D20FA4E8@oracle.com> References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> <5E496DB2-90CA-4490-85E1-6267D20FA4E8@oracle.com> Message-ID: <5aab7fcb-a980-1283-6542-1d07b9add22f@openssl.org> On 29/11/2019 08:38, Dr Paul Dale wrote: > I?d prefer option 1 or the middle ground. I?ve lost count of the > number of times I?ve seen programs crashing in the crypto library > which required mammoth debugging efforts to irrefutably demonstrate > that the caller is at fault rather than the crypto library :( > > Option 1 would be preferable from this point of view but it can cause > a performance hit ? most of the time it wouldn?t matter but when it > does it would be a big deal. The middle ground doesn?t entail any > performance loss in production code (it does in debug but that > shouldn?t be relevant). I think you misunderstand the middle ground option: if (!ossl_assert(ptr != NULL)) { ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); return 0; } In debug code this will crash if ptr is NULL. In production code this acts exactly like option 1 - so has exactly the same performance hit. For the record my preference is the middle ground option as being the norm for new code and where we make a significant refactor of old code. If something truly is performance critical then we can choose not to do it in those cases. Matt From tjh at cryptsoft.com Fri Nov 29 09:23:48 2019 From: tjh at cryptsoft.com (Tim Hudson) Date: Fri, 29 Nov 2019 19:23:48 +1000 Subject: Check NULL pointers or not... In-Reply-To: References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> Message-ID: On Fri, Nov 29, 2019 at 7:08 PM Tomas Mraz wrote: > The "always check for NULL pointers" approach does not avoid > catastrophical errors in applications. I didn't say it avoided all errors (nor did anyone else on the thread that I've read) - but it does avoid a whole class of errors. And for that particular context there are many things you can do to mitigate it - and incorrect handling of EVP_CipherUpdate itself is very common - where error returns are completely ignored. We could reasonably define that it should wipe out the output buffer on any error condition - that would make the function safer in a whole pile of contexts. However that is talking about a different issue IMHO. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.dale at oracle.com Fri Nov 29 09:40:14 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Fri, 29 Nov 2019 19:40:14 +1000 Subject: Check NULL pointers or not... In-Reply-To: <5aab7fcb-a980-1283-6542-1d07b9add22f@openssl.org> References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> <5E496DB2-90CA-4490-85E1-6267D20FA4E8@oracle.com> <5aab7fcb-a980-1283-6542-1d07b9add22f@openssl.org> Message-ID: Oops, you are correct. I was under the mistaken impression that ossl_assert compiled to nothing outside of debug mode. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 29 Nov 2019, at 7:22 pm, Matt Caswell wrote: > > > > On 29/11/2019 08:38, Dr Paul Dale wrote: >> I?d prefer option 1 or the middle ground. I?ve lost count of the >> number of times I?ve seen programs crashing in the crypto library >> which required mammoth debugging efforts to irrefutably demonstrate >> that the caller is at fault rather than the crypto library :( >> >> Option 1 would be preferable from this point of view but it can cause >> a performance hit ? most of the time it wouldn?t matter but when it >> does it would be a big deal. The middle ground doesn?t entail any >> performance loss in production code (it does in debug but that >> shouldn?t be relevant). > > > I think you misunderstand the middle ground option: > > if (!ossl_assert(ptr != NULL)) { > ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); > return 0; > } > > In debug code this will crash if ptr is NULL. In production code this > acts exactly like option 1 - so has exactly the same performance hit. > > For the record my preference is the middle ground option as being the > norm for new code and where we make a significant refactor of old code. > If something truly is performance critical then we can choose not to do > it in those cases. > > Matt > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Matthias.St.Pierre at ncp-e.com Fri Nov 29 10:29:39 2019 From: Matthias.St.Pierre at ncp-e.com (Matthias St. Pierre) Date: Fri, 29 Nov 2019 11:29:39 +0100 Subject: Check NULL pointers or not... In-Reply-To: <5aab7fcb-a980-1283-6542-1d07b9add22f@openssl.org> References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> <5E496DB2-90CA-4490-85E1-6267D20FA4E8@oracle.com> <5aab7fcb-a980-1283-6542-1d07b9add22f@openssl.org> Message-ID: On 29.11.19 10:22, Matt Caswell wrote: > > if (!ossl_assert(ptr != NULL)) { > ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); > return 0; > } > I still dislike the odd way in which the assertion needs to be formulated, with the double negation. With the `ossl_is_null()` macro, which I proposed in #7218, the same condition would read if (ossl_is_null(ptr)) { ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); return 0; } Isn't that much better readable and easier to understand? For more examples like that, see the change set https://github.com/openssl/openssl/pull/7218/files Matthias From Matthias.St.Pierre at ncp-e.com Fri Nov 29 10:32:06 2019 From: Matthias.St.Pierre at ncp-e.com (Matthias St. Pierre) Date: Fri, 29 Nov 2019 11:32:06 +0100 Subject: Check NULL pointers or not... In-Reply-To: References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> <5E496DB2-90CA-4490-85E1-6267D20FA4E8@oracle.com> <5aab7fcb-a980-1283-6542-1d07b9add22f@openssl.org> Message-ID: <0c7a76e8-5a45-1889-ea29-468aa37e7fe7@ncp-e.com> On 29.11.19 11:29, Matthias St. Pierre wrote: > > > On 29.11.19 10:22, Matt Caswell wrote: >> >> ???? if (!ossl_assert(ptr != NULL)) { >> ???????? ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); >> ???????? return 0; >> ???? } >> > > I still dislike the odd way in which the assertion needs to be formulated, > with the double negation. With the `ossl_is_null()` macro, which I proposed > in #7218, the same condition would read > > ??? if (ossl_is_null(ptr)) { > ??????? ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); > ??????? return 0; > ??? } > > > Isn't that much better readable and easier to understand? > > > For more examples like that, see the change set > > https://github.com/openssl/openssl/pull/7218/files > > > Matthias > > Moreover, in the debug build you get the error message? "Invalid NULL pointer:" instead of a generic "Assertion Failed:" https://github.com/openssl/openssl/pull/7218/files#diff-6e9d962dc8c30948fdf827ad471ec11dR41-R44 From matt at openssl.org Fri Nov 29 10:35:20 2019 From: matt at openssl.org (Matt Caswell) Date: Fri, 29 Nov 2019 10:35:20 +0000 Subject: Check NULL pointers or not... In-Reply-To: References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> <5E496DB2-90CA-4490-85E1-6267D20FA4E8@oracle.com> <5aab7fcb-a980-1283-6542-1d07b9add22f@openssl.org> Message-ID: On 29/11/2019 10:29, Matthias St. Pierre wrote: > > > On 29.11.19 10:22, Matt Caswell wrote: >> >> ???? if (!ossl_assert(ptr != NULL)) { >> ???????? ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); >> ???????? return 0; >> ???? } >> > > I still dislike the odd way in which the assertion needs to be formulated, > with the double negation. With the `ossl_is_null()` macro, which I proposed > in #7218, the same condition would read > > ??? if (ossl_is_null(ptr)) { > ??????? ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); > ??????? return 0; > ??? } > > > Isn't that much better readable and easier to understand? Yes, I quite like that approach. This is just a refinement of the "middle ground" option though. Matt From rsalz at akamai.com Fri Nov 29 18:02:02 2019 From: rsalz at akamai.com (Salz, Rich) Date: Fri, 29 Nov 2019 18:02:02 +0000 Subject: Check NULL pointers or not... In-Reply-To: References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> Message-ID: <50E24421-57C1-469D-85A3-12C779D37C05@akamai.com> * In a production environment, it is almost never appropriate to simply crash in an uncontrolled manner (i.e. dereferencing a NULL pointer). Applications that want this can check parameters themselves before calling the function. Saying ?C arguments don?t hold? is only because it goes against your position :) FILE *f = fopen(?/?, ?w?); fprintf(f, ?hello world?); Is a programming error. No two ways about it. The C runtime doesn?t protect against that programming error. The OpenSSL contract says things like ?foo points to a {some type of object}? Except for the free routines ? which was new in 1.1 ? it says nothing about NULL. (i2d/d2i also excepted) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Fri Nov 29 18:06:46 2019 From: rsalz at akamai.com (Salz, Rich) Date: Fri, 29 Nov 2019 18:06:46 +0000 Subject: Check NULL pointers or not... In-Reply-To: <5aab7fcb-a980-1283-6542-1d07b9add22f@openssl.org> References: <87r21ty7z2.wl-levitte@openssl.org> <20191128191932.GT32847@mit.edu> <5E496DB2-90CA-4490-85E1-6267D20FA4E8@oracle.com> <5aab7fcb-a980-1283-6542-1d07b9add22f@openssl.org> Message-ID: <8002DFE6-1AB8-4193-BDBA-AFC302742E31@akamai.com> if (!ossl_assert(ptr != NULL)) { ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER); return 0; } If the project decides to do this, I would like a configure option to ONLY turn on ossl_assert so that previous behavior can be restored without debug code penalties.