From matt at openssl.org Mon Sep 3 16:54:52 2018 From: matt at openssl.org (Matt Caswell) Date: Mon, 3 Sep 2018 17:54:52 +0100 Subject: [openssl-project] Current status of our release criteria Message-ID: <2af40e6d-ee72-d2cc-8f99-233133889c2e@openssl.org> We are currently 1 week away from release, so I've assessed the release criteria below. TL;DR summary: Mostly we are green but we have 9 outstanding PRs to get closed. There are specific questions/actions for the following people below: @levitte, @paulidale, @t-j-h, @kroeckx All of OMC (for #7014) All of the following release criteria are either met, or are currently "green": - Clean builds in Travis and Appveyor for two days - run-checker.sh to be showing as clean 2 days before release - No open Coverity issues (not flagged as "False Positive" or "Ignore") - TLSv1.3 RFC published (with at least one beta release after the publicaction) The only criterion not currently met is this one: - All open github issues/PRs older than 2 weeks at the time of release to be assessed for relevance to 1.1.1. Any flagged with the 1.1.1 milestone to be closed I have been through all new Issues/PRs raised that had not been assessed for relevance to 1.1.1 and have done that assessment. There are currently 9 outstanding PRs that are considered release blockers. 2 of the 9 PRs have the "ready" label: #7028 (@levitte) and #6972 (@paulidale) 3 are "active" having been raised in the last 24 hours: #7095, #7097, #7099 The remaining 4 are all awaiting sign off: #6757: Add support for setting SM2 id field Hold label has been applied and there seems to be confusion in the specs about whether UIDs are required and whether a default UID should be applied. I'm worried about this one. @t-j-h has been looking into this. #6944: Ignore EPIPE when sending NewSessionTickets in TLSv1.3 Updates applied following review. Awaiting further feedback from @t-j-h #7058: Process handshake messages after we've send a shutdown Awaiting updates from @kroeckx...Kurt will you able to do that soon? Or alternatively (if you prefer) I can take this one over. #7073: Support EdDSA in apps/speed Awaiting review...anyone? There are also 6 release blocking issues. Of those 6 issues, 5 of them are fixed by one of the 9 PRs. There is 1 with no associated PR: #7014: TLSv1.2 SNI hostname works in 1.1.0h, not in 1.1.1 master (as of 18 Ben has asked for input from the OMC on this one Matt From kurt at roeckx.be Mon Sep 3 21:06:22 2018 From: kurt at roeckx.be (Kurt Roeckx) Date: Mon, 3 Sep 2018 23:06:22 +0200 Subject: [openssl-project] Current status of our release criteria In-Reply-To: <2af40e6d-ee72-d2cc-8f99-233133889c2e@openssl.org> References: <2af40e6d-ee72-d2cc-8f99-233133889c2e@openssl.org> Message-ID: <20180903210621.GA19002@roeckx.be> On Mon, Sep 03, 2018 at 05:54:52PM +0100, Matt Caswell wrote: > > #7058: Process handshake messages after we've send a shutdown > > Awaiting updates from @kroeckx...Kurt will you able to do that soon? Or > alternatively (if you prefer) I can take this one over. I was waiting for your feedback, but feel free to take it over. I'm happy to review such changes. From kurt at roeckx.be Mon Sep 3 21:42:19 2018 From: kurt at roeckx.be (Kurt Roeckx) Date: Mon, 3 Sep 2018 23:42:19 +0200 Subject: [openssl-project] Current status of our release criteria In-Reply-To: <2af40e6d-ee72-d2cc-8f99-233133889c2e@openssl.org> References: <2af40e6d-ee72-d2cc-8f99-233133889c2e@openssl.org> Message-ID: <20180903214219.GB19002@roeckx.be> On Mon, Sep 03, 2018 at 05:54:52PM +0100, Matt Caswell wrote: > > #7014: TLSv1.2 SNI hostname works in 1.1.0h, not in 1.1.1 master (as of 18 > > Ben has asked for input from the OMC on this one So SSL_get_servername() was not documented in 1.1.0, but did exist in it. It's currently documented as: SSL_get_servername() returns a servername extension value of the specified type if provided in the Client Hello or NULL. It's clearly a function intended to be used to select which certificate should be used, during the negotation. But it seems that someone uses it after the negotation, or the SNI callback, and now gets NULL in case SNI was sent but not used. It at least looks like a misuse of the API, but the documentation does not say when you can call this function, unlike the SSL_CTX_set_client_hello_cb() documentation. Kurt From matt at openssl.org Tue Sep 4 15:39:00 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 4 Sep 2018 16:39:00 +0100 Subject: [openssl-project] Monthly Status Report (August) Message-ID: <98ea46a5-54a2-9cf2-6440-afa9649b7d25@openssl.org> As well as normal reviews, responding to user queries, wiki user requests, OMC business, handling security reports, etc., key activities this month: - Attended a number of conference calls related to FIPS - Attended the week long FIPS summit in Brisbane. A lot was achieved and write ups of the various discussions that were held will appear over the coming weeks and months. - Various activities trying to shepherd the 1.1.1 release towards a conclusion - Produced a PR to revert the TLSv1.2 to TLSv1.3 PSK reuse. Ultimately it was decided to retain the reuse and the PR was closed without merge. - Performed the 1.1.0i and 1.0.2p releases - Fixed an issue where under certain error conditions a call to SSLfatal could be missed - Resolved some issues with TLSv1.3 alerts after early_data - Fixed compilation with no-comp - Implemented some improvements to the TLSv1.3 fallback protection - Implemented some improvements for the usability of the ca app using EdDSA - Fixed some documentation that incorrectly stated we used md5 as a default in the req app - Created a PR to a fix problems in a scenario with a client that only writes and a server that only reads - Implemented updates required for the final TLSv1.3 version (RFC8446) - Changed post-handshake auth so that it is opt-in only - Attended the OSTIF advisory committee meeting - Fixed an issue in certain circumstances where a fallback/downgrade was incorrectly being reported - Performed the 1.1.1-pre9 release - Made some clarifications to the EVP_DigestSign*/EVP_DigestVerify* docs - Fixed some mem leaks on error paths Matt From matt at openssl.org Tue Sep 4 16:11:41 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 4 Sep 2018 17:11:41 +0100 Subject: [openssl-project] Release Criteria Update Message-ID: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> Current status of the 1.1.1 PRs/issues: There are currently 6 open PRs for 1.1.1. However in 2 cases there are 2 alternative implementations for the same thing - so really there are only 4 issues being addressed. One of these is in the "ready" state. The remaining 3 are: #7114 Process KeyUpdate and NewSessionTicket messages after a close_notify (an alternative to the older PR, #7058) Awaiting review Owner: Matt #7113 An alternative to address the SM2 ID issues (an alternative to the older PR, #6757) Currently being reviewed Owner: Paul Yang #7073 Support EdDSA in apps/speed Awaiting updates following review comments Owner: Paul Yang There are 2 open issues for 1.1.1. One of these is being addressed by PR#7073 above. The other one is: #7014 TLSv1.2 SNI hostname works in 1.1.0h, not in 1.1.1 master (as of 18-Aug) This one seems stuck!! No clear way forward as yet. Ben - any views? Matt From kaduk at mit.edu Tue Sep 4 16:34:19 2018 From: kaduk at mit.edu (Benjamin Kaduk) Date: Tue, 4 Sep 2018 11:34:19 -0500 Subject: [openssl-project] Release Criteria Update In-Reply-To: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> References: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> Message-ID: <20180904163419.GH91593@kduck.kaduk.org> On Tue, Sep 04, 2018 at 05:11:41PM +0100, Matt Caswell wrote: > There are 2 open issues for 1.1.1. One of these is being addressed by > PR#7073 above. The other one is: > > #7014 TLSv1.2 SNI hostname works in 1.1.0h, not in 1.1.1 master (as of > 18-Aug) > > This one seems stuck!! No clear way forward as yet. > > Ben - any views? I'm thinking that the ABI stability argument is going to win me over and we should continue to return the client's offered SNI in all cases until 1.2.0. Hoping to get a patch out this morning (US pacific) -- yesterday was a national holiday. -Ben From matt at openssl.org Wed Sep 5 22:59:34 2018 From: matt at openssl.org (Matt Caswell) Date: Wed, 5 Sep 2018 23:59:34 +0100 Subject: [openssl-project] Release Criteria Update In-Reply-To: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> References: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> Message-ID: <16a4a124-8b67-2016-9b2f-caa32cb2a27b@openssl.org> Today's update is that we still have 6 open PRs for 1.1.1. 5 of these are the same as yesterday. The 1 that was marked as "ready" yesterday has now been merged, and a new PR addressing issue #7014 has been opened. There are still 2 open issues for 1.1.1 but both of these are now being addressed by one of the open PRs. That means there are still 4 "critical path" PRs open: #7115 Restore historical SSL_get_servername() behavior Updates made following earlier review. Ready for another round of reviews?? Owner: Ben. #7114 Process KeyUpdate and NewSessionTicket messages after a close_notify (an alternative to the older PR, #7058) Currently in review. Awaiting some updates following review feedback. Owner: Matt. #7113 An alternative to address the SM2 ID issues (an alternative to the older PR, #6757) Updates made following earlier review. Awaiting another round of reviews. Owner: Paul Yang #7073 Support EdDSA in apps/speed Updates made following earlier review. Awaiting another round of reviews. Owner: Paul Yang Matt On 04/09/18 17:11, Matt Caswell wrote: > Current status of the 1.1.1 PRs/issues: > > There are currently 6 open PRs for 1.1.1. However in 2 cases there are 2 > alternative implementations for the same thing - so really there are > only 4 issues being addressed. One of these is in the "ready" state. > > The remaining 3 are: > > #7114 Process KeyUpdate and NewSessionTicket messages after a close_notify > (an alternative to the older PR, #7058) > > Awaiting review > Owner: Matt > > #7113 An alternative to address the SM2 ID issues > (an alternative to the older PR, #6757) > > Currently being reviewed > Owner: Paul Yang > > #7073 Support EdDSA in apps/speed > > Awaiting updates following review comments > Owner: Paul Yang > > > There are 2 open issues for 1.1.1. One of these is being addressed by > PR#7073 above. The other one is: > > #7014 TLSv1.2 SNI hostname works in 1.1.0h, not in 1.1.1 master (as of > 18-Aug) > > This one seems stuck!! No clear way forward as yet. > > Ben - any views? > > > Matt > From kaduk at mit.edu Wed Sep 5 23:04:08 2018 From: kaduk at mit.edu (Benjamin Kaduk) Date: Wed, 5 Sep 2018 18:04:08 -0500 Subject: [openssl-project] Release Criteria Update In-Reply-To: <16a4a124-8b67-2016-9b2f-caa32cb2a27b@openssl.org> References: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> <16a4a124-8b67-2016-9b2f-caa32cb2a27b@openssl.org> Message-ID: <20180905230408.GJ73164@kduck.kaduk.org> On Wed, Sep 05, 2018 at 11:59:34PM +0100, Matt Caswell wrote: > Today's update is that we still have 6 open PRs for 1.1.1. 5 of these > are the same as yesterday. The 1 that was marked as "ready" yesterday > has now been merged, and a new PR addressing issue #7014 has been opened. > > There are still 2 open issues for 1.1.1 but both of these are now being > addressed by one of the open PRs. > > That means there are still 4 "critical path" PRs open: > > #7115 Restore historical SSL_get_servername() behavior > > Updates made following earlier review. Ready for another round of reviews?? > Owner: Ben. I believe it's ready for another round of reviews, yes. Do we think we want to wait for confirmation from @MSP-Greg? -Ben From tjh at cryptsoft.com Thu Sep 6 00:54:06 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Thu, 6 Sep 2018 10:54:06 +1000 Subject: [openssl-project] Release Criteria Update In-Reply-To: <16a4a124-8b67-2016-9b2f-caa32cb2a27b@openssl.org> References: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> <16a4a124-8b67-2016-9b2f-caa32cb2a27b@openssl.org> Message-ID: On Thu, Sep 6, 2018 at 8:59 AM, Matt Caswell wrote: > #7113 An alternative to address the SM2 ID issues > (an alternative to the older PR, #6757) > > Updates made following earlier review. Awaiting another round of reviews. > Owner: Paul Yang All the previous comments have been addressed. I noted two missing SM2err calls on malloc failure and a typo in SM2.pod. I've approved it conditional on those being fixed. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kaduk at mit.edu Thu Sep 6 16:07:55 2018 From: kaduk at mit.edu (Benjamin Kaduk) Date: Thu, 6 Sep 2018 11:07:55 -0500 Subject: [openssl-project] Release Criteria Update In-Reply-To: <20180905230408.GJ73164@kduck.kaduk.org> References: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> <16a4a124-8b67-2016-9b2f-caa32cb2a27b@openssl.org> <20180905230408.GJ73164@kduck.kaduk.org> Message-ID: <20180906160755.GN73164@kduck.kaduk.org> On Wed, Sep 05, 2018 at 06:04:08PM -0500, Benjamin Kaduk wrote: > On Wed, Sep 05, 2018 at 11:59:34PM +0100, Matt Caswell wrote: > > Today's update is that we still have 6 open PRs for 1.1.1. 5 of these > > are the same as yesterday. The 1 that was marked as "ready" yesterday > > has now been merged, and a new PR addressing issue #7014 has been opened. > > > > There are still 2 open issues for 1.1.1 but both of these are now being > > addressed by one of the open PRs. > > > > That means there are still 4 "critical path" PRs open: > > > > #7115 Restore historical SSL_get_servername() behavior > > > > Updates made following earlier review. Ready for another round of reviews?? > > Owner: Ben. > > I believe it's ready for another round of reviews, yes. > Do we think we want to wait for confirmation from @MSP-Greg? I see that Matt has marked this one as Ready. I'm going to be "on a plane" (not exactly, but effectively so) for the next 9-ish hours and am not confident that I'll be able to merge it until tomorrow. I also see that the original reporter is still not having success; is anyone in a position to try to set up those Ruby EventMachine tests (it's unclear if it needs to be on windows or not)? -Ben From kurt at roeckx.be Thu Sep 6 16:32:05 2018 From: kurt at roeckx.be (Kurt Roeckx) Date: Thu, 6 Sep 2018 18:32:05 +0200 Subject: [openssl-project] Release Criteria Update In-Reply-To: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> References: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> Message-ID: <20180906163205.GA18943@roeckx.be> On Tue, Sep 04, 2018 at 05:11:41PM +0100, Matt Caswell wrote: > Current status of the 1.1.1 PRs/issues: Since we did make a lot of changes, including things that applications can run into, would it make sense to have an other beta release? Kurt From matt at openssl.org Thu Sep 6 22:25:13 2018 From: matt at openssl.org (Matt Caswell) Date: Thu, 6 Sep 2018 23:25:13 +0100 Subject: [openssl-project] Release Criteria Update In-Reply-To: <20180906163205.GA18943@roeckx.be> References: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> <20180906163205.GA18943@roeckx.be> Message-ID: <72eb3d2a-6748-48e1-a531-98a12075c85a@openssl.org> On 06/09/18 17:32, Kurt Roeckx wrote: > On Tue, Sep 04, 2018 at 05:11:41PM +0100, Matt Caswell wrote: >> Current status of the 1.1.1 PRs/issues: > > Since we did make a lot of changes, including things that > applications can run into, would it make sense to have an other > beta release? I'm not keen on that. What do others think? Matt From tjh at cryptsoft.com Thu Sep 6 22:28:59 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Fri, 7 Sep 2018 08:28:59 +1000 Subject: [openssl-project] Release Criteria Update In-Reply-To: <72eb3d2a-6748-48e1-a531-98a12075c85a@openssl.org> References: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> <20180906163205.GA18943@roeckx.be> <72eb3d2a-6748-48e1-a531-98a12075c85a@openssl.org> Message-ID: We need to get this release out and available - there are a lot of people waiting on the "production"release - and who won't go forward on a beta (simple fact of life there). I don't see the outstanding items as release blockers - and they will be wrapped up in time. Having the release date as a drive I think helps for a lot of focus - and more stuff has gone into 1.1.1 that we originally anticipated because we held it open waiting on TLSv1.3 finalisation. So a +1 for keeping to the release date. Tim. On Fri, Sep 7, 2018 at 8:25 AM, Matt Caswell wrote: > > > On 06/09/18 17:32, Kurt Roeckx wrote: > > On Tue, Sep 04, 2018 at 05:11:41PM +0100, Matt Caswell wrote: > >> Current status of the 1.1.1 PRs/issues: > > > > Since we did make a lot of changes, including things that > > applications can run into, would it make sense to have an other > > beta release? > > I'm not keen on that. What do others think? > > Matt > > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Thu Sep 6 22:33:20 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Thu, 6 Sep 2018 18:33:20 -0400 Subject: [openssl-project] Release Criteria Update In-Reply-To: <72eb3d2a-6748-48e1-a531-98a12075c85a@openssl.org> References: <6fc1338c-1e2b-3984-5272-36ad5a37034d@openssl.org> <20180906163205.GA18943@roeckx.be> <72eb3d2a-6748-48e1-a531-98a12075c85a@openssl.org> Message-ID: <9AEC5997-5565-4ED4-9076-F10294594043@dukhovni.org> > On Sep 6, 2018, at 6:25 PM, Matt Caswell wrote: > > I'm not keen on that. What do others think? No objections to issuing a release. We're unlikely to have to change the API/ABI or feature set based on further beta feedback. Any late bugs can be fixed in 1.1.1a, and unless they trigger CVEs, there's no compelling reason to wait. Barring specific concerns, I am not opposed to release as planned. -- Viktor. From matt at openssl.org Thu Sep 6 22:41:59 2018 From: matt at openssl.org (Matt Caswell) Date: Thu, 6 Sep 2018 23:41:59 +0100 Subject: [openssl-project] Release Criteria Update Message-ID: We currently have 8 1.1.1 PRs that are open. 3 of which are in the "ready" state. There are 2 which are alternative implementations of the same thing - so there are really on 4 issues currently being addressed: #7145 SipHash: add separate setter for the hash size Owner: Richard Awaiting review (CIs are failing) #7141 Ensure certificate callbacks work correctly in TLSv1.3 Owner: Matt Trivial change. Awaiting review #7139 Remove a reference to SSL_force_post_handshake_auth() Owner: Matt Trivial change. Awaiting review #7114 Process KeyUpdate and NewSessionTicket messages after a close_notify Alternative implementation for #7058 Owner: Matt Awaiting review. Anyone? There 5 1.1.1 issues open - 3 of which should be solved by outstanding PRS. The remaining 2 are: #7014 TLSv1.2 SNI hostname works in 1.1.0h, not in 1.1.1 master (as of 18-Aug) We thought we had a fix for this, but the PR in question does not seem to have solved the OPs issue #7133 X509_sign SIGSEGVs with NULL private key Should be an easy fix Matt From tjh at cryptsoft.com Thu Sep 6 22:50:46 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Fri, 7 Sep 2018 08:50:46 +1000 Subject: [openssl-project] Release Criteria Update In-Reply-To: References: Message-ID: All PRs except #7145 now reviewed and marked ready. Tim On Fri, Sep 7, 2018 at 8:41 AM, Matt Caswell wrote: > We currently have 8 1.1.1 PRs that are open. 3 of which are in the > "ready" state. There are 2 which are alternative implementations of the > same thing - so there are really on 4 issues currently being addressed: > > #7145 SipHash: add separate setter for the hash size > > Owner: Richard > Awaiting review (CIs are failing) > > > #7141 Ensure certificate callbacks work correctly in TLSv1.3 > > Owner: Matt > Trivial change. Awaiting review > > > #7139 Remove a reference to SSL_force_post_handshake_auth() > > Owner: Matt > Trivial change. Awaiting review > > > #7114 Process KeyUpdate and NewSessionTicket messages after a close_notify > Alternative implementation for #7058 > > Owner: Matt > Awaiting review. Anyone? > > > There 5 1.1.1 issues open - 3 of which should be solved by outstanding > PRS. The remaining 2 are: > > > #7014 TLSv1.2 SNI hostname works in 1.1.0h, not in 1.1.1 master (as of > 18-Aug) > > We thought we had a fix for this, but the PR in question does not seem > to have solved the OPs issue > > > #7133 X509_sign SIGSEGVs with NULL private key > > Should be an easy fix > > > Matt > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.dale at oracle.com Thu Sep 6 23:08:07 2018 From: paul.dale at oracle.com (Paul Dale) Date: Thu, 6 Sep 2018 16:08:07 -0700 (PDT) Subject: [openssl-project] Release Criteria Update In-Reply-To: References: Message-ID: PR for 7133 submitted. ? ? Pauli -- Oracle Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia ? From: Tim Hudson [mailto:tjh at cryptsoft.com] Sent: Friday, 7 September 2018 8:51 AM To: openssl-project at openssl.org Subject: Re: [openssl-project] Release Criteria Update ? All PRs except?#7145 now reviewed and marked ready. ? Tim? ? On Fri, Sep 7, 2018 at 8:41 AM, Matt Caswell wrote: We currently have 8 1.1.1 PRs that are open. 3 of which are in the "ready" state. There are 2 which are alternative implementations of the same thing - so there are really on 4 issues currently being addressed: #7145 SipHash: add separate setter for the hash size Owner: Richard Awaiting review (CIs are failing) #7141 Ensure certificate callbacks work correctly in TLSv1.3 Owner: Matt Trivial change. Awaiting review #7139 Remove a reference to SSL_force_post_handshake_auth() Owner: Matt Trivial change. Awaiting review #7114 Process KeyUpdate and NewSessionTicket messages after a close_notify Alternative implementation for #7058 Owner: Matt Awaiting review. Anyone? There 5 1.1.1 issues open - 3 of which should be solved by outstanding PRS. The remaining 2 are: #7014 TLSv1.2 SNI hostname works in 1.1.0h, not in 1.1.1 master (as of 18-Aug) We thought we had a fix for this, but the PR in question does not seem to have solved the OPs issue #7133 X509_sign SIGSEGVs with NULL private key Should be an easy fix Matt _______________________________________________ openssl-project mailing list HYPERLINK "mailto:openssl-project at openssl.org"openssl-project at openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Fri Sep 7 00:51:52 2018 From: levitte at openssl.org (Richard Levitte) Date: Fri, 07 Sep 2018 02:51:52 +0200 (CEST) Subject: [openssl-project] Release Criteria Update In-Reply-To: References: Message-ID: <20180907.025152.1131079938025695690.levitte@openssl.org> I think this one should be part of the lot as well: #7144 ASN.1 DER: Make INT32 / INT64 types read badly encoded LONG zeroes For example, *all* two-prime RSA keys from pre-1.1.1 become unreadable in 1.1.1, because pre-1.1.1 encodes the version indicator (zero) as 02 00 (zero length INTEGER, which is invalid) instead of 02 01 00 (proper zero). That's simply because the internal version number was changed from a LONG (custom ASN.1 type, mapping to a C long) to a INT32 (new custom ASN.1 type, mapping to a C int32). (no, we don't want to go back to using LONG) Cheers, Richard In message on Thu, 6 Sep 2018 23:41:59 +0100, Matt Caswell said: > We currently have 8 1.1.1 PRs that are open. 3 of which are in the > "ready" state. There are 2 which are alternative implementations of the > same thing - so there are really on 4 issues currently being addressed: > > #7145 SipHash: add separate setter for the hash size > > Owner: Richard > Awaiting review (CIs are failing) > > > #7141 Ensure certificate callbacks work correctly in TLSv1.3 > > Owner: Matt > Trivial change. Awaiting review > > > #7139 Remove a reference to SSL_force_post_handshake_auth() > > Owner: Matt > Trivial change. Awaiting review > > > #7114 Process KeyUpdate and NewSessionTicket messages after a close_notify > Alternative implementation for #7058 > > Owner: Matt > Awaiting review. Anyone? > > > There 5 1.1.1 issues open - 3 of which should be solved by outstanding > PRS. The remaining 2 are: > > > #7014 TLSv1.2 SNI hostname works in 1.1.0h, not in 1.1.1 master (as of > 18-Aug) > > We thought we had a fix for this, but the PR in question does not seem > to have solved the OPs issue > > > #7133 X509_sign SIGSEGVs with NULL private key > > Should be an easy fix > > > Matt > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > From levitte at openssl.org Fri Sep 7 00:56:59 2018 From: levitte at openssl.org (Richard Levitte) Date: Fri, 07 Sep 2018 02:56:59 +0200 (CEST) Subject: [openssl-project] Release Criteria Update In-Reply-To: <20180907.025152.1131079938025695690.levitte@openssl.org> References: <20180907.025152.1131079938025695690.levitte@openssl.org> Message-ID: <20180907.025659.1202615710975962410.levitte@openssl.org> In message <20180907.025152.1131079938025695690.levitte at openssl.org> on Fri, 07 Sep 2018 02:51:52 +0200 (CEST), Richard Levitte said: > For example, *all* two-prime RSA keys from pre-1.1.1 become unreadable That was a bit of an over-statement... but it seems that there are things in the wild that were accepted in 1.1.0 (because LONG is used) that aren't accepted in 1.1.1. A regression still, even though with less drama. -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Fri Sep 7 08:56:01 2018 From: matt at openssl.org (Matt Caswell) Date: Fri, 7 Sep 2018 09:56:01 +0100 Subject: [openssl-project] Release Criteria Update In-Reply-To: <20180907.025152.1131079938025695690.levitte@openssl.org> References: <20180907.025152.1131079938025695690.levitte@openssl.org> Message-ID: On 07/09/18 01:51, Richard Levitte wrote: > I think this one should be part of the lot as well: > > #7144 > ASN.1 DER: Make INT32 / INT64 types read badly encoded LONG zeroes > > For example, *all* two-prime RSA keys from pre-1.1.1 become unreadable > in 1.1.1, because pre-1.1.1 encodes the version indicator (zero) as > 02 00 (zero length INTEGER, which is invalid) instead of 02 01 00 > (proper zero). That's simply because the internal version number was > changed from a LONG (custom ASN.1 type, mapping to a C long) to a INT32 > (new custom ASN.1 type, mapping to a C int32). > (no, we don't want to go back to using LONG) So...that PR seems to be labelled for 1.1.0 too? So why is the problem specific to 1.1.1? Matt > > Cheers, > Richard > > In message on Thu, 6 Sep 2018 23:41:59 +0100, Matt Caswell said: > >> We currently have 8 1.1.1 PRs that are open. 3 of which are in the >> "ready" state. There are 2 which are alternative implementations of the >> same thing - so there are really on 4 issues currently being addressed: >> >> #7145 SipHash: add separate setter for the hash size >> >> Owner: Richard >> Awaiting review (CIs are failing) >> >> >> #7141 Ensure certificate callbacks work correctly in TLSv1.3 >> >> Owner: Matt >> Trivial change. Awaiting review >> >> >> #7139 Remove a reference to SSL_force_post_handshake_auth() >> >> Owner: Matt >> Trivial change. Awaiting review >> >> >> #7114 Process KeyUpdate and NewSessionTicket messages after a close_notify >> Alternative implementation for #7058 >> >> Owner: Matt >> Awaiting review. Anyone? >> >> >> There 5 1.1.1 issues open - 3 of which should be solved by outstanding >> PRS. The remaining 2 are: >> >> >> #7014 TLSv1.2 SNI hostname works in 1.1.0h, not in 1.1.1 master (as of >> 18-Aug) >> >> We thought we had a fix for this, but the PR in question does not seem >> to have solved the OPs issue >> >> >> #7133 X509_sign SIGSEGVs with NULL private key >> >> Should be an easy fix >> >> >> Matt >> _______________________________________________ >> openssl-project mailing list >> openssl-project at openssl.org >> https://mta.openssl.org/mailman/listinfo/openssl-project >> > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > From levitte at openssl.org Fri Sep 7 09:09:43 2018 From: levitte at openssl.org (Richard Levitte) Date: Fri, 07 Sep 2018 11:09:43 +0200 (CEST) Subject: [openssl-project] Release Criteria Update In-Reply-To: References: <20180907.025152.1131079938025695690.levitte@openssl.org> Message-ID: <20180907.110943.896486182899407748.levitte@openssl.org> In message on Fri, 7 Sep 2018 09:56:01 +0100, Matt Caswell said: > > > On 07/09/18 01:51, Richard Levitte wrote: > > I think this one should be part of the lot as well: > > > > #7144 > > ASN.1 DER: Make INT32 / INT64 types read badly encoded LONG zeroes > > > > For example, *all* two-prime RSA keys from pre-1.1.1 become unreadable > > in 1.1.1, because pre-1.1.1 encodes the version indicator (zero) as > > 02 00 (zero length INTEGER, which is invalid) instead of 02 01 00 > > (proper zero). That's simply because the internal version number was > > changed from a LONG (custom ASN.1 type, mapping to a C long) to a INT32 > > (new custom ASN.1 type, mapping to a C int32). > > (no, we don't want to go back to using LONG) > > So...that PR seems to be labelled for 1.1.0 too? So why is the problem > specific to 1.1.1? Because of commit 6a32a3c058dbd9fa7cec5b020e4f027808972e4a, which is only present in master. In that commit, we switch a number of uses of LONGs (all the remaining) to INT32. Of course, one way would be to revert that commit, but that doesn't fix the actual issue with INT32 not reading in a backward compatible way (that issue exists in 1.1.0 as well). So yeah, in summary, it's a regression that exists only in 1.1.1, but is really caused by a bug that exists in 1.1.0 as well. I hope that's a good enough explanation. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Fri Sep 7 09:17:11 2018 From: matt at openssl.org (Matt Caswell) Date: Fri, 7 Sep 2018 10:17:11 +0100 Subject: [openssl-project] Release Criteria Update In-Reply-To: <20180907.110943.896486182899407748.levitte@openssl.org> References: <20180907.025152.1131079938025695690.levitte@openssl.org> <20180907.110943.896486182899407748.levitte@openssl.org> Message-ID: <371e26f9-7c60-9dc9-3af5-5f9c5ea7ae68@openssl.org> On 07/09/18 10:09, Richard Levitte wrote: > In message on Fri, 7 Sep 2018 09:56:01 +0100, Matt Caswell said: > >> >> >> On 07/09/18 01:51, Richard Levitte wrote: >>> I think this one should be part of the lot as well: >>> >>> #7144 >>> ASN.1 DER: Make INT32 / INT64 types read badly encoded LONG zeroes >>> >>> For example, *all* two-prime RSA keys from pre-1.1.1 become unreadable >>> in 1.1.1, because pre-1.1.1 encodes the version indicator (zero) as >>> 02 00 (zero length INTEGER, which is invalid) instead of 02 01 00 >>> (proper zero). That's simply because the internal version number was >>> changed from a LONG (custom ASN.1 type, mapping to a C long) to a INT32 >>> (new custom ASN.1 type, mapping to a C int32). >>> (no, we don't want to go back to using LONG) >> >> So...that PR seems to be labelled for 1.1.0 too? So why is the problem >> specific to 1.1.1? > > Because of commit 6a32a3c058dbd9fa7cec5b020e4f027808972e4a, which is > only present in master. In that commit, we switch a number of uses of > LONGs (all the remaining) to INT32. > > Of course, one way would be to revert that commit, but that doesn't > fix the actual issue with INT32 not reading in a backward compatible > way (that issue exists in 1.1.0 as well). > > So yeah, in summary, it's a regression that exists only in 1.1.1, but > is really caused by a bug that exists in 1.1.0 as well. > > I hope that's a good enough explanation. Makes sense. Matt From matt at openssl.org Fri Sep 7 22:25:27 2018 From: matt at openssl.org (Matt Caswell) Date: Fri, 7 Sep 2018 23:25:27 +0100 Subject: [openssl-project] Release Criteria Update Message-ID: <9592039b-c5d1-f32e-cf18-ec2f49824d2b@openssl.org> We have 2 outstanding 1.1.1 PRs. These are: #7144 ASN.1 DER: Make INT32 / INT64 types read badly encoded LONG zeroes Owner: Richard Awaiting updates following review feedback #7145 SipHash: add separate setter for the hash size Owner: Richard Awaiting updates following review feedback There are also 2 outstanding 1.1.1 issues. Both of those are addressed by the above PRs. Strictly speaking we are now "green" for the open issues/PR part of the release criteria. The above two PRs/issues will be less than 2 weeks old at the time of release: "All open github issues/PRs older than 2 weeks at the time of release to be assessed for relevance to 1.1.1. Any flagged with the 1.1.1 milestone to be closed (see below)" We should still aim to close them though if at all possible. However, travis has gone red. This looks like an environmental issue since we're getting lots of lines like this all of a sudden in the pyca external tests: /home/travis/build/openssl/openssl/pyca-cryptography/tests/conftest.py:22: RemovedInPytest4Warning: MarkInfo objects are deprecated as they contain merged marks which are hard to deal with correctly. Please use node.get_closest_marker(name) or node.iter_markers(name). Docs: https://docs.pytest.org/en/latest/mark.html#updating-code for mark in request.node.get_marker("requires_backend_interface") Until eventually we get this: "The log length has exceeded the limit of 4 MB (this usually means that the test suite is raising the same exception over and over). The job has been terminated" Ideally we should see what we can do to fix this. I don't think it should hold up our release though. I intend to freeze the repo 48 hours before the release (i.e. on Sunday) to ensure stability of the build. Matt From matt at openssl.org Sun Sep 9 10:34:18 2018 From: matt at openssl.org (Matt Caswell) Date: Sun, 9 Sep 2018 11:34:18 +0100 Subject: [openssl-project] Please freeze the repo Message-ID: <22962ad7-6232-dcd7-4ec4-11544360fddd@openssl.org> Please can someone freeze the repo: ssh openssl-git at git.openssl.org freeze openssl matt Thanks Matt From tjh at cryptsoft.com Sun Sep 9 10:34:57 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Sun, 9 Sep 2018 20:34:57 +1000 Subject: [openssl-project] Please freeze the repo In-Reply-To: <22962ad7-6232-dcd7-4ec4-11544360fddd@openssl.org> References: <22962ad7-6232-dcd7-4ec4-11544360fddd@openssl.org> Message-ID: Done. Tim. On Sun, Sep 9, 2018 at 8:34 PM, Matt Caswell wrote: > Please can someone freeze the repo: > > ssh openssl-git at git.openssl.org freeze openssl matt > > > Thanks > > Matt > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Sun Sep 9 10:35:15 2018 From: levitte at openssl.org (Richard Levitte) Date: Sun, 09 Sep 2018 12:35:15 +0200 (CEST) Subject: [openssl-project] Please freeze the repo In-Reply-To: <22962ad7-6232-dcd7-4ec4-11544360fddd@openssl.org> References: <22962ad7-6232-dcd7-4ec4-11544360fddd@openssl.org> Message-ID: <20180909.123515.1119130387233727043.levitte@openssl.org> In message <22962ad7-6232-dcd7-4ec4-11544360fddd at openssl.org> on Sun, 9 Sep 2018 11:34:18 +0100, Matt Caswell said: > Please can someone freeze the repo: > > ssh openssl-git at git.openssl.org freeze openssl matt Done -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From kaduk at mit.edu Sun Sep 9 16:04:16 2018 From: kaduk at mit.edu (Benjamin Kaduk) Date: Sun, 9 Sep 2018 11:04:16 -0500 Subject: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) In-Reply-To: <5b94e7ab7da92_4ed72b08291e2f60744eb@node1.mail> References: <5b94e7ab7da92_4ed72b08291e2f60744eb@node1.mail> Message-ID: <20180909160416.GV73164@kduck.kaduk.org> I see that Matthias has opened pull requests for a couple of these already; are you planning to work through the rest of them as well? -Ben On Sun, Sep 09, 2018 at 09:28:12AM +0000, scan-admin at coverity.com wrote: > Hi, > > Please find the latest report on new defect(s) introduced to openssl/openssl found with Coverity Scan. > > 6 new defect(s) introduced to openssl/openssl found with Coverity Scan. > > > New defect(s) Reported-by: Coverity Scan > Showing 6 of 6 defect(s) > > > ** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > > > ________________________________________________________________________________________________________ > *** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > /crypto/rsa/rsa_pss.c: 247 in RSA_padding_add_PKCS1_PSS_mgf1() > 241 EM[emLen - 1] = 0xbc; > 242 > 243 ret = 1; > 244 > 245 err: > 246 EVP_MD_CTX_free(ctx); > >>> CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > >>> "sLen" is passed to a parameter that cannot be negative. > 247 OPENSSL_clear_free(salt, sLen); > 248 > 249 return ret; > 250 > 251 } > 252 > 253 #if defined(_MSC_VER) > 254 # pragma optimize("",on) > > ** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > > > ________________________________________________________________________________________________________ > *** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > /crypto/sm2/sm2_pmeth.c: 277 in pkey_sm2_digest_custom() > 271 } > 272 > 273 /* get hashed prefix 'z' of tbs message */ > 274 if (!sm2_compute_z_digest(z, md, smctx->id, smctx->id_len, ec)) > 275 return 0; > 276 > >>> CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > >>> "EVP_MD_size(md)" is passed to a parameter that cannot be negative. > 277 return EVP_DigestUpdate(mctx, z, EVP_MD_size(md)); > 278 } > 279 > 280 const EVP_PKEY_METHOD sm2_pkey_meth = { > 281 EVP_PKEY_SM2, > 282 0, > > ** CID 1439136: Resource leaks (RESOURCE_LEAK) > /test/dhtest.c: 202 in dh_test() > > > ________________________________________________________________________________________________________ > *** CID 1439136: Resource leaks (RESOURCE_LEAK) > /test/dhtest.c: 202 in dh_test() > 196 BN_free(bp); > 197 BN_free(bg); > 198 BN_free(cpriv_key); > 199 BN_GENCB_free(_cb); > 200 DH_free(dh); > 201 > >>> CID 1439136: Resource leaks (RESOURCE_LEAK) > >>> Variable "priv_key" going out of scope leaks the storage it points to. > 202 return ret; > 203 } > 204 > 205 static int cb(int p, int n, BN_GENCB *arg) > 206 { > 207 return 1; > > ** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > > > ________________________________________________________________________________________________________ > *** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > /apps/speed.c: 3105 in speed_main() > 3099 ERR_print_errors(bio_err); > 3100 rsa_count = 1; > 3101 } else { > 3102 for (i = 0; i < loopargs_len; i++) { > 3103 /* Perform EdDSA signature test */ > 3104 loopargs[i].siglen = test_ed_curves[testnum].siglen; > >>> CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > >>> Pointer "&loopargs[i].siglen" points to an object whose effective type is "unsigned int" (32 bits, unsigned) but is dereferenced as a wider "unsigned long" (64 bits, unsigned). This may lead to memory corruption. > 3105 st = EVP_DigestSign(loopargs[i].eddsa_ctx[testnum], > 3106 loopargs[i].buf2, (size_t *)&loopargs[i].siglen, > 3107 loopargs[i].buf, 20); > 3108 if (st == 0) > 3109 break; > 3110 } > > ** CID 1423323: Null pointer dereferences (FORWARD_NULL) > > > ________________________________________________________________________________________________________ > *** CID 1423323: Null pointer dereferences (FORWARD_NULL) > /test/evp_extra_test.c: 894 in test_EVP_PKEY_check() > 888 > 889 if (!TEST_int_eq(EVP_PKEY_param_check(ctx), expected_param_check)) > 890 goto done; > 891 > 892 ctx2 = EVP_PKEY_CTX_new_id(0xdefaced, NULL); > 893 /* assign the pkey directly, as an internal test */ > >>> CID 1423323: Null pointer dereferences (FORWARD_NULL) > >>> Passing null pointer "pkey" to "EVP_PKEY_up_ref", which dereferences it. > 894 EVP_PKEY_up_ref(pkey); > 895 ctx2->pkey = pkey; > 896 > 897 if (!TEST_int_eq(EVP_PKEY_check(ctx2), 0xbeef)) > 898 goto done; > 899 > > ** CID 1201571: Error handling issues (CHECKED_RETURN) > /crypto/pkcs12/p12_init.c: 25 in PKCS12_init() > > > ________________________________________________________________________________________________________ > *** CID 1201571: Error handling issues (CHECKED_RETURN) > /crypto/pkcs12/p12_init.c: 25 in PKCS12_init() > 19 PKCS12 *pkcs12; > 20 > 21 if ((pkcs12 = PKCS12_new()) == NULL) { > 22 PKCS12err(PKCS12_F_PKCS12_INIT, ERR_R_MALLOC_FAILURE); > 23 return NULL; > 24 } > >>> CID 1201571: Error handling issues (CHECKED_RETURN) > >>> Calling "ASN1_INTEGER_set" without checking return value (as is done elsewhere 30 out of 37 times). > 25 ASN1_INTEGER_set(pkcs12->version, 3); > 26 pkcs12->authsafes->type = OBJ_nid2obj(mode); > 27 switch (mode) { > 28 case NID_pkcs7_data: > 29 if ((pkcs12->authsafes->d.data = ASN1_OCTET_STRING_new()) == NULL) { > 30 PKCS12err(PKCS12_F_PKCS12_INIT, ERR_R_MALLOC_FAILURE); > > > ________________________________________________________________________________________________________ > To view the defects in Coverity Scan visit, https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F-2BfV0V05UPxvVjWch-2Bd2MGckcRakUl6QyjujEohY7rPpoYUE4H-2Fm-2BeoDOl8jw7bf4Z78hw-3D-3D_bpOft2V4l9NXEcTx5CnNFJqpP-2F8a09dz6vsuNilvAJgBy9hWgnGhTAFGZnkvhcJuSQocoiCV36Dw66FwvViDOF-2BGQbzbMH8LM1tsnputryXt7SEgZZ-2FmpoWsuVr91UzOFBmmlL0bipzCjL7WfoT7QvLLnFuGxTjboshY44ftCBEhW8TAZR-2B1c1y7JdbYkdSXw-2B7Vmts-2F-2BitkvIjISgebBlgXuThX1DnzutpYSf00XD0-3D > > To manage Coverity Scan email notifications for "kaduk-github at mit.edu", click https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F-2BfV0V05UPxvVjWch-2Bd2MGckcRbVDbis712qZDP-2FA8y06Nq414hC6p-2BsqBEViFMJYotwSt4SYNeSzd6tPCdCHgDzpHIBW-2Fr0I0sQJCop-2Fx5Lu2ueYFxYqLmFh7APZbTTED-2B53KXZ2qVo0Y2q2bUC-2BpL2TzE-3D_bpOft2V4l9NXEcTx5CnNFJqpP-2F8a09dz6vsuNilvAJgBy9hWgnGhTAFGZnkvhcJu7xxYKPr1HkiPh-2BL3MaUbhQMZae3MPjv9c6bU6U4uhOZEhiS1P-2BwpukQ4-2BcSzk5FouA75ij0odEEgZcWTB05BKimz0wg0Y8JsC1Izz20-2FpfRp2kjWD47vvs4NmxuDPkNqvS3qoLRQ0vIXW8CFF339G-2B7jGolZ214Wxo3Gh6Hc0HY-3D > From Matthias.St.Pierre at ncp-e.com Sun Sep 9 18:31:18 2018 From: Matthias.St.Pierre at ncp-e.com (Dr. Matthias St. Pierre) Date: Sun, 9 Sep 2018 18:31:18 +0000 Subject: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) In-Reply-To: <20180909160416.GV73164@kduck.kaduk.org> References: <5b94e7ab7da92_4ed72b08291e2f60744eb@node1.mail> <20180909160416.GV73164@kduck.kaduk.org> Message-ID: <513e30fed5f843ac90fc9d17e046352a@Ex13.ncp.local> I am currently occupied with other things, so I won't be able to look at it before later this evening or tomorrow. I also had a quick look at CID 1423323 (see below) but I was unable to see why 'pkey' would be a NULL pointer when passed to 'EVP_PKEY_up_ref'. So I'm not sure yet whether this is a false positive. Matthias > -----Urspr?ngliche Nachricht----- > Von: openssl-project Im Auftrag von Benjamin Kaduk > Gesendet: Sonntag, 9. September 2018 18:04 > An: openssl-project at openssl.org > Betreff: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) > > I see that Matthias has opened pull requests for a couple of these already; > are you planning to work through the rest of them as well? > > -Ben > > On Sun, Sep 09, 2018 at 09:28:12AM +0000, scan-admin at coverity.com wrote: > > Hi, > > > > Please find the latest report on new defect(s) introduced to openssl/openssl found with Coverity Scan. > > > > 6 new defect(s) introduced to openssl/openssl found with Coverity Scan. > > > > > > New defect(s) Reported-by: Coverity Scan > > Showing 6 of 6 defect(s) > > > > > > ** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > > > > > > ________________________________________________________________________________________________________ > > *** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > > /crypto/rsa/rsa_pss.c: 247 in RSA_padding_add_PKCS1_PSS_mgf1() > > 241 EM[emLen - 1] = 0xbc; > > 242 > > 243 ret = 1; > > 244 > > 245 err: > > 246 EVP_MD_CTX_free(ctx); > > >>> CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > > >>> "sLen" is passed to a parameter that cannot be negative. > > 247 OPENSSL_clear_free(salt, sLen); > > 248 > > 249 return ret; > > 250 > > 251 } > > 252 > > 253 #if defined(_MSC_VER) > > 254 # pragma optimize("",on) > > > > ** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > > > > > > ________________________________________________________________________________________________________ > > *** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > > /crypto/sm2/sm2_pmeth.c: 277 in pkey_sm2_digest_custom() > > 271 } > > 272 > > 273 /* get hashed prefix 'z' of tbs message */ > > 274 if (!sm2_compute_z_digest(z, md, smctx->id, smctx->id_len, ec)) > > 275 return 0; > > 276 > > >>> CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > > >>> "EVP_MD_size(md)" is passed to a parameter that cannot be negative. > > 277 return EVP_DigestUpdate(mctx, z, EVP_MD_size(md)); > > 278 } > > 279 > > 280 const EVP_PKEY_METHOD sm2_pkey_meth = { > > 281 EVP_PKEY_SM2, > > 282 0, > > > > ** CID 1439136: Resource leaks (RESOURCE_LEAK) > > /test/dhtest.c: 202 in dh_test() > > > > > > ________________________________________________________________________________________________________ > > *** CID 1439136: Resource leaks (RESOURCE_LEAK) > > /test/dhtest.c: 202 in dh_test() > > 196 BN_free(bp); > > 197 BN_free(bg); > > 198 BN_free(cpriv_key); > > 199 BN_GENCB_free(_cb); > > 200 DH_free(dh); > > 201 > > >>> CID 1439136: Resource leaks (RESOURCE_LEAK) > > >>> Variable "priv_key" going out of scope leaks the storage it points to. > > 202 return ret; > > 203 } > > 204 > > 205 static int cb(int p, int n, BN_GENCB *arg) > > 206 { > > 207 return 1; > > > > ** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > > > > > > ________________________________________________________________________________________________________ > > *** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > > /apps/speed.c: 3105 in speed_main() > > 3099 ERR_print_errors(bio_err); > > 3100 rsa_count = 1; > > 3101 } else { > > 3102 for (i = 0; i < loopargs_len; i++) { > > 3103 /* Perform EdDSA signature test */ > > 3104 loopargs[i].siglen = test_ed_curves[testnum].siglen; > > >>> CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > > >>> Pointer "&loopargs[i].siglen" points to an object whose effective type is "unsigned int" (32 bits, unsigned) but is dereferenced as a > wider "unsigned long" (64 bits, unsigned). This may lead to memory corruption. > > 3105 st = EVP_DigestSign(loopargs[i].eddsa_ctx[testnum], > > 3106 loopargs[i].buf2, (size_t *)&loopargs[i].siglen, > > 3107 loopargs[i].buf, 20); > > 3108 if (st == 0) > > 3109 break; > > 3110 } > > > > ** CID 1423323: Null pointer dereferences (FORWARD_NULL) > > > > > > ________________________________________________________________________________________________________ > > *** CID 1423323: Null pointer dereferences (FORWARD_NULL) > > /test/evp_extra_test.c: 894 in test_EVP_PKEY_check() > > 888 > > 889 if (!TEST_int_eq(EVP_PKEY_param_check(ctx), expected_param_check)) > > 890 goto done; > > 891 > > 892 ctx2 = EVP_PKEY_CTX_new_id(0xdefaced, NULL); > > 893 /* assign the pkey directly, as an internal test */ > > >>> CID 1423323: Null pointer dereferences (FORWARD_NULL) > > >>> Passing null pointer "pkey" to "EVP_PKEY_up_ref", which dereferences it. > > 894 EVP_PKEY_up_ref(pkey); > > 895 ctx2->pkey = pkey; > > 896 > > 897 if (!TEST_int_eq(EVP_PKEY_check(ctx2), 0xbeef)) > > 898 goto done; > > 899 > > > > ** CID 1201571: Error handling issues (CHECKED_RETURN) > > /crypto/pkcs12/p12_init.c: 25 in PKCS12_init() > > > > > > ________________________________________________________________________________________________________ > > *** CID 1201571: Error handling issues (CHECKED_RETURN) > > /crypto/pkcs12/p12_init.c: 25 in PKCS12_init() > > 19 PKCS12 *pkcs12; > > 20 > > 21 if ((pkcs12 = PKCS12_new()) == NULL) { > > 22 PKCS12err(PKCS12_F_PKCS12_INIT, ERR_R_MALLOC_FAILURE); > > 23 return NULL; > > 24 } > > >>> CID 1201571: Error handling issues (CHECKED_RETURN) > > >>> Calling "ASN1_INTEGER_set" without checking return value (as is done elsewhere 30 out of 37 times). > > 25 ASN1_INTEGER_set(pkcs12->version, 3); > > 26 pkcs12->authsafes->type = OBJ_nid2obj(mode); > > 27 switch (mode) { > > 28 case NID_pkcs7_data: > > 29 if ((pkcs12->authsafes->d.data = ASN1_OCTET_STRING_new()) == NULL) { > > 30 PKCS12err(PKCS12_F_PKCS12_INIT, ERR_R_MALLOC_FAILURE); > > > > > > ________________________________________________________________________________________________________ > > To view the defects in Coverity Scan visit, https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F- > 2BfV0V05UPxvVjWch-2Bd2MGckcRakUl6QyjujEohY7rPpoYUE4H-2Fm-2BeoDOl8jw7bf4Z78hw-3D-3D_bpOft2V4l9NXEcTx5CnNFJqpP- > 2F8a09dz6vsuNilvAJgBy9hWgnGhTAFGZnkvhcJuSQocoiCV36Dw66FwvViDOF-2BGQbzbMH8LM1tsnputryXt7SEgZZ- > 2FmpoWsuVr91UzOFBmmlL0bipzCjL7WfoT7QvLLnFuGxTjboshY44ftCBEhW8TAZR-2B1c1y7JdbYkdSXw-2B7Vmts-2F- > 2BitkvIjISgebBlgXuThX1DnzutpYSf00XD0-3D > > > > To manage Coverity Scan email notifications for "kaduk-github at mit.edu", click > https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F-2BfV0V05UPxvVjWch-2Bd2MGckcRbVDbis712qZDP- > 2FA8y06Nq414hC6p-2BsqBEViFMJYotwSt4SYNeSzd6tPCdCHgDzpHIBW-2Fr0I0sQJCop-2Fx5Lu2ueYFxYqLmFh7APZbTTED- > 2B53KXZ2qVo0Y2q2bUC-2BpL2TzE-3D_bpOft2V4l9NXEcTx5CnNFJqpP-2F8a09dz6vsuNilvAJgBy9hWgnGhTAFGZnkvhcJu7xxYKPr1HkiPh- > 2BL3MaUbhQMZae3MPjv9c6bU6U4uhOZEhiS1P-2BwpukQ4-2BcSzk5FouA75ij0odEEgZcWTB05BKimz0wg0Y8JsC1Izz20- > 2FpfRp2kjWD47vvs4NmxuDPkNqvS3qoLRQ0vIXW8CFF339G-2B7jGolZ214Wxo3Gh6Hc0HY-3D > > > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project From Matthias.St.Pierre at ncp-e.com Sun Sep 9 22:38:50 2018 From: Matthias.St.Pierre at ncp-e.com (Dr. Matthias St. Pierre) Date: Sun, 9 Sep 2018 22:38:50 +0000 Subject: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) In-Reply-To: <20180909160416.GV73164@kduck.kaduk.org> References: <5b94e7ab7da92_4ed72b08291e2f60744eb@node1.mail> <20180909160416.GV73164@kduck.kaduk.org> Message-ID: <7e5dd21c6f9d40fdad601188be9209ce@Ex13.ncp.local> preliminary status report: *** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) see https://github.com/openssl/openssl/pull/7156 *** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) work in progress... *** CID 1439136: Resource leaks (RESOURCE_LEAK) see https://github.com/openssl/openssl/pull/7155 *** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) todo *** CID 1423323: Null pointer dereferences (FORWARD_NULL) see https://github.com/openssl/openssl/pull/7158 *** CID 1201571: Error handling issues (CHECKED_RETURN) todo if anybody wants to fix one of the CIDs marked 'todo', no problem. Just drop a note on the openssl-project list. Matthias > -----Urspr?ngliche Nachricht----- > Von: openssl-project Im Auftrag von Benjamin Kaduk > Gesendet: Sonntag, 9. September 2018 18:04 > An: openssl-project at openssl.org > Betreff: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) > > I see that Matthias has opened pull requests for a couple of these already; > are you planning to work through the rest of them as well? > > -Ben > > On Sun, Sep 09, 2018 at 09:28:12AM +0000, scan-admin at coverity.com wrote: > > Hi, > > > > Please find the latest report on new defect(s) introduced to openssl/openssl found with Coverity Scan. > > > > 6 new defect(s) introduced to openssl/openssl found with Coverity Scan. > > > > > > New defect(s) Reported-by: Coverity Scan > > Showing 6 of 6 defect(s) > > > > > > ** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > > > > > > ________________________________________________________________________________________________________ > > *** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > > /crypto/rsa/rsa_pss.c: 247 in RSA_padding_add_PKCS1_PSS_mgf1() > > 241 EM[emLen - 1] = 0xbc; > > 242 > > 243 ret = 1; > > 244 > > 245 err: > > 246 EVP_MD_CTX_free(ctx); > > >>> CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > > >>> "sLen" is passed to a parameter that cannot be negative. > > 247 OPENSSL_clear_free(salt, sLen); > > 248 > > 249 return ret; > > 250 > > 251 } > > 252 > > 253 #if defined(_MSC_VER) > > 254 # pragma optimize("",on) > > > > ** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > > > > > > ________________________________________________________________________________________________________ > > *** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > > /crypto/sm2/sm2_pmeth.c: 277 in pkey_sm2_digest_custom() > > 271 } > > 272 > > 273 /* get hashed prefix 'z' of tbs message */ > > 274 if (!sm2_compute_z_digest(z, md, smctx->id, smctx->id_len, ec)) > > 275 return 0; > > 276 > > >>> CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > > >>> "EVP_MD_size(md)" is passed to a parameter that cannot be negative. > > 277 return EVP_DigestUpdate(mctx, z, EVP_MD_size(md)); > > 278 } > > 279 > > 280 const EVP_PKEY_METHOD sm2_pkey_meth = { > > 281 EVP_PKEY_SM2, > > 282 0, > > > > ** CID 1439136: Resource leaks (RESOURCE_LEAK) > > /test/dhtest.c: 202 in dh_test() > > > > > > ________________________________________________________________________________________________________ > > *** CID 1439136: Resource leaks (RESOURCE_LEAK) > > /test/dhtest.c: 202 in dh_test() > > 196 BN_free(bp); > > 197 BN_free(bg); > > 198 BN_free(cpriv_key); > > 199 BN_GENCB_free(_cb); > > 200 DH_free(dh); > > 201 > > >>> CID 1439136: Resource leaks (RESOURCE_LEAK) > > >>> Variable "priv_key" going out of scope leaks the storage it points to. > > 202 return ret; > > 203 } > > 204 > > 205 static int cb(int p, int n, BN_GENCB *arg) > > 206 { > > 207 return 1; > > > > ** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > > > > > > ________________________________________________________________________________________________________ > > *** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > > /apps/speed.c: 3105 in speed_main() > > 3099 ERR_print_errors(bio_err); > > 3100 rsa_count = 1; > > 3101 } else { > > 3102 for (i = 0; i < loopargs_len; i++) { > > 3103 /* Perform EdDSA signature test */ > > 3104 loopargs[i].siglen = test_ed_curves[testnum].siglen; > > >>> CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > > >>> Pointer "&loopargs[i].siglen" points to an object whose effective type is "unsigned int" (32 bits, unsigned) but is dereferenced as a > wider "unsigned long" (64 bits, unsigned). This may lead to memory corruption. > > 3105 st = EVP_DigestSign(loopargs[i].eddsa_ctx[testnum], > > 3106 loopargs[i].buf2, (size_t *)&loopargs[i].siglen, > > 3107 loopargs[i].buf, 20); > > 3108 if (st == 0) > > 3109 break; > > 3110 } > > > > ** CID 1423323: Null pointer dereferences (FORWARD_NULL) > > > > > > ________________________________________________________________________________________________________ > > *** CID 1423323: Null pointer dereferences (FORWARD_NULL) > > /test/evp_extra_test.c: 894 in test_EVP_PKEY_check() > > 888 > > 889 if (!TEST_int_eq(EVP_PKEY_param_check(ctx), expected_param_check)) > > 890 goto done; > > 891 > > 892 ctx2 = EVP_PKEY_CTX_new_id(0xdefaced, NULL); > > 893 /* assign the pkey directly, as an internal test */ > > >>> CID 1423323: Null pointer dereferences (FORWARD_NULL) > > >>> Passing null pointer "pkey" to "EVP_PKEY_up_ref", which dereferences it. > > 894 EVP_PKEY_up_ref(pkey); > > 895 ctx2->pkey = pkey; > > 896 > > 897 if (!TEST_int_eq(EVP_PKEY_check(ctx2), 0xbeef)) > > 898 goto done; > > 899 > > > > ** CID 1201571: Error handling issues (CHECKED_RETURN) > > /crypto/pkcs12/p12_init.c: 25 in PKCS12_init() > > > > > > ________________________________________________________________________________________________________ > > *** CID 1201571: Error handling issues (CHECKED_RETURN) > > /crypto/pkcs12/p12_init.c: 25 in PKCS12_init() > > 19 PKCS12 *pkcs12; > > 20 > > 21 if ((pkcs12 = PKCS12_new()) == NULL) { > > 22 PKCS12err(PKCS12_F_PKCS12_INIT, ERR_R_MALLOC_FAILURE); > > 23 return NULL; > > 24 } > > >>> CID 1201571: Error handling issues (CHECKED_RETURN) > > >>> Calling "ASN1_INTEGER_set" without checking return value (as is done elsewhere 30 out of 37 times). > > 25 ASN1_INTEGER_set(pkcs12->version, 3); > > 26 pkcs12->authsafes->type = OBJ_nid2obj(mode); > > 27 switch (mode) { > > 28 case NID_pkcs7_data: > > 29 if ((pkcs12->authsafes->d.data = ASN1_OCTET_STRING_new()) == NULL) { > > 30 PKCS12err(PKCS12_F_PKCS12_INIT, ERR_R_MALLOC_FAILURE); > > > > > > ________________________________________________________________________________________________________ > > To view the defects in Coverity Scan visit, https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F- > 2BfV0V05UPxvVjWch-2Bd2MGckcRakUl6QyjujEohY7rPpoYUE4H-2Fm-2BeoDOl8jw7bf4Z78hw-3D-3D_bpOft2V4l9NXEcTx5CnNFJqpP- > 2F8a09dz6vsuNilvAJgBy9hWgnGhTAFGZnkvhcJuSQocoiCV36Dw66FwvViDOF-2BGQbzbMH8LM1tsnputryXt7SEgZZ- > 2FmpoWsuVr91UzOFBmmlL0bipzCjL7WfoT7QvLLnFuGxTjboshY44ftCBEhW8TAZR-2B1c1y7JdbYkdSXw-2B7Vmts-2F- > 2BitkvIjISgebBlgXuThX1DnzutpYSf00XD0-3D > > > > To manage Coverity Scan email notifications for "kaduk-github at mit.edu", click > https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F-2BfV0V05UPxvVjWch-2Bd2MGckcRbVDbis712qZDP- > 2FA8y06Nq414hC6p-2BsqBEViFMJYotwSt4SYNeSzd6tPCdCHgDzpHIBW-2Fr0I0sQJCop-2Fx5Lu2ueYFxYqLmFh7APZbTTED- > 2B53KXZ2qVo0Y2q2bUC-2BpL2TzE-3D_bpOft2V4l9NXEcTx5CnNFJqpP-2F8a09dz6vsuNilvAJgBy9hWgnGhTAFGZnkvhcJu7xxYKPr1HkiPh- > 2BL3MaUbhQMZae3MPjv9c6bU6U4uhOZEhiS1P-2BwpukQ4-2BcSzk5FouA75ij0odEEgZcWTB05BKimz0wg0Y8JsC1Izz20- > 2FpfRp2kjWD47vvs4NmxuDPkNqvS3qoLRQ0vIXW8CFF339G-2B7jGolZ214Wxo3Gh6Hc0HY-3D > > > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project From matt at openssl.org Sun Sep 9 22:44:33 2018 From: matt at openssl.org (Matt Caswell) Date: Sun, 9 Sep 2018 23:44:33 +0100 Subject: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) In-Reply-To: <513e30fed5f843ac90fc9d17e046352a@Ex13.ncp.local> References: <5b94e7ab7da92_4ed72b08291e2f60744eb@node1.mail> <20180909160416.GV73164@kduck.kaduk.org> <513e30fed5f843ac90fc9d17e046352a@Ex13.ncp.local> Message-ID: <9d14fb20-9ac8-be8e-d669-1c5e4944bfb1@openssl.org> On 09/09/18 19:31, Dr. Matthias St. Pierre wrote: > I am currently occupied with other things, so I won't be able to look at it before later this evening or tomorrow. > > I also had a quick look at CID 1423323 (see below) but I was unable to see why 'pkey' would be a NULL pointer > when passed to 'EVP_PKEY_up_ref'. So I'm not sure yet whether this is a false positive. It's because Coverity doesn't know that "type" will only ever be 0, 1 or 2. If it wasn't one of those then pkey would be NULL. BTW, note that there is an oddity in the way we have Coverity set up. It's my understanding that we filter out defect reports in the tests and only worry about issues found in the main code base. If you look in the Coverity tool itself you will never see any issues in the test suite (at least I've never figured out how to see them if they are there). Nonetheless it still reports them in the emails it sends out. As far as the release criteria go we only count the ones shown in the Coverity tool. That's not to say we shouldn't fix issues in the tests as well (and actually I'd suggest we stop filtering out problems in the tests if anyone knows how to do that...perhaps Tim?). Matt > > Matthias > > >> -----Urspr?ngliche Nachricht----- >> Von: openssl-project Im Auftrag von Benjamin Kaduk >> Gesendet: Sonntag, 9. September 2018 18:04 >> An: openssl-project at openssl.org >> Betreff: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) >> >> I see that Matthias has opened pull requests for a couple of these already; >> are you planning to work through the rest of them as well? >> >> -Ben >> >> On Sun, Sep 09, 2018 at 09:28:12AM +0000, scan-admin at coverity.com wrote: >>> Hi, >>> >>> Please find the latest report on new defect(s) introduced to openssl/openssl found with Coverity Scan. >>> >>> 6 new defect(s) introduced to openssl/openssl found with Coverity Scan. >>> >>> >>> New defect(s) Reported-by: Coverity Scan >>> Showing 6 of 6 defect(s) >>> >>> >>> ** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) >>> >>> >>> ________________________________________________________________________________________________________ >>> *** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) >>> /crypto/rsa/rsa_pss.c: 247 in RSA_padding_add_PKCS1_PSS_mgf1() >>> 241 EM[emLen - 1] = 0xbc; >>> 242 >>> 243 ret = 1; >>> 244 >>> 245 err: >>> 246 EVP_MD_CTX_free(ctx); >>>>>> CID 1439138: Integer handling issues (NEGATIVE_RETURNS) >>>>>> "sLen" is passed to a parameter that cannot be negative. >>> 247 OPENSSL_clear_free(salt, sLen); >>> 248 >>> 249 return ret; >>> 250 >>> 251 } >>> 252 >>> 253 #if defined(_MSC_VER) >>> 254 # pragma optimize("",on) >>> >>> ** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) >>> >>> >>> ________________________________________________________________________________________________________ >>> *** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) >>> /crypto/sm2/sm2_pmeth.c: 277 in pkey_sm2_digest_custom() >>> 271 } >>> 272 >>> 273 /* get hashed prefix 'z' of tbs message */ >>> 274 if (!sm2_compute_z_digest(z, md, smctx->id, smctx->id_len, ec)) >>> 275 return 0; >>> 276 >>>>>> CID 1439137: Integer handling issues (NEGATIVE_RETURNS) >>>>>> "EVP_MD_size(md)" is passed to a parameter that cannot be negative. >>> 277 return EVP_DigestUpdate(mctx, z, EVP_MD_size(md)); >>> 278 } >>> 279 >>> 280 const EVP_PKEY_METHOD sm2_pkey_meth = { >>> 281 EVP_PKEY_SM2, >>> 282 0, >>> >>> ** CID 1439136: Resource leaks (RESOURCE_LEAK) >>> /test/dhtest.c: 202 in dh_test() >>> >>> >>> ________________________________________________________________________________________________________ >>> *** CID 1439136: Resource leaks (RESOURCE_LEAK) >>> /test/dhtest.c: 202 in dh_test() >>> 196 BN_free(bp); >>> 197 BN_free(bg); >>> 198 BN_free(cpriv_key); >>> 199 BN_GENCB_free(_cb); >>> 200 DH_free(dh); >>> 201 >>>>>> CID 1439136: Resource leaks (RESOURCE_LEAK) >>>>>> Variable "priv_key" going out of scope leaks the storage it points to. >>> 202 return ret; >>> 203 } >>> 204 >>> 205 static int cb(int p, int n, BN_GENCB *arg) >>> 206 { >>> 207 return 1; >>> >>> ** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) >>> >>> >>> ________________________________________________________________________________________________________ >>> *** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) >>> /apps/speed.c: 3105 in speed_main() >>> 3099 ERR_print_errors(bio_err); >>> 3100 rsa_count = 1; >>> 3101 } else { >>> 3102 for (i = 0; i < loopargs_len; i++) { >>> 3103 /* Perform EdDSA signature test */ >>> 3104 loopargs[i].siglen = test_ed_curves[testnum].siglen; >>>>>> CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) >>>>>> Pointer "&loopargs[i].siglen" points to an object whose effective type is "unsigned int" (32 bits, unsigned) but is dereferenced as a >> wider "unsigned long" (64 bits, unsigned). This may lead to memory corruption. >>> 3105 st = EVP_DigestSign(loopargs[i].eddsa_ctx[testnum], >>> 3106 loopargs[i].buf2, (size_t *)&loopargs[i].siglen, >>> 3107 loopargs[i].buf, 20); >>> 3108 if (st == 0) >>> 3109 break; >>> 3110 } >>> >>> ** CID 1423323: Null pointer dereferences (FORWARD_NULL) >>> >>> >>> ________________________________________________________________________________________________________ >>> *** CID 1423323: Null pointer dereferences (FORWARD_NULL) >>> /test/evp_extra_test.c: 894 in test_EVP_PKEY_check() >>> 888 >>> 889 if (!TEST_int_eq(EVP_PKEY_param_check(ctx), expected_param_check)) >>> 890 goto done; >>> 891 >>> 892 ctx2 = EVP_PKEY_CTX_new_id(0xdefaced, NULL); >>> 893 /* assign the pkey directly, as an internal test */ >>>>>> CID 1423323: Null pointer dereferences (FORWARD_NULL) >>>>>> Passing null pointer "pkey" to "EVP_PKEY_up_ref", which dereferences it. >>> 894 EVP_PKEY_up_ref(pkey); >>> 895 ctx2->pkey = pkey; >>> 896 >>> 897 if (!TEST_int_eq(EVP_PKEY_check(ctx2), 0xbeef)) >>> 898 goto done; >>> 899 >>> >>> ** CID 1201571: Error handling issues (CHECKED_RETURN) >>> /crypto/pkcs12/p12_init.c: 25 in PKCS12_init() >>> >>> >>> ________________________________________________________________________________________________________ >>> *** CID 1201571: Error handling issues (CHECKED_RETURN) >>> /crypto/pkcs12/p12_init.c: 25 in PKCS12_init() >>> 19 PKCS12 *pkcs12; >>> 20 >>> 21 if ((pkcs12 = PKCS12_new()) == NULL) { >>> 22 PKCS12err(PKCS12_F_PKCS12_INIT, ERR_R_MALLOC_FAILURE); >>> 23 return NULL; >>> 24 } >>>>>> CID 1201571: Error handling issues (CHECKED_RETURN) >>>>>> Calling "ASN1_INTEGER_set" without checking return value (as is done elsewhere 30 out of 37 times). >>> 25 ASN1_INTEGER_set(pkcs12->version, 3); >>> 26 pkcs12->authsafes->type = OBJ_nid2obj(mode); >>> 27 switch (mode) { >>> 28 case NID_pkcs7_data: >>> 29 if ((pkcs12->authsafes->d.data = ASN1_OCTET_STRING_new()) == NULL) { >>> 30 PKCS12err(PKCS12_F_PKCS12_INIT, ERR_R_MALLOC_FAILURE); >>> >>> >>> ________________________________________________________________________________________________________ >>> To view the defects in Coverity Scan visit, https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F- >> 2BfV0V05UPxvVjWch-2Bd2MGckcRakUl6QyjujEohY7rPpoYUE4H-2Fm-2BeoDOl8jw7bf4Z78hw-3D-3D_bpOft2V4l9NXEcTx5CnNFJqpP- >> 2F8a09dz6vsuNilvAJgBy9hWgnGhTAFGZnkvhcJuSQocoiCV36Dw66FwvViDOF-2BGQbzbMH8LM1tsnputryXt7SEgZZ- >> 2FmpoWsuVr91UzOFBmmlL0bipzCjL7WfoT7QvLLnFuGxTjboshY44ftCBEhW8TAZR-2B1c1y7JdbYkdSXw-2B7Vmts-2F- >> 2BitkvIjISgebBlgXuThX1DnzutpYSf00XD0-3D >>> >>> To manage Coverity Scan email notifications for "kaduk-github at mit.edu", click >> https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F-2BfV0V05UPxvVjWch-2Bd2MGckcRbVDbis712qZDP- >> 2FA8y06Nq414hC6p-2BsqBEViFMJYotwSt4SYNeSzd6tPCdCHgDzpHIBW-2Fr0I0sQJCop-2Fx5Lu2ueYFxYqLmFh7APZbTTED- >> 2B53KXZ2qVo0Y2q2bUC-2BpL2TzE-3D_bpOft2V4l9NXEcTx5CnNFJqpP-2F8a09dz6vsuNilvAJgBy9hWgnGhTAFGZnkvhcJu7xxYKPr1HkiPh- >> 2BL3MaUbhQMZae3MPjv9c6bU6U4uhOZEhiS1P-2BwpukQ4-2BcSzk5FouA75ij0odEEgZcWTB05BKimz0wg0Y8JsC1Izz20- >> 2FpfRp2kjWD47vvs4NmxuDPkNqvS3qoLRQ0vIXW8CFF339G-2B7jGolZ214Wxo3Gh6Hc0HY-3D >>> >> _______________________________________________ >> openssl-project mailing list >> openssl-project at openssl.org >> https://mta.openssl.org/mailman/listinfo/openssl-project > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > From kaduk at mit.edu Sun Sep 9 23:02:25 2018 From: kaduk at mit.edu (Benjamin Kaduk) Date: Sun, 9 Sep 2018 18:02:25 -0500 Subject: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) In-Reply-To: <7e5dd21c6f9d40fdad601188be9209ce@Ex13.ncp.local> References: <5b94e7ab7da92_4ed72b08291e2f60744eb@node1.mail> <20180909160416.GV73164@kduck.kaduk.org> <7e5dd21c6f9d40fdad601188be9209ce@Ex13.ncp.local> Message-ID: <20180909230225.GX73164@kduck.kaduk.org> On Sun, Sep 09, 2018 at 10:38:50PM +0000, Dr. Matthias St. Pierre wrote: > preliminary status report: > > *** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > see https://github.com/openssl/openssl/pull/7156 > > *** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > work in progress... I think this one may be a false positive -- it's worried that EVP_MD_size() will return -1, but we've essentially already validated that the md is valid by the time we get there. I didn't do a full check, though. -Ben > *** CID 1439136: Resource leaks (RESOURCE_LEAK) > see https://github.com/openssl/openssl/pull/7155 > > *** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > todo > > *** CID 1423323: Null pointer dereferences (FORWARD_NULL) > see https://github.com/openssl/openssl/pull/7158 > > *** CID 1201571: Error handling issues (CHECKED_RETURN) > todo > > if anybody wants to fix one of the CIDs marked 'todo', no problem. Just drop a note on the openssl-project list. > > Matthias > > > > -----Urspr?ngliche Nachricht----- > > Von: openssl-project Im Auftrag von Benjamin Kaduk > > Gesendet: Sonntag, 9. September 2018 18:04 > > An: openssl-project at openssl.org > > Betreff: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) > > > > I see that Matthias has opened pull requests for a couple of these already; > > are you planning to work through the rest of them as well? > > > > -Ben > > > > On Sun, Sep 09, 2018 at 09:28:12AM +0000, scan-admin at coverity.com wrote: > > > Hi, > > > > > > Please find the latest report on new defect(s) introduced to openssl/openssl found with Coverity Scan. > > > > > > 6 new defect(s) introduced to openssl/openssl found with Coverity Scan. > > > > > > > > > New defect(s) Reported-by: Coverity Scan > > > Showing 6 of 6 defect(s) > > > > > > > > > ** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > > > > > > > > > ________________________________________________________________________________________________________ > > > *** CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > > > /crypto/rsa/rsa_pss.c: 247 in RSA_padding_add_PKCS1_PSS_mgf1() > > > 241 EM[emLen - 1] = 0xbc; > > > 242 > > > 243 ret = 1; > > > 244 > > > 245 err: > > > 246 EVP_MD_CTX_free(ctx); > > > >>> CID 1439138: Integer handling issues (NEGATIVE_RETURNS) > > > >>> "sLen" is passed to a parameter that cannot be negative. > > > 247 OPENSSL_clear_free(salt, sLen); > > > 248 > > > 249 return ret; > > > 250 > > > 251 } > > > 252 > > > 253 #if defined(_MSC_VER) > > > 254 # pragma optimize("",on) > > > > > > ** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > > > > > > > > > ________________________________________________________________________________________________________ > > > *** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > > > /crypto/sm2/sm2_pmeth.c: 277 in pkey_sm2_digest_custom() > > > 271 } > > > 272 > > > 273 /* get hashed prefix 'z' of tbs message */ > > > 274 if (!sm2_compute_z_digest(z, md, smctx->id, smctx->id_len, ec)) > > > 275 return 0; > > > 276 > > > >>> CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > > > >>> "EVP_MD_size(md)" is passed to a parameter that cannot be negative. > > > 277 return EVP_DigestUpdate(mctx, z, EVP_MD_size(md)); > > > 278 } > > > 279 > > > 280 const EVP_PKEY_METHOD sm2_pkey_meth = { > > > 281 EVP_PKEY_SM2, > > > 282 0, > > > > > > ** CID 1439136: Resource leaks (RESOURCE_LEAK) > > > /test/dhtest.c: 202 in dh_test() > > > > > > > > > ________________________________________________________________________________________________________ > > > *** CID 1439136: Resource leaks (RESOURCE_LEAK) > > > /test/dhtest.c: 202 in dh_test() > > > 196 BN_free(bp); > > > 197 BN_free(bg); > > > 198 BN_free(cpriv_key); > > > 199 BN_GENCB_free(_cb); > > > 200 DH_free(dh); > > > 201 > > > >>> CID 1439136: Resource leaks (RESOURCE_LEAK) > > > >>> Variable "priv_key" going out of scope leaks the storage it points to. > > > 202 return ret; > > > 203 } > > > 204 > > > 205 static int cb(int p, int n, BN_GENCB *arg) > > > 206 { > > > 207 return 1; > > > > > > ** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > > > > > > > > > ________________________________________________________________________________________________________ > > > *** CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > > > /apps/speed.c: 3105 in speed_main() > > > 3099 ERR_print_errors(bio_err); > > > 3100 rsa_count = 1; > > > 3101 } else { > > > 3102 for (i = 0; i < loopargs_len; i++) { > > > 3103 /* Perform EdDSA signature test */ > > > 3104 loopargs[i].siglen = test_ed_curves[testnum].siglen; > > > >>> CID 1439135: Memory - illegal accesses (INCOMPATIBLE_CAST) > > > >>> Pointer "&loopargs[i].siglen" points to an object whose effective type is "unsigned int" (32 bits, unsigned) but is dereferenced as a > > wider "unsigned long" (64 bits, unsigned). This may lead to memory corruption. > > > 3105 st = EVP_DigestSign(loopargs[i].eddsa_ctx[testnum], > > > 3106 loopargs[i].buf2, (size_t *)&loopargs[i].siglen, > > > 3107 loopargs[i].buf, 20); > > > 3108 if (st == 0) > > > 3109 break; > > > 3110 } > > > > > > ** CID 1423323: Null pointer dereferences (FORWARD_NULL) > > > > > > > > > ________________________________________________________________________________________________________ > > > *** CID 1423323: Null pointer dereferences (FORWARD_NULL) > > > /test/evp_extra_test.c: 894 in test_EVP_PKEY_check() > > > 888 > > > 889 if (!TEST_int_eq(EVP_PKEY_param_check(ctx), expected_param_check)) > > > 890 goto done; > > > 891 > > > 892 ctx2 = EVP_PKEY_CTX_new_id(0xdefaced, NULL); > > > 893 /* assign the pkey directly, as an internal test */ > > > >>> CID 1423323: Null pointer dereferences (FORWARD_NULL) > > > >>> Passing null pointer "pkey" to "EVP_PKEY_up_ref", which dereferences it. > > > 894 EVP_PKEY_up_ref(pkey); > > > 895 ctx2->pkey = pkey; > > > 896 > > > 897 if (!TEST_int_eq(EVP_PKEY_check(ctx2), 0xbeef)) > > > 898 goto done; > > > 899 > > > > > > ** CID 1201571: Error handling issues (CHECKED_RETURN) > > > /crypto/pkcs12/p12_init.c: 25 in PKCS12_init() > > > > > > > > > ________________________________________________________________________________________________________ > > > *** CID 1201571: Error handling issues (CHECKED_RETURN) > > > /crypto/pkcs12/p12_init.c: 25 in PKCS12_init() > > > 19 PKCS12 *pkcs12; > > > 20 > > > 21 if ((pkcs12 = PKCS12_new()) == NULL) { > > > 22 PKCS12err(PKCS12_F_PKCS12_INIT, ERR_R_MALLOC_FAILURE); > > > 23 return NULL; > > > 24 } > > > >>> CID 1201571: Error handling issues (CHECKED_RETURN) > > > >>> Calling "ASN1_INTEGER_set" without checking return value (as is done elsewhere 30 out of 37 times). > > > 25 ASN1_INTEGER_set(pkcs12->version, 3); > > > 26 pkcs12->authsafes->type = OBJ_nid2obj(mode); > > > 27 switch (mode) { > > > 28 case NID_pkcs7_data: > > > 29 if ((pkcs12->authsafes->d.data = ASN1_OCTET_STRING_new()) == NULL) { > > > 30 PKCS12err(PKCS12_F_PKCS12_INIT, ERR_R_MALLOC_FAILURE); > > > > > > > > > ________________________________________________________________________________________________________ > > > To view the defects in Coverity Scan visit, https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F- > > 2BfV0V05UPxvVjWch-2Bd2MGckcRakUl6QyjujEohY7rPpoYUE4H-2Fm-2BeoDOl8jw7bf4Z78hw-3D-3D_bpOft2V4l9NXEcTx5CnNFJqpP- > > 2F8a09dz6vsuNilvAJgBy9hWgnGhTAFGZnkvhcJuSQocoiCV36Dw66FwvViDOF-2BGQbzbMH8LM1tsnputryXt7SEgZZ- > > 2FmpoWsuVr91UzOFBmmlL0bipzCjL7WfoT7QvLLnFuGxTjboshY44ftCBEhW8TAZR-2B1c1y7JdbYkdSXw-2B7Vmts-2F- > > 2BitkvIjISgebBlgXuThX1DnzutpYSf00XD0-3D > > > > > > To manage Coverity Scan email notifications for "kaduk-github at mit.edu", click > > https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F-2BfV0V05UPxvVjWch-2Bd2MGckcRbVDbis712qZDP- > > 2FA8y06Nq414hC6p-2BsqBEViFMJYotwSt4SYNeSzd6tPCdCHgDzpHIBW-2Fr0I0sQJCop-2Fx5Lu2ueYFxYqLmFh7APZbTTED- > > 2B53KXZ2qVo0Y2q2bUC-2BpL2TzE-3D_bpOft2V4l9NXEcTx5CnNFJqpP-2F8a09dz6vsuNilvAJgBy9hWgnGhTAFGZnkvhcJu7xxYKPr1HkiPh- > > 2BL3MaUbhQMZae3MPjv9c6bU6U4uhOZEhiS1P-2BwpukQ4-2BcSzk5FouA75ij0odEEgZcWTB05BKimz0wg0Y8JsC1Izz20- > > 2FpfRp2kjWD47vvs4NmxuDPkNqvS3qoLRQ0vIXW8CFF339G-2B7jGolZ214Wxo3Gh6Hc0HY-3D > > > > > _______________________________________________ > > openssl-project mailing list > > openssl-project at openssl.org > > https://mta.openssl.org/mailman/listinfo/openssl-project > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project From Matthias.St.Pierre at ncp-e.com Sun Sep 9 23:10:36 2018 From: Matthias.St.Pierre at ncp-e.com (Dr. Matthias St. Pierre) Date: Sun, 9 Sep 2018 23:10:36 +0000 Subject: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) In-Reply-To: <20180909230225.GX73164@kduck.kaduk.org> References: <5b94e7ab7da92_4ed72b08291e2f60744eb@node1.mail> <20180909160416.GV73164@kduck.kaduk.org> <7e5dd21c6f9d40fdad601188be9209ce@Ex13.ncp.local> <20180909230225.GX73164@kduck.kaduk.org> Message-ID: <140adc73f29042bdbb1e6aba6534020e@Ex13.ncp.local> > > *** CID 1439137: Integer handling issues (NEGATIVE_RETURNS) > > work in progress... > > I think this one may be a false positive -- it's worried that EVP_MD_size() > will return -1, but we've essentially already validated that the md is > valid by the time we get there. I didn't do a full check, though. > > -Ben Yes, that's my suspicion, too. But I am also not sure yet. As far as I understand it, EVP_MD_size() will be negative only if md == NULL. So it boils down to the question whether one can assert that mctx always contains a valid md in line 261: const EVP_MD *md = EVP_MD_CTX_md(mctx); If that is the case, then one can silence coverity by casting the sign of the return value of EVP_MD_size(). But if not, some error handling is missing. Matthias From kurt at roeckx.be Mon Sep 10 07:12:29 2018 From: kurt at roeckx.be (Kurt Roeckx) Date: Mon, 10 Sep 2018 09:12:29 +0200 Subject: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) In-Reply-To: <9d14fb20-9ac8-be8e-d669-1c5e4944bfb1@openssl.org> References: <5b94e7ab7da92_4ed72b08291e2f60744eb@node1.mail> <20180909160416.GV73164@kduck.kaduk.org> <513e30fed5f843ac90fc9d17e046352a@Ex13.ncp.local> <9d14fb20-9ac8-be8e-d669-1c5e4944bfb1@openssl.org> Message-ID: <20180910071229.GA5516@roeckx.be> On Sun, Sep 09, 2018 at 11:44:33PM +0100, Matt Caswell wrote: > > As far as the release criteria go we only count the ones shown in the > Coverity tool. That's not to say we shouldn't fix issues in the tests as > well (and actually I'd suggest we stop filtering out problems in the > tests if anyone knows how to do that...perhaps Tim?). You can change that in the project settings, where it splits the paths into different groups. Kurt From tjh at cryptsoft.com Mon Sep 10 07:28:19 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Mon, 10 Sep 2018 17:28:19 +1000 Subject: [openssl-project] coverity defect release criteria (Fwd: New Defects reported by Coverity Scan for openssl/openssl) In-Reply-To: <9d14fb20-9ac8-be8e-d669-1c5e4944bfb1@openssl.org> References: <5b94e7ab7da92_4ed72b08291e2f60744eb@node1.mail> <20180909160416.GV73164@kduck.kaduk.org> <513e30fed5f843ac90fc9d17e046352a@Ex13.ncp.local> <9d14fb20-9ac8-be8e-d669-1c5e4944bfb1@openssl.org> Message-ID: On Mon, Sep 10, 2018 at 8:44 AM, Matt Caswell wrote: > As far as the release criteria go we only count the ones shown in the > Coverity tool. That's not to say we shouldn't fix issues in the tests as > well (and actually I'd suggest we stop filtering out problems in the > tests if anyone knows how to do that...perhaps Tim?). > I have changed things to no longer exclude the tests area from the online reports (that was a historical setting on the original project from pre-2014). The second project which I got added to track master has always emailed all issues. I have also now changed the OpenSSL-1.0.2 project to report all errors as well via email - no longer excluding various components. So now we should have both the online tool and the emails for both projects configured the same and including the test programs. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Mon Sep 10 21:14:52 2018 From: matt at openssl.org (Matt Caswell) Date: Mon, 10 Sep 2018 22:14:52 +0100 Subject: [openssl-project] Final check against the release criteria Message-ID: A final check against the release criteria: - All open github issues/PRs older than 2 weeks at the time of release to be assessed for relevance to 1.1.1. Any flagged with the 1.1.1 milestone to be closed (see below) There are no 1.1.1 flagged issues. There is one 1.1.1 flagged PR which was opened yesterday. It is a trivial fix in the ossltest engine, currently blocked on the author updating it to add "CLA:trivial". This is not a significant issue so I consider this criterion met. - Clean builds in Travis and Appveyor for two days AppVeyor has been green for more than two days. Travis is currently green. It was red earlier due to an environmental change on Travis - not due to any code issue in OpenSSL. It also briefly went red due to a failure in travis to start the build properly (i.e. not an OpenSSL issue). I don't consider either of these to be blockers for the release. - run-checker.sh to be showing as clean 2 days before release run-checker is currently green and has been for some while. - No open Coverity issues (not flagged as "False Positive" or "Ignore") There are no open Coverity issues not in the test suite that are not flagged as false positive or ignore. Until yesterday the project settings filtered out issues in the test suite. That setting has been changed and now we have a whole bunch of them to fix. I don't think that should stop us from releasing. - TLSv1.3 RFC published (with at least one beta release after the publicaction) RFC is published! In summary I consider the release criteria to be met and so I will be going ahead with the release tomorrow. Matt From openssl at openssl.org Tue Sep 11 13:42:31 2018 From: openssl at openssl.org (OpenSSL) Date: Tue, 11 Sep 2018 13:42:31 +0000 Subject: [openssl-project] OpenSSL version 1.1.1 published Message-ID: <20180911134231.GA11632@openssl.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 OpenSSL version 1.1.1 released =============================== OpenSSL - The Open Source toolkit for SSL/TLS https://www.openssl.org/ The OpenSSL project team is pleased to announce the release of version 1.1.1 of our open source toolkit for SSL/TLS. For details of changes and known issues see the release notes at: https://www.openssl.org/news/openssl-1.1.1-notes.html OpenSSL 1.1.1 is available for download via HTTP and FTP from the following master locations (you can find the various FTP mirrors under https://www.openssl.org/source/mirror.html): * https://www.openssl.org/source/ * ftp://ftp.openssl.org/source/ The distribution file name is: o openssl-1.1.1.tar.gz Size: 8337920 SHA1 checksum: e4559f31dca37ce815e0c7135488b747745a056d SHA256 checksum: 2836875a0f89c03d0fdf483941512613a50cfb421d6fd94b9f41d7279d586a3d The checksums were calculated using the following commands: openssl sha1 openssl-1.1.1.tar.gz openssl sha256 openssl-1.1.1.tar.gz Yours, The OpenSSL Project Team. -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEhlersmDwVrHlGQg52cTSbQ5gRJEFAluXuZ8ACgkQ2cTSbQ5g RJFPFQf9G1LopuN1P3tIUTgps9Z1SS+TuC7OeRPu9TCEqOR0yO8WGyTCfLZnoXZ7 0BqFASYW4VbPCy8LH3glHLBe64NApdoA1HoMmHCvd+TxPQHEvhc0OejSaOGZKY/r 2LGUvEguiyYpjQS4bQmsl8wNl3CrYRGSMqBcbFj+qF/Rrlpa1hpKGnH4ooMxe7Nx /Ro4AjMe46vQL/RU980yFl+JTkhAvSOxw0cltbILPO2MP6Fo4QZqMO8mYRjEnqUZ E/Ixl/dIkSWjPC8pkkRS9FmMQHHYe66S20OK7V2Zl3Zd88FrNI+qeKgEF3ABGknR 6vR0kPkddRl43JktQ4B1QKS+GcwzHw== =fvfm -----END PGP SIGNATURE----- From matt at openssl.org Tue Sep 11 13:52:22 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 11 Sep 2018 14:52:22 +0100 Subject: [openssl-project] 1.1.1 is released! Message-ID: <140fdca5-bbed-0d3e-5715-cf2aba74add7@openssl.org> I've just finished the 1.1.1 release process and the repo is now unfrozen. There is now a new OpenSSL_1_1_1-stable branch. 1.1.0 is officially in security fixes only mode so generally we should not be cherry-picking fixes to OpenSSL_1_1_0-stable. Congratulations and thanks to everyone who has contributed to this release!!! Matt From matt at openssl.org Tue Sep 11 13:56:19 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 11 Sep 2018 14:56:19 +0100 Subject: [openssl-project] OpenSSL 1.1.1 Blog Message-ID: Our new Long Term Support release, OpenSSL 1.1.1, including TLSv1.3, has been released today. Please download and upgrade! There is a blog post about the new release and the status of the older releases here: https://www.openssl.org/blog/blog/2018/09/11/release111/ Matt From bernd.edlinger at hotmail.de Wed Sep 12 10:57:10 2018 From: bernd.edlinger at hotmail.de (Bernd Edlinger) Date: Wed, 12 Sep 2018 10:57:10 +0000 Subject: [openssl-project] OpenSSL 1.1.1 Blog In-Reply-To: References: Message-ID: Hi, I just read this in the blog article: New OMC Member and New Committers https://www.openssl.org/blog/blog/2018/08/22/updates/ "The latest additions to the committers (in alphabetical order) are: Paul Yang Nicola Tuveri " aehm, maybe we should fix the alphabetical order ? :-) Bernd. ________________________________________ From: openssl-project on behalf of Matt Caswell Sent: Tuesday, September 11, 2018 15:56 To: openssl-announce at openssl.org; openssl-project at openssl.org; openssl-users at openssl.org Subject: [openssl-project] OpenSSL 1.1.1 Blog Our new Long Term Support release, OpenSSL 1.1.1, including TLSv1.3, has been released today. Please download and upgrade! There is a blog post about the new release and the status of the older releases here: https://www.openssl.org/blog/blog/2018/09/11/release111/ Matt _______________________________________________ openssl-project mailing list openssl-project at openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project From matt at openssl.org Wed Sep 12 10:58:19 2018 From: matt at openssl.org (Matt Caswell) Date: Wed, 12 Sep 2018 11:58:19 +0100 Subject: [openssl-project] OpenSSL 1.1.1 Blog In-Reply-To: References: Message-ID: On 12/09/18 11:57, Bernd Edlinger wrote: > Hi, > > I just read this in the blog article: New OMC Member and New Committers > https://www.openssl.org/blog/blog/2018/08/22/updates/ > > "The latest additions to the committers (in alphabetical order) are: > > Paul Yang > Nicola Tuveri > " > > aehm, maybe we should fix the alphabetical order ? :-) Tim tells me it is alphabetic by github user id! ;-) Matt > > Bernd. > > ________________________________________ > From: openssl-project on behalf of Matt Caswell > Sent: Tuesday, September 11, 2018 15:56 > To: openssl-announce at openssl.org; openssl-project at openssl.org; openssl-users at openssl.org > Subject: [openssl-project] OpenSSL 1.1.1 Blog > > Our new Long Term Support release, OpenSSL 1.1.1, including TLSv1.3, has > been released today. Please download and upgrade! > > There is a blog post about the new release and the status of the older > releases here: > https://www.openssl.org/blog/blog/2018/09/11/release111/ > > Matt > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > From kurt at roeckx.be Tue Sep 18 18:06:12 2018 From: kurt at roeckx.be (Kurt Roeckx) Date: Tue, 18 Sep 2018 20:06:12 +0200 Subject: [openssl-project] Current priorities Message-ID: <20180918180611.GA2053@roeckx.be> Hi, Now that 1.1.1 is released, and before everybody starts to work on new features, I would like to suggest that we work on reducing the number of open pull requests and issues. Fixing issues that are regressions in the 1.1.1 version is something we really should be doing, and I think that should currently be our top priority. We should probably also do a new release after a month or something. The open PRs were around 100 when 1.1.0 was released, and have been around 120 for a very long time, but the last few months it has grown to around 150. I guess we have some that are waiting because they are new features that didn't go in 1.1.1. But I would like to get the number of open PRs to around 100, but even lower is of course even better. I guess the open issues are also also growing, but I don't have any numbers for that. Kurt From levitte at openssl.org Fri Sep 21 09:58:31 2018 From: levitte at openssl.org (Richard Levitte) Date: Fri, 21 Sep 2018 11:58:31 +0200 (CEST) Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) Message-ID: <20180921.115831.1818111203113256464.levitte@openssl.org> Hi, I've worked on a proposal for an update of the OpenSSL version scheme, to basically align ourselves with semantic versioning, most of all in presentation, but also in spirit, while retaining numeric compatibility. This proposal is currently in the form of a Google Document: https://docs.google.com/document/d/1H3HSLxHzg7Ca3WQEU-zsOd1lUP_uJieHw5Dae34KPBs/ One notable thing with semantic versioning is that it's much stricter about the meaning of a major release regarding incompatible API/ABI changes. Our FAQ says that such changes *may* be part of a major release (we don't guarantee that breaking changes won't happen), while semantic versioning says that major releases *do* incur backward incompatible API changes. See https://semver.org/ for the full definition and description of semantic versioning. Please discuss. I intend to finalize the document at some point and make it a PDF stored among the policies on our web (submitted as a PR of course) and start a vote on its final inclusion. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From tjh at cryptsoft.com Fri Sep 21 10:48:34 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Fri, 21 Sep 2018 20:48:34 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180921.115831.1818111203113256464.levitte@openssl.org> References: <20180921.115831.1818111203113256464.levitte@openssl.org> Message-ID: On Fri, Sep 21, 2018 at 7:58 PM Richard Levitte wrote: > Our FAQ says that such changes *may* be part of a major > release (we don't guarantee that breaking changes won't happen), while > semantic versioning says that major releases *do* incur backward > incompatible API changes. > I think you are misreading the semantic versioning usage - it states when things MUST happen. It does not state that you MUST NOT change a version if the trigger event has not occurred. Semantic versioning also requires you to explicitly declare what your public API is in a "precise and comprehensive" manner. What do you consider the public API of OpenSSL? That is pretty much a prerequisite for actually adopting semantic versioning. I also think the concept of reinterpreting the current major version number into an epoch as you propose is not something that we should be doing. We have defined the first digit as our major version number - and changing that in my view at least would be going completely against the principles of semantic versioning. The version itself is meant to be purely X.Y.Z[-PRERELEASE] or X.Y.Z[+BUILDMETA] and your suggested encoding is not that at all. What you have is EPOCH.X.Y.Z.FIX.STATUS - where EPOCH and STATUS are not concepts contained within semantic versioning. Basically adopting semantic versioning actually requires something different to what has been proposed in my view. I would suggest it means our current version encoding in an integer of MNNFFPPS becomes simply MNNFF000 and the information for PP and S is moved elsewhere as semantic versioning defines those concepts differently (as noted above). Part of our challenge is ensuring we don't cause unnecessary breakage for users: Vendors change the text string to add additional indicators for their variations. Otherwise developers use the current integer version for feature testing - and it needs to remain compatible enough. I haven't seen any code actually testing the S field within the version or doing anything specific with the PP version - other than reporting it to the user. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Fri Sep 21 11:02:15 2018 From: matt at openssl.org (Matt Caswell) Date: Fri, 21 Sep 2018 12:02:15 +0100 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.115831.1818111203113256464.levitte@openssl.org> Message-ID: On 21/09/18 11:48, Tim Hudson wrote: > On Fri, Sep 21, 2018 at 7:58 PM Richard Levitte > wrote: > > Our FAQ says that such changes *may* be part of a major > release (we don't guarantee that breaking changes won't happen), while > semantic versioning says that major releases *do* incur backward > incompatible API changes. > > > I think you are misreading the semantic versioning usage - it states > when things MUST happen. > It does not state that you MUST NOT change a version if the trigger > event has not occurred. > > Semantic versioning also requires you to explicitly declare what your > public API is in a "precise and comprehensive" manner. > What do you consider the public API of OpenSSL? > > That is pretty much a prerequisite for actually adopting semantic > versioning. > > I also think the concept of reinterpreting the current major version > number into an epoch as you propose is not something that we should be > doing. > We have defined the first digit as our major version number - and > changing that in my view at least would be going completely against the > principles of semantic versioning. > The version itself is meant to be purely X.Y.Z[-PRERELEASE] or > X.Y.Z[+BUILDMETA] and your suggested encoding is not that at all. > > What you have is EPOCH.X.Y.Z.FIX.STATUS - where EPOCH and STATUS are not > concepts contained within semantic versioning. I think this is an incorrect interpretation of Richard's proposal. The OPENSSL_VERSION_NUMBER value is an *integer* value. It does not and cannot ever conform to semantic versioning because, because version numbers in that scheme are *strings* in a specific format, where characters such as "." and "-" have special meanings. The semantic version for openssl would be the value of the OPENSSL_VERSION_NUMBER_TEXT macro (if PR #7285 gets merged). If we were to adopt semantic versioning then the OPENSSL_VERSION_NUMBER macro would be an OpenSSL specific encoding of the semantic version number, useful for the purpose of comparing different version numbers such that the precedence rules of semantic versioning still apply. As an OpenSSL specific encoding, we can choose to encode it in any way we like. This *includes* having a status value if we so wish (and that actually fits quite nicely with the semantic versioning concept of pre-releases) > > Basically adopting semantic versioning actually requires something > different to what has been proposed in my view. > > I would suggest it means our current version encoding in an integer > of?MNNFFPPS becomes simply MNNFF000 and the information for PP and S is > moved elsewhere as semantic versioning defines those concepts > differently (as noted above). > > Part of our challenge is ensuring we don't cause unnecessary breakage > for users: > > Vendors change the text string to add additional indicators for their > variations. > Otherwise developers use the current integer version for feature testing > - and it needs to remain compatible enough. > > I haven't seen any code actually testing the S field within the version > or doing anything specific with the PP version - other than reporting it > to the user. The S field remains useful for checking precedence, i.e. knowing that 1.1.1-pre1 is an older version that 1.1.1. Matt From tjh at cryptsoft.com Fri Sep 21 11:17:12 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Fri, 21 Sep 2018 21:17:12 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.115831.1818111203113256464.levitte@openssl.org> Message-ID: On Fri, Sep 21, 2018 at 9:02 PM Matt Caswell wrote: > I think this is an incorrect interpretation of Richard's proposal. The > OPENSSL_VERSION_NUMBER value is an *integer* value. It does not and > cannot ever conform to semantic versioning because, because version > numbers in that scheme are *strings* in a specific format, where > characters such as "." and "-" have special meanings. > It is the version number. We have it in two forms within OpenSSL from a code perspective - we have an integer encoding and a text string. They are precisely what semantic versioning is about - making sure the versioning concept is represented in what you are using versioning for. For OpenSSL, codewise we have two macros and two functions that let us access the build-time version of the macros: OPENSSL_VERSION_NUMBER OPENSSL_VERSION_TEXT OpenSSL_version_num() OpenSSL_version() We also separately have another form of version number - for shared libraries: The macro: SHLIB_VERSION_NUMBER We also encode the version number in release packages too. What semantic versioning is about is sorting out how we represent the version. It should impact both OPENSSL_VERSION_NUMBER and OPENSSL_VERSION_TEXT and it should be consistent. For the semantic versioning document the status indicator is handled in the pre-release indicator. We could limit that to a numeric number and have it in the OPENSSL_VERSION_NUMBER but I don't think that has helped and semantic versioning strictly defines precedence handling. So I would see the simple mapping from semantic versioning very differently to Richard's write up - and in fact encoding something rather differently into the OPENSSL_VERSION_NUMBER to my reading and thinking actually goes against the principles of semantic versioning. i.e. OPENSSL_VERSION_NUMBER should be X.Y.Z and OPENSSL_VERSION_TEXT should be "X.Y.Z[-patch][+buildmeta]" and that would be a simple, direct, and expected mapping to OpenSSL for semantic versioning. A merged approach or keeping parts of our (non-semantic) approach while not fully adopting semantic versioning to me at least would be missing the point. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjh at cryptsoft.com Fri Sep 21 11:28:55 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Fri, 21 Sep 2018 21:28:55 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.115831.1818111203113256464.levitte@openssl.org> Message-ID: So as a concrete example - taking master and the current OPENSSL_VERSION_TEXT value. "OpenSSL 1.1.2-dev xx XXX xxxx" would become "1.1.2-dev+xx.XXX.xxxx" That is what I understand is the point of semantic versioning. You know how to pull apart the version string. -dev indicates a pre-release version known as "dev" +xx.XXX.xxxx indicates build metadata. The underlying release is major 1, minor 1, patch 2. But for semantic versioning where we allow API breakage in our current minor version we would have to shift everything left. And we would then have "1.2.0-dev+xx.XXX.xxxx" if the planned new release wasn't guaranteed API non-breaking with the previous release (1.1.1). Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Fri Sep 21 12:52:37 2018 From: levitte at openssl.org (Richard Levitte) Date: Fri, 21 Sep 2018 14:52:37 +0200 (CEST) Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.115831.1818111203113256464.levitte@openssl.org> Message-ID: <20180921.145237.999607809821393632.levitte@openssl.org> In message on Fri, 21 Sep 2018 20:48:34 +1000, Tim Hudson said: > On Fri, Sep 21, 2018 at 7:58 PM Richard Levitte wrote: > > Our FAQ says that such changes *may* be part of a major > release (we don't guarantee that breaking changes won't happen), while > semantic versioning says that major releases *do* incur backward > incompatible API changes. > > I think you are misreading the semantic versioning usage - it states > when things MUST happen. It does not state that you MUST NOT change > a version if the trigger event has not occurred. Fair point... so OK, we're then even more compatible with semver than I though. > Semantic versioning also requires you to explicitly declare what > your public API is in a "precise and comprehensive" manner. > What do you consider the public API of OpenSSL? The collection of stuff in our public headers, i.e. all headers that we end up installing in {prefix}/include/openssl. I thought we all agreed on that. If we don't, then we need to discuss that as well... in a different thread. > That is pretty much a prerequisite for actually adopting semantic > versioning. > I also think the concept of reinterpreting the current major version > number into an epoch as you propose is not something that we should > be doing. The way we've been changing the version number and the way we describe our current version number scheme in the FAQ (since 2012, but this had already happened in spirit before that), that first 1 Is Not The Major Version Number. It's just a 1 that sits there with no real purpose any more. Last time it served as a major version number was when we released 1.0.0. In my mind, that 1 needs to go away. Do you want me to pull out the full history lesson? You know I can. So the say we current do things, the version number is 1.MAJOR.MINORfix-letter, and I'm trying to change that to MAJOR.MINOR.PATCH, where PATCH takes the role we have currently assigned to letters. Note that this is for the text form, which is separate from our numeric encoding (something that isn't covered by semver at all). That is the only place where I propose to have an epoch, and it's for one purpose only, that the value of that number should be greater in the next version number. If it wasn't for that, I would drop it entirely and just have 0 in its place in the number, i.e. the proposed 2.0.0 would be encoded to 0x00200000L (formalised encoding would then be 0x0MMNNFFSL). > We have defined the first digit as our major version number - and > changing that in my view at least would be going completely against > the principles of semantic versioning. > The version itself is meant to be purely X.Y.Z[-PRERELEASE] or > X.Y.Z[+BUILDMETA] and your suggested encoding is not that at all. > > What you have is EPOCH.X.Y.Z.FIX.STATUS - where EPOCH and STATUS are > not concepts contained within semantic versioning. I don't know how you got that into so many numbers. In 0xEMMNNFFSL, you have Epoch(E), Major(X), miNor(Y), Fix(Z) and Status(S). With Fix already finding a place in Z, I dunno why you need another FIX, and I have no idea why you want the status (prerelease number) to not be separated from the rest of the version with a dash. I'll repeat this again: semantic version Does Not Talk About Numeric Encoding At All. It only covers text. Numeric encoding is *entirely* left to us, so we're entirely free to only use MM, NN, FF and S, becoming MM.NN.FF-S (translating to semver X.Y.Z-STATUS). In my proposal, the epoch is silent in the text / semantic form of the version number. But OK, I get the message, I need to flesh out the document a bit more. > Basically adopting semantic versioning actually requires something > different to what has been proposed in my view. > > I would suggest it means our current version encoding in an integer > of MNNFFPPS becomes simply MNNFF000 and the information for PP and S > is moved elsewhere as semantic versioning defines those concepts > differently (as noted above). Just let me hammer in yet again that the numeric encoding is out of scope re semantic versioning, we're free to do what the hell we want, including have a *silent* epoch that simply doesn't show in the text form version. > Part of our challenge is ensuring we don't cause unnecessary > breakage for users: > > Vendors change the text string to add additional indicators for > their variations. > Otherwise developers use the current integer version for feature > testing - and it needs to remain compatible enough. And that, especially the last part is exactly what I'm trying to accomplish. Your proposed change of the encoded number to 0xMNNFF000L makes it incompatible. > I haven't seen any code actually testing the S field within the > version or doing anything specific with the PP version - other than > reporting it to the user. OpenSSH did check the full number for a while, at least during the 0.9.6 series (and I suspect all the 0.9.x versions). They did that because we broken API compatibility in the middle of 0.9.6. This I know for fact. Actually, they still do, but after 1.0.0, only what they have perceived as major version (including what my proposal calls the epoch): https://github.com/openssh/openssh-portable/blob/master/openbsd-compat/openssl-compat.c#L42-L67 So basically, they don't trust us to do shared library version bumping right yet... and all things considered, that's fair. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From tjh at cryptsoft.com Fri Sep 21 13:01:03 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Fri, 21 Sep 2018 23:01:03 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180921.145237.999607809821393632.levitte@openssl.org> References: <20180921.115831.1818111203113256464.levitte@openssl.org> <20180921.145237.999607809821393632.levitte@openssl.org> Message-ID: Semantic versioning is about a consistent concept of version handling. And that concept of consistency should be in a forms of the version - be it text string or numberic. That you see them as two somewhat independent concepts isn't something I support or thing makes sense at all. Our users code checks version information using the integer representation and it should be in semantic form as such - i.e. the pure numeric parts of the semantic version. This is the major point I've been trying to get across. Semantic versioning isn't about just one identifier in text format - it is about how you handle versioning in general. And consistency is its purpose. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Fri Sep 21 13:02:29 2018 From: levitte at openssl.org (Richard Levitte) Date: Fri, 21 Sep 2018 15:02:29 +0200 (CEST) Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: Message-ID: <20180921.150229.501400413700937229.levitte@openssl.org> In message on Fri, 21 Sep 2018 21:17:12 +1000, Tim Hudson said: > i.e. OPENSSL_VERSION_NUMBER should be X.Y.Z So you mean we'd lose the integer form entirely? Hey, I have zero issue with that! However, that's going to make life hard for everyone that wants to be able to build against different OpenSSL versions and use this macro in pre-processing their source. I don't think we'll get many happy faces with such a move (talk about an API break and making life hard on people). I hope I misunderstand what you're going for... (hey, if we want to drop that integer, or make it something different, maybe we should follow the POSIX example and make that YYYYMM (year+month) of the release? It's incremental and certainly usable with pre-processing! .... and less hard on our users than dropping the number altogether) > and OPENSSL_VERSION_TEXT should be "X.Y.Z [-patch][+buildmeta]" and > that would be a simple, direct, and expected mapping to OpenSSL for > semantic versioning. Semantic versioning says MAJOR.MINOR.PATCH, i.e. Z would be the patch number. Why you want to have the patch indicator where semantic versioning has pre-release information (which is exactly what we do with the -dev and -prex suffixes), I do not understand. -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Fri Sep 21 13:02:44 2018 From: matt at openssl.org (Matt Caswell) Date: Fri, 21 Sep 2018 14:02:44 +0100 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180921.145237.999607809821393632.levitte@openssl.org> References: <20180921.115831.1818111203113256464.levitte@openssl.org> <20180921.145237.999607809821393632.levitte@openssl.org> Message-ID: <55d62f6f-ef4b-686f-44ee-17ae4ef169f3@openssl.org> On 21/09/18 13:52, Richard Levitte wrote: > Note that this is for the text form, which is separate from our > numeric encoding (something that isn't covered by semver at all). > That is the only place where I propose to have an epoch, and it's for > one purpose only, that the value of that number should be greater in > the next version number. I see no reason for the epoch value to be greater. Equal is sufficient (i.e. setting it to 1 is fine). 0 would not be fine. There are two requirements for OPENSSL_VERSION_NUMBER afaict: 1) The encoded number should obey the same precedence rules as semantic versioning, so that equality and inequality tests between version numbers work as you might expect. 2) We don't break things when compared to how this version number is used in practice today Matt From matt at openssl.org Fri Sep 21 13:08:35 2018 From: matt at openssl.org (Matt Caswell) Date: Fri, 21 Sep 2018 14:08:35 +0100 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.115831.1818111203113256464.levitte@openssl.org> <20180921.145237.999607809821393632.levitte@openssl.org> Message-ID: On 21/09/18 14:01, Tim Hudson wrote: > Semantic versioning is about a consistent concept of version handling. > > And that concept of consistency should be in a forms of the version - be > it text string or numberic. > > That you see them as two somewhat independent concepts isn't something I > support or thing makes sense at all. The text form is fully consistent with semantic versioning. Richard's proposal for the integer form is also fully consistent with the text form, albeit encoded in a different way to the way you think it should be encoded. > > Our users code checks version information using the integer > representation and it should be in semantic form as such - i.e. the pure > numeric parts of the semantic version. There we disagree. Richard's integer form *is* a representation of the semantic form. There is nothing in semantic versioning that requires us to only consider the numeric parts when encoding it. > > This is the major point I've been trying to get across. Semantic > versioning isn't about just one identifier in text format - it is about > how you handle versioning in general. And consistency is its purpose. It *is* consistent IMO. Your proposal is just different - it isn't any more or less consistent - but it has the disadvantage of breaking stuff. I don't think we are going to get to agreement on this point. Matt From levitte at openssl.org Fri Sep 21 13:14:33 2018 From: levitte at openssl.org (Richard Levitte) Date: Fri, 21 Sep 2018 15:14:33 +0200 (CEST) Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: Message-ID: <20180921.151433.1906006932935413051.levitte@openssl.org> In message on Fri, 21 Sep 2018 21:28:55 +1000, Tim Hudson said: > So as a concrete example - taking master and the current OPENSSL_VERSION_TEXT value. > > "OpenSSL 1.1.2-dev xx XXX xxxx" > > would become > > "1.1.2-dev+xx.XXX.xxxx" No, it would become "1.1.2-dev" I have no idea why you want to shove the date template into the version number. > That is what I understand is the point of semantic versioning. You know how to pull apart the > version string. > -dev indicates a pre-release version known as "dev" > +xx.XXX.xxxx indicates build metadata. I haven't seen "xx XXX xxxx" as build metadata, only as a template of the release date, which is part of how we display the version (with 'openssl version'). I don't see it as part of the version string, nor have our script that do parse out the version from that string *ever* done that. You'd have to argue why we should start making it part of the version now... > The underlying release is major 1, minor 1, patch 2. No. 1.1.1 is a minor (feature) release visavi 1.1.0. 1.1.0 is a major release visavi 1.0.x. We've talked about 1.2.0 as the next major release. Semantically (!) speaking, that tells me that the middle number is the major version number, the last number is the minor version number, and the letters after that represent the patch version number. This is how we have acted, and this is how our FAQ describes our version scheme since April 2012. > But for semantic versioning where we allow API breakage in our > current minor version we would have to shift everything left. > And we would then have "1.2.0-dev+xx.XXX.xxxx" if the planned new > release wasn't guaranteed API non-breaking with the previous release > (1.1.1). Yes (except the '+xx.XXX.xxxx' part, just drop that please). But I'm not proposing that we do this change as a part of a minor release, I propose that we do this on the next major release. In other words, this would allow us to continue according to the current scheme for any 1.1.x if we would choose to release one, but that the next major release would be called 2.0.0 instead of 1.2.0, that the next minor (feature) release after that would be called 2.1.0 instead of 1.2.1, and that the fix releases would be called 2.0.1 instead of 1.2.0a, 2.1.1 instead of 1.2.1a, etc etc etc. Is this clearer to you? Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Fri Sep 21 13:19:06 2018 From: matt at openssl.org (Matt Caswell) Date: Fri, 21 Sep 2018 14:19:06 +0100 Subject: [openssl-project] Release strategy updates Message-ID: <75452fd2-9d54-3b43-8daf-24961c7af1e0@openssl.org> I am very concerned about stability of our API moving forwards. There are various discussions about changing the version number to 1.2.0 (or possibly 2.0.0) - which according to our versioning scheme would allow breaking changes. Whilst this is true I think we need to be very wary about "opening the flood gates" for breaking changes. The move from 1.0.x to 1.1.0 was hard on our users and we should avoid that again. With that in mind I have opened the following PR (based largely on wording suggested by Viktor): https://github.com/openssl/web/pull/82 At the same time I have taken the opportunity to clean up some out-of-date stuff in the release strategy. This is independent of the semantic versioning discussion which may itself see further changes being made to the release strategy. Thoughts? Matt From tjh at cryptsoft.com Fri Sep 21 13:28:01 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Fri, 21 Sep 2018 23:28:01 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180921.151433.1906006932935413051.levitte@openssl.org> References: <20180921.151433.1906006932935413051.levitte@openssl.org> Message-ID: Now I get the conceptual issue that Richard and Matt are differing on - and it is about actually replacing OpenSSL's versioning concept with semantic versioning compared to adopting semantic versioning principles without actually being precisely a semantic version approach. The whole concept of semantic versioning is that you define *precisely *what you mean by a version. Everywhere you have the concept of a version you must use the semantic form of a *version encoding*. That is the X.Y.Z[-prerelease][+buildmeta] format that is documented along with the rules for X.Y.Z in terms of your public API. And all other information about a version goes into prerelease and into buildmeta. Both prerelease and buildmeta are allowed to be a sequence of dot separated alphanumerichyphen combinations. This is the point of semantic versioning. All versions for all products are all represented with the same sort of concepts and you know what the rules are for the numeric X.Y.Z handling and the parsing rules for prerelease and buildmeta. Our concepts of versioning within OpenSSL if expressed in semantic form MUST fit into this approach. No prefixes. No suffixes. Not special additional encoding The idea is consistency. When dealing with API issues you only ever need to see X.Y.Z for any code related testing - it precisely identifies a point in time API release. There should never be any code ever that varies that requires looking at prerelease or buildmeta in order to perform any action in terms of the code. That maps to our concept of OPENSSL_VERSION_NUMBER For the human reporting we should have the fill concept which is a text string - and that should be OPENSSL_VERSION_TEXT and that means not having anything else within the text other than the actual semantic version. The syntax of the semantic version is fixed. If you want to keep the concept of a build date in the macro you are calling the version then it must be encoded in that format - or you move that concept out of the macro that is the version. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Fri Sep 21 13:35:56 2018 From: levitte at openssl.org (Richard Levitte) Date: Fri, 21 Sep 2018 15:35:56 +0200 (CEST) Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> Message-ID: <20180921.153556.2187782107479473679.levitte@openssl.org> In message on Fri, 21 Sep 2018 23:01:03 +1000, Tim Hudson said: > Semantic versioning is about a consistent concept of version handling. > > And that concept of consistency should be in a forms of the version > - be it text string or numberic. > > That you see them as two somewhat independent concepts isn't > something I support or thing makes sense at all. In that case, we should probably just thrown away OPENSSL_VERSION_NUMBER and come up with a different name. If we keep that macro around, it needs to be consistent with its semantics as we've done it since that FAQ update. Otherwise, I fear we're making life much harder on those who want to use it for pre-processing, and those who want to check the encoded version number. I do get what you're after... a clean 1:1 mapping between the version number in text form and in numeric encoding. I get that. The trouble is the incompatibilities that introduces, and I'm trying to take the middle ground. > Our users code checks version information using the integer representation and it should be in > semantic form as such - i.e. the pure numeric parts of the semantic version. > > This is the major point I've been trying to get across. Semantic versioning isn't about just one > identifier in text format - it is about how you handle versioning in general. And consistency is its > purpose. Sure. Would you mind writing up a quick proposal on a new encoding of the version? (and just so you don't limit yourself too much, it's fine by me if that includes abandoning the macro OPENSSL_VERSION_NUMBER and inventing a new one, a better one, with a definition that we can keep more consistent than our current mess) Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From openssl-users at dukhovni.org Fri Sep 21 13:40:35 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 09:40:35 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180921.115831.1818111203113256464.levitte@openssl.org> References: <20180921.115831.1818111203113256464.levitte@openssl.org> Message-ID: <71645FCF-99DC-4EA2-B8BD-673A3776E28C@dukhovni.org> I support Richard's numeric scheme as proposed, with one small change. I think that setting the epoch to 8 to set the high bit is neither necessary nor wise. Though the numeric version constant should be manifestly unsigned (UL suffix not L), and having the top 3 nibbles for the "effective" major version maximizes compatibility with prior practice for most users, making the number negative if treated as signed is not a good idea at this time. Since we're making a change to semantic versioning, I'd bump the epoch to 2. This means that some (not common) software that reconstructs the version string from the numeric constant (e.g. Postfix when warning about run-time vs. compile-time mismatch) gets something vaguely sensible: 0x2020000FL -> "2.2.0" (rather than "2.0.0") I'll have to adjust Postfix to produce better version mismatch reports (which really should not happen since the SONAME prevents running with an incompatible shared library, so perhaps remove the check, but Wietse may be difficult to convince, my problem...) > On Sep 21, 2018, at 5:58 AM, Richard Levitte wrote: > > I've worked on a proposal for an update of the OpenSSL version scheme, > to basically align ourselves with semantic versioning, most of all in > presentation, but also in spirit, while retaining numeric > compatibility. This proposal is currently in the form of a Google > Document: > > https://docs.google.com/document/d/1H3HSLxHzg7Ca3WQEU-zsOd1lUP_uJieHw5Dae34KPBs/ -- Viktor. From openssl-users at dukhovni.org Fri Sep 21 13:59:37 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 09:59:37 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180921.153556.2187782107479473679.levitte@openssl.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> Message-ID: <637F23FA-99D6-4480-817D-047642DADA37@dukhovni.org> > On Sep 21, 2018, at 9:35 AM, Richard Levitte wrote: > > In that case, we should probably just thrown away > OPENSSL_VERSION_NUMBER. Sorry, that must not t happen and there's no need. My sense is that Tim may end up "in the rough" on this issue, so unless there's more evidence of support for his "strict" interpretation, I don't think you need to consider any radical changes as yet. Just set the high nibble to 2 for backwards compatibility, and as long as we stick with semantic versioning, we never change it (at least not until OpenSSL major number 256). The "epoch" nibble is then signifies the version of the versioning scheme, and as long as we're doing semantic versioning and the major number is 255 or less, it does not change. If we wanted to make a concession to co?xistence with LibreSSL (which I do not), we could go with an initial epoch of "3" rather than 2. Personally, I think that making it clear that OPENSSL_VERSION_NUMBER is a name in the OpenSSL and not LibreSSL namespace is a good thing. LibreSSL should have stayed with "1.0.2" and encoded their version elsewhere. Their squatting on "2.x.y" is their fault, and I don't see any need to make concessions to avoid "conflicts". Software that wants to be compatible with both, and wants to use the version number to select implementations of various features, needs to make LibreSSL-specific choices only when some LibreSSL-specific macro indicates that it is compiled with LibreSSL. -- -- Viktor. From tjh at cryptsoft.com Fri Sep 21 14:07:47 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Sat, 22 Sep 2018 00:07:47 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180921.153556.2187782107479473679.levitte@openssl.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> Message-ID: I think we have to keep OPENSSL_VERSON_NUMBER and it has to have MAJOR.MINOR.FIX in it encoded as we currently have it (where FIX is PATCH in semantic terms and our current alpha PATCH is left blank). That is what I've been saying in the various emails - because we precisely need to not change the definition of what that macro is - as people interpret it. I suggest we zero out all the other information in the OPENSSL_VERSION_NUMBER macro. And I did also suggest we make the OPENSSL_VERSION_TEXT field precisely what semantic versioning would have us do - and either drop the things we have that don't fit or encode them following the rules. I would also suggest we make that macro up using macros that use the semantic version terminology directly. i.e. something like the following. And the version number is encoded that way to not break the existing usage (except that what we currently call a fix is actually semantically named a patch). One of the critically important parts of semantic versioning is that the API is precisely only about the major.minor.patch. The examples for pre-release and build-metadata are just showing that one goes first with a hyphen and can have dot separated things, the other goes second with a plus and also can have dot separated things. If we wanted to keep the date concept in the version text macro then we encode it correctly - or we can stop doing that sort of thing and leave it out. The pre-release can be blank. The build metadata can be blank. In semantic versioning terms this is what it would mean. And if you want to check release/alpha/beta status you look at the OPENSSL_VERSION_PRE_RELEASE macro and we stop the release+alpha+beta indicator usage in the OPENSSL_VERSION_NUMBER macro. It was rather limiting in its encoding format. That more rightly belongs in the semantic version string format. #include #define OPENSSL_MSTR_HELPER(x) #x #define OPENSSL_MSTR(x) OPENSSL_MSTR_HELPER(x) #define OPENSSL_VERSION_MAJOR 1 #define OPENSSL_VERSION_MINOR 1 #define OPENSSL_VERSION_PATCH 2 #define OPENSSL_VERSION_PRE_RELEASE "-beta1.special" #define OPENSSL_VERSION_BUILD_METADATA "+21Sep2018.optbuild.arm" #define OPENSSL_VERSION_NUMBER (long)((OPENSSL_VERSION_MAJOR<<28)|(OPENSSL_VERSION_MINOR<<20)|(OPENSSL_VERSION_PATCH<<12)) #define OPENSSL_VERSION_TEXT OPENSSL_MSTR(OPENSSL_VERSION_MAJOR) "." OPENSSL_MSTR(OPENSSL_VERSION_MINOR) "." OPENSSL_MSTR(OPENSSL_VERSION_PATCH) OPENSSL_VERSION_PRE_RELEASE OPENSSL_VERSION_BUILD_METADATA int main(void) { printf("0x%8lx\n",OPENSSL_VERSION_NUMBER); printf("%d.%d.%d\n",OPENSSL_VERSION_MAJOR,OPENSSL_VERSION_MINOR,OPENSSL_VERSION_PATCH); printf("%s\n",OPENSSL_VERSION_TEXT); } And the output you get: 0x10102000 1.1.2 1.1.2-beta1+21Sep2018.optbuild.arm Tim. On Fri, Sep 21, 2018 at 11:36 PM Richard Levitte wrote: > In message w2o_njr8bFOorsmw at mail.gmail.com> on Fri, 21 Sep 2018 23:01:03 +1000, Tim > Hudson said: > > > Semantic versioning is about a consistent concept of version handling. > > > > And that concept of consistency should be in a forms of the version > > - be it text string or numberic. > > > > That you see them as two somewhat independent concepts isn't > > something I support or thing makes sense at all. > > In that case, we should probably just thrown away > OPENSSL_VERSION_NUMBER and come up with a different name. If we keep > that macro around, it needs to be consistent with its semantics as > we've done it since that FAQ update. Otherwise, I fear we're making > life much harder on those who want to use it for pre-processing, and > those who want to check the encoded version number. > > I do get what you're after... a clean 1:1 mapping between the version > number in text form and in numeric encoding. I get that. The trouble > is the incompatibilities that introduces, and I'm trying to take the > middle ground. > > > Our users code checks version information using the integer > representation and it should be in > > semantic form as such - i.e. the pure numeric parts of the semantic > version. > > > > This is the major point I've been trying to get across. Semantic > versioning isn't about just one > > identifier in text format - it is about how you handle versioning in > general. And consistency is its > > purpose. > > Sure. > > Would you mind writing up a quick proposal on a new encoding of the > version? (and just so you don't limit yourself too much, it's fine by > me if that includes abandoning the macro OPENSSL_VERSION_NUMBER and > inventing a new one, a better one, with a definition that we can keep > more consistent than our current mess) > > Cheers, > Richard > > -- > Richard Levitte levitte at openssl.org > OpenSSL Project http://www.openssl.org/~levitte/ > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Fri Sep 21 14:32:46 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 10:32:46 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> Message-ID: <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> > On Sep 21, 2018, at 10:07 AM, Tim Hudson wrote: > > And the output you get: > > 0x10102000 The trouble is that existing software expects to potential ABI changes resulting from changes in the 2nd and 3rd nibbles, and if the major version is just in the first nibble, our minor version changes will look like major number changes to such software. One could take the view that software that uses the OpenSSL version number for more than inequalities is in a state of sin, and should stop doing that, and perhaps doing that is not typical application behaviour, but what Richard is trying to do is embed the semantic version number in a wider field that allows us to keep the pre-release bits (which are useful), to have an epoch nibble for versioning the version format, and also keep the "significance" of the existing nibbles with the 2nd/3rd nibble signalling major changes while the 4th/5th are minor version feature additions and 6th/7th are micro fix versions. the 8th nibble indicates dev/pre with 0xF signalling release. This does not violate semantic versioning, if I only want to support the *released* version of version 1.2.3, I'll test for >= 0x?010203FUL, with "?" the epoch nibble (2 or 3). If I am planning to test pre-release features I can compare with >= 0x?0102030UL. We might not have done it this way if this were the first even release of OpenSSL, but I think it is a find proposal. -- -- Viktor. From tjh at cryptsoft.com Fri Sep 21 15:00:29 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Sat, 22 Sep 2018 01:00:29 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> Message-ID: On Sat, Sep 22, 2018 at 12:32 AM Viktor Dukhovni wrote: > > On Sep 21, 2018, at 10:07 AM, Tim Hudson wrote: > > > > And the output you get: > > > > 0x10102000 > > The trouble is that existing software expects to potential ABI changes > resulting from changes in the 2nd and 3rd nibbles, and if the major > version is just in the first nibble, our minor version changes will > look like major number changes to such software. > The problem is that we documented a major.minor.fix.patch as our versioning scheme and semantic versioning is different. Semantic versioning does not allow for breaking changes on the minor version - and that is what we do in OpenSSL currently. Part of the requests that have come in for semantic versioning is to actually do what is "expected". Basically our significant version changes alter the minor version field in the OPENSSL_VERSION_NUMBER structure. And we have loosely referred to those as major releases but not changed the actual major version field - but the minor. The simple way to look at this: for OpenSSL-1.0.2a: - what is the major version number - the answer is clearly "1". - what is the minor version number - the answer is clearly "0" - what is the fix version number - the answer is clear "2" - what is the patch version number - the answer is clearly "a" If you repeat that in semantic versioning concepts just using the labels for mapping you get: - what is the major version number - the answer is clearly "1". - what is the minor version number - the answer is clearly "0" - what is the fix version number - there is no such thing - what is the patch version number - the answer is clearly "2" (reusing the fix version) If you repeat that in semantic versioning concepts just using the actual definitions: - what is the major version number - the answer is clearly "1". - what is the minor version number - the answer is probably "2" - what is the fix version number - there is no such thing - what is the patch version number - the answer is probably "0" Semantic versioning does not have the concept of a fix version separate from a patch version - those are just a fix version. If you add or deprecate APIs you have a new minor version. If you fix bugs without touching the API is it a patch version. If you break anything in the API is a major version. It is completely totally about the API itself. And for OpenSSL we are API compatible for most releases - except when we did the major change for 1.1.0 which in reality semantic versioning would have called 2.0.0 Semantic versioning does not have the concept of an API and a ABI distinction. In OpenSSL we have if the major.minor is the same then we are ABI compatible. That is what doesn't exist in the semantic versioning approach - it is about the API. Effectively the current "minor" version disappears. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Matthias.St.Pierre at ncp-e.com Fri Sep 21 15:13:21 2018 From: Matthias.St.Pierre at ncp-e.com (Matthias St. Pierre) Date: Fri, 21 Sep 2018 17:13:21 +0200 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180921.153556.2187782107479473679.levitte@openssl.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> Message-ID: I like Richard's approach (with the '8' or another number) and? I don't think it is contradicting semantic versioning. Maybe a good compromise between? your two opposing views would be to make the encoding irrelevant to our users by introducing version check macros like ?? ??? OPENSSL_MAKE_VERSION(maj,min,patch)??? and ?? ??? OPENSSL_VERSION_AT_LEAST(maj,min) (note: the patch level was omitted from the second macro on purpose) which enable the application programmer to write code like ??? #if OPENSSL_MAKE_VERSION(2.0.0) <= OPENSSL_VERSION_NUMBER ??????? ... ??? #endif or ??? #if OPENSSL_VERSION_AT_LEAST(2,0) ??????? ... ??? #endif This would work both for programs built for old and new openssl versions. There was a first failed attempt to introduce such macros, see the threads https://github.com/openssl/openssl/issues/5961 https://github.com/openssl/openssl/pull/5968 Matthias From openssl-users at dukhovni.org Fri Sep 21 15:16:46 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 11:16:46 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> Message-ID: <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> > On Sep 21, 2018, at 11:00 AM, Tim Hudson wrote: > > If you repeat that in semantic versioning concepts just using the labels for mapping you get: > - what is the major version number - the answer is clearly "1". > - what is the minor version number - the answer is clearly "0" > - what is the fix version number - there is no such thing > - what is the patch version number - the answer is clearly "2" (reusing the fix version) I'm afraid that's where you're simply wrong. Ever since 1.0.0, OpenSSL has promised (and I think delivered) ABI stability for the *minor* version and feature stability (bug fixes only) for the patch letters. Therefore, the semantic version number of "1.0.2a" is "1.0", its minor number is 2 and its fix number is 1 ("a"). Now of course "1.0" is not a valid semantic version number, but fortunately, we've since released "1.1" and are now considering "1.2" which with semantic versioning, is ready to become simply "2" as the "1." prefix is not needed, and "2" is conveniently larger than the current "not-really major" leading "1". > Effectively the current "minor" version disappears. No, effectively the current "major" number disappears, with the minor assuming the major role it already had. -- Viktor. From openssl-users at dukhovni.org Fri Sep 21 15:20:58 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 11:20:58 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> Message-ID: <79349BB9-8F45-4F50-B8F2-5D42F589988B@dukhovni.org> > On Sep 21, 2018, at 11:13 AM, Matthias St. Pierre wrote: > > I like Richard's approach (with the '8' or another number) and I don't think it is > contradicting semantic versioning. Maybe a good compromise between your two > opposing views would be to make the encoding irrelevant to our users by introducing > version check macros like > > OPENSSL_MAKE_VERSION(maj,min,patch) and > OPENSSL_VERSION_AT_LEAST(maj,min) > > (note: the patch level was omitted from the second macro on purpose) > > which enable the application programmer to write code like > > > #if OPENSSL_MAKE_VERSION(2,0,0) <= OPENSSL_VERSION_NUMBER > ... > #endif The macros would help new software, and should be added, but existing software should continue to work with the version-dependent bits unmodified. To that end, Richard's encoding does the job (modulo a minor quibble over the high bit vs. just an epoch nibble of 0x2 or 0x3). -- Viktor. From openssl-users at dukhovni.org Fri Sep 21 15:26:59 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 11:26:59 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> Message-ID: > On Sep 21, 2018, at 11:16 AM, Viktor Dukhovni wrote: > > I'm afraid that's where you're simply wrong. Ever since 1.0.0, OpenSSL > has promised (and I think delivered) ABI stability for the *minor* version > and feature stability (bug fixes only) for the patch letters. Therefore, > the semantic version number of "1.0.2a" is "1.0", its minor number is 2 > and its fix number is 1 ("a"). > > Now of course "1.0" is not a valid semantic version number, but fortunately, > we've since released "1.1" and are now considering "1.2" which with semantic > versioning, is ready to become simply "2" as the "1." prefix is not needed, > and "2" is conveniently larger than the current "not-really major" leading "1". To clarify, in the above I said "semantic version number" a few times where I meant to say "semantic major version number"... -- Viktor. From tjh at cryptsoft.com Fri Sep 21 15:27:28 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Sat, 22 Sep 2018 01:27:28 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> Message-ID: On Sat, Sep 22, 2018 at 1:16 AM Viktor Dukhovni wrote: > > > > On Sep 21, 2018, at 11:00 AM, Tim Hudson wrote: > > > > If you repeat that in semantic versioning concepts just using the labels > for mapping you get: > > - what is the major version number - the answer is clearly "1". > > - what is the minor version number - the answer is clearly "0" > > - what is the fix version number - there is no such thing > > - what is the patch version number - the answer is clearly "2" (reusing > the fix version) > > I'm afraid that's where you're simply wrong. Ever since 1.0.0, OpenSSL > has promised (and I think delivered) ABI stability for the *minor* version > and feature stability (bug fixes only) for the patch letters. Therefore, > the semantic version number of "1.0.2a" is "1.0", its minor number is 2 > and its fix number is 1 ("a"). > No it isn't - as you note that isn't a valid mapping - 1.0 isn't a semantic version and there is no such thing as a fix number, You get three concepts and then on top of that the pre-release and the build-metadata. Semantic versioning is about the API not the ABI. So we could redefine what we have been telling our user all along and combine our current major+minor in a new major version. Making 1.0.2a be 10.2.0 in semantic version terms. We cannot remove the current major version number - as that concept exists and we have used it all along. We don't just get to tell our users for the last 20+ years what we called the major version (which was 0 for the first half and 1 for the second half) doesn't exist. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Matthias.St.Pierre at ncp-e.com Fri Sep 21 15:34:04 2018 From: Matthias.St.Pierre at ncp-e.com (Matthias St. Pierre) Date: Fri, 21 Sep 2018 17:34:04 +0200 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> Message-ID: On 21.09.2018 17:27, Tim Hudson wrote: > > We cannot remove the current major version number - as that concept exists and we have used it all along.? > We don't just get to tell our users for the last 20+ years what we called the major version (which was 0 for the first half and 1 for the second half) doesn't exist. > There has been a famous precedent in history though:? Java dropped the "1." prefix when going from "1.4" to "5.0" (https://en.wikipedia.org/wiki/Java_version_history#Versioning_change) Matthias From openssl-users at dukhovni.org Fri Sep 21 15:39:16 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 11:39:16 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> Message-ID: > On Sep 21, 2018, at 11:27 AM, Tim Hudson wrote: > > No it isn't - as you note that isn't a valid mapping - 1.0 isn't a semantic version and there is no such thing as a fix number, > You get three concepts and then on top of that the pre-release and the build-metadata. > > Semantic versioning is about the API not the ABI. > > So we could redefine what we have been telling our user all along and combine our current major+minor in a new major version. > Making 1.0.2a be 10.2.0 in semantic version terms. Now that we've agreed on the semantic minor number, it is easy to note that 1.0.2a's micro number is "1" not 0, since the .0 release with 1.0.2 (no letter). As for the major number being "10", that's silly, we can just subtract 8, and land on 2 which is the next major number value we've not used. There's no reason to jump to 10. > We cannot remove the current major version number - as that concept exists > and we have used it all along. And it can now become "2". > We don't just get to tell our users for the last 20+ years what we called the > major version (which was 0 for the first half and 1 for the second half) doesn't > exist. It still exists, it becomes 2. And an "epoch nibble" of 0x2 in OPENSSL_VERSION_NUMBER maintains proper ordering. The users see a sane versioning scheme, that is backwards compatible. You seem to be somewhat fixated (pardon the language, no disrespect intended) on some unnecessary constraint. The encoding: 0x2UL just works to represent semantic version MM.NN.FF with == 0xF for final release, and 0x0--0xE for pre-releases. There is no disruption. The only change needed is a minor one in applications that actually parse the nibbles, ... most don't, they just use the version number as an ordinal for conditional compilation. -- Viktor. From tjh at cryptsoft.com Fri Sep 21 15:40:48 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Sat, 22 Sep 2018 01:40:48 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> Message-ID: On Sat, Sep 22, 2018 at 1:34 AM Matthias St. Pierre < Matthias.St.Pierre at ncp-e.com> wrote: > > On 21.09.2018 17:27, Tim Hudson wrote: > > > > We cannot remove the current major version number - as that concept > exists and we have used it all along. > > We don't just get to tell our users for the last 20+ years what we > called the major version (which was 0 for the first half and 1 for the > second half) doesn't exist. > > > > There has been a famous precedent in history though: Java dropped the > "1." prefix when going from "1.4" to "5.0" > (https://en.wikipedia.org/wiki/Java_version_history#Versioning_change) > Unfortunately that isn't a good example - as the version number didn't actually change - just the marketing and download packaging. The APIs continue to return 1.5.0 etc. And many Java developers continue to use the real version numbers. That is something I wouldn't suggest makes sense as an approach - to change the tarfile name and leave all the internals the same achieves nothing. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Fri Sep 21 15:53:46 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 11:53:46 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> Message-ID: <09E7EDE2-5BC8-48B2-96D9-F2B837564812@dukhovni.org> > On Sep 21, 2018, at 11:40 AM, Tim Hudson wrote: > > That is something I wouldn't suggest makes sense as an approach - to change the tarfile name and leave all the internals the same achieves nothing. And that's not the proposal. The proposal is that the new major number is 2 (or 3 if someone sees a compelling need to skip over LibreSSL), changing not just the marketing name but also the internal version. The *integer* representing the combined major.minor.micro (+ status) is not a data structure. Its encoding has changed multiple times over the years, and will now change once more, in a way that preserves order. That's all. We'll just need to update: https://www.openssl.org/docs/manmaster/man3/OPENSSL_VERSION_NUMBER.html to indicate a new encoding as of 0x20000000UL and up. -- Viktor. From tjh at cryptsoft.com Fri Sep 21 15:56:14 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Sat, 22 Sep 2018 01:56:14 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> Message-ID: On Sat, Sep 22, 2018 at 1:39 AM Viktor Dukhovni wrote: > The only change needed is a minor one in applications that actually > parse the nibbles What I was suggesting is that we don't need to break the current encoding at all. We have a major.minor.fix version encoded and documented in the header file that we can continue to represent in semantic terms. So anyone decoding those items will actually get the correct result (from a user perspective). Those using it for conditional compilation checks continue to work. BTW thanks for the correction on "a" to "1" for the patch release (wasn't being careful there). Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Fri Sep 21 16:04:48 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 12:04:48 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> Message-ID: > On Sep 21, 2018, at 11:56 AM, Tim Hudson wrote: > > What I was suggesting is that we don't need to break the current encoding at all. Not changing the encoding has a downside. * The bits that represent ABI stability would shift from the 2nd/3rd nibbles to just the first nibble. * We lose the option of changing the encoding in the future unless we start requiring 64-bit longs. * We end up with 3 status nibbles, two of which some applications may misinterpret as holding a patch level: 0xUL On the whole maintaining the current placement of the major number in the encoding makes it less not more natural for holding the new semantic versions in a backwards-compatible way. I think I've said everything I have to say on this topic. So I'll stop for now. I continue to support Richard's proposal, but with an epoch smaller than 8. -- Viktor. From matt at openssl.org Fri Sep 21 16:14:27 2018 From: matt at openssl.org (Matt Caswell) Date: Fri, 21 Sep 2018 17:14:27 +0100 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> Message-ID: <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> On 21/09/18 17:04, Viktor Dukhovni wrote: > I think I've said everything I have to say on this topic. So I'll stop > for now. I continue to support Richard's proposal, but with an epoch > smaller than 8. > To summarise my position: I support Richard's proposal with an epoch of 1. Grudgingly I would accept an epoch in the 3-8 range. I would oppose an epoch of 2. Matt From openssl-users at dukhovni.org Fri Sep 21 16:29:43 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 12:29:43 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> Message-ID: <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> > On Sep 21, 2018, at 12:14 PM, Matt Caswell wrote: > > I support Richard's proposal with an epoch of 1. > Grudgingly I would accept an epoch in the 3-8 range. > I would oppose an epoch of 2. I can live with that, though it might mean that a minority of applications will interpret (based on obsolete nibble extraction) that OpenSSL 2.0.0 (0x1020000FUL) is actually OpenSSL 1.2.0, but that's probably harmless. It does mean we might never reclaim the version numbers with 0x2 for the high nibble. -- Viktor. From matt at openssl.org Fri Sep 21 16:35:46 2018 From: matt at openssl.org (Matt Caswell) Date: Fri, 21 Sep 2018 17:35:46 +0100 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> Message-ID: On 21/09/18 17:29, Viktor Dukhovni wrote: > > >> On Sep 21, 2018, at 12:14 PM, Matt Caswell wrote: >> >> I support Richard's proposal with an epoch of 1. >> Grudgingly I would accept an epoch in the 3-8 range. >> I would oppose an epoch of 2. > > I can live with that, though it might mean that a minority of > applications will interpret (based on obsolete nibble extraction) > that OpenSSL 2.0.0 (0x1020000FUL) is actually OpenSSL 1.2.0, but > that's probably harmless. It does mean we might never reclaim the > version numbers with 0x2 for the high nibble. > I'd further add that I don't think we should change the version number until we have agreement on the stability principles: https://github.com/openssl/web/pull/82 Matt From tjh at cryptsoft.com Fri Sep 21 16:50:08 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Sat, 22 Sep 2018 02:50:08 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> Message-ID: On Sat, 22 Sep. 2018, 2:29 am Viktor Dukhovni, wrote: > > > > On Sep 21, 2018, at 12:14 PM, Matt Caswell wrote: > > > > I support Richard's proposal with an epoch of 1. > > Grudgingly I would accept an epoch in the 3-8 range. > > I would oppose an epoch of 2. > > I can live with that, though it might mean that a minority of > applications will interpret (based on obsolete nibble extraction) > It isn't obsolete nibble extraction - that is the documented way to get the major version. In the header file and in the API docs. This is changing things as we document the format of that number currently and some code does use that. What Richard proposes is a breaking change conceptually and it doesn't need to be one. We can get to semantic versioning and also not break existing documented usage. But what we will be doing is following a versioning scheme that states precisely what it means for API changes. It says nothing specifically about ABI changes unless you take the view that ABI == API when reading the document too. If that is the case then our current practice of allowing ABI breakage with minor release changes (the middle number we document as the minor release number) has to change. And anyone depending on checking the minor version number for ABI compatibility will break. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Fri Sep 21 17:24:33 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 13:24:33 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> Message-ID: <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> > On Sep 21, 2018, at 12:50 PM, Tim Hudson wrote: > If that is the case then our current practice of allowing ABI breakage with > minor release changes (the middle number we document as the minor release number) > has to change. CORRECTION: As advertised when 1.0.0 first came out, and repeated in our release policy: As of release 1.0.0 the OpenSSL versioning scheme was improved to better meet developers' and vendors' expectations. Letter releases, such as 1.0.2a, exclusively contain bug and security fixes and no new features. Minor releases that change the last digit, e.g. 1.1.0 vs. 1.1.1, can and are likely to contain new features, but in a way that does not break binary compatibility. This means that an application compiled and dynamically linked with 1.1.0 does not need to be recompiled when the shared library is updated to 1.1.1. It should be noted that some features are transparent to the application such as the maximum negotiated TLS version and cipher suites, performance improvements and so on. There is no need to recompile applications to benefit from these features. The text says that the *last* digit changes in "minor releases". Presently, the combination of the first and 2nd digits are effectively the major release number (1.1 at the moment). Indeed well before this discussion (for a few years now), I've been supporting OpenSSL at $WORK by installing both 1.0.2x and 1.1.0x in: /opt/openssl/1.0/ - 1.0.2x directory prefix /opt/openssl/1.1/ - 1.1.0x directory prefix and I plan to upgrade to shortly upgrade the 1.1.0x tree to 1.1.1x in place, without changing the prefix. The present major versions are clearly two-digits, with the "1." not carrying any substantive meaning. Your proposal gives us only one nibble for the major release, and forces bumps in that nibble whenever the ABI changes. > And anyone depending on checking the minor version number for ABI compatibility will break. The 2nd/3rd nibbles in OPENSSL_VERSION_NUMBER are not "the minor version number", they are just some bits in an ordinal. The mapping from that ordinal to the .../major/minor/micro/patch/status/... has evolved in the past and can evolve again. Applications don't generally extract these nibbles at all, they only compare ordinals at compile time as an alternative to dynamic (autoconf) feature test macros. We can at some point provide functions to return extracted major/minor/micro versions (for the runtime versions) and macros (for the compile-time versions). Which then abstract-away the encoding, which retains meaning only as an ordinal, and we're still free to change the encoding when requirements change, provided we don't break strict linear ordering. -- Viktor. From levitte at openssl.org Fri Sep 21 20:01:43 2018 From: levitte at openssl.org (Richard Levitte) Date: Fri, 21 Sep 2018 22:01:43 +0200 (CEST) Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> References: <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> Message-ID: <20180921.220143.1977024067978896739.levitte@openssl.org> In message <22DFFD58-5686-4FD6-BBB7-00512CC88035 at dukhovni.org> on Fri, 21 Sep 2018 12:29:43 -0400, Viktor Dukhovni said: > > > > On Sep 21, 2018, at 12:14 PM, Matt Caswell wrote: > > > > I support Richard's proposal with an epoch of 1. > > Grudgingly I would accept an epoch in the 3-8 range. > > I would oppose an epoch of 2. > > I can live with that, though it might mean that a minority of > applications will interpret (based on obsolete nibble extraction) > that OpenSSL 2.0.0 (0x1020000FUL) is actually OpenSSL 1.2.0, but > that's probably harmless. It does mean we might never reclaim the > version numbers with 0x2 for the high nibble. The applications who currently care since the release of 1.0.0 use 0xfff0000fL as a mask, meaning that they might as well view 0x102 (258 decimal) as the internal major version number. I haven't seen any example where anyone cares about the exact nibbles, just that 0xfff00000L gives them something to compare. For pre-processing, that might be different, I have no clue how people figure out the exact number to compare OPENSSL_VERSION_NUMBER... I assume, though, that they follow the path of least resistance and simply pick the current number from opensslv.h. -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From openssl-users at dukhovni.org Fri Sep 21 20:09:24 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 16:09:24 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180921.220143.1977024067978896739.levitte@openssl.org> References: <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> <20180921.220143.1977024067978896739.levitte@openssl.org> Message-ID: <00BD925F-CBF3-4B55-AB14-3DF7AF083F1E@dukhovni.org> > On Sep 21, 2018, at 4:01 PM, Richard Levitte wrote: > > For pre-processing, that might be different, I have no clue how people > figure out the exact number to compare OPENSSL_VERSION_NUMBER... I > assume, though, that they follow the path of least resistance and > simply pick the current number from opensslv.h. I've always used the actual number from the final (0xf in the status nibble) release that introduced the feature. This is basically copying the value from , though often I know what it is and don't need to look. -- Viktor. From levitte at openssl.org Fri Sep 21 20:55:05 2018 From: levitte at openssl.org (Richard Levitte) Date: Fri, 21 Sep 2018 22:55:05 +0200 (CEST) Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.153556.2187782107479473679.levitte@openssl.org> Message-ID: <20180921.225505.1801618997584155803.levitte@openssl.org> In message on Sat, 22 Sep 2018 00:07:47 +1000, Tim Hudson said: > In semantic versioning terms this is what it would mean. > And if you want to check release/alpha/beta status you look at the > OPENSSL_VERSION_PRE_RELEASE macro and we stop the release+alpha+beta > indicator usage in the OPENSSL_VERSION_NUMBER macro. > It was rather limiting in its encoding format. That more rightly > belongs in the semantic version string format. > > #include > > #define OPENSSL_MSTR_HELPER(x) #x > #define OPENSSL_MSTR(x) OPENSSL_MSTR_HELPER(x) > > #define OPENSSL_VERSION_MAJOR 1 > #define OPENSSL_VERSION_MINOR 1 > #define OPENSSL_VERSION_PATCH 2 > #define OPENSSL_VERSION_PRE_RELEASE "-beta1.special" > #define OPENSSL_VERSION_BUILD_METADATA "+21Sep2018.optbuild.arm" I like this. This makes everything very clear to me > #define OPENSSL_VERSION_NUMBER (long)((OPENSSL_VERSION_MAJOR<<28)| > (OPENSSL_VERSION_MINOR<<20)|(OPENSSL_VERSION_PATCH<<12)) I think we need to get rid of OPENSSL_VERSION_NUMBER. It's very obvious that there are different and quite confusing view of what that number represents. It makes the number rubbish... I've heard your arguments for your view, and you've been hammered with my view, so here's the summary as I see it: 1. There's how we have acted and what we have told our users. That corresponds to the FAQ entry that was introduced in April 2012 and the spelled out how things were from 1.0.0, which was released in March 2010. This part supports the view that our versions are 1.MAJOR.MINORpatch (where 'patch' is a letter). 2. There's a comment in opensslv.h and documentation in OPENSSL_VERSIO_NUMBER.pod that both stipulates that the version number is in fact MAJOR.MINOR.FIXpatch (where 'patch' is a letter). Just for reference, I'll point out that the last updates that affected the encoded version number format was in July 2001 (which is between releases 0.9.7 and 0.9.8) and September 2000 (by yours truly, somewhere between 0.9.5 and 0.9.6). In both descriptions, and with the way we have encoded the version as a number turns out to be the same, if we stop looking too hard at the labels. For clarity, I'll just rename them to X.Y.Zp ('p' for the patch letter), so whether you interpret 1.0.2a according to the first or the second interpretation, it has numerically been translated to the same, 0x1000201fL (*). I hope we can all agree so far. So in essence, we have two competting views on the current semantics of our versioning, where one is about the documentation in opensslv.h comment and in pod, while the other is about an FAQ entry and how we actually acted. Neither of these two views are untrue per se, however one is more current than the other (frankly, considering when they were last updated and that we didn't follow that form in practice, I consider the documentation in opensslv.h and pod outdated). I wouldn't mind going back to acting according to the docs, not in the least! The only thing that troubles me there is that the mask people used to compare the "major" part of the numeric encoding of the version will have to change, i.e. it's in fact going to break what we have told everyone since at least 2012. And it seems to be quite controversial to try and adapt to current usage, so perhaps the wiser thing is to get rid of OPENSSL_VERSION_NUMBER and start over with a differently named macro. The three macro solution Tim presented is good enough in my view. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From openssl-users at dukhovni.org Fri Sep 21 21:23:35 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 17:23:35 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180921.225505.1801618997584155803.levitte@openssl.org> References: <20180921.153556.2187782107479473679.levitte@openssl.org> <20180921.225505.1801618997584155803.levitte@openssl.org> Message-ID: > On Sep 21, 2018, at 4:55 PM, Richard Levitte wrote: > > I think we need to get rid of OPENSSL_VERSION_NUMBER. Absolutely not. That's the one thing we must keep, and keep monotone so that applications continue to compile and build. Sure we need to communicate our ABI/API stability policies, and publish release numbers and all that, but the API must provide a sensible backwards compatible macro for testing whether the compile-time version exceeds various threshold values. We should also provide various helper macros, but NOT remove the legacy encoded number. That's a non-starter. -- Viktor. From tjh at cryptsoft.com Sat Sep 22 01:28:44 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Sat, 22 Sep 2018 11:28:44 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> Message-ID: On Sat, 22 Sep. 2018, 3:24 am Viktor Dukhovni, wrote: > > On Sep 21, 2018, at 12:50 PM, Tim Hudson wrote: > > If that is the case then our current practice of allowing ABI breakage > with > > minor release changes (the middle number we document as the minor > release number) > > has to change. > CORRECTION: As advertised when 1.0.0 first came out, and repeated in our > release > policy: > As of release 1.0.0 the OpenSSL versioning scheme was improved > to better meet developers' and vendors' expectations. Letter > releases, such as 1.0.2a, exclusively contain bug and security > fixes and no new features. Minor releases that change the > last digit, e.g. 1.1.0 vs. 1.1.1, can and are likely to > contain new features, but in a way that does not break binary > compatibility. This means that an application compiled and > dynamically linked with 1.1.0 does not need to be recompiled > when the shared library is updated to 1.1.1. It should be > noted that some features are transparent to the application > such as the maximum negotiated TLS version and cipher suites, > performance improvements and so on. There is no need to > recompile applications to benefit from these features. > I have to disagree here. We said things about *minor releases* and *major releases* but we didn't say things about the version numbers or change the documentation or code comments to say the first digit was meaningless and that we have shifted the definition of what X.Y.Z means. That parsing of history I think is at best a stretch and not supportable and also not what our *users* think. Our users don't call 1.0.2 the 0.2 release of OpenSSL. Our users don't call 1.0.0 the 0.0 release. There isn't a short hand or acceptance or a decision communicated to conceptually drop the 1. off the front of our versioning. Your logic and practice is saying you see the first two digits as the major version number - that's fine - you are welcome to take an ABI compatible short hand to refer to version - but that doesn't mean we changed the definition of our versioning system. What you are tracking there is effectively the SHLIB version. So if our users don't think that, and our code comments don't stay that, and our pod documentation doesn't say that and users have the first part in their masks and don't just ignore it then I don't think it is supportable to claim we told everyone it was a meaningless first digit and changed our definition of our versioning scheme back at the 1.0.0 release. Currently when we make minor API changes that aren't breaking we update the fix version number. When we make major API changes which we expect to be breaking we update the minor version number. Now think about the transition from 1.1.0 to 1.1.1 - that is by any reasonable definition *a major release* - but we don't update the major version number by either definition of the major version number. We call it in our website a "feature release" - yet more terminology to dance around our version numbering scheme. Read the actual blog post and try to parse that as *a minor release* - it isn't - it is *a major release* - but we did not change the *major release number *even if we accepted the theory and your definition that the first number is meaningless and it is only the second number that is the real major version and the third number isn't a fix version is it actually the minor version. The implications of that view point aren't supported by our actions. We did not say we are redefining the concept of our release numbering scheme - we just defined what API impacts there were and we used "major" and "minor" about *release efforts and impacts* - not about changing the definition of the parts of the OPENSSL_VERSION_NUMBER macro and the corresponding meaning of what SSLeay_version_num() and now OpenSSL_version_num() returns. This is a core part of our API. Remember that SSLeay_version_num() was only renamed OpenSSL_version_num() in 2015 in commit b0700d2c8de79252ba605748a075cf2e5d670da1 as part of 1.1.0 The first update for the top nibble indicating the major version number had changed was back in 2009 in commit 093f5d2c152489dd7733dcbb68cbf654988a496c which is when 1.0.0 started. Saying that our documentation in doc/crypto/OPENSSL_VERSION_NUMBER.pod was just forgotten to be updated and happens to be wrong (in order to support that view point) isn't matched by the actual update history - it was updated in 2014 in commit 9208640a36228b10fcdf75c8853d9410aaff19a3 and it actually changed the documentation of what the major version number encoding is - from the top two nibbles to just the top nibble. If your assertions about the history were accurate then I would expect to see a comment there that the top nibble is a meaningless epoch and the major version is encoded elsewhere. But what was done there was to *bring the documentation in line with the code comment* and also in line with reality and what our users perceive things as. > The 2nd/3rd nibbles in OPENSSL_VERSION_NUMBER are not "the minor version > number", > they are just some bits in an ordinal. Our documentation says otherwise. Our code comments say otherwise. Our users (I assert) say otherwise. And your rationale for this in the website statements in their plain reading also support that - you have to insert "number" in quite a few places in order to read the context of those posts as meaning we have a change in the encoding scheme for numbers. We didn't do that. So in summary, don't mix major and minor in the sense of the scope of a change in with the major version number and minor version number - that is I think where the confusion comes from in terms of your interpretation (and Richard's proposal). We did not explicitly disconnect those concepts from their encoding in OPENSSL_VERSION_NUMBER, we did not redefine what we meant by version. We also did not say the first two digits are the major version number and the next digit is the minor version number. One of those two leaps has to be made to support your view point and I see no evidence to actually support that without inserting meaning into the release strategy that isn't actually there and conflicts with multiple other sources of information. Remember our users read the code and sometimes the documentation when using OpenSSL - a user doesn't go and read our release strategy to see if there is some wording that happens to have updated meaning. And for amusement value, I reference the "ultimate source of truth" - our entry on Wikipedia which lists our "major version releases" using the same sort of loose release effort scope definition with no connection to the major version number. For wikipedia, all major.minor.fix releases are "major version releases" - and that I think actually matches our users view point. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Sat Sep 22 01:55:14 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 21 Sep 2018 21:55:14 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> Message-ID: <39463F9F-1AE7-4E16-85F2-2AE147C383E7@dukhovni.org> > On Sep 21, 2018, at 9:28 PM, Tim Hudson wrote: > > That parsing of history I think is at best a stretch and not supportable and also not what our users think. This isn't a mathematical theorem or even a legal debate. We should do what makes the most sense going forward. Richar proposes, and I agree that it makes more sense to leave room for two nibbles of major number, and that for better or worse applications expect to see the same top 3 nibbles in a given stable ABI. The proposal to lose many of the low bits in the ordinal, and lose the status bits, ... is just not appealing, and there is no API/ABI reason to retain the verbatim mapping from notional major/minor/micro/... into the ordinal, this is an ad-hoc encoding with monitonicity as the the only constraint. I don't agree that anything about semantic versioning, or our past practice constrains the encoding beyond that sole constraint. So I'm afraid there are no arguments that will convince me that what appears to be a less flexible encoding is somehow a forced choice. We'll have to disagree on this one, and see where the consensus falls. -- Viktor. From tjh at cryptsoft.com Sat Sep 22 04:50:43 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Sat, 22 Sep 2018 14:50:43 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <39463F9F-1AE7-4E16-85F2-2AE147C383E7@dukhovni.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> <39463F9F-1AE7-4E16-85F2-2AE147C383E7@dukhovni.org> Message-ID: On Sat, Sep 22, 2018 at 11:55 AM Viktor Dukhovni wrote: > this is an ad-hoc encoding with monitonicity as the > the only constraint. > If you start from the position that the encoding of OPENSSL_VERSION_NUMBER is free to change so long as the resulting value is larger than what we have been using for all previous versions then a whole range of options come into play. But we shouldn't assert that we aren't changing the meaning of OPENSSL_VERSION_NUMBER - and that is what others have been doing on this thread - and what your previous email also asserted. It is a *breaking* change in our comments and our documentation and in what our users are expecting. Basically it is an API and ABI change - we said what the number means and we are changing that. The impact of the breaking change for those using it for pure feature testing for API difference handling (where it isn't actually parsed) can be minimised just by always having a larger number that all previous uses. The impact of the breaking change on anyone actually following our documented encoding cannot. i.e. openssh as one example Richard pointed out. That isn't the only code that actually believes our header files and documentation :-) Semantic versioning is about MAJOR.MINOR.PATCH with specific meanings. There is no FIX concept as such. There is PRERELEASE and METADATA. One of our existing concepts disappears - we have MAJOR.MINOR.FIX.PATCH currently and we also encode STATUS as a concept (which we can map mostly to PRERELEASE). And if we are changing to fit into semantic versioning we need to use its concepts and naming and not our present ones. A merge of concepts pretty much goes against the point of semantic versioning. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Sat Sep 22 04:59:02 2018 From: levitte at openssl.org (Richard Levitte) Date: Sat, 22 Sep 2018 06:59:02 +0200 (CEST) Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> Message-ID: <20180922.065902.132678184516757017.levitte@openssl.org> This was a lot. Can I start with asking that we stop referring to enigmatic users? They are diverse and so are their views on this (if not just utter confusion when they can't make sense of how our version numbers change... The Wikipedia article may be reflection of that, or it might be that its authors are smarter than us). The message I get from all of this -- *and this is something I agree with* -- is that we've treated the semantics of our version number wrong ever since 1.0.0 was released, or at least since that FAQ entry came up. Or to put it differently, our actions and how we have talked about major releases and minor / feature releases don't match the documentation in opensslv.h comment and in pod on how our versioning works. I also get the message that semantic versioning per se is good and that we actually have something close according to documentation. I have not seen anyone saying that we shouldn't go with semantic versioning, even though we may have different views on timing. It seems to me that this boils down to a disagreement over technicalities, how to use OPENSSL_VERSION_NUMBER, what bits to look at to figure out compatibility with what the application was built against. (this reminded me that there are those who dlopen libcrypto and libssl rather than linking with it at build time... it makes sense to compare numbers in that case) So in summary, do we agree on this, and that it's a good path forward? - semantic versioning scheme good, we should adopt it. - we need to agree on how to translate that in code. - we need to stop fighting about history. Cheers, Richard In message on Sat, 22 Sep 2018 11:28:44 +1000, Tim Hudson said: > On Sat, 22 Sep. 2018, 3:24 am Viktor Dukhovni, wrote: > > > On Sep 21, 2018, at 12:50 PM, Tim Hudson wrote: > > If that is the case then our current practice of allowing ABI breakage with > > minor release changes (the middle number we document as the minor release number) > > has to change. > CORRECTION: As advertised when 1.0.0 first came out, and repeated in our release > policy: > As of release 1.0.0 the OpenSSL versioning scheme was improved > to better meet developers' and vendors' expectations. Letter > releases, such as 1.0.2a, exclusively contain bug and security > fixes and no new features. Minor releases that change the > last digit, e.g. 1.1.0 vs. 1.1.1, can and are likely to > contain new features, but in a way that does not break binary > compatibility. This means that an application compiled and > dynamically linked with 1.1.0 does not need to be recompiled > when the shared library is updated to 1.1.1. It should be > noted that some features are transparent to the application > such as the maximum negotiated TLS version and cipher suites, > performance improvements and so on. There is no need to > recompile applications to benefit from these features. > > I have to disagree here. We said things about minor releases and major releases but we didn't say > things about the version numbers or change the documentation or code comments to say the first > digit was meaningless and that we have shifted the definition of what X.Y.Z means. > > That parsing of history I think is at best a stretch and not supportable and also not what our users > think. > > Our users don't call 1.0.2 the 0.2 release of OpenSSL. Our users don't call 1.0.0 the 0.0 release. > There isn't a short hand or acceptance or a decision communicated to conceptually drop the 1. off > the front of our versioning. > Your logic and practice is saying you see the first two digits as the major version number - that's > fine - you are welcome to take an ABI compatible short hand to refer to version - but that doesn't > mean we changed the definition of our versioning system. > What you are tracking there is effectively the SHLIB version. > > So if our users don't think that, and our code comments don't stay that, and our pod > documentation doesn't say that and users have the first part in their masks and don't just ignore it > then I don't think it is supportable to claim we told everyone it was a meaningless first digit and > changed our definition of our versioning scheme back at the 1.0.0 release. > > Currently when we make minor API changes that aren't breaking we update the fix version > number. When we make major API changes which we expect to be breaking we update the minor > version number. > Now think about the transition from 1.1.0 to 1.1.1 - that is by any reasonable definition a major > release - but we don't update the major version number by either definition of the major version > number. > We call it in our website a "feature release" - yet more terminology to dance around our version > numbering scheme. > Read the actual blog post and try to parse that as a minor release - it isn't - it is a major > release - but we did not change the major release number even if we accepted the theory and > your definition that the first number is meaningless and it is only the second number that is the > real major version and the third number isn't a fix version is it actually the minor version. The > implications of that view point aren't supported by our actions. > > We did not say we are redefining the concept of our release numbering scheme - we just defined > what API impacts there were and we used "major" and "minor" about release efforts and > impacts - not about changing the definition of the parts of the OPENSSL_VERSION_NUMBER > macro and the corresponding meaning of what SSLeay_version_num() and now > OpenSSL_version_num() returns. This is a core part of our API. > > Remember that SSLeay_version_num() was only renamed OpenSSL_version_num() in 2015 in > commit b0700d2c8de79252ba605748a075cf2e5d670da1 as part of 1.1.0 > The first update for the top nibble indicating the major version number had changed was back in > 2009 in commit 093f5d2c152489dd7733dcbb68cbf654988a496c which is when 1.0.0 started. > > Saying that our documentation in doc/crypto/OPENSSL_VERSION_NUMBER.pod was just forgotten > to be updated and happens to be wrong (in order to support that view point) isn't matched by the > actual update history - it was updated in 2014 in commit > 9208640a36228b10fcdf75c8853d9410aaff19a3 and it actually changed the documentation of > what the major version number encoding is - from the top two nibbles to just the top nibble. If your > assertions about the history were accurate then I would expect to see a comment there that the > top nibble is a meaningless epoch and the major version is encoded elsewhere. But what was done > there was to bring the documentation in line with the code comment and also in line with > reality and what our users perceive things as. > > The 2nd/3rd nibbles in OPENSSL_VERSION_NUMBER are not "the minor version number", > they are just some bits in an ordinal. > > Our documentation says otherwise. Our code comments say otherwise. Our users (I assert) say > otherwise. > And your rationale for this in the website statements in their plain reading also support that - you > have to insert "number" in quite a few places in order to read the context of those posts as > meaning we have a change in the encoding scheme for numbers. > We didn't do that. > > So in summary, don't mix major and minor in the sense of the scope of a change in with the major > version number and minor version number - that is I think where the confusion comes from in > terms of your interpretation (and Richard's proposal). > We did not explicitly disconnect those concepts from their encoding in > OPENSSL_VERSION_NUMBER, we did not redefine what we meant by version. > We also did not say the first two digits are the major version number and the next digit is the > minor version number. > > One of those two leaps has to be made to support your view point and I see no evidence to > actually support that without inserting meaning into the release strategy that isn't actually there > and conflicts with multiple other sources of information. > Remember our users read the code and sometimes the documentation when using OpenSSL - a > user doesn't go and read our release strategy to see if there is some wording that happens to have > updated meaning. > > And for amusement value, I reference the "ultimate source of truth" - our entry on Wikipedia > which lists our "major version releases" using the same sort of loose release effort scope definition > with no connection to the major version number. > For wikipedia, all major.minor.fix releases are "major version releases" - and that I think actually > matches our users view point. > > Tim. > From openssl-users at dukhovni.org Sat Sep 22 05:02:29 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Sat, 22 Sep 2018 01:02:29 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180922.065902.132678184516757017.levitte@openssl.org> References: <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> <20180922.065902.132678184516757017.levitte@openssl.org> Message-ID: <056A59CE-2F19-43AE-B2E8-290ECB3D0D77@dukhovni.org> > On Sep 22, 2018, at 12:59 AM, Richard Levitte wrote: > > So in summary, do we agree on this, and that it's a good path forward? > > - semantic versioning scheme good, we should adopt it. > - we need to agree on how to translate that in code. > - we need to stop fighting about history. Yes. -- Viktor. From openssl-users at dukhovni.org Sat Sep 22 05:12:21 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Sat, 22 Sep 2018 01:12:21 -0400 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> <39463F9F-1AE7-4E16-85F2-2AE147C383E7@dukhovni.org> Message-ID: <27A0FBB1-F577-4A3C-A62A-947D6F251A63@dukhovni.org> > On Sep 22, 2018, at 12:50 AM, Tim Hudson wrote: > > The impact of the breaking change on anyone actually following our documented encoding cannot. > i.e. openssh as one example Richard pointed out. The only use of OPENSSL_VERSION_NUMBER bits in OpenSSH (which is not yet ported to 1.1.x upstream BTW, so hardly relevant really) is: ssh_compatible_openssl(long headerver, long libver) { long mask, hfix, lfix; /* exact match is always OK */ if (headerver == libver) return 1; /* for versions < 1.0.0, major,minor,fix,status must match */ if (headerver < 0x1000000f) { mask = 0xfffff00fL; /* major,minor,fix,status */ return (headerver & mask) == (libver & mask); } /* * For versions >= 1.0.0, major,minor,status must match and library * fix version must be equal to or newer than the header. */ mask = 0xfff0000fL; /* major,minor,status */ hfix = (headerver & 0x000ff000) >> 12; lfix = (libver & 0x000ff000) >> 12; if ( (headerver & mask) == (libver & mask) && lfix >= hfix) return 1; return 0; } all other uses as a simple ordinal. In the above function they expect stability of the ABI for matching first three nibbles and release status. Which makes a case for Richard's encoding scheme as being more compatible with one of the more prominent applications that depends on the encoding. The proposal to move the minor version into nibbles 2 and 3 breaks this OpenSSH function. -- Viktor. From tjh at cryptsoft.com Sat Sep 22 05:22:49 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Sat, 22 Sep 2018 15:22:49 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180922.065902.132678184516757017.levitte@openssl.org> References: <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> <20180922.065902.132678184516757017.levitte@openssl.org> Message-ID: If you accept that we have a MAJOR.MINOR.FIX.PATCH encoding currently documented and in place and that the encoding in OPENSSL_VERSION_NUMBER is documented then the semantic versioning is an easy change. We do not need to change our encoding of MAJOR.MINOR - that is documented and fixed. We just need to start using it the way semantic versioning says we should. We need to eliminate FIX - there is no such concept in semantic versioning. And PATCH exists as a concept but in semantic versioning terms it is a number. That leads to two main choices in our current encoding as to how we achieve that: 1) leave old FIX blank 2) move old PATCH to old FIX and leave old PATCH blank (i.e. old FIX is renamed to new PATCH and old PATCH is eliminated) Option 2 offers the *least surprise* to users - in that it is how most users already think of OpenSSL in terms of a stable API - i.e. our current MAJOR.MINOR.FIX is actually read as MAJOR.MINOR.PATCH or to be more precise our users see FIX and PATCH as combined things - they don't see our MAJOR and MINOR as combined things. Under option 2 it does mean anyone that thinks a change of our current MINOR means an ABI break would be interpreting things incorrectly - but that interpretation is *entirely safe* - in that they would be assuming something more conservative rather than less conservative. And leaving the old PATCH blank doesn't hurt anyone. I don't think that moving to semantic versioning requires us to change our encoding of OPENSSL_VERSION_NUMBER except to move PATCH to FIX - as one of those concepts disappears. And then when we do a major release update (like 1.1.1) we end up with a MAJOR version change. Alternatively, we can elect to effectively say the OPENSSL_VERSION_NUMBER encoding we have documented and supported is subject to yet another change where we reinterpret things. That is doable - but I also think entirely unnecessary and confusing for our users. Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjh at cryptsoft.com Sat Sep 22 05:30:13 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Sat, 22 Sep 2018 15:30:13 +1000 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <27A0FBB1-F577-4A3C-A62A-947D6F251A63@dukhovni.org> References: <20180921.145237.999607809821393632.levitte@openssl.org> <20180921.153556.2187782107479473679.levitte@openssl.org> <7DFA82AE-9E7D-4A77-A503-472842837122@dukhovni.org> <68676EFA-A7E8-4546-8D14-75E311E58128@dukhovni.org> <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> <39463F9F-1AE7-4E16-85F2-2AE147C383E7@dukhovni.org> <27A0FBB1-F577-4A3C-A62A-947D6F251A63@dukhovni.org> Message-ID: On Sat, Sep 22, 2018 at 3:12 PM Viktor Dukhovni wrote: > The proposal to move the minor version into nibbles 2 and 3 breaks this > OpenSSH function. > No it doesn't - because I'm not talking about moving *anything* in the current encoding for major and minor - see earlier post. The positions don't change for minor version. We just stop using the current PATCH and use the current FIX as PATCH. And the logical test there remains valid in that it detects all incompatible versions correctly - what changes is that some versions that are compatible are seen as incompatible - but that is an incorrect interpretation that is *safe.* And note that the openssh code there is actually more conservative than it needs to be already. And under semantic versioning, it is only the MAJOR that changes when breaking changes happen. But what I've been referring to is that there is other code that uses our documented encoding and parses it ... this isn't just an openssh issue. Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Sat Sep 22 05:59:55 2018 From: levitte at openssl.org (Richard Levitte) Date: Sat, 22 Sep 2018 07:59:55 +0200 (CEST) Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <056A59CE-2F19-43AE-B2E8-290ECB3D0D77@dukhovni.org> References: <20180922.065902.132678184516757017.levitte@openssl.org> <056A59CE-2F19-43AE-B2E8-290ECB3D0D77@dukhovni.org> Message-ID: <20180922.075955.1307478942356074896.levitte@openssl.org> In message <056A59CE-2F19-43AE-B2E8-290ECB3D0D77 at dukhovni.org> on Sat, 22 Sep 2018 01:02:29 -0400, Viktor Dukhovni said: > > > > On Sep 22, 2018, at 12:59 AM, Richard Levitte wrote: > > > > So in summary, do we agree on this, and that it's a good path forward? > > > > - semantic versioning scheme good, we should adopt it. > > - we need to agree on how to translate that in code. > > - we need to stop fighting about history. > > Yes. Cool. On to how to translate that in code. As a basis, it makes sense to have a base with three macros to express the three numbers of a semantic version and calculate things from there. If it were just me, I'd adopt that as is. The problematic part of this, as far as I've seen, is how our users have treated the integer (numeric encoding of the version) that we've given them, and what we want to say to them going forward. For the use in pre-processing C, what I've seen treats the number as an incrementing entity without putting actual meaning into the bits. If they can check the version with < and >= operators, they should be fine. As long as the number we give them continues to be useful in this manner, we're good. For checking if they can rely on ABI compatibility between the application and the library, what I've seen is that they compare certain bits from the result of calling OpenSSL_version_num() with the same bits from OPENSSL_VERSION_NUMBER. This is where things get tricky and we must ask ourselves how we want things to be done going forward. Let me expand on that last bit for a moment. It's important for some (those that dlopen libcrypto rather than linking at build time, for example) to be able to check at run time that they can rely on our functionality compared to what they built for. Masking and checking for equality has been the method to do so, as far as I've seen. Do we agree so far? So far, we've basically seen two proposed solutions to the problem: - one is a method that tries to match what we have done de-facto for the last 6 years and how those that check for compatibility have used the encoded version number, with specific masks. - the other goes to the semantics of the encoded version number according to documentation in opensslv.h and pod and drops the bits that don't correspond to semantic versioning. This will require those that check for compatibility to change their mask to match. In essence, openssh will have to introduce something like this: if (headerver >= 0x20000000L) mask = 0xf0000000L; ... and adjust their lfix and hfix masks to match. A third solution could be to add an API that returns the three semantic versioning numbers (MAJOR, MINOR, PATCH), to be compared against Tim's proposed OPENSSL_VERSION_MAJOR, OPENSSL_VERSION_MINOR and OPENSSL_VERSION_PATCH. That is a solution that, even though *new*, would solve quite a lot of problems, beginning with all the masking nonsense... and OPENSSL_VERSION_NUMBER could still exist for those using it for pre-processing. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From levitte at openssl.org Sat Sep 22 06:30:05 2018 From: levitte at openssl.org (Richard Levitte) Date: Sat, 22 Sep 2018 08:30:05 +0200 (CEST) Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <20180922.075955.1307478942356074896.levitte@openssl.org> References: <20180922.065902.132678184516757017.levitte@openssl.org> <056A59CE-2F19-43AE-B2E8-290ECB3D0D77@dukhovni.org> <20180922.075955.1307478942356074896.levitte@openssl.org> Message-ID: <20180922.083005.1933665223962995133.levitte@openssl.org> In message <20180922.075955.1307478942356074896.levitte at openssl.org> on Sat, 22 Sep 2018 07:59:55 +0200 (CEST), Richard Levitte said: > So far, we've basically seen two proposed solutions to the problem: > ... > > - the other goes to the semantics of the encoded version number > according to documentation in opensslv.h and pod and drops the bits > that don't correspond to semantic versioning. This will require > those that check for compatibility to change their mask to match. > In essence, openssh will have to introduce something like this: > > if (headerver >= 0x20000000L) > mask = 0xf0000000L; > > ... and adjust their lfix and hfix masks to match. After writing this, I caught up on Tim's last post on the matter, and this caught my eye: tim> And the logical test there remains valid in that it detects all tim> incompatible versions correctly - what changes is that some tim> versions that are compatible are seen as incompatible - but that tim> is an incorrect interpretation that is safe. The brutal interpretation is "yeah ok, them guys are a bit overly cautious / paranoid, but it's not a real problem visavi compatibility, so uhmm *shrug*". Put like that, it does resolve at least my concern. It will mean that those who use a mask that covers a bit more than just the major version number will get to rebuild their application a bit more often than absolutely necessary (as long as we stick to the social contract we make), until they adapt their checking. It doesn't actually break anything. This is something I can go with. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From kaduk at mit.edu Mon Sep 24 00:04:52 2018 From: kaduk at mit.edu (Benjamin Kaduk) Date: Sun, 23 Sep 2018 19:04:52 -0500 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <27A0FBB1-F577-4A3C-A62A-947D6F251A63@dukhovni.org> References: <605550d9-bb84-5125-de5f-f3b4ac4585ab@openssl.org> <22DFFD58-5686-4FD6-BBB7-00512CC88035@dukhovni.org> <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> <39463F9F-1AE7-4E16-85F2-2AE147C383E7@dukhovni.org> <27A0FBB1-F577-4A3C-A62A-947D6F251A63@dukhovni.org> Message-ID: <20180924000452.GC24695@kduck.kaduk.org> On Sat, Sep 22, 2018 at 01:12:21AM -0400, Viktor Dukhovni wrote: > > > > On Sep 22, 2018, at 12:50 AM, Tim Hudson wrote: > > > > The impact of the breaking change on anyone actually following our documented encoding cannot. > > i.e. openssh as one example Richard pointed out. > > The only use of OPENSSL_VERSION_NUMBER bits in OpenSSH (which is not yet ported to > 1.1.x upstream BTW, so hardly relevant really) is: It seems that they have done the porting just in the past couple weeks: 482d23bcac upstream: hold our collective noses and use the openssl-1.1.x 48f54b9d12 adapt -portable to OpenSSL 1.1x API 86e0a9f3d2 upstream: use only openssl-1.1.x API here too a3fd8074e2 upstream: missed a bit of openssl-1.0.x API in this unittest cce8cbe0ed Fix openssl-1.1 fallout for --without-openssl. -Ben From kaduk at mit.edu Mon Sep 24 00:05:36 2018 From: kaduk at mit.edu (Benjamin Kaduk) Date: Sun, 23 Sep 2018 19:05:36 -0500 Subject: [openssl-project] A proposal for an updated OpenSSL version scheme (v2) In-Reply-To: <056A59CE-2F19-43AE-B2E8-290ECB3D0D77@dukhovni.org> References: <89DA2699-D306-46F9-9A06-106CFBE21AD9@dukhovni.org> <20180922.065902.132678184516757017.levitte@openssl.org> <056A59CE-2F19-43AE-B2E8-290ECB3D0D77@dukhovni.org> Message-ID: <20180924000535.GD24695@kduck.kaduk.org> On Sat, Sep 22, 2018 at 01:02:29AM -0400, Viktor Dukhovni wrote: > > > > On Sep 22, 2018, at 12:59 AM, Richard Levitte wrote: > > > > So in summary, do we agree on this, and that it's a good path forward? > > > > - semantic versioning scheme good, we should adopt it. > > - we need to agree on how to translate that in code. > > - we need to stop fighting about history. > > Yes. +100. -Ben From paul.dale at oracle.com Mon Sep 24 03:37:46 2018 From: paul.dale at oracle.com (Paul Dale) Date: Sun, 23 Sep 2018 20:37:46 -0700 (PDT) Subject: [openssl-project] Release strategy updates & other policies Message-ID: <79590c7b-98a4-4816-9eea-52f21b5e23e1@default> While reading through the lively versioning discussions, it appears that there are some fundamentals that seem to lack surrounding policies and guidance. The referenced PR https://github.com/openssl/web/pull/82 starts by asking what is our public API? I'm in the "what's in the header files" bucket, which appears to be the consensus. That's over 5000 functions and some number of macros. Is this too many to support long term? I suspect it might be. The discussion also highlights the amount of missing documentation, which is also a concern but not the topic here. If we have too many APIs, what can we do about it? The low level APIs account for about 10% of the total and the FIPS project should help clean up some of these. That still leaves a lot. Deprecation seems like a beginning, however I've not seen a policy that specifies what or how this is done. Is there one? If not, it seems prudent to try to create one. Some thoughts about this: Should we deprecate functions at a major release or with LTS releases? I'll just use the generic "release" in absence to an answer to this. Is it sensible to deprecate functions in one release and remove them in the next? Deprecate and remove two releases later? Would it make more sense to deprecate functions in one release, move them to a legacy library in the next and remove them in the third? Or even deprecate in the first, move to legacy in the second and let them languish there thereafter? Does it make sense to announce APIs that will be deprecated one release cycle ahead of doing so? Macro deprecation might need to be slightly different. Do we have a policy for API changes? Have we defined what an API change is? I like a distinction between adding a new API and changing an existing one. Putting the latter to a vote seems sensible, the former probably doesn't need it so long as too many don't start appearing. Would an API deprecation be considered a change? I'd think not if there is a deprecation policy and it is followed. Does defining otherwise undefined behaviour constitute a change in the API? Does documenting undefined behaviour constitute a change in the API? While I wouldn't consider adding a NULL check to be an API change, but what about removing one? I'd think the latter is. Pauli -- Oracle Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia -----Original Message----- From: Matt Caswell [mailto:matt at openssl.org] Sent: Friday, 21 September 2018 11:19 PM To: openssl-project at openssl.org Subject: [openssl-project] Release strategy updates I am very concerned about stability of our API moving forwards. There are various discussions about changing the version number to 1.2.0 (or possibly 2.0.0) - which according to our versioning scheme would allow breaking changes. Whilst this is true I think we need to be very wary about "opening the flood gates" for breaking changes. The move from 1.0.x to 1.1.0 was hard on our users and we should avoid that again. With that in mind I have opened the following PR (based largely on wording suggested by Viktor): https://github.com/openssl/web/pull/82 At the same time I have taken the opportunity to clean up some out-of-date stuff in the release strategy. This is independent of the semantic versioning discussion which may itself see further changes being made to the release strategy. Thoughts? Matt _______________________________________________ openssl-project mailing list openssl-project at openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project From levitte at openssl.org Mon Sep 24 08:00:03 2018 From: levitte at openssl.org (Richard Levitte) Date: Mon, 24 Sep 2018 10:00:03 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <79590c7b-98a4-4816-9eea-52f21b5e23e1@default> References: <79590c7b-98a4-4816-9eea-52f21b5e23e1@default> Message-ID: <20180924.100003.2155153469056385673.levitte@openssl.org> In message <79590c7b-98a4-4816-9eea-52f21b5e23e1 at default> on Sun, 23 Sep 2018 20:37:46 -0700 (PDT), Paul Dale said: > While reading through the lively versioning discussions, it appears > that there are some fundamentals that seem to lack surrounding > policies and guidance. True, we do need some. At the same time, let's be careful not to get too regulated... but we're not quite there, so I'm not worried yet. > The referenced PR https://github.com/openssl/web/pull/82 starts by > asking what is our public API? I'm in the "what's in the header > files" bucket, which appears to be the consensus. That's over 5000 > functions and some number of macros. Is this too many to support > long term? I suspect it might be. The discussion also highlights > the amount of missing documentation, which is also a concern but not > the topic here. It also depends on what the functions are. Quite a few of them are very simple getters and setters, or converters (d2i_ and i2d_ functions, which are documented all in one go), i.e. convenience names that keep the more difficult to use generic functions at bay. It's quite possible that we can find more user friendly ways to generalise a lot of them... > If we have too many APIs, what can we do about it? > The low level APIs account for about 10% of the total and the FIPS > project should help clean up some of these. > > That still leaves a lot. Deprecation seems like a beginning, > however I've not seen a policy that specifies what or how this is > done. Is there one? If not, it seems prudent to try to create > one. There isn't really one written out, as far as I recall. There is some kind of concensus among some (I have stayed out of it so far, but am obviously getting interested...) > Some thoughts about this: > Should we deprecate functions at a major release or with LTS > releases? I'm not sure I'd tie that to LTS releases specifically. Our current practice (as seen in the name of our deprecation macros) is to deprecate stuff (not just functions... there are certain openssl app commands that I'd like to get rid of) at major release boundary. (Incidently, semantic versioning FAQ suggests that deprecation can as well happen in a minor release, possibly even preceding a major release) Also, we must consider if we see deprecation as an incompatible API change. If we do, deprecation can most certainly only happen at major release boundary. My 0.02 SEK is that we should, if we consider the configuration option 'no-deprecated' and how deprecation at minor release boundary could lead to surprise breakage. > I'll just use the generic "release" in absence to an answer to this. > Is it sensible to deprecate functions in one release and remove them > in the next? Deprecate and remove two releases later? Removal is definitely something we only at major release boundary, and if we continue to do deprecation at major release boundary as well, it means that deprecation and removal are at least a major release away from each other. So the question is what we think we need, or what the users need. The current concensus, last I heard it, is that everything deprecated at 1.1.0 or older will be removed in the next major release. Should we reconsider this? > Would it make more sense to deprecate functions in one release, move > them to a legacy library in the next and remove them in the third? > Or even deprecate in the first, move to legacy in the second and let > them languish there thereafter? I'm thinking that as long as we can get the users to think that to reach anything "old" or deprecated, they must link with a legacy library as well, then I think it should happen immediately, i.e. when deprecation starts (incidently, advance deprecation, like we've done with some stuff, will present some build challenges, at least if we want things maximally automated). Frankly, though, I don't think we absolutely need to move deprecated stuff at all. Something to consider is that there might be stuff that we want to remove from public API, but still want to use internally... > Does it make sense to announce APIs that will be deprecated one > release cycle ahead of doing so? Uhmmm, not sure I understand. Isn't deprecation itself an announcement? So announcing it in advance, wouldn't that be something like announcing that we're going to have an announcement soon? > Macro deprecation might need to be slightly different. Why? > Do we have a policy for API changes? Have we defined what an API > change is? No, and not exactly. The FAQ entry on our current versioning scheme has an answer, though, says this: Changes to the middle number are considered major releases and neither source nor binary compatibility is guaranteed. I think that's the closest to a written out definition that we have, and it gives the message that for major releases, any change is fine on principle, while for minor releases, only compatible source changes are allowed (for example, constification of function parameters is fine). Different members have had different opinions on this, though... some have been more strict and said that staying compatible means Do Not Touch Public Headers (except for additions). (I'm not commenting on binary compatibility... you did ask about API, not ABI ;-)) > Would an API deprecation be considered a change? I'd think not if > there is a deprecation policy and it is followed. The 'no-deprecated' configuration option might make you change your mind ;-) > Does defining otherwise undefined behaviour constitute a change in > the API? Good question... one could ask the same question in terms of stability, if something undefined becomes stable, is it less undefined? We could also ask the question in terms of API compatibility. If something undefined becomes defined, is that an API compatible change or not? > Does documenting undefined behaviour constitute a change in the API? Not sure what you mean... if something is undefined and we amend the documentation to say that it's undefined, did the API change at all? > While I wouldn't consider adding a NULL check to be an API change, > but what about removing one? I'd think the latter is. Both are. Are you really asking about compatible vs incompatible API changes? In that case, I agree that the latter is incompatible. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Mon Sep 24 14:25:48 2018 From: matt at openssl.org (Matt Caswell) Date: Mon, 24 Sep 2018 15:25:48 +0100 Subject: [openssl-project] [openssl-commits] FAILED build of OpenSSL branch master with options -d --strict-warnings enable-asan no-shared -DOPENSSL_SMALL_FOOTPRINT In-Reply-To: <1537741888.232961.31151.nullmailer@run.openssl.org> References: <1537741888.232961.31151.nullmailer@run.openssl.org> Message-ID: I'm getting strange results for this. I can't recreate this locally. When I run this on the run-checker box every test fails. Running a test with V=1, give this: $ make TESTS=test_sanity V=1 test make depend && make _tests make[1]: Entering directory '/home/matt/enable-asan' make[1]: Leaving directory '/home/matt/enable-asan' make[1]: Entering directory '/home/matt/enable-asan' ( cd test; \ mkdir -p test-runs; \ SRCTOP=../../openssl \ BLDTOP=../. \ RESULT_D=test-runs \ PERL="/usr/bin/perl" \ EXE_EXT= \ OPENSSL_ENGINES=`cd .././engines 2>/dev/null && pwd` \ OPENSSL_DEBUG_MEMORY=on \ /usr/bin/perl ../../openssl/test/run_tests.pl test_sanity ) ../../openssl/test/recipes/01-test_sanity.t .. 1..1 ==2035==AddressSanitizer CHECK failed: /build/llvm-toolchain-3.8-_PD09B/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/asan/asan_rtl.cc:407 "((!asan_init_is_running && "ASan init calls itself!")) != (0)" (0x0, 0x0) ../../util/shlib_wrap.sh ../../test/sanitytest => 1 not ok 1 - running sanitytest # Failed test 'running sanitytest' # at /home/matt/enable-asan/test/../../openssl/util/perl/OpenSSL/Test/Simple.pm line 77. # Looks like you failed 1 test of 1. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/1 subtests Test Summary Report ------------------- ../../openssl/test/recipes/01-test_sanity.t (Wstat: 256 Tests: 1 Failed: 1) Failed test: 1 Non-zero exit status: 1 Files=1, Tests=1, 1 wallclock secs ( 0.04 usr 0.00 sys + 0.16 cusr 0.02 csys = 0.22 CPU) Result: FAIL Makefile:204: recipe for target '_tests' failed make[1]: *** [_tests] Error 1 make[1]: Leaving directory '/home/matt/enable-asan' Makefile:202: recipe for target 'tests' failed make: *** [tests] Error 2 Which makes no sense to me at all. I don't know why this has suddenly started happening. Maybe something environmental has changed? Matt On 23/09/18 23:31, OpenSSL run-checker wrote: > Platform and configuration command: > > $ uname -a > Linux run 4.4.0-135-generic #161-Ubuntu SMP Mon Aug 27 10:45:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux > $ CC=clang ../openssl/config -d --strict-warnings enable-asan no-shared -DOPENSSL_SMALL_FOOTPRINT > > Commit log since last time: > > 0f58220973 Create the .rnd file it it does not exist > 46d085096c typo-fixes: miscellaneous typo fixes > f39a02c68a Fix the max psk len for TLSv1.3 > cd6fe29f5b Add a test for the certificate callback > 524006dd1b Delay setting the sig algs until after the cert_cb has been called > dda5396aae crypto/bn/asm/x86_64-gcc.c: remove unnecessary redefinition of BN_ULONG > > Build log ended with (last 100 lines): > > ../../openssl/test/recipes/80-test_dtls_mtu.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/80-test_dtlsv1listen.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/80-test_ocsp.t (Wstat: 768 Tests: 11 Failed: 3) > Failed tests: 1, 10-11 > Non-zero exit status: 3 > ../../openssl/test/recipes/80-test_pkcs12.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/80-test_ssl_new.t (Wstat: 6656 Tests: 27 Failed: 26) > Failed tests: 1-21, 23-27 > Non-zero exit status: 26 > ../../openssl/test/recipes/80-test_ssl_old.t (Wstat: 1280 Tests: 6 Failed: 5) > Failed tests: 1-2, 4-6 > Non-zero exit status: 5 > ../../openssl/test/recipes/80-test_ssl_test_ctx.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/80-test_sslcorrupt.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/80-test_tsa.t (Wstat: 256 Tests: 20 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/80-test_x509aux.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_asn1_time.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_async.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_bio_enc.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_constant_time.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_fatalerr.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_gmdiff.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_ige.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_includes.t (Wstat: 768 Tests: 3 Failed: 3) > Failed tests: 1-3 > Non-zero exit status: 3 > ../../openssl/test/recipes/90-test_memleak.t (Wstat: 512 Tests: 2 Failed: 2) > Failed tests: 1-2 > Non-zero exit status: 2 > ../../openssl/test/recipes/90-test_overhead.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_secmem.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_srp.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_sslapi.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_sslbuffers.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_sysdefault.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_threads.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_time_offset.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_tls13ccs.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_tls13encryption.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/90-test_v3name.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > ../../openssl/test/recipes/99-test_fuzz.t (Wstat: 2816 Tests: 11 Failed: 11) > Failed tests: 1-11 > Non-zero exit status: 11 > Files=152, Tests=860, 46 wallclock secs ( 0.96 usr 0.35 sys + 31.51 cusr 5.57 csys = 38.39 CPU) > Result: FAIL > Makefile:204: recipe for target '_tests' failed > make[1]: *** [_tests] Error 1 > make[1]: Leaving directory '/home/openssl/run-checker/enable-asan' > Makefile:202: recipe for target 'tests' failed > make: *** [tests] Error 2 > _____ > openssl-commits mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-commits > From levitte at openssl.org Mon Sep 24 14:30:14 2018 From: levitte at openssl.org (Richard Levitte) Date: Mon, 24 Sep 2018 16:30:14 +0200 Subject: [openssl-project] [openssl-commits] FAILED build of OpenSSL branch master with options -d --strict-warnings enable-asan no-shared -DOPENSSL_SMALL_FOOTPRINT In-Reply-To: References: <1537741888.232961.31151.nullmailer@run.openssl.org> Message-ID: <8A29BE88-25F9-44B1-8768-27A5CDBA7DB6@openssl.org> The box got upgraded to Ubuntu 18.04 LTS yesterday. I'll turn off test building and see if I can figure it out. Cheers Richard Matt Caswell skrev: (24 september 2018 16:25:48 CEST) >I'm getting strange results for this. I can't recreate this locally. > >When I run this on the run-checker box every test fails. Running a test >with V=1, give this: > >$ make TESTS=test_sanity V=1 test >make depend && make _tests >make[1]: Entering directory '/home/matt/enable-asan' >make[1]: Leaving directory '/home/matt/enable-asan' >make[1]: Entering directory '/home/matt/enable-asan' >( cd test; \ > mkdir -p test-runs; \ > SRCTOP=../../openssl \ > BLDTOP=../. \ > RESULT_D=test-runs \ > PERL="/usr/bin/perl" \ > EXE_EXT= \ > OPENSSL_ENGINES=`cd .././engines 2>/dev/null && pwd` \ > OPENSSL_DEBUG_MEMORY=on \ > /usr/bin/perl ../../openssl/test/run_tests.pl test_sanity ) >../../openssl/test/recipes/01-test_sanity.t .. >1..1 >==2035==AddressSanitizer CHECK failed: >/build/llvm-toolchain-3.8-_PD09B/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/asan/asan_rtl.cc:407 >"((!asan_init_is_running && "ASan init calls itself!")) != (0)" (0x0, >0x0) > > >../../util/shlib_wrap.sh ../../test/sanitytest => 1 >not ok 1 - running sanitytest > ># Failed test 'running sanitytest' ># at >/home/matt/enable-asan/test/../../openssl/util/perl/OpenSSL/Test/Simple.pm >line 77. ># Looks like you failed 1 test of 1. >Dubious, test returned 1 (wstat 256, 0x100) >Failed 1/1 subtests > >Test Summary Report >------------------- >../../openssl/test/recipes/01-test_sanity.t (Wstat: 256 Tests: 1 >Failed: 1) > Failed test: 1 > Non-zero exit status: 1 >Files=1, Tests=1, 1 wallclock secs ( 0.04 usr 0.00 sys + 0.16 cusr >0.02 csys = 0.22 CPU) >Result: FAIL >Makefile:204: recipe for target '_tests' failed >make[1]: *** [_tests] Error 1 >make[1]: Leaving directory '/home/matt/enable-asan' >Makefile:202: recipe for target 'tests' failed >make: *** [tests] Error 2 > > >Which makes no sense to me at all. I don't know why this has suddenly >started happening. Maybe something environmental has changed? > >Matt > > > > >On 23/09/18 23:31, OpenSSL run-checker wrote: >> Platform and configuration command: >> >> $ uname -a >> Linux run 4.4.0-135-generic #161-Ubuntu SMP Mon Aug 27 10:45:01 UTC >2018 x86_64 x86_64 x86_64 GNU/Linux >> $ CC=clang ../openssl/config -d --strict-warnings enable-asan >no-shared -DOPENSSL_SMALL_FOOTPRINT >> >> Commit log since last time: >> >> 0f58220973 Create the .rnd file it it does not exist >> 46d085096c typo-fixes: miscellaneous typo fixes >> f39a02c68a Fix the max psk len for TLSv1.3 >> cd6fe29f5b Add a test for the certificate callback >> 524006dd1b Delay setting the sig algs until after the cert_cb has >been called >> dda5396aae crypto/bn/asm/x86_64-gcc.c: remove unnecessary >redefinition of BN_ULONG >> >> Build log ended with (last 100 lines): >> >> ../../openssl/test/recipes/80-test_dtls_mtu.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/80-test_dtlsv1listen.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/80-test_ocsp.t (Wstat: >768 Tests: 11 Failed: 3) >> Failed tests: 1, 10-11 >> Non-zero exit status: 3 >> ../../openssl/test/recipes/80-test_pkcs12.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/80-test_ssl_new.t (Wstat: >6656 Tests: 27 Failed: 26) >> Failed tests: 1-21, 23-27 >> Non-zero exit status: 26 >> ../../openssl/test/recipes/80-test_ssl_old.t (Wstat: >1280 Tests: 6 Failed: 5) >> Failed tests: 1-2, 4-6 >> Non-zero exit status: 5 >> ../../openssl/test/recipes/80-test_ssl_test_ctx.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/80-test_sslcorrupt.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/80-test_tsa.t (Wstat: >256 Tests: 20 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/80-test_x509aux.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_asn1_time.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_async.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_bio_enc.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_constant_time.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_fatalerr.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_gmdiff.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_ige.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_includes.t (Wstat: >768 Tests: 3 Failed: 3) >> Failed tests: 1-3 >> Non-zero exit status: 3 >> ../../openssl/test/recipes/90-test_memleak.t (Wstat: >512 Tests: 2 Failed: 2) >> Failed tests: 1-2 >> Non-zero exit status: 2 >> ../../openssl/test/recipes/90-test_overhead.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_secmem.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_srp.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_sslapi.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_sslbuffers.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_sysdefault.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_threads.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_time_offset.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_tls13ccs.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_tls13encryption.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/90-test_v3name.t (Wstat: >256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> ../../openssl/test/recipes/99-test_fuzz.t (Wstat: >2816 Tests: 11 Failed: 11) >> Failed tests: 1-11 >> Non-zero exit status: 11 >> Files=152, Tests=860, 46 wallclock secs ( 0.96 usr 0.35 sys + 31.51 >cusr 5.57 csys = 38.39 CPU) >> Result: FAIL >> Makefile:204: recipe for target '_tests' failed >> make[1]: *** [_tests] Error 1 >> make[1]: Leaving directory '/home/openssl/run-checker/enable-asan' >> Makefile:202: recipe for target 'tests' failed >> make: *** [tests] Error 2 >> _____ >> openssl-commits mailing list >> To unsubscribe: >https://mta.openssl.org/mailman/listinfo/openssl-commits >> >_______________________________________________ >openssl-project mailing list >openssl-project at openssl.org >https://mta.openssl.org/mailman/listinfo/openssl-project -- Skickat fr?n min Android-enhet med K-9 Mail. Urs?kta min f?ordighet. From levitte at openssl.org Mon Sep 24 15:27:06 2018 From: levitte at openssl.org (Richard Levitte) Date: Mon, 24 Sep 2018 17:27:06 +0200 (CEST) Subject: [openssl-project] [openssl-commits] FAILED build of OpenSSL branch master with options -d --strict-warnings enable-asan no-shared -DOPENSSL_SMALL_FOOTPRINT In-Reply-To: <8A29BE88-25F9-44B1-8768-27A5CDBA7DB6@openssl.org> References: <1537741888.232961.31151.nullmailer@run.openssl.org> <8A29BE88-25F9-44B1-8768-27A5CDBA7DB6@openssl.org> Message-ID: <20180924.172706.449881421252007055.levitte@openssl.org> Apparently, I did a sloppy job. One more 'apt update && apt upgrade' was needed. So very sorry for the noise it created. I've now verified that everything is back to normal. Cheers, Richard In message <8A29BE88-25F9-44B1-8768-27A5CDBA7DB6 at openssl.org> on Mon, 24 Sep 2018 16:30:14 +0200, Richard Levitte said: > The box got upgraded to Ubuntu 18.04 LTS yesterday. > I'll turn off test building and see if I can figure it out. > > Cheers > Richard > > > Matt Caswell skrev: (24 september 2018 16:25:48 CEST) > >I'm getting strange results for this. I can't recreate this locally. > > > >When I run this on the run-checker box every test fails. Running a test > >with V=1, give this: > > > >$ make TESTS=test_sanity V=1 test > >make depend && make _tests > >make[1]: Entering directory '/home/matt/enable-asan' > >make[1]: Leaving directory '/home/matt/enable-asan' > >make[1]: Entering directory '/home/matt/enable-asan' > >( cd test; \ > > mkdir -p test-runs; \ > > SRCTOP=../../openssl \ > > BLDTOP=../. \ > > RESULT_D=test-runs \ > > PERL="/usr/bin/perl" \ > > EXE_EXT= \ > > OPENSSL_ENGINES=`cd .././engines 2>/dev/null && pwd` \ > > OPENSSL_DEBUG_MEMORY=on \ > > /usr/bin/perl ../../openssl/test/run_tests.pl test_sanity ) > >../../openssl/test/recipes/01-test_sanity.t .. > >1..1 > >==2035==AddressSanitizer CHECK failed: > >/build/llvm-toolchain-3.8-_PD09B/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/asan/asan_rtl.cc:407 > >"((!asan_init_is_running && "ASan init calls itself!")) != (0)" (0x0, > >0x0) > > > > > >../../util/shlib_wrap.sh ../../test/sanitytest => 1 > >not ok 1 - running sanitytest > > > ># Failed test 'running sanitytest' > ># at > >/home/matt/enable-asan/test/../../openssl/util/perl/OpenSSL/Test/Simple.pm > >line 77. > ># Looks like you failed 1 test of 1. > >Dubious, test returned 1 (wstat 256, 0x100) > >Failed 1/1 subtests > > > >Test Summary Report > >------------------- > >../../openssl/test/recipes/01-test_sanity.t (Wstat: 256 Tests: 1 > >Failed: 1) > > Failed test: 1 > > Non-zero exit status: 1 > >Files=1, Tests=1, 1 wallclock secs ( 0.04 usr 0.00 sys + 0.16 cusr > >0.02 csys = 0.22 CPU) > >Result: FAIL > >Makefile:204: recipe for target '_tests' failed > >make[1]: *** [_tests] Error 1 > >make[1]: Leaving directory '/home/matt/enable-asan' > >Makefile:202: recipe for target 'tests' failed > >make: *** [tests] Error 2 > > > > > >Which makes no sense to me at all. I don't know why this has suddenly > >started happening. Maybe something environmental has changed? > > > >Matt > > > > > > > > > >On 23/09/18 23:31, OpenSSL run-checker wrote: > >> Platform and configuration command: > >> > >> $ uname -a > >> Linux run 4.4.0-135-generic #161-Ubuntu SMP Mon Aug 27 10:45:01 UTC > >2018 x86_64 x86_64 x86_64 GNU/Linux > >> $ CC=clang ../openssl/config -d --strict-warnings enable-asan > >no-shared -DOPENSSL_SMALL_FOOTPRINT > >> > >> Commit log since last time: > >> > >> 0f58220973 Create the .rnd file it it does not exist > >> 46d085096c typo-fixes: miscellaneous typo fixes > >> f39a02c68a Fix the max psk len for TLSv1.3 > >> cd6fe29f5b Add a test for the certificate callback > >> 524006dd1b Delay setting the sig algs until after the cert_cb has > >been called > >> dda5396aae crypto/bn/asm/x86_64-gcc.c: remove unnecessary > >redefinition of BN_ULONG > >> > >> Build log ended with (last 100 lines): > >> > >> ../../openssl/test/recipes/80-test_dtls_mtu.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/80-test_dtlsv1listen.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/80-test_ocsp.t (Wstat: > >768 Tests: 11 Failed: 3) > >> Failed tests: 1, 10-11 > >> Non-zero exit status: 3 > >> ../../openssl/test/recipes/80-test_pkcs12.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/80-test_ssl_new.t (Wstat: > >6656 Tests: 27 Failed: 26) > >> Failed tests: 1-21, 23-27 > >> Non-zero exit status: 26 > >> ../../openssl/test/recipes/80-test_ssl_old.t (Wstat: > >1280 Tests: 6 Failed: 5) > >> Failed tests: 1-2, 4-6 > >> Non-zero exit status: 5 > >> ../../openssl/test/recipes/80-test_ssl_test_ctx.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/80-test_sslcorrupt.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/80-test_tsa.t (Wstat: > >256 Tests: 20 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/80-test_x509aux.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_asn1_time.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_async.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_bio_enc.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_constant_time.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_fatalerr.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_gmdiff.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_ige.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_includes.t (Wstat: > >768 Tests: 3 Failed: 3) > >> Failed tests: 1-3 > >> Non-zero exit status: 3 > >> ../../openssl/test/recipes/90-test_memleak.t (Wstat: > >512 Tests: 2 Failed: 2) > >> Failed tests: 1-2 > >> Non-zero exit status: 2 > >> ../../openssl/test/recipes/90-test_overhead.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_secmem.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_srp.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_sslapi.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_sslbuffers.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_sysdefault.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_threads.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_time_offset.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_tls13ccs.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_tls13encryption.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/90-test_v3name.t (Wstat: > >256 Tests: 1 Failed: 1) > >> Failed test: 1 > >> Non-zero exit status: 1 > >> ../../openssl/test/recipes/99-test_fuzz.t (Wstat: > >2816 Tests: 11 Failed: 11) > >> Failed tests: 1-11 > >> Non-zero exit status: 11 > >> Files=152, Tests=860, 46 wallclock secs ( 0.96 usr 0.35 sys + 31.51 > >cusr 5.57 csys = 38.39 CPU) > >> Result: FAIL > >> Makefile:204: recipe for target '_tests' failed > >> make[1]: *** [_tests] Error 1 > >> make[1]: Leaving directory '/home/openssl/run-checker/enable-asan' > >> Makefile:202: recipe for target 'tests' failed > >> make: *** [tests] Error 2 > >> _____ > >> openssl-commits mailing list > >> To unsubscribe: > >https://mta.openssl.org/mailman/listinfo/openssl-commits > >> > >_______________________________________________ > >openssl-project mailing list > >openssl-project at openssl.org > >https://mta.openssl.org/mailman/listinfo/openssl-project > > -- > Skickat fr?n min Android-enhet med K-9 Mail. Urs?kta min f?ordighet. From levitte at openssl.org Mon Sep 24 15:41:14 2018 From: levitte at openssl.org (Richard Levitte) Date: Mon, 24 Sep 2018 17:41:14 +0200 (CEST) Subject: [openssl-project] NEW: A proposal for an updated OpenSSL version scheme (v3.0-dev) In-Reply-To: <20180924000535.GD24695@kduck.kaduk.org> References: <20180922.065902.132678184516757017.levitte@openssl.org> <056A59CE-2F19-43AE-B2E8-290ECB3D0D77@dukhovni.org> <20180924000535.GD24695@kduck.kaduk.org> Message-ID: <20180924.174114.1855742985540546357.levitte@openssl.org> Following the discussion that we had on the previous documents and on all the input I got, I created a new version (v3.0-dev) for this proposal: https://docs.google.com/document/d/1p6l7VYn176JKzOtERdp9OG0HcyhnJZnVdRLD07L_1wE/ It's written from the point of view that the comment in opensslv.h and the documentation in OPENSSL_VERSION_NUMBER.pod are correct as to what the components in the version number are, and that we simply didn't do as the docs said since 1.0.0. So the idea is to simply reset, and then synthesize the value of existing macros (especially OPENSSL_VERSION_NUMBER) to be safe to use as we have observed that users do. This document leaves a few questions open: 1. what version will the next major release actually be? 2.0.0 has been suggested, and 3.0.0 as well. I see that as out of scope for this document, and should simply be voted on by the OMC at some point. 2. how should we handle ABI compatibility / incompatibility? It's possible that it's out of scope here, I'm unsure... 3. policies on what should and should not go into what version number level? I see that as out of scope for this document, but is definitly a related topic to discuss. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From Matthias.St.Pierre at ncp-e.com Mon Sep 24 16:09:07 2018 From: Matthias.St.Pierre at ncp-e.com (Dr. Matthias St. Pierre) Date: Mon, 24 Sep 2018 16:09:07 +0000 Subject: [openssl-project] NEW: A proposal for an updated OpenSSL version scheme (v3.0-dev) In-Reply-To: <20180924.174114.1855742985540546357.levitte@openssl.org> References: <20180922.065902.132678184516757017.levitte@openssl.org> <056A59CE-2F19-43AE-B2E8-290ECB3D0D77@dukhovni.org> <20180924000535.GD24695@kduck.kaduk.org> <20180924.174114.1855742985540546357.levitte@openssl.org> Message-ID: > -----Urspr?ngliche Nachricht----- > Von: openssl-project Im Auftrag von Richard Levitte > Gesendet: Montag, 24. September 2018 17:41 > An: openssl-project at openssl.org > Betreff: [openssl-project] NEW: A proposal for an updated OpenSSL version scheme (v3.0-dev) > > Following the discussion that we had on the previous documents and on > all the input I got, I created a new version (v3.0-dev) for this proposal: > > https://docs.google.com/document/d/1p6l7VYn176JKzOtERdp9OG0HcyhnJZnVdRLD07L_1wE/ > > It's written from the point of view that the comment in opensslv.h and > the documentation in OPENSSL_VERSION_NUMBER.pod are correct as to what > the components in the version number are, and that we simply didn't do > as the docs said since 1.0.0. So the idea is to simply reset, and > then synthesize the value of existing macros (especially > OPENSSL_VERSION_NUMBER) to be safe to use as we have observed that > users do. I'm not sure about the implication of the new document v3 on the two proposals from document v2. Does it mean you dropped your own proposal in favour of Tim's proposal? Or will there be two competing proposals, each described in its own document? Matthias From levitte at openssl.org Mon Sep 24 16:14:58 2018 From: levitte at openssl.org (Richard Levitte) Date: Mon, 24 Sep 2018 18:14:58 +0200 (CEST) Subject: [openssl-project] NEW: A proposal for an updated OpenSSL version scheme (v3.0-dev) In-Reply-To: References: <20180924000535.GD24695@kduck.kaduk.org> <20180924.174114.1855742985540546357.levitte@openssl.org> Message-ID: <20180924.181458.639508651382253699.levitte@openssl.org> In message on Mon, 24 Sep 2018 16:09:07 +0000, "Dr. Matthias St. Pierre" said: > > > -----Urspr?ngliche Nachricht----- > > Von: openssl-project Im Auftrag von Richard Levitte > > Gesendet: Montag, 24. September 2018 17:41 > > An: openssl-project at openssl.org > > Betreff: [openssl-project] NEW: A proposal for an updated OpenSSL version scheme (v3.0-dev) > > > > Following the discussion that we had on the previous documents and on > > all the input I got, I created a new version (v3.0-dev) for this proposal: > > > > https://docs.google.com/document/d/1p6l7VYn176JKzOtERdp9OG0HcyhnJZnVdRLD07L_1wE/ > > > > It's written from the point of view that the comment in opensslv.h and > > the documentation in OPENSSL_VERSION_NUMBER.pod are correct as to what > > the components in the version number are, and that we simply didn't do > > as the docs said since 1.0.0. So the idea is to simply reset, and > > then synthesize the value of existing macros (especially > > OPENSSL_VERSION_NUMBER) to be safe to use as we have observed that > > users do. > > I'm not sure about the implication of the new document v3 on the two > proposals from document v2. Does it mean you dropped your own > proposal in favour of Tim's proposal? Or will there be two competing > proposals, each described in its own document? v3 is a new version that replaces v2. So yeah, I'm going with Tim's proposal. That takes us on a path where we don't try to preserve historical habits ad nauseum, but rather do a good enough job for a period of time while fully switching to semantic versioning. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From paul.dale at oracle.com Mon Sep 24 21:55:00 2018 From: paul.dale at oracle.com (Paul Dale) Date: Mon, 24 Sep 2018 14:55:00 -0700 (PDT) Subject: [openssl-project] NEW: A proposal for an updated OpenSSL version scheme (v3.0-dev) In-Reply-To: <20180924.174114.1855742985540546357.levitte@openssl.org> References: <20180922.065902.132678184516757017.levitte@openssl.org> <056A59CE-2F19-43AE-B2E8-290ECB3D0D77@dukhovni.org> <20180924000535.GD24695@kduck.kaduk.org> <20180924.174114.1855742985540546357.levitte@openssl.org> Message-ID: <5e1608d5-ab1d-4ad4-af04-3af95d34fae1@default> Looks great Richard. I'd support that I think. Pauli -- Oracle Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia -----Original Message----- From: Richard Levitte [mailto:levitte at openssl.org] Sent: Tuesday, 25 September 2018 1:41 AM To: openssl-project at openssl.org Subject: [openssl-project] NEW: A proposal for an updated OpenSSL version scheme (v3.0-dev) Following the discussion that we had on the previous documents and on all the input I got, I created a new version (v3.0-dev) for this proposal: https://docs.google.com/document/d/1p6l7VYn176JKzOtERdp9OG0HcyhnJZnVdRLD07L_1wE/ It's written from the point of view that the comment in opensslv.h and the documentation in OPENSSL_VERSION_NUMBER.pod are correct as to what the components in the version number are, and that we simply didn't do as the docs said since 1.0.0. So the idea is to simply reset, and then synthesize the value of existing macros (especially OPENSSL_VERSION_NUMBER) to be safe to use as we have observed that users do. This document leaves a few questions open: 1. what version will the next major release actually be? 2.0.0 has been suggested, and 3.0.0 as well. I see that as out of scope for this document, and should simply be voted on by the OMC at some point. 2. how should we handle ABI compatibility / incompatibility? It's possible that it's out of scope here, I'm unsure... 3. policies on what should and should not go into what version number level? I see that as out of scope for this document, but is definitly a related topic to discuss. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ _______________________________________________ openssl-project mailing list openssl-project at openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project From levitte at openssl.org Tue Sep 25 08:26:16 2018 From: levitte at openssl.org (Richard Levitte) Date: Tue, 25 Sep 2018 10:26:16 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180924.100003.2155153469056385673.levitte@openssl.org> References: <79590c7b-98a4-4816-9eea-52f21b5e23e1@default> <20180924.100003.2155153469056385673.levitte@openssl.org> Message-ID: <20180925.102616.1672292553491401454.levitte@openssl.org> In message <20180924.100003.2155153469056385673.levitte at openssl.org> on Mon, 24 Sep 2018 10:00:03 +0200 (CEST), Richard Levitte said: > In message <79590c7b-98a4-4816-9eea-52f21b5e23e1 at default> on Sun, 23 Sep 2018 20:37:46 -0700 (PDT), Paul Dale said: > ... > > Some thoughts about this: > > Should we deprecate functions at a major release or with LTS > > releases? > > I'm not sure I'd tie that to LTS releases specifically. Our current > practice (as seen in the name of our deprecation macros) is to > deprecate stuff (not just functions... there are certain openssl app > commands that I'd like to get rid of) at major release boundary. > > (Incidently, semantic versioning FAQ suggests that deprecation can as > well happen in a minor release, possibly even preceding a major > release) > > Also, we must consider if we see deprecation as an incompatible API > change. If we do, deprecation can most certainly only happen at major > release boundary. My 0.02 SEK is that we should, if we consider the > configuration option 'no-deprecated' and how deprecation at minor > release boundary could lead to surprise breakage. I've been thinking a bit more on this part, and am wavering... with our slow pace (it's usually a couple of years between MINOR releases), deprecating only at MAJOR release boundaries may be quite a stretch to predict what should go and what shouldn't. After all, deprecation is only tacking on a warning that something is going to disappear in a predictable future, if at all possible to do. We may have to regard 'no-deprecated' as a development configuration option only, basically classifying it as "all bets are off". Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From tjh at cryptsoft.com Tue Sep 25 08:39:43 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Tue, 25 Sep 2018 18:39:43 +1000 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180925.102616.1672292553491401454.levitte@openssl.org> References: <79590c7b-98a4-4816-9eea-52f21b5e23e1@default> <20180924.100003.2155153469056385673.levitte@openssl.org> <20180925.102616.1672292553491401454.levitte@openssl.org> Message-ID: A fairly common approach that is used is that you can only remove something that has been marked for deprecation at a MAJOR release version boundary. That is entirely independent of the semantic versioning view of things - which also happens to say the same thing (that adding a deprecation indication is at least a MINOR release and the removal is at a MAJOR release). PATCH versions should never change an API. So we start warning whenever it is that we decide to deprecate in a MINOR release, but any actual removal (actual non-backwards compatible API change) doesn't come in place until a MAJOR release. I see marking things for deprecation is a warning of a pending non-backwards-compatible API change - i.e. that there is another way to handle the functionality which is preferred which you should have switched across to - but for now we are warning you as the final step before removing an API at the next MAJOR release. We haven't committed we will actually remove it - but we have warned you that we *expect* to remove it. Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Tue Sep 25 09:23:30 2018 From: levitte at openssl.org (Richard Levitte) Date: Tue, 25 Sep 2018 11:23:30 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <20180924.100003.2155153469056385673.levitte@openssl.org> <20180925.102616.1672292553491401454.levitte@openssl.org> Message-ID: <20180925.112330.1851743819010702859.levitte@openssl.org> In message on Tue, 25 Sep 2018 18:39:43 +1000, Tim Hudson said: > A fairly common approach that is used is that you can only remove something that has been > marked for deprecation at a MAJOR release version boundary. > > That is entirely independent of the semantic versioning view of things - which also happens to say > the same thing (that adding a deprecation indication is at least a MINOR release and the removal > is at a MAJOR release). Yes, hence the subject change (good call on Paul) > PATCH versions should never change an API. Yes. > So we start warning whenever it is that we decide to deprecate in a MINOR release, but any actual > removal (actual non-backwards compatible API change) doesn't come in place until a MAJOR > release. Well, *so far* we have done deprecation at MAJOR release boundary only, at east post 1.0.0. The diverse DEPRECATEDIN_ macros show that. So what you suggest (and what I'm leaning toward) means that we will change our habits. > I see marking things for deprecation is a warning of a pending non-backwards-compatible API > change - i.e. that there is another way to handle the functionality which is preferred which you > should have switched across to - but for now we are warning you as the final step before removing > an API at the next MAJOR release. We haven't committed we will actually remove it - but we have > warned you that we expect to remove it. Yes. -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From tjh at cryptsoft.com Tue Sep 25 09:58:54 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Tue, 25 Sep 2018 19:58:54 +1000 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180925.112330.1851743819010702859.levitte@openssl.org> References: <20180924.100003.2155153469056385673.levitte@openssl.org> <20180925.102616.1672292553491401454.levitte@openssl.org> <20180925.112330.1851743819010702859.levitte@openssl.org> Message-ID: On Tue, Sep 25, 2018 at 7:23 PM Richard Levitte wrote: > So what you suggest (and what I'm leaning toward) means that we will > change our habits. > Adoption of semantic versioning will indeed require us to change our habits in a number of areas - that is the point of having a single clear definition of what our version numbers mean. I also do think we need to document a few things like what we mean by bug fix compared to a feature update as there are items which we could argue could either be a PATCH or a MINOR update depending on how we describe such things. Getting those things documented so we can be consistent is a good thing IMHO. The specifics of which we place in PATCH and which we place in MINOR are less important than being consistent in handling the same item. For example: - adding an ASM implementation for performance reasons - is that PATCH or MINOR - changing an ASM implementation for performance release - is that PATCH or MINOR - adding an ASM implementation to get constant time behaviour - is that PATCH or MINOR - changing an ASM implementation for constant time behaviour - is that PATCH or MINOR For all four of the above examples the API is the same (assuming that the low-level APIs are not actually exposed in the public interface for any of these). And deciding on those depends how you view performance - is it a bug that something runs slower than it could - or is it a feature. Good arguments can be made for always MINOR or for PATCH - but I think we should have a clear view on how we will handle such things going forward given the OMC members have differing views on the topic and we shouldn't end up with different handling depending on which members in which timezone are awake for a given pull request :-) Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Tue Sep 25 10:07:10 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 25 Sep 2018 11:07:10 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <20180924.100003.2155153469056385673.levitte@openssl.org> <20180925.102616.1672292553491401454.levitte@openssl.org> <20180925.112330.1851743819010702859.levitte@openssl.org> Message-ID: <567a955a-916a-7a4a-dd98-e0137426c878@openssl.org> On 25/09/18 10:58, Tim Hudson wrote: > On Tue, Sep 25, 2018 at 7:23 PM Richard Levitte > wrote: > > So what you suggest (and what I'm leaning toward) means that we will > change our habits. > > > Adoption of semantic versioning will indeed require us to change our > habits in a number of areas - that is the point of having a single clear > definition of what our version numbers mean. > > I also do think we need to document a few things like what we mean by > bug fix compared to a feature update as there are items which we could > argue could either be a PATCH or a MINOR update depending on how we > describe such things. Sometimes we need to add a function in order to fix a bug. How should this be handled? e.g. there are 60 functions defined in util/libcrypto.num in the current 1.1.0i release that weren't there in the original 1.1.0 release. Matt > Getting those things documented so we can be consistent is a good thing > IMHO. The specifics of which we place in PATCH and which we place in > MINOR are less important than being consistent in handling the same item. > > For example: > - adding an ASM implementation for performance reasons - is that PATCH > or MINOR > - changing an ASM implementation for performance release - is that PATCH > or MINOR > - adding an ASM implementation to get constant time behaviour - is that > PATCH or MINOR > - changing an ASM implementation for constant time behaviour - is that > PATCH or MINOR > > For all four of the above examples the API is the same (assuming that > the low-level APIs are not actually exposed in the public interface for > any of these). > And deciding on those depends how you view performance - is it a bug > that something runs slower than it could - or is it a feature. > > Good arguments can be made for always MINOR or for PATCH - but I think > we should have a clear view on how we will handle such things going > forward given the OMC members have differing views on the topic and we > shouldn't end up with different handling depending on which members in > which timezone are awake for a given pull request :-) > > Tim. > > > > > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > From tjh at cryptsoft.com Tue Sep 25 10:13:53 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Tue, 25 Sep 2018 20:13:53 +1000 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <567a955a-916a-7a4a-dd98-e0137426c878@openssl.org> References: <20180924.100003.2155153469056385673.levitte@openssl.org> <20180925.102616.1672292553491401454.levitte@openssl.org> <20180925.112330.1851743819010702859.levitte@openssl.org> <567a955a-916a-7a4a-dd98-e0137426c878@openssl.org> Message-ID: On Tue, Sep 25, 2018 at 8:07 PM Matt Caswell wrote: > On 25/09/18 10:58, Tim Hudson wrote: > > On Tue, Sep 25, 2018 at 7:23 PM Richard Levitte > > wrote: > > > > So what you suggest (and what I'm leaning toward) means that we will > > change our habits. > > > > > > Adoption of semantic versioning will indeed require us to change our > > habits in a number of areas - that is the point of having a single clear > > definition of what our version numbers mean. > > > > I also do think we need to document a few things like what we mean by > > bug fix compared to a feature update as there are items which we could > > argue could either be a PATCH or a MINOR update depending on how we > > describe such things. > > Sometimes we need to add a function in order to fix a bug. How should > this be handled? e.g. there are 60 functions defined in > util/libcrypto.num in the current 1.1.0i release that weren't there in > the original 1.1.0 release. > In semantic versioning those are MINOR releases. The API has changed in a backward compatible manner. They cannot be called PATCH releases - those are only for internal changes that fix incorrect behaviour. If you change the API you are either doing a MINOR or a MAJOR release. Now I think the flexibility we added during 1.1.0 when the MAJOR change was to make APIs opaque was a different context where our API remained unstable (in semantic terms) yet we did a release (for other reasons). Under semantic versioning, 1.1.0 would have been called 2.0.0 and we would have had to batch up the accessor API additions into a 2.1.0 release and not have those included in any 2.0.X PATCH release. It is quite a change under semantic versioning in some areas as it basically requires the version to reflect API guarantees. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Tue Sep 25 10:30:39 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 25 Sep 2018 11:30:39 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <20180924.100003.2155153469056385673.levitte@openssl.org> <20180925.102616.1672292553491401454.levitte@openssl.org> <20180925.112330.1851743819010702859.levitte@openssl.org> <567a955a-916a-7a4a-dd98-e0137426c878@openssl.org> Message-ID: On 25/09/18 11:13, Tim Hudson wrote: > On Tue, Sep 25, 2018 at 8:07 PM Matt Caswell > wrote: > > On 25/09/18 10:58, Tim Hudson wrote: > > On Tue, Sep 25, 2018 at 7:23 PM Richard Levitte > > > >> wrote: > > > >? ? ?So what you suggest (and what I'm leaning toward) means that > we will > >? ? ?change our habits. > > > > > > Adoption of semantic versioning will indeed require us to change our > > habits in a number of areas - that is the point of having a single > clear > > definition of what our version numbers mean. > > > > I also do think we need to document a few things like what we mean by > > bug fix compared to a feature update as there are items which we could > > argue could either be a PATCH or a MINOR update depending on how we > > describe such things. > > Sometimes we need to add a function in order to fix a bug. How should > this be handled? e.g. there are 60 functions defined in > util/libcrypto.num in the current 1.1.0i release that weren't there in > the original 1.1.0 release. > > > > In semantic versioning those are MINOR releases. The API has changed in > a backward compatible manner. > They cannot be called PATCH releases - those are only for internal > changes that fix incorrect behaviour. > If you change the API you are either doing a MINOR or a MAJOR release. > > Now I think the flexibility we added during 1.1.0 when the MAJOR change > was to make APIs opaque was a different context where our API remained > unstable (in semantic terms) yet we did a release (for other reasons). > Under semantic versioning, 1.1.0 would have been called 2.0.0 and we > would have had to batch up the accessor API additions into a 2.1.0 > release and not have those included in any 2.0.X PATCH release. > > It is quite a change under semantic versioning in some areas as it > basically requires the version to reflect API guarantees.? I don't think we should batch up accessor API additions. Or in fact any others. We should release them at the right time as we do now. A move to semantic versioning shouldn't change that. That has implications for the way we manage branches and support periods/LTS. I suggest that we only create new branches for a MAJOR version number change. We define the support periods/LTS status relative to the major version number. For a given supported major version we may update the MINOR or PATCH number at any time dependent on whether we've added any new functions or not. What we now think of as a "letter" release could be either a MINOR or a PATCH release under semantic versioning (dependent on what we've changed/added). We continue with the same policy of not adding new features to a stable branch (except where necessary to fix a bug). Matt From levitte at openssl.org Tue Sep 25 10:48:45 2018 From: levitte at openssl.org (Richard Levitte) Date: Tue, 25 Sep 2018 12:48:45 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <567a955a-916a-7a4a-dd98-e0137426c878@openssl.org> Message-ID: <20180925.124845.722451152680932252.levitte@openssl.org> In message on Tue, 25 Sep 2018 11:30:39 +0100, Matt Caswell said: > > > On 25/09/18 11:13, Tim Hudson wrote: > > On Tue, Sep 25, 2018 at 8:07 PM Matt Caswell > > wrote: > > > > On 25/09/18 10:58, Tim Hudson wrote: > > > On Tue, Sep 25, 2018 at 7:23 PM Richard Levitte > > > > > >> wrote: > > > > > >? ? ?So what you suggest (and what I'm leaning toward) means that > > we will > > >? ? ?change our habits. > > > > > > > > > Adoption of semantic versioning will indeed require us to change our > > > habits in a number of areas - that is the point of having a single > > clear > > > definition of what our version numbers mean. > > > > > > I also do think we need to document a few things like what we mean by > > > bug fix compared to a feature update as there are items which we could > > > argue could either be a PATCH or a MINOR update depending on how we > > > describe such things. > > > > Sometimes we need to add a function in order to fix a bug. How should > > this be handled? e.g. there are 60 functions defined in > > util/libcrypto.num in the current 1.1.0i release that weren't there in > > the original 1.1.0 release. > > > > > > > > In semantic versioning those are MINOR releases. The API has changed in > > a backward compatible manner. > > They cannot be called PATCH releases - those are only for internal > > changes that fix incorrect behaviour. > > If you change the API you are either doing a MINOR or a MAJOR release. > > > > Now I think the flexibility we added during 1.1.0 when the MAJOR change > > was to make APIs opaque was a different context where our API remained > > unstable (in semantic terms) yet we did a release (for other reasons). > > Under semantic versioning, 1.1.0 would have been called 2.0.0 and we > > would have had to batch up the accessor API additions into a 2.1.0 > > release and not have those included in any 2.0.X PATCH release. > > > > It is quite a change under semantic versioning in some areas as it > > basically requires the version to reflect API guarantees.? > > I don't think we should batch up accessor API additions. Or in fact any > others. We should release them at the right time as we do now. A move to > semantic versioning shouldn't change that. That has implications for the > way we manage branches and support periods/LTS. Most of all, it may mean more frequent MINOR releases. > I suggest that we only create new branches for a MAJOR version number > change. We define the support periods/LTS status relative to the major > version number. For a given supported major version we may update the > MINOR or PATCH number at any time dependent on whether we've added any > new functions or not. What we now think of as a "letter" release could > be either a MINOR or a PATCH release under semantic versioning > (dependent on what we've changed/added). We continue with the same > policy of not adding new features to a stable branch (except where > necessary to fix a bug). You are contradicting yourself. If we only make new branches for MAJOR version number changes, then it will be allowed to add new functionality in them, because that's allowed with a MINOR version number change. I understand the wish to reduce the number of branches to maintain, we must just make sure we know what we're talking about. If we would start branching MAJOR releases only, then we'll need some kind of policy on what branches are still being developed (i.e new MINOR releases are allowed in that branch) and what branches are in maintenance mode only (i.e. only new PATCH releases allowed). On the other hand, that would simplify our view of 'master', that will always be the development of the next major release, which I would say is a good thing, that will reduce the number of PRs just hanging on github, waiting for us to decide that we switch master to "next major release development". So a generic idea could be that: - master is always the development of next MAJOR release - the last current MAJOR release branch can have new functionality added, i.e. can have new MINOR releases. - the second, third etc MAJOR release branch is maintenance only. >From a version number perspective, this will lead to much more rapid development (good thing in my mind), and a reduction of MINOR release branches to maintain (hooray!!!). Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Tue Sep 25 10:53:48 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 25 Sep 2018 11:53:48 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180925.124845.722451152680932252.levitte@openssl.org> References: <567a955a-916a-7a4a-dd98-e0137426c878@openssl.org> <20180925.124845.722451152680932252.levitte@openssl.org> Message-ID: <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> On 25/09/18 11:48, Richard Levitte wrote: > In message on Tue, 25 Sep 2018 11:30:39 +0100, Matt Caswell said: > >> >> >> On 25/09/18 11:13, Tim Hudson wrote: >>> On Tue, Sep 25, 2018 at 8:07 PM Matt Caswell >> > wrote: >>> >>> On 25/09/18 10:58, Tim Hudson wrote: >>> > On Tue, Sep 25, 2018 at 7:23 PM Richard Levitte >>> >>> > >> wrote: >>> > >>> >? ? ?So what you suggest (and what I'm leaning toward) means that >>> we will >>> >? ? ?change our habits. >>> > >>> > >>> > Adoption of semantic versioning will indeed require us to change our >>> > habits in a number of areas - that is the point of having a single >>> clear >>> > definition of what our version numbers mean. >>> > >>> > I also do think we need to document a few things like what we mean by >>> > bug fix compared to a feature update as there are items which we could >>> > argue could either be a PATCH or a MINOR update depending on how we >>> > describe such things. >>> >>> Sometimes we need to add a function in order to fix a bug. How should >>> this be handled? e.g. there are 60 functions defined in >>> util/libcrypto.num in the current 1.1.0i release that weren't there in >>> the original 1.1.0 release. >>> >>> >>> >>> In semantic versioning those are MINOR releases. The API has changed in >>> a backward compatible manner. >>> They cannot be called PATCH releases - those are only for internal >>> changes that fix incorrect behaviour. >>> If you change the API you are either doing a MINOR or a MAJOR release. >>> >>> Now I think the flexibility we added during 1.1.0 when the MAJOR change >>> was to make APIs opaque was a different context where our API remained >>> unstable (in semantic terms) yet we did a release (for other reasons). >>> Under semantic versioning, 1.1.0 would have been called 2.0.0 and we >>> would have had to batch up the accessor API additions into a 2.1.0 >>> release and not have those included in any 2.0.X PATCH release. >>> >>> It is quite a change under semantic versioning in some areas as it >>> basically requires the version to reflect API guarantees.? >> >> I don't think we should batch up accessor API additions. Or in fact any >> others. We should release them at the right time as we do now. A move to >> semantic versioning shouldn't change that. That has implications for the >> way we manage branches and support periods/LTS. > > Most of all, it may mean more frequent MINOR releases. > >> I suggest that we only create new branches for a MAJOR version number >> change. We define the support periods/LTS status relative to the major >> version number. For a given supported major version we may update the >> MINOR or PATCH number at any time dependent on whether we've added any >> new functions or not. What we now think of as a "letter" release could >> be either a MINOR or a PATCH release under semantic versioning >> (dependent on what we've changed/added). We continue with the same >> policy of not adding new features to a stable branch (except where >> necessary to fix a bug). > > You are contradicting yourself. If we only make new branches for > MAJOR version number changes, then it will be allowed to add new > functionality in them, because that's allowed with a MINOR version > number change. You misunderstand me. Yes, its allowed that we can under semantic versioning rules. I'm saying we shouldn't. > > I understand the wish to reduce the number of branches to maintain, we > must just make sure we know what we're talking about. > > If we would start branching MAJOR releases only, then we'll need some > kind of policy on what branches are still being developed (i.e new > MINOR releases are allowed in that branch) and what branches are in > maintenance mode only (i.e. only new PATCH releases allowed). All branches are in maintenance mode except master. Typically we only do PATCH releases for a branch. If we've needed to add a function to a branch (e.g. a missing accessor) then we make it a MINOR release. Otherwise we don't add new functionality (as now). Matt From levitte at openssl.org Tue Sep 25 11:12:56 2018 From: levitte at openssl.org (Richard Levitte) Date: Tue, 25 Sep 2018 13:12:56 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> References: <20180925.124845.722451152680932252.levitte@openssl.org> <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> Message-ID: <20180925.131256.2234906856880302119.levitte@openssl.org> In message <98774a3e-d127-dcd9-c835-3b359d69b449 at openssl.org> on Tue, 25 Sep 2018 11:53:48 +0100, Matt Caswell said: > > > On 25/09/18 11:48, Richard Levitte wrote: > > In message on Tue, 25 Sep 2018 11:30:39 +0100, Matt Caswell said: > > > >> > >> > >> On 25/09/18 11:13, Tim Hudson wrote: > >>> On Tue, Sep 25, 2018 at 8:07 PM Matt Caswell >>> > wrote: > >>> > >>> On 25/09/18 10:58, Tim Hudson wrote: > >>> > On Tue, Sep 25, 2018 at 7:23 PM Richard Levitte > >>> > >>> > >> wrote: > >>> > > >>> >? ? ?So what you suggest (and what I'm leaning toward) means that > >>> we will > >>> >? ? ?change our habits. > >>> > > >>> > > >>> > Adoption of semantic versioning will indeed require us to change our > >>> > habits in a number of areas - that is the point of having a single > >>> clear > >>> > definition of what our version numbers mean. > >>> > > >>> > I also do think we need to document a few things like what we mean by > >>> > bug fix compared to a feature update as there are items which we could > >>> > argue could either be a PATCH or a MINOR update depending on how we > >>> > describe such things. > >>> > >>> Sometimes we need to add a function in order to fix a bug. How should > >>> this be handled? e.g. there are 60 functions defined in > >>> util/libcrypto.num in the current 1.1.0i release that weren't there in > >>> the original 1.1.0 release. > >>> > >>> > >>> > >>> In semantic versioning those are MINOR releases. The API has changed in > >>> a backward compatible manner. > >>> They cannot be called PATCH releases - those are only for internal > >>> changes that fix incorrect behaviour. > >>> If you change the API you are either doing a MINOR or a MAJOR release. > >>> > >>> Now I think the flexibility we added during 1.1.0 when the MAJOR change > >>> was to make APIs opaque was a different context where our API remained > >>> unstable (in semantic terms) yet we did a release (for other reasons). > >>> Under semantic versioning, 1.1.0 would have been called 2.0.0 and we > >>> would have had to batch up the accessor API additions into a 2.1.0 > >>> release and not have those included in any 2.0.X PATCH release. > >>> > >>> It is quite a change under semantic versioning in some areas as it > >>> basically requires the version to reflect API guarantees.? > >> > >> I don't think we should batch up accessor API additions. Or in fact any > >> others. We should release them at the right time as we do now. A move to > >> semantic versioning shouldn't change that. That has implications for the > >> way we manage branches and support periods/LTS. > > > > Most of all, it may mean more frequent MINOR releases. > > > >> I suggest that we only create new branches for a MAJOR version number > >> change. We define the support periods/LTS status relative to the major > >> version number. For a given supported major version we may update the > >> MINOR or PATCH number at any time dependent on whether we've added any > >> new functions or not. What we now think of as a "letter" release could > >> be either a MINOR or a PATCH release under semantic versioning > >> (dependent on what we've changed/added). We continue with the same > >> policy of not adding new features to a stable branch (except where > >> necessary to fix a bug). > > > > You are contradicting yourself. If we only make new branches for > > MAJOR version number changes, then it will be allowed to add new > > functionality in them, because that's allowed with a MINOR version > > number change. > > You misunderstand me. Yes, its allowed that we can under semantic > versioning rules. I'm saying we shouldn't. You're saying that we should only release new functionality (new APIs) in MAJOR releases? Mind telling us why? > > I understand the wish to reduce the number of branches to maintain, we > > must just make sure we know what we're talking about. > > > > If we would start branching MAJOR releases only, then we'll need some > > kind of policy on what branches are still being developed (i.e new > > MINOR releases are allowed in that branch) and what branches are in > > maintenance mode only (i.e. only new PATCH releases allowed). > > All branches are in maintenance mode except master. Typically we only do > PATCH releases for a branch. If we've needed to add a function to a > branch (e.g. a missing accessor) then we make it a MINOR release. > Otherwise we don't add new functionality (as now). "as now" is false, we *do* release new functionality in minor releases. 1.1.1 was a minor release, even though given an incorrect version number from a semver point of view. Had we given 1.1.0 and so on semantically correct version numbers, we would have versioned like this: 1.1.0 => 2.0.0 (MAJOR release, has API incompatibilities) 1.1.1 => 2.1.0 (MINOR release, supposedly has new compatible API) Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Tue Sep 25 11:22:44 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 25 Sep 2018 12:22:44 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180925.131256.2234906856880302119.levitte@openssl.org> References: <20180925.124845.722451152680932252.levitte@openssl.org> <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> <20180925.131256.2234906856880302119.levitte@openssl.org> Message-ID: On 25/09/18 12:12, Richard Levitte wrote: > In message <98774a3e-d127-dcd9-c835-3b359d69b449 at openssl.org> on Tue, 25 Sep 2018 11:53:48 +0100, Matt Caswell said: > >> >> >> On 25/09/18 11:48, Richard Levitte wrote: >>> In message on Tue, 25 Sep 2018 11:30:39 +0100, Matt Caswell said: >>> >>>> >>>> >>>> On 25/09/18 11:13, Tim Hudson wrote: >>>>> On Tue, Sep 25, 2018 at 8:07 PM Matt Caswell >>>> > wrote: >>>>> >>>>> On 25/09/18 10:58, Tim Hudson wrote: >>>>> > On Tue, Sep 25, 2018 at 7:23 PM Richard Levitte >>>>> >>>>> > >> wrote: >>>>> > >>>>> >? ? ?So what you suggest (and what I'm leaning toward) means that >>>>> we will >>>>> >? ? ?change our habits. >>>>> > >>>>> > >>>>> > Adoption of semantic versioning will indeed require us to change our >>>>> > habits in a number of areas - that is the point of having a single >>>>> clear >>>>> > definition of what our version numbers mean. >>>>> > >>>>> > I also do think we need to document a few things like what we mean by >>>>> > bug fix compared to a feature update as there are items which we could >>>>> > argue could either be a PATCH or a MINOR update depending on how we >>>>> > describe such things. >>>>> >>>>> Sometimes we need to add a function in order to fix a bug. How should >>>>> this be handled? e.g. there are 60 functions defined in >>>>> util/libcrypto.num in the current 1.1.0i release that weren't there in >>>>> the original 1.1.0 release. >>>>> >>>>> >>>>> >>>>> In semantic versioning those are MINOR releases. The API has changed in >>>>> a backward compatible manner. >>>>> They cannot be called PATCH releases - those are only for internal >>>>> changes that fix incorrect behaviour. >>>>> If you change the API you are either doing a MINOR or a MAJOR release. >>>>> >>>>> Now I think the flexibility we added during 1.1.0 when the MAJOR change >>>>> was to make APIs opaque was a different context where our API remained >>>>> unstable (in semantic terms) yet we did a release (for other reasons). >>>>> Under semantic versioning, 1.1.0 would have been called 2.0.0 and we >>>>> would have had to batch up the accessor API additions into a 2.1.0 >>>>> release and not have those included in any 2.0.X PATCH release. >>>>> >>>>> It is quite a change under semantic versioning in some areas as it >>>>> basically requires the version to reflect API guarantees.? >>>> >>>> I don't think we should batch up accessor API additions. Or in fact any >>>> others. We should release them at the right time as we do now. A move to >>>> semantic versioning shouldn't change that. That has implications for the >>>> way we manage branches and support periods/LTS. >>> >>> Most of all, it may mean more frequent MINOR releases. >>> >>>> I suggest that we only create new branches for a MAJOR version number >>>> change. We define the support periods/LTS status relative to the major >>>> version number. For a given supported major version we may update the >>>> MINOR or PATCH number at any time dependent on whether we've added any >>>> new functions or not. What we now think of as a "letter" release could >>>> be either a MINOR or a PATCH release under semantic versioning >>>> (dependent on what we've changed/added). We continue with the same >>>> policy of not adding new features to a stable branch (except where >>>> necessary to fix a bug). >>> >>> You are contradicting yourself. If we only make new branches for >>> MAJOR version number changes, then it will be allowed to add new >>> functionality in them, because that's allowed with a MINOR version >>> number change. >> >> You misunderstand me. Yes, its allowed that we can under semantic >> versioning rules. I'm saying we shouldn't. > > You're saying that we should only release new functionality (new APIs) > in MAJOR releases? Mind telling us why? Lets imagine we release version 5.0.0. We create a branch for it and declare a support period. Its an LTS release. This is a *stable* release, so we shouldn't de-stabilise it by adding new features. Later we do some work on some new features in master. They are backwards compatible in terms of API so we release it as version 5.1.0. Its got a separate support period to the LTS release. We fix some bugs in 5.0.0, and release the bug fixes as 5.0.1. All fine so far. But then we realise there is a missing accessor in it. Its an LTS release so its going to be around for a long time - we really need to add the accessor. But we *can't* do it!! Technically speaking, according to the rules of semantic versioning, that is a change to our API - so it needs to be released as a MINOR version change. That would make the new version 5.1.0....but we already used that number for our backwards compatible feature release! Matt > >>> I understand the wish to reduce the number of branches to maintain, we >>> must just make sure we know what we're talking about. >>> >>> If we would start branching MAJOR releases only, then we'll need some >>> kind of policy on what branches are still being developed (i.e new >>> MINOR releases are allowed in that branch) and what branches are in >>> maintenance mode only (i.e. only new PATCH releases allowed). >> >> All branches are in maintenance mode except master. Typically we only do >> PATCH releases for a branch. If we've needed to add a function to a >> branch (e.g. a missing accessor) then we make it a MINOR release. >> Otherwise we don't add new functionality (as now). > > "as now" is false, we *do* release new functionality in minor > releases. 1.1.1 was a minor release, even though given an incorrect > version number from a semver point of view. > > Had we given 1.1.0 and so on semantically correct version numbers, we > would have versioned like this: > > 1.1.0 => 2.0.0 (MAJOR release, has API incompatibilities) > 1.1.1 => 2.1.0 (MINOR release, supposedly has new compatible API) > > Cheers, > Richard > From tjh at cryptsoft.com Tue Sep 25 12:03:07 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Tue, 25 Sep 2018 22:03:07 +1000 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <20180925.124845.722451152680932252.levitte@openssl.org> <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> <20180925.131256.2234906856880302119.levitte@openssl.org> Message-ID: On Tue, Sep 25, 2018 at 9:22 PM Matt Caswell wrote: > Lets imagine we release version 5.0.0. We create a branch for it and > declare a support period. Its an LTS release. This is a *stable* > release, so we shouldn't de-stabilise it by adding new features. > > Later we do some work on some new features in master. They are backwards > compatible in terms of API so we release it as version 5.1.0. Its got a > separate support period to the LTS release. > > We fix some bugs in 5.0.0, and release the bug fixes as 5.0.1. All fine > so far. But then we realise there is a missing accessor in it. Its an > LTS release so its going to be around for a long time - we really need > to add the accessor. But we *can't* do it!! Technically speaking, > according to the rules of semantic versioning, that is a change to our > API - so it needs to be released as a MINOR version change. That would > make the new version 5.1.0....but we already used that number for our > backwards compatible feature release! > And that is what semantic versioning is about - it is about the API. So if you add to the API you change either MINOR or MAJOR. In your scenario the moment you need to add an API you are doing a MINOR release and if you already have a MINOR release under the MAJOR then you need to add a MINOR on top of the latest MINOR that you have. You don't get to make API changing releases that expand the API behind the existing releases that are out there. That is not how a semantically versioned project behaves. The rules are strict for a reason to stop some of the practices that we have - where PATCH releases add APIs. Part of the precondition for a semantically versioned project is that the API (and in this sense this is the public API) is under "control" as such. I think there are very few circumstances under which we have needed to add APIs - and I think outside of accessor functions during the opacity changes - I don't know that there were any API additions that weren't actually avoidable by solving the problem without adding an API. I don't know - I haven't checked - but none leap to the front on mind. We have done that simply because we didn't have a strict rule in place about API additions - we do about changes or deletions - but we tend to view additions as relatively safe (and they are from a backwards compatible perspective - but they are not from a semantic versioning point of view). Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Tue Sep 25 12:25:54 2018 From: levitte at openssl.org (Richard Levitte) Date: Tue, 25 Sep 2018 14:25:54 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> <20180925.131256.2234906856880302119.levitte@openssl.org> Message-ID: <20180925.142554.316001523155186922.levitte@openssl.org> In message on Tue, 25 Sep 2018 12:22:44 +0100, Matt Caswell said: > > > On 25/09/18 12:12, Richard Levitte wrote: > > In message <98774a3e-d127-dcd9-c835-3b359d69b449 at openssl.org> on Tue, 25 Sep 2018 11:53:48 +0100, Matt Caswell said: > > > >> > >> > >> On 25/09/18 11:48, Richard Levitte wrote: > >>> You are contradicting yourself. If we only make new branches for > >>> MAJOR version number changes, then it will be allowed to add new > >>> functionality in them, because that's allowed with a MINOR version > >>> number change. > >> > >> You misunderstand me. Yes, its allowed that we can under semantic > >> versioning rules. I'm saying we shouldn't. > > > > You're saying that we should only release new functionality (new APIs) > > in MAJOR releases? Mind telling us why? > > Lets imagine we release version 5.0.0. We create a branch for it and > declare a support period. Its an LTS release. This is a *stable* > release, so we shouldn't de-stabilise it by adding new features. Side note: I would never make a x.0.0 release an LTS. That's very risky business, and considering things like missing accessors and stuff, it would be downright stupid. > Later we do some work on some new features in master. They are backwards > compatible in terms of API so we release it as version 5.1.0. Its got a > separate support period to the LTS release. > > We fix some bugs in 5.0.0, and release the bug fixes as 5.0.1. All fine > so far. But then we realise there is a missing accessor in it. Its an > LTS release so its going to be around for a long time - we really need > to add the accessor. But we *can't* do it!! Technically speaking, > according to the rules of semantic versioning, that is a change to our > API - so it needs to be released as a MINOR version change. That would > make the new version 5.1.0....but we already used that number for our > backwards compatible feature release! Added accessors is additions to out API, not a change of our existing API, let's make that clear. The choice we can make in the scenario is to either release 5.2.0 or 6.0.0. In my mind, both are viable, but for different reasons. I understand that your idea of having our release branches based on the major releases is what's getting you stuck here, 'cause it basically forces you do have everything in one timeline (unless we do sub-branches, and uhmmm, just nooooo! m'kay?). So we would essentially have a 5.y.z branch where we would have this straight timeline (as an example): 5.0.0 5.0.1 5.0.3 5.1.0 (this stops the 5.0.z series) 5.1.2 5.1.3 5.2.0 (this stops the 5.1.z series) ... With this type of branch, your scenario is already impossible, 'cause you can't release 5.0.1 after 5.1.0 LTS, you'll be forced to release 5.1.1, and if you add accessors after that, you could release 5.2.0, but that means buh-bye for the idea of the 5.1.0 LTS, 'cause you can't make any more patch releases on it. So yeah, I agree in this case that we'd be forced to go to a new major release rather than 5.2.0. What you got here is a mixup between branch policy and semantic versioning. It got lost in translation... Our current pattern is actually to make new stable branches on minor releases. In that case, it would be perfectly feasible to release 5.0.1 after 5.1.0 LTS was released ('cause one is on the 5.0 branch and the other on the 5.1 branch), and even to release 5.2.0 with new accessors (forming the 5.2 branch). So generally speaking, it should still be possible to create new minor releases in a branch based on major release, but with the caveat that it works like an update of the previous minor release (including its patch releases), AND that any LTS release basically stops that branch from getting new API added ever again. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Tue Sep 25 12:37:45 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 25 Sep 2018 13:37:45 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <20180925.124845.722451152680932252.levitte@openssl.org> <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> <20180925.131256.2234906856880302119.levitte@openssl.org> Message-ID: <896ece72-8923-801e-b97d-5a1b21c9c815@openssl.org> On 25/09/18 13:03, Tim Hudson wrote: > On Tue, Sep 25, 2018 at 9:22 PM Matt Caswell > wrote: > > Lets imagine we release version 5.0.0. We create a branch for it and > declare a support period. Its an LTS release. This is a *stable* > release, so we shouldn't de-stabilise it by adding new features. > > Later we do some work on some new features in master. They are backwards > compatible in terms of API so we release it as version 5.1.0. Its got a > separate support period to the LTS release. > > We fix some bugs in 5.0.0, and release the bug fixes as 5.0.1. All fine > so far. But then we realise there is a missing accessor in it. Its an > LTS release so its going to be around for a long time - we really need > to add the accessor. But we *can't* do it!! Technically speaking, > according to the rules of semantic versioning, that is a change to our > API - so it needs to be released as a MINOR version change. That would > make the new version 5.1.0....but we already used that number for our > backwards compatible feature release! > > > And that is what semantic versioning is about - it is about the API. > So if you add to the API you change either MINOR or MAJOR. > In your scenario the moment you need to add an API you are doing a MINOR > release and if you already have a MINOR release under the MAJOR then you > need to add a MINOR on top of the latest MINOR that you have. > You don't get to make API changing releases that expand the API behind > the existing releases that are out there. Exactly. That is why I am proposing that each time we create a branch it is associated with a major release ONLY. You can't have two branches with the same major release because that means you cannot make MINOR changes on the older branch - even ones that we would historically have allowed. > > That is not how a semantically versioned project behaves. > > The rules are strict for a reason to stop some of the practices that we > have - where PATCH releases add APIs.? > > Part of the precondition for a semantically versioned project is that > the API (and in this sense this is the public API) is under "control" as > such.? > > I think there are very few circumstances under which we have needed to > add APIs - and I think outside of accessor functions during the opacity Looking through changes we have made to 1.1.0 headers there are more than just accessor functions: - Add "const" qualifiers to functions - Add error function or reason codes - Not sure what to classify this change as:https://github.com/openssl/openssl/pull/6874 - This change modified some public macro values in 1.1.0: https://github.com/openssl/openssl/pull/6075/files - This change modified the way declaration warnings are handled in 1.1.0 public headers: https://github.com/openssl/openssl/pull/6688 - We deprecated a function which was documented as deprecated, should have been deprecated, but wasn't inside the deprecated guards: https://github.com/openssl/openssl/pull/6588 - We added a whole set of functions for creating X509_LOOKUP_METHODS: https://github.com/openssl/openssl/pull/6152 - Added some new macros: https://github.com/openssl/openssl/pull/6037 - We removed a macro added in a previous "letter" release because we realised it was a mistake: https://github.com/openssl/openssl/pull/5968 - Fixed a typo in a macro name: https://github.com/openssl/openssl/pull/3292 - Added a new SSL_OP_NO_ code https://github.com/openssl/openssl/pull/4901 There's a stack load more changes than this. I stopped looking after a relatively short time. Probably (almost) all of these would have to be released as a new MINOR version number under semantic versioning. I don't see this changing as we move into the future. Matt From levitte at openssl.org Tue Sep 25 12:54:20 2018 From: levitte at openssl.org (Richard Levitte) Date: Tue, 25 Sep 2018 14:54:20 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <896ece72-8923-801e-b97d-5a1b21c9c815@openssl.org> References: <896ece72-8923-801e-b97d-5a1b21c9c815@openssl.org> Message-ID: <20180925.145420.222921852290192351.levitte@openssl.org> In message <896ece72-8923-801e-b97d-5a1b21c9c815 at openssl.org> on Tue, 25 Sep 2018 13:37:45 +0100, Matt Caswell said: > > And that is what semantic versioning is about - it is about the API. > > So if you add to the API you change either MINOR or MAJOR. > > In your scenario the moment you need to add an API you are doing a MINOR > > release and if you already have a MINOR release under the MAJOR then you > > need to add a MINOR on top of the latest MINOR that you have. > > You don't get to make API changing releases that expand the API behind > > the existing releases that are out there. > > Exactly. That is why I am proposing that each time we create a branch it > is associated with a major release ONLY. You can't have two branches > with the same major release because that means you cannot make MINOR > changes on the older branch - even ones that we would historically have > allowed. Hmmmm? If we have three major branches 5.0.0, 6.0.0 and 7.0.0, I don't quite see what would stop us, technically. 5.0.0 5.0.1 5.1.0 5.1.1 6.0.0 (new branch) 6.0.1 6.0.2 7.0.0 (new branch) 6.0.3 6.1.0 5.1.2 7.0.1 7.1.0 5.2.0 6.1.2 6.2.0 Nothing in major based branching stops this from happening. The only thing you stop is new patches on the next to last minor release in any given branch. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From tjh at cryptsoft.com Tue Sep 25 12:56:39 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Tue, 25 Sep 2018 22:56:39 +1000 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <896ece72-8923-801e-b97d-5a1b21c9c815@openssl.org> References: <20180925.124845.722451152680932252.levitte@openssl.org> <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> <20180925.131256.2234906856880302119.levitte@openssl.org> <896ece72-8923-801e-b97d-5a1b21c9c815@openssl.org> Message-ID: On Tue, Sep 25, 2018 at 10:37 PM Matt Caswell wrote: > - Added some new macros: > https://github.com/openssl/openssl/pull/6037 No we didn't change our public API for this one - we changed *internals*. The change to include/openssl/crypto.h was entirely changing *comments* to document that the internals used more of a reserved space - but the API wasn't changed. And it isn't a matter of what we did for each item - it is a matter of could we have fixed certainly things that we would classify as *bugs* without impacting the API - we didn't have that as a rule - but if we did then what would we have done. And where we missed an API then it is not a PATCH. Where we got something that wasn't spelled correctly - it has to wait for a MINOR to be fixed. A patch doesn't change APIs. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Tue Sep 25 13:01:59 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 25 Sep 2018 14:01:59 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <20180925.124845.722451152680932252.levitte@openssl.org> <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> <20180925.131256.2234906856880302119.levitte@openssl.org> <896ece72-8923-801e-b97d-5a1b21c9c815@openssl.org> Message-ID: <4c48337a-4e40-c6bf-2263-7a965dcce75f@openssl.org> On 25/09/18 13:56, Tim Hudson wrote: > On Tue, Sep 25, 2018 at 10:37 PM Matt Caswell > wrote: > > - Added some new macros: > https://github.com/openssl/openssl/pull/6037 > > > No we didn't change our public API for this one - we changed *internals*. > The change to include/openssl/crypto.h was entirely changing *comments* > to document that the internals used more of a reserved space - but the > API wasn't changed. You're right on this one. I misread the diff. > > And it isn't a matter of what we did for each item - it is a matter of > could we have fixed certainly things that we would classify as *bugs* > without impacting the API - we didn't have that as a rule - but if we > did then what would we have done. > And where we missed an API then it is not a PATCH. Where we got > something that wasn't spelled correctly - it has to wait for a MINOR to > be fixed. A patch doesn't change APIs. > > Tim. > ? > > > _______________________________________________ > openssl-project mailing list > openssl-project at openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project > From tjh at cryptsoft.com Tue Sep 25 13:09:31 2018 From: tjh at cryptsoft.com (Tim Hudson) Date: Tue, 25 Sep 2018 23:09:31 +1000 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <4c48337a-4e40-c6bf-2263-7a965dcce75f@openssl.org> References: <20180925.124845.722451152680932252.levitte@openssl.org> <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> <20180925.131256.2234906856880302119.levitte@openssl.org> <896ece72-8923-801e-b97d-5a1b21c9c815@openssl.org> <4c48337a-4e40-c6bf-2263-7a965dcce75f@openssl.org> Message-ID: On Tue, Sep 25, 2018 at 11:02 PM Matt Caswell wrote: > You're right on this one. I misread the diff. > Not a problem - you are doing the look-at-what-we-did and how it would be impacted - and that is certainly what we should be doing - working through what impact this would have had. Semantic versioning is a major change in behaviour with a focus on the API being directly reflected in the versioning scheme. It does mean that a lot of how we have handled things in the past in terms of what was okay to go into a patch release changes. Patch release become* pure bug fix *with no API changes (of any form) releases and that is very different. We have taken a relatively flexible interpretation - and put in a lot more than bug fixes into the letter (patch) releases - we have added upwards compatible API additions. It would also mean our LTS releases are MAJOR.MINOR - as the PATCH is the fixes we will apply - so it isn't part of the LTS designation as such. e.g. 5.0.x would be the marker - not 5.0.0 - so 5.0 in shorthand form. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Tue Sep 25 13:11:11 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 25 Sep 2018 14:11:11 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180925.145420.222921852290192351.levitte@openssl.org> References: <896ece72-8923-801e-b97d-5a1b21c9c815@openssl.org> <20180925.145420.222921852290192351.levitte@openssl.org> Message-ID: On 25/09/18 13:54, Richard Levitte wrote: > In message <896ece72-8923-801e-b97d-5a1b21c9c815 at openssl.org> on Tue, 25 Sep 2018 13:37:45 +0100, Matt Caswell said: > >>> And that is what semantic versioning is about - it is about the API. >>> So if you add to the API you change either MINOR or MAJOR. >>> In your scenario the moment you need to add an API you are doing a MINOR >>> release and if you already have a MINOR release under the MAJOR then you >>> need to add a MINOR on top of the latest MINOR that you have. >>> You don't get to make API changing releases that expand the API behind >>> the existing releases that are out there. >> >> Exactly. That is why I am proposing that each time we create a branch it >> is associated with a major release ONLY. You can't have two branches >> with the same major release because that means you cannot make MINOR >> changes on the older branch - even ones that we would historically have >> allowed. > > Hmmmm? If we have three major branches 5.0.0, 6.0.0 and 7.0.0, I > don't quite see what would stop us, technically. Is this exactly what I'm proposing? There is no problem if each major release is associated with a different branch. The problem comes if there are two branches on the SAME major version. Matt > > 5.0.0 > 5.0.1 > 5.1.0 > 5.1.1 > 6.0.0 (new branch) > 6.0.1 > 6.0.2 > 7.0.0 (new branch) > 6.0.3 > 6.1.0 > 5.1.2 > 7.0.1 > 7.1.0 > 5.2.0 > 6.1.2 > 6.2.0 > > Nothing in major based branching stops this from happening. The only > thing you stop is new patches on the next to last minor release in any > given branch. > > Cheers, > Richard > From matt at openssl.org Tue Sep 25 13:15:32 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 25 Sep 2018 14:15:32 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <20180925.124845.722451152680932252.levitte@openssl.org> <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> <20180925.131256.2234906856880302119.levitte@openssl.org> <896ece72-8923-801e-b97d-5a1b21c9c815@openssl.org> <4c48337a-4e40-c6bf-2263-7a965dcce75f@openssl.org> Message-ID: On 25/09/18 14:09, Tim Hudson wrote: > On Tue, Sep 25, 2018 at 11:02 PM Matt Caswell > wrote: > > You're right on this one. I misread the diff. > > > Not a problem - you are doing the look-at-what-we-did and how it would > be impacted - and that is certainly what we should be doing - working > through what impact this would have had. > Semantic versioning is a major change in behaviour with a focus on the > API being directly reflected in the versioning scheme. > > It does mean that a lot of how we have handled things in the past in > terms of what was okay to go into a patch release changes. > Patch release become*pure bug fix *with no API changes (of any form) > releases and that is very different.? > We have taken a relatively flexible interpretation - and put in a lot > more than bug fixes into the letter (patch) releases - we have added > upwards compatible API additions. > > It would also mean our LTS releases are MAJOR.MINOR - as the PATCH is > the fixes we will apply - so it isn't part of the LTS designation as such. > e.g. 5.0.x would be the marker - not 5.0.0 - so 5.0 in shorthand form.? This is where we disagree. My proposal is that the LTS designation would be 5 (not 5.0.x or 5.0.0). We would continue to do updates as we have done except we would have to classify our changes on the LTS branch as either API affecting (e.g. new accessor) or just a bug fix. If the former then the new version becomes an update to the MINOR number, otherwise it is an update to the PATCH number. I think it is an absolute non-starter to disallow all API affecting changes in an LTS release (or indeed any stable release). Matt From matt at openssl.org Tue Sep 25 13:18:17 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 25 Sep 2018 14:18:17 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180925.142554.316001523155186922.levitte@openssl.org> References: <98774a3e-d127-dcd9-c835-3b359d69b449@openssl.org> <20180925.131256.2234906856880302119.levitte@openssl.org> <20180925.142554.316001523155186922.levitte@openssl.org> Message-ID: <989c8546-78e5-8bae-29d7-c9abf1bf79f1@openssl.org> On 25/09/18 13:25, Richard Levitte wrote: > In message on Tue, 25 Sep 2018 12:22:44 +0100, Matt Caswell said: > >> >> >> On 25/09/18 12:12, Richard Levitte wrote: >>> In message <98774a3e-d127-dcd9-c835-3b359d69b449 at openssl.org> on Tue, 25 Sep 2018 11:53:48 +0100, Matt Caswell said: >>> >>>> >>>> >>>> On 25/09/18 11:48, Richard Levitte wrote: >>>>> You are contradicting yourself. If we only make new branches for >>>>> MAJOR version number changes, then it will be allowed to add new >>>>> functionality in them, because that's allowed with a MINOR version >>>>> number change. >>>> >>>> You misunderstand me. Yes, its allowed that we can under semantic >>>> versioning rules. I'm saying we shouldn't. >>> >>> You're saying that we should only release new functionality (new APIs) >>> in MAJOR releases? Mind telling us why? >> >> Lets imagine we release version 5.0.0. We create a branch for it and >> declare a support period. Its an LTS release. This is a *stable* >> release, so we shouldn't de-stabilise it by adding new features. > > Side note: I would never make a x.0.0 release an LTS. That's very > risky business, and considering things like missing accessors and > stuff, it would be downright stupid. > >> Later we do some work on some new features in master. They are backwards >> compatible in terms of API so we release it as version 5.1.0. Its got a >> separate support period to the LTS release. >> >> We fix some bugs in 5.0.0, and release the bug fixes as 5.0.1. All fine >> so far. But then we realise there is a missing accessor in it. Its an >> LTS release so its going to be around for a long time - we really need >> to add the accessor. But we *can't* do it!! Technically speaking, >> according to the rules of semantic versioning, that is a change to our >> API - so it needs to be released as a MINOR version change. That would >> make the new version 5.1.0....but we already used that number for our >> backwards compatible feature release! > > Added accessors is additions to out API, not a change of our existing > API, let's make that clear. The choice we can make in the scenario is > to either release 5.2.0 or 6.0.0. In my mind, both are viable, but > for different reasons. Neither seems viable to me. That would mean you have to add all the features from 5.1.0 into an LTS release. That can't happen. Matt > > I understand that your idea of having our release branches based on > the major releases is what's getting you stuck here, 'cause it > basically forces you do have everything in one timeline (unless we do > sub-branches, and uhmmm, just nooooo! m'kay?). So we would > essentially have a 5.y.z branch where we would have this straight > timeline (as an example): > > 5.0.0 > 5.0.1 > 5.0.3 > 5.1.0 (this stops the 5.0.z series) > 5.1.2 > 5.1.3 > 5.2.0 (this stops the 5.1.z series) > ... > > With this type of branch, your scenario is already impossible, 'cause > you can't release 5.0.1 after 5.1.0 LTS, you'll be forced to release > 5.1.1, and if you add accessors after that, you could release 5.2.0, > but that means buh-bye for the idea of the 5.1.0 LTS, 'cause you can't > make any more patch releases on it. So yeah, I agree in this case > that we'd be forced to go to a new major release rather than 5.2.0. > > What you got here is a mixup between branch policy and semantic > versioning. It got lost in translation... > > Our current pattern is actually to make new stable branches on minor > releases. In that case, it would be perfectly feasible to release > 5.0.1 after 5.1.0 LTS was released ('cause one is on the 5.0 branch > and the other on the 5.1 branch), and even to release 5.2.0 with new > accessors (forming the 5.2 branch). > > So generally speaking, it should still be possible to create new minor > releases in a branch based on major release, but with the caveat that > it works like an update of the previous minor release (including its > patch releases), AND that any LTS release basically stops that branch > from getting new API added ever again. > > Cheers, > Richard > From levitte at openssl.org Tue Sep 25 13:21:55 2018 From: levitte at openssl.org (Richard Levitte) Date: Tue, 25 Sep 2018 15:21:55 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <896ece72-8923-801e-b97d-5a1b21c9c815@openssl.org> <20180925.145420.222921852290192351.levitte@openssl.org> Message-ID: <20180925.152155.1990641124977502005.levitte@openssl.org> In message on Tue, 25 Sep 2018 14:11:11 +0100, Matt Caswell said: > > > On 25/09/18 13:54, Richard Levitte wrote: > > In message <896ece72-8923-801e-b97d-5a1b21c9c815 at openssl.org> on Tue, 25 Sep 2018 13:37:45 +0100, Matt Caswell said: > > > >>> And that is what semantic versioning is about - it is about the API. > >>> So if you add to the API you change either MINOR or MAJOR. > >>> In your scenario the moment you need to add an API you are doing a MINOR > >>> release and if you already have a MINOR release under the MAJOR then you > >>> need to add a MINOR on top of the latest MINOR that you have. > >>> You don't get to make API changing releases that expand the API behind > >>> the existing releases that are out there. > >> > >> Exactly. That is why I am proposing that each time we create a branch it > >> is associated with a major release ONLY. You can't have two branches > >> with the same major release because that means you cannot make MINOR > >> changes on the older branch - even ones that we would historically have > >> allowed. > > > > Hmmmm? If we have three major branches 5.0.0, 6.0.0 and 7.0.0, I > > don't quite see what would stop us, technically. > > Is this exactly what I'm proposing? There is no problem if each major > release is associated with a different branch. The problem comes if > there are two branches on the SAME major version. It seems I misread you, then. The more I think about it, the more I agree with the MAJOR release base for our branches, what I still don't quite catch on is your thinking around the release on new MINOR releases... you seem to say that they shouldn't be allowed, at least under certain conditions, or always, and question is then, what's the actual difference? Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From levitte at openssl.org Tue Sep 25 13:30:24 2018 From: levitte at openssl.org (Richard Levitte) Date: Tue, 25 Sep 2018 15:30:24 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <989c8546-78e5-8bae-29d7-c9abf1bf79f1@openssl.org> References: <4c48337a-4e40-c6bf-2263-7a965dcce75f@openssl.org> Message-ID: <20180925.153024.156547327021891373.levitte@openssl.org> In message on Tue, 25 Sep 2018 14:15:32 +0100, Matt Caswell said: > On 25/09/18 14:09, Tim Hudson wrote: > > It would also mean our LTS releases are MAJOR.MINOR - as the PATCH is > > the fixes we will apply - so it isn't part of the LTS designation as such. > > e.g. 5.0.x would be the marker - not 5.0.0 - so 5.0 in shorthand form.? > > This is where we disagree. My proposal is that the LTS designation would > be 5 (not 5.0.x or 5.0.0). We would continue to do updates as we have > done except we would have to classify our changes on the LTS branch as > either API affecting (e.g. new accessor) or just a bug fix. If the > former then the new version becomes an update to the MINOR number, > otherwise it is an update to the PATCH number. I *like* the idea of an LTS designation on the major number only. However, the rest leaves me utterly confused. Here, it seems that you would allow a 5.1.0 minor update in the v5 LTS branch, and yet, you say this when responding to my posting: In message <989c8546-78e5-8bae-29d7-c9abf1bf79f1 at openssl.org> on Tue, 25 Sep 2018 14:18:17 +0100, Matt Caswell said: > On 25/09/18 13:25, Richard Levitte wrote: > > Added accessors is additions to out API, not a change of our existing > > API, let's make that clear. The choice we can make in the scenario is > > to either release 5.2.0 or 6.0.0. In my mind, both are viable, but > > for different reasons. > > Neither seems viable to me. That would mean you have to add all the > features from 5.1.0 into an LTS release. That can't happen. I think I need an example timeline from you, 'cause I can see clearly how you look at it... Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Tue Sep 25 13:31:56 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 25 Sep 2018 14:31:56 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180925.152155.1990641124977502005.levitte@openssl.org> References: <896ece72-8923-801e-b97d-5a1b21c9c815@openssl.org> <20180925.145420.222921852290192351.levitte@openssl.org> <20180925.152155.1990641124977502005.levitte@openssl.org> Message-ID: On 25/09/18 14:21, Richard Levitte wrote: > In message on Tue, 25 Sep 2018 14:11:11 +0100, Matt Caswell said: > >> >> >> On 25/09/18 13:54, Richard Levitte wrote: >>> In message <896ece72-8923-801e-b97d-5a1b21c9c815 at openssl.org> on Tue, 25 Sep 2018 13:37:45 +0100, Matt Caswell said: >>> >>>>> And that is what semantic versioning is about - it is about the API. >>>>> So if you add to the API you change either MINOR or MAJOR. >>>>> In your scenario the moment you need to add an API you are doing a MINOR >>>>> release and if you already have a MINOR release under the MAJOR then you >>>>> need to add a MINOR on top of the latest MINOR that you have. >>>>> You don't get to make API changing releases that expand the API behind >>>>> the existing releases that are out there. >>>> >>>> Exactly. That is why I am proposing that each time we create a branch it >>>> is associated with a major release ONLY. You can't have two branches >>>> with the same major release because that means you cannot make MINOR >>>> changes on the older branch - even ones that we would historically have >>>> allowed. >>> >>> Hmmmm? If we have three major branches 5.0.0, 6.0.0 and 7.0.0, I >>> don't quite see what would stop us, technically. >> >> Is this exactly what I'm proposing? There is no problem if each major >> release is associated with a different branch. The problem comes if >> there are two branches on the SAME major version. > > It seems I misread you, then. > > The more I think about it, the more I agree with the MAJOR release > base for our branches, what I still don't quite catch on is your > thinking around the release on new MINOR releases... you seem to say > that they shouldn't be allowed, at least under certain conditions, or > always, and question is then, what's the actual difference? Semantic versioning says that a change to your API means the release is at least MINOR. Or it could be MAJOR. Up until now we've had a subtly different rule. Adding a new feature cannot go into a stable release. We *do* however allow some changes to the API in a stable release, e.g. new accessors, some limited constifying, error codes etc (see my other email for other examples). I'll collectively call these things "trivial API changes". So, the rules would be: 1) Each MAJOR version has a branch associated with it 2) New features only ever get merged to master. 3) When master is eventually released, a new branch is created for it and it becomes a new MAJOR version. 4) Trivial API changes can go to any branch 5) When creating a new release from a branch, if there have been any trivial API changes then we update the MINOR version. Otherwise we update the PATCH version. Matt From matt at openssl.org Tue Sep 25 13:51:00 2018 From: matt at openssl.org (Matt Caswell) Date: Tue, 25 Sep 2018 14:51:00 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180925.153024.156547327021891373.levitte@openssl.org> References: <4c48337a-4e40-c6bf-2263-7a965dcce75f@openssl.org> <20180925.153024.156547327021891373.levitte@openssl.org> Message-ID: <674a7e48-8d5b-c652-ac14-f3e7e51538ef@openssl.org> On 25/09/18 14:30, Richard Levitte wrote: > In message on Tue, 25 Sep 2018 14:15:32 +0100, Matt Caswell said: > >> On 25/09/18 14:09, Tim Hudson wrote: >>> It would also mean our LTS releases are MAJOR.MINOR - as the PATCH is >>> the fixes we will apply - so it isn't part of the LTS designation as such. >>> e.g. 5.0.x would be the marker - not 5.0.0 - so 5.0 in shorthand form.? >> >> This is where we disagree. My proposal is that the LTS designation would >> be 5 (not 5.0.x or 5.0.0). We would continue to do updates as we have >> done except we would have to classify our changes on the LTS branch as >> either API affecting (e.g. new accessor) or just a bug fix. If the >> former then the new version becomes an update to the MINOR number, >> otherwise it is an update to the PATCH number. > > I *like* the idea of an LTS designation on the major number only. > However, the rest leaves me utterly confused. Here, it seems that you > would allow a 5.1.0 minor update in the v5 LTS branch, and yet, you > say this when responding to my posting: > > In message <989c8546-78e5-8bae-29d7-c9abf1bf79f1 at openssl.org> on Tue, 25 Sep 2018 14:18:17 +0100, Matt Caswell said: > >> On 25/09/18 13:25, Richard Levitte wrote: >>> Added accessors is additions to out API, not a change of our existing >>> API, let's make that clear. The choice we can make in the scenario is >>> to either release 5.2.0 or 6.0.0. In my mind, both are viable, but >>> for different reasons. >> >> Neither seems viable to me. That would mean you have to add all the >> features from 5.1.0 into an LTS release. That can't happen. > > I think I need an example timeline from you, 'cause I can see clearly > how you look at it... The above was in response to my example of the problem that happens if you have multiple branches associated with a single MAJOR version number. In my example 5.0.0 was an LTS release, and 5.1.0 was some other feature release. The problem was that, in that scenario, if you need to add an accessor to the LTS release, you can't do it because 5.1.0 is already used for some other feature release. You can't just call it 5.2.0 and that's the new incarnation of the LTS release because that means adding all the features from 5.1.0 into the LTS release in the middle of the LTS cycle. I see all MAJOR versions as stable releases with a one-to-one correspondence with a branch. A timeline might look like this 5.0.0 5.0.1 (bug fix) 5.1.0 (add accessor) 6.0.0 (new feature) 6.0.1 (bug fix) 5.1.1 (bug fix) 6.0.2 (bug fix) 5.2.1 (add accessor) 6.1.0 (add accessor) And so on. Matt From matt at openssl.org Wed Sep 26 09:01:22 2018 From: matt at openssl.org (Matt Caswell) Date: Wed, 26 Sep 2018 10:01:22 +0100 Subject: [openssl-project] Fwd: Release strategy updates & other policies In-Reply-To: <29977.1537897068@localhost> References: <29977.1537897068@localhost> Message-ID: FYI -------- Forwarded Message -------- Subject: Re: [openssl-project] Release strategy updates & other policies Date: Tue, 25 Sep 2018 13:37:48 -0400 From: Michael Richardson To: Matt Caswell replying directly, because the list is closed, but this is not private. Matt Caswell wrote: >> I also do think we need to document a few things like what we mean by >> bug fix compared to a feature update as there are items which we could >> argue could either be a PATCH or a MINOR update depending on how we >> describe such things. > Sometimes we need to add a function in order to fix a bug. How should > this be handled? e.g. there are 60 functions defined in > util/libcrypto.num in the current 1.1.0i release that weren't there in > the original 1.1.0 release. As the thread shows this was problematic. My answer is that applications either want 1.1.0, and should have a #define that asks for that version only, and the headers would omit the 60 new functions so that the new APIs would not compile. (That let's 1.1.0LETTER be alpha released in attempts to fix things without commiting to a particular API change) Applications that want these 60 new functions need to ask for 1.1.0i by "name", and get the new values. Or need to declare "I'm a developer" This probably means that 1.1.0i should have been 1.1.1 or 1.2.0, as semantic versioning wants to ignore that letter when looking for ABI changes. (Still looking for a VAX/VMS machine to test my unit tests on...) -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works | network architect [ ] mcr at sandelman.ca http://www.sandelman.ca/ | ruby on rails [ From kurt at roeckx.be Wed Sep 26 16:58:26 2018 From: kurt at roeckx.be (Kurt Roeckx) Date: Wed, 26 Sep 2018 18:58:26 +0200 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <20180924.100003.2155153469056385673.levitte@openssl.org> <20180925.102616.1672292553491401454.levitte@openssl.org> <20180925.112330.1851743819010702859.levitte@openssl.org> <567a955a-916a-7a4a-dd98-e0137426c878@openssl.org> Message-ID: <20180926165825.GA14262@roeckx.be> On Tue, Sep 25, 2018 at 08:13:53PM +1000, Tim Hudson wrote: > On Tue, Sep 25, 2018 at 8:07 PM Matt Caswell wrote: > > > On 25/09/18 10:58, Tim Hudson wrote: > > > On Tue, Sep 25, 2018 at 7:23 PM Richard Levitte > > > wrote: > > > > > > So what you suggest (and what I'm leaning toward) means that we will > > > change our habits. > > > > > > > > > Adoption of semantic versioning will indeed require us to change our > > > habits in a number of areas - that is the point of having a single clear > > > definition of what our version numbers mean. > > > > > > I also do think we need to document a few things like what we mean by > > > bug fix compared to a feature update as there are items which we could > > > argue could either be a PATCH or a MINOR update depending on how we > > > describe such things. > > > > Sometimes we need to add a function in order to fix a bug. How should > > this be handled? e.g. there are 60 functions defined in > > util/libcrypto.num in the current 1.1.0i release that weren't there in > > the original 1.1.0 release. > > > > > In semantic versioning those are MINOR releases. The API has changed in a > backward compatible manner. > They cannot be called PATCH releases - those are only for internal changes > that fix incorrect behaviour. > If you change the API you are either doing a MINOR or a MAJOR release. We might need to add this to multiple branches at the same time. Assume that we have a 2.0.5 and 2.1.2 version. And in both versions we need to add that a new function, what should the new versions be? My understanding is that you can't actually do it. Kurt From levitte at openssl.org Wed Sep 26 17:06:18 2018 From: levitte at openssl.org (Richard Levitte) Date: Wed, 26 Sep 2018 19:06:18 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180926165825.GA14262@roeckx.be> References: <567a955a-916a-7a4a-dd98-e0137426c878@openssl.org> <20180926165825.GA14262@roeckx.be> Message-ID: <20180926.190618.1899126142080789973.levitte@openssl.org> In message <20180926165825.GA14262 at roeckx.be> on Wed, 26 Sep 2018 18:58:26 +0200, Kurt Roeckx said: > On Tue, Sep 25, 2018 at 08:13:53PM +1000, Tim Hudson wrote: > > On Tue, Sep 25, 2018 at 8:07 PM Matt Caswell wrote: > > > > > On 25/09/18 10:58, Tim Hudson wrote: > > > > On Tue, Sep 25, 2018 at 7:23 PM Richard Levitte > > > > wrote: > > > > > > > > So what you suggest (and what I'm leaning toward) means that we will > > > > change our habits. > > > > > > > > > > > > Adoption of semantic versioning will indeed require us to change our > > > > habits in a number of areas - that is the point of having a single clear > > > > definition of what our version numbers mean. > > > > > > > > I also do think we need to document a few things like what we mean by > > > > bug fix compared to a feature update as there are items which we could > > > > argue could either be a PATCH or a MINOR update depending on how we > > > > describe such things. > > > > > > Sometimes we need to add a function in order to fix a bug. How should > > > this be handled? e.g. there are 60 functions defined in > > > util/libcrypto.num in the current 1.1.0i release that weren't there in > > > the original 1.1.0 release. > > > > > > > > > In semantic versioning those are MINOR releases. The API has changed in a > > backward compatible manner. > > They cannot be called PATCH releases - those are only for internal changes > > that fix incorrect behaviour. > > If you change the API you are either doing a MINOR or a MAJOR release. > > We might need to add this to multiple branches at the same time. > > Assume that we have a 2.0.5 and 2.1.2 version. And in both > versions we need to add that a new function, what should the > new versions be? My understanding is that you can't actually do > it. If we have both 2.0.5 and 2.1.2 as current releases, then it means we've branched at MINOR releases, i.e. we have a 2.0.x branch and a 2.1.x branch. Semver says that when you add new functionality, the next release MUST get its MINOR version number increased. Since the last MINOR release is 2.1.0, it means that we must release 2.2.0, and start a new branch for 2.2.x. If we branch on MAJOR releases, then that situation isn't even possible, you just cannot have 2.0.5 and 2.1.2 at the same time... unless you sub-branch, an endeavour I was stand very far away from. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From levitte at openssl.org Wed Sep 26 17:11:37 2018 From: levitte at openssl.org (Richard Levitte) Date: Wed, 26 Sep 2018 19:11:37 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180926.190618.1899126142080789973.levitte@openssl.org> References: <20180926165825.GA14262@roeckx.be> <20180926.190618.1899126142080789973.levitte@openssl.org> Message-ID: <20180926.191137.2088733449129089505.levitte@openssl.org> Speaking of this, I've started on a document that explores the meaning of changes and releases. Its objective is to create discussion and debate, and hopefully, we will eventually be able to create a working policy with sufficient detail. https://docs.google.com/document/d/1fCt_tgK9V1pp1aeR80sMmyH1h74cZ_fQg2b76z560UY/ There are some example cases that I could think of, please feel free to add. It also discusses ABI changes, and also mentions the CLI. Cheers, Richard ( currently brain freeze, can't think of more to write ) -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From kurt at roeckx.be Wed Sep 26 17:19:57 2018 From: kurt at roeckx.be (Kurt Roeckx) Date: Wed, 26 Sep 2018 19:19:57 +0200 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <20180926.190618.1899126142080789973.levitte@openssl.org> References: <567a955a-916a-7a4a-dd98-e0137426c878@openssl.org> <20180926165825.GA14262@roeckx.be> <20180926.190618.1899126142080789973.levitte@openssl.org> Message-ID: <20180926171956.GA15122@roeckx.be> On Wed, Sep 26, 2018 at 07:06:18PM +0200, Richard Levitte wrote: > In message <20180926165825.GA14262 at roeckx.be> on Wed, 26 Sep 2018 18:58:26 +0200, Kurt Roeckx said: > > > On Tue, Sep 25, 2018 at 08:13:53PM +1000, Tim Hudson wrote: > > > On Tue, Sep 25, 2018 at 8:07 PM Matt Caswell wrote: > > > > > > > On 25/09/18 10:58, Tim Hudson wrote: > > > > > On Tue, Sep 25, 2018 at 7:23 PM Richard Levitte > > > > > wrote: > > > > > > > > > > So what you suggest (and what I'm leaning toward) means that we will > > > > > change our habits. > > > > > > > > > > > > > > > Adoption of semantic versioning will indeed require us to change our > > > > > habits in a number of areas - that is the point of having a single clear > > > > > definition of what our version numbers mean. > > > > > > > > > > I also do think we need to document a few things like what we mean by > > > > > bug fix compared to a feature update as there are items which we could > > > > > argue could either be a PATCH or a MINOR update depending on how we > > > > > describe such things. > > > > > > > > Sometimes we need to add a function in order to fix a bug. How should > > > > this be handled? e.g. there are 60 functions defined in > > > > util/libcrypto.num in the current 1.1.0i release that weren't there in > > > > the original 1.1.0 release. > > > > > > > > > > > > > In semantic versioning those are MINOR releases. The API has changed in a > > > backward compatible manner. > > > They cannot be called PATCH releases - those are only for internal changes > > > that fix incorrect behaviour. > > > If you change the API you are either doing a MINOR or a MAJOR release. > > > > We might need to add this to multiple branches at the same time. > > > > Assume that we have a 2.0.5 and 2.1.2 version. And in both > > versions we need to add that a new function, what should the > > new versions be? My understanding is that you can't actually do > > it. > > If we have both 2.0.5 and 2.1.2 as current releases, then it means > we've branched at MINOR releases, i.e. we have a 2.0.x branch and a > 2.1.x branch. Semver says that when you add new functionality, the > next release MUST get its MINOR version number increased. Since the > last MINOR release is 2.1.0, it means that we must release 2.2.0, and > start a new branch for 2.2.x. 2.0.5 could for instance be something like the current version 1.0.0e, and 2.1.2 the current 1.0.1b. So 2.0.5 can't get fixed, because we can't give it a new number? Does that then mean we should leave numbers free, so that it's possible we can use them later? And that 2.1.2 should have been named something like 2.10.2 instead? > If we branch on MAJOR releases, then that situation isn't even > possible, you just cannot have 2.0.5 and 2.1.2 at the same time... > unless you sub-branch, an endeavour I was stand very far away from. So you're saying we should always increase the major number, not the minor number, in case we make a release with new features? Minor numbers are in case we need to add new features to an older version? Kurt From openssl-users at dukhovni.org Wed Sep 26 17:24:11 2018 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Wed, 26 Sep 2018 13:24:11 -0400 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <674a7e48-8d5b-c652-ac14-f3e7e51538ef@openssl.org> References: <4c48337a-4e40-c6bf-2263-7a965dcce75f@openssl.org> <20180925.153024.156547327021891373.levitte@openssl.org> <674a7e48-8d5b-c652-ac14-f3e7e51538ef@openssl.org> Message-ID: <73E9CD06-575E-4CDE-B137-95179B07B591@dukhovni.org> > On Sep 25, 2018, at 9:51 AM, Matt Caswell wrote: > > 5.0.0 > 5.0.1 (bug fix) > 5.1.0 (add accessor) > 6.0.0 (new feature) > 6.0.1 (bug fix) > 5.1.1 (bug fix) 6.0.2 (bug fix) > 5.2.1 (add accessor) > 6.1.0 (add accessor) Previously, we could add non-triviall features in "z+1" of "x.y.z", with a stable ABI moving forward from "x.y.z" to "x.y.(z+1)". Thus, e.g. 1.1.1 is a feature evolution of 1.1.0. With this, we seem to lose the ability to produce a manifestly (forward) ABI-compatible feature release, that's a drop-in replacement for a previous release. I would have expected to have 5.1.x as an ABI compatible upgrade of 5.0 with non-trivial new features. The trivial API updates in stable releases (new accessors for forward compatibility, ...) would go into the micro version along with the bug fixes. And should be made for the same reason. In the case of new accessors, their visibility should conditioned the user defining a suitable macro to make them visible. Their purpose is to facilitate compiling code that's forward-ported to a later release, but needs to still work with the stable release. Otherwise, there really should be no feature changes in stable releases. So, Matt, we're not on the same page just yet... -- -- Viktor. From levitte at openssl.org Wed Sep 26 18:00:10 2018 From: levitte at openssl.org (Richard Levitte) Date: Wed, 26 Sep 2018 20:00:10 +0200 (CEST) Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <73E9CD06-575E-4CDE-B137-95179B07B591@dukhovni.org> References: <20180925.153024.156547327021891373.levitte@openssl.org> <674a7e48-8d5b-c652-ac14-f3e7e51538ef@openssl.org> <73E9CD06-575E-4CDE-B137-95179B07B591@dukhovni.org> Message-ID: <20180926.200010.365511971016697361.levitte@openssl.org> In message <73E9CD06-575E-4CDE-B137-95179B07B591 at dukhovni.org> on Wed, 26 Sep 2018 13:24:11 -0400, Viktor Dukhovni said: > > > > On Sep 25, 2018, at 9:51 AM, Matt Caswell wrote: > > > > 5.0.0 > > 5.0.1 (bug fix) > > 5.1.0 (add accessor) > > 6.0.0 (new feature) > > 6.0.1 (bug fix) > > 5.1.1 (bug fix) 6.0.2 (bug fix) > > 5.2.1 (add accessor) > > 6.1.0 (add accessor) > > Previously, we could add non-triviall features in "z+1" of "x.y.z", > with a stable ABI moving forward from "x.y.z" to "x.y.(z+1)". > > Thus, e.g. 1.1.1 is a feature evolution of 1.1.0. With this, we seem > to lose the ability to produce a manifestly (forward) ABI-compatible > feature release, that's a drop-in replacement for a previous release. Your interpretation is incorrect, but that's because you're mixing up our current practice (which is wrong according to our docs), which is shited right compared to semantic versioning (and our docs). We have done none-trivial feature additions in "z+1" BECAUSE WE CALLED THAT A MINOR FEATURE RELEASE. In semver terms, we should have done "y+1" instead. (incidently, our letter release correspond to "z+1" in semver terms) BTW, "add non-trivial features" isn't really a useful terminology. The semver trigger for MINOR version bump is "add new functionality to the public API", regardless of if it's trivial or not. > I would have expected to have 5.1.x as an ABI compatible upgrade of > 5.0 with non-trivial new features. > > The trivial API updates in stable releases (new accessors for forward > compatibility, ...) would go into the micro version along with the > bug fixes. And should be made for the same reason. This is absolutely wrong in semantic versioning. Compatible APi *changes* are allowed in PATCH level releases (what you call micro versions). And strictly speaking, this doesn't even necessarily comply with our current practice, see this FAQ text: Letter releases (e.g. 1.0.1a) can only contain bug and security fixes and no new features. Minor releases change the last number (e.g. 1.0.2) and can contain new features that retain binary compatibility. I noticed that you mentioned "drop-in"... the way we've designed things so far, our minor releases produce libraries that can replace the same libraries from earlier minor releases in the same major series. > In the case of new accessors, their visibility should conditioned > the user defining a suitable macro to make them visible. Their > purpose is to facilitate compiling code that's forward-ported > to a later release, but needs to still work with the stable > release. Otherwise, there really should be no feature changes > in stable releases. This is where opinions seem to diverge. I have had a change of heart, new accessors should have trigger a new minor release and not been added as a letter (micro) release. What I see happening is, of course, that we will have more frequent minor releases, and that minor release may work as an upgrade from an earlier patch release. The latter will essentially depending on how we decide to branch going forward (Matt, branching on major releases starts to make sense, incidently... I'm not totally convinced yet, but...) Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Thu Sep 27 09:48:35 2018 From: matt at openssl.org (Matt Caswell) Date: Thu, 27 Sep 2018 10:48:35 +0100 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: <73E9CD06-575E-4CDE-B137-95179B07B591@dukhovni.org> References: <4c48337a-4e40-c6bf-2263-7a965dcce75f@openssl.org> <20180925.153024.156547327021891373.levitte@openssl.org> <674a7e48-8d5b-c652-ac14-f3e7e51538ef@openssl.org> <73E9CD06-575E-4CDE-B137-95179B07B591@dukhovni.org> Message-ID: On 26/09/18 18:24, Viktor Dukhovni wrote: > > >> On Sep 25, 2018, at 9:51 AM, Matt Caswell wrote: >> >> 5.0.0 >> 5.0.1 (bug fix) >> 5.1.0 (add accessor) >> 6.0.0 (new feature) >> 6.0.1 (bug fix) >> 5.1.1 (bug fix) 6.0.2 (bug fix) >> 5.2.1 (add accessor) >> 6.1.0 (add accessor) > > Previously, we could add non-triviall features in "z+1" of "x.y.z", > with a stable ABI moving forward from "x.y.z" to "x.y.(z+1)". > > Thus, e.g. 1.1.1 is a feature evolution of 1.1.0. With this, we seem > to lose the ability to produce a manifestly (forward) ABI-compatible > feature release, that's a drop-in replacement for a previous release. Yes. This is a consequence of going with semver that I don't see any way of avoiding. At least not without a radical shift in the way we support releases. > > I would have expected to have 5.1.x as an ABI compatible upgrade of > 5.0 with non-trivial new features. > > The trivial API updates in stable releases (new accessors for forward > compatibility, ...) would go into the micro version along with the > bug fixes. And should be made for the same reason. > > In the case of new accessors, their visibility should conditioned > the user defining a suitable macro to make them visible. Their > purpose is to facilitate compiling code that's forward-ported > to a later release, but needs to still work with the stable > release. Otherwise, there really should be no feature changes > in stable releases. If I'm reading you correctly then you seem to be suggesting that we avoid the semver rules through some clever slight-of-hand based on conditional macros, i.e. we add new accessors into PATCH releases, but we hide them away behind a conditional macro. It means, effectively, that people cannot rely on the semver versioning if they happen to be using those macros. This seems to make the whole exercise utterly pointless in my mind. Either we go with semver and totally commit to it - or we stick with what we've already got. No half-way, "well we're kind of doing semver, but not really". Matt From tjh at openssl.org Fri Sep 28 07:05:48 2018 From: tjh at openssl.org (Tim Hudson) Date: Fri, 28 Sep 2018 17:05:48 +1000 Subject: [openssl-project] Release strategy updates & other policies In-Reply-To: References: <4c48337a-4e40-c6bf-2263-7a965dcce75f@openssl.org> <20180925.153024.156547327021891373.levitte@openssl.org> <674a7e48-8d5b-c652-ac14-f3e7e51538ef@openssl.org> <73E9CD06-575E-4CDE-B137-95179B07B591@dukhovni.org> Message-ID: On Fri, Sep 28, 2018 at 4:55 PM Matt Caswell wrote: > Either we go with semver and totally commit to it - or we stick with what > we've already got. No > half-way, "well we're kind of doing semver, but not really". > +1 I see no point in changing what we are doing *without* getting the benefit of following the semantic versioning approach. Right now things like 1.1.0 with a major API change and 1.1.1 with a major feature update of TLSv1.3 are confusing to those who haven't been along for the journey. Our current handling of version numbering surprises developers and requires careful explanation. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: