From levitte at openssl.org Sat Jun 1 16:31:36 2019 From: levitte at openssl.org (Richard Levitte) Date: Sat, 01 Jun 2019 18:31:36 +0200 Subject: Server shutdown and move on Monday Message-ID: <87o93hz2qf.wl-levitte@openssl.org> Hi, This is a notification that our main server will be shutdown on Monday, between 08:00 and 11:00 UTC. The reason is that the server will move to a different data center. Cheers, Richard ( your friendly sysadmin ) -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Mon Jun 3 10:26:18 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 3 Jun 2019 11:26:18 +0100 Subject: Monthly Status Report (May) Message-ID: <41872622-1762-6a13-a3b5-583d14d28c55@openssl.org> As well as normal reviews, responding to user queries, wiki user requests, OMC business, handling security reports, etc., key activities this month: - Moved various global data items to use OPENSSL_CTX instead in preparation for making EVP available from within the FIPS module - Preparation for 2 ICMC panels (OpenSSL panel and TLS panel) - Improved DSA to reject obviously invalid parameters during signing - Significant review of the CMP PRs - Attended the OMC f2f in Vancouver - Attended the ICMC conference in Vancouver - Enabled EVP from within the FIPS module - Proposed alternative fix to the problem where NULL is passed to EVP_EncryptUpdate - Added CAVS test vectors for CCM - Assisted with the releases of 1.1.1c, 1.1.0k and 1.0.2s - Significant work to make the RAND code available from within the FIPS Module - PR to make the basic AES ciphers available from within FIPS module - PR to make the threading code provider aware - Fixed an error in the test framework that meant, for some tests, a failure didn't really mean failure Matt From paul.dale at oracle.com Tue Jun 4 06:12:50 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Tue, 4 Jun 2019 16:12:50 +1000 Subject: OSSL_PARAMs Message-ID: A question for the committers and beyond. Should the OSSL_PARAM concept be used for more than just the libcrypto ? provider interface? The one that springs to mind is a more general form of the various ctrl calls, there are some string ctrl calls that attempt to do the same thing in a less structured manner. Thoughts? Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From shane.lontis at oracle.com Tue Jun 4 06:30:35 2019 From: shane.lontis at oracle.com (SHANE LONTIS) Date: Tue, 4 Jun 2019 16:30:35 +1000 Subject: OSSL_PARAMs In-Reply-To: References: Message-ID: The controls are easier to understand and use currently. The advantage of params is that you can batch them, so that the order no longer matters during setup (e.g- I need to set EVP_MD first). The params are not being used in anger yet. My main problem with the params is that no data is set into them - only pointers to the data blob. I find this to be a confusing interface which hurts my (tiny) brain (especially with types like strings) I would prefer base types (like int to just be embedded in the set data - even if that meant allocs are needed). My concern is that is the end user is supposed to use what we have currently, there are going to be many issues. Shane > On 4 Jun 2019, at 4:12 pm, Dr Paul Dale wrote: > > A question for the committers and beyond. > > Should the OSSL_PARAM concept be used for more than just the libcrypto ? provider interface? > > The one that springs to mind is a more general form of the various ctrl calls, there are some string ctrl calls that attempt to do the same thing in a less structured manner. > > > Thoughts? > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Tue Jun 4 08:42:35 2019 From: levitte at openssl.org (Richard Levitte) Date: Tue, 04 Jun 2019 10:42:35 +0200 Subject: OSSL_PARAMs In-Reply-To: References: Message-ID: <87muixzqpw.wl-levitte@openssl.org> Part of the idea was that this would be a means of communication between application and provider, just like controls are with libcrypto sub-systems. The main differences are that we aren't stuck with the limitations of the control functions, that the interface is uniform (if you look at the control functions, they have variety, all depending on when they were crafted and by who), and that you can pass or retrieve several parameters in one go. Also, it's extensible (i.e. it's always possible to add new data types if we feel the need to). Cheers, Richard On Tue, 04 Jun 2019 08:12:50 +0200, Dr Paul Dale wrote: > > > A question for the committers and beyond. > > Should the OSSL_PARAM concept be used for more than just the libcrypto ? provider interface? > > The one that springs to mind is a more general form of the various ctrl calls, there are some > string ctrl calls that attempt to do the same thing in a less structured manner. > > Thoughts? > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From mattlindn at gmail.com Mon Jun 3 23:08:24 2019 From: mattlindn at gmail.com (Matthew Lindner) Date: Mon, 3 Jun 2019 16:08:24 -0700 Subject: OpenSSL 3.0.0 FIPS Validation Message-ID: I notice that the OpenSSL 3.0.0 design page https://www.openssl.org/docs/OpenSSL300Design.html still references "CAVS testing" even though CAVS testing is shortly ending with the release of ACVP testing. See: https://csrc.nist.gov/Projects/Automated-Cryptographic-Validation-Testing The new format is JSON based and entirely different from the previous CAVS file format. Is OpenSSL team simply using "CAVS Testing" generically or is OpenSSL team missing some information? - Matthew From matt at openssl.org Tue Jun 4 09:40:31 2019 From: matt at openssl.org (Matt Caswell) Date: Tue, 4 Jun 2019 10:40:31 +0100 Subject: OpenSSL 3.0.0 FIPS Validation In-Reply-To: References: Message-ID: <4da99465-c25e-b70f-9336-11b2031cdaa0@openssl.org> On 04/06/2019 00:08, Matthew Lindner wrote: > I notice that the OpenSSL 3.0.0 design page > https://www.openssl.org/docs/OpenSSL300Design.html still references > "CAVS testing" even though CAVS testing is shortly ending with the > release of ACVP testing. See: > https://csrc.nist.gov/Projects/Automated-Cryptographic-Validation-Testing > The new format is JSON based and entirely different from the previous > CAVS file format. > > Is OpenSSL team simply using "CAVS Testing" generically or is OpenSSL > team missing some information? We're well aware of the ACVP testing that is coming down the line. At the time we made the design decisions it was not clear whether ACVP would be available and sufficiently mature within the timescales we needed it to be. Therefore we made the decision to go with CAVS testing for the initial 3.0 release. Matt From rsalz at akamai.com Tue Jun 4 12:57:00 2019 From: rsalz at akamai.com (Salz, Rich) Date: Tue, 4 Jun 2019 12:57:00 +0000 Subject: OSSL_PARAMs In-Reply-To: <87muixzqpw.wl-levitte@openssl.org> References: <87muixzqpw.wl-levitte@openssl.org> Message-ID: <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> > Part of the idea was that this would be a means of communication between application and provider, just like controls are with libcrypto sub-systems. I can probably find the email thread (or maybe it was a GitHub comment on my proposal for params), where you said, quite definitively, that this was *not* a general-purpose mechanism but rather a way to expose the necessary internals for opaque objects like RSA keys. What changed your mind? Perhaps not surprisingly, I agree with Shane's assessment and am strongly opposed to the project foisting this on everyone at this time. @DavidBen, your thoughts? From levitte at openssl.org Tue Jun 4 13:26:30 2019 From: levitte at openssl.org (Richard Levitte) Date: Tue, 04 Jun 2019 15:26:30 +0200 Subject: OSSL_PARAMs In-Reply-To: <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> Message-ID: <87lfyhzdkp.wl-levitte@openssl.org> On Tue, 04 Jun 2019 14:57:00 +0200, Salz, Rich wrote: > > > > Part of the idea was that this would be a means of communication > between application and provider, just like controls are with > libcrypto sub-systems. > > > I can probably find the email thread (or maybe it was a GitHub > comment on my proposal for params), where you said, quite > definitively, that this was *not* a general-purpose mechanism but > rather a way to expose the necessary internals for opaque objects > like RSA keys. Either I misunderstood what you said at the time, or you misunderstood what I said... there's definitely a disconnect here somewhere. What I wonder is why it should be exclusively only one of those options? Either way, the OSSL_PARAM is defined publically and openly (i.e. non-opaque), and we currently have the following functions in the public API: EVP_MD_CTX_set_params EVP_MD_CTX_get_params OSSL_PROVIDER_get_params I fully expect that more will come. I have a branch where I've EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before long (I'm actually rather surprised they haven't already), and I'm absolutely sure we will see similar functions for asymmetric algorithms. > What changed your mind? > > Perhaps not surprisingly, I agree with Shane's assessment and am > strongly opposed to the project foisting this on everyone at this > time. @DavidBen, your thoughts? Maybe we're reading differently, I didn't see Shane being opposed to parameter passing in this way per se, just the exact form of the OSSL_PARAM structure, which is different. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From paul.dale at oracle.com Tue Jun 4 23:18:56 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Wed, 5 Jun 2019 09:18:56 +1000 Subject: OSSL_PARAMs In-Reply-To: <87lfyhzdkp.wl-levitte@openssl.org> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> Message-ID: <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> Shane?s major complaints are about the indirection the OSSL_PARAM structure forces ? for integers and return lengths and the necessity of allocating additional memory in parallel with the OSSL_PARAM. The extra indirection was intended to support const arrays of OSSL_PARAM, which turn out to be a rarity because they aren?t thread safe. With most OSSL_PARAM structure being dynamically created, the need for the indirection seems redundant. E.g. could the return length be moved into OSSL_PARAM? I think so. Moving integral values into the structure is more difficult because BIGNUMs will always need to be references. Allocating additional memory will still be required. I?ve got three obvious solutions: 1. include a void * in the OSSL_PARAM structure that needs to be freed when the structure is destroyed or 2. have a block of data in the OSSL_PARAM structure that can be used for native types (OSSL_UNION_ALIGN works perfectly for this) or 3. add a flag field to the OSSL_PARAM to indicate that the referenced value needs to be freed. The memory allocation comes to the for when reading e.g. a file and extracting data ? either the reader needs a lot of local variables to hold everything or it has to allocated for each. The file?s data is transient in memory. For the most part, the receiver side APIs seem reasonable. It is the owning side that has the complications. I think I might be able come up with some owner side routines that assist here but allowing changes to the params structure would be far easier. I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl functions if not ctrl as well (subject to backward compatibility concerns). Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 4 Jun 2019, at 11:26 pm, Richard Levitte wrote: > > On Tue, 04 Jun 2019 14:57:00 +0200, > Salz, Rich wrote: >> >> >>> Part of the idea was that this would be a means of communication >> between application and provider, just like controls are with >> libcrypto sub-systems. >> >> >> I can probably find the email thread (or maybe it was a GitHub >> comment on my proposal for params), where you said, quite >> definitively, that this was *not* a general-purpose mechanism but >> rather a way to expose the necessary internals for opaque objects >> like RSA keys. > > Either I misunderstood what you said at the time, or you misunderstood > what I said... there's definitely a disconnect here somewhere. > > What I wonder is why it should be exclusively only one of those > options? > > Either way, the OSSL_PARAM is defined publically and openly (i.e. > non-opaque), and we currently have the following functions in the > public API: > > EVP_MD_CTX_set_params > EVP_MD_CTX_get_params > OSSL_PROVIDER_get_params > > I fully expect that more will come. I have a branch where I've > EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if > EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before > long (I'm actually rather surprised they haven't already), and I'm > absolutely sure we will see similar functions for asymmetric > algorithms. > >> What changed your mind? >> >> Perhaps not surprisingly, I agree with Shane's assessment and am >> strongly opposed to the project foisting this on everyone at this >> time. @DavidBen, your thoughts? > > Maybe we're reading differently, I didn't see Shane being opposed to > parameter passing in this way per se, just the exact form of the > OSSL_PARAM structure, which is different. > > Cheers, > Richard > > -- > Richard Levitte levitte at openssl.org > OpenSSL Project http://www.openssl.org/~levitte/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From shane.lontis at oracle.com Wed Jun 5 00:31:52 2019 From: shane.lontis at oracle.com (SHANE LONTIS) Date: Wed, 5 Jun 2019 10:31:52 +1000 Subject: OSSL_PARAMs In-Reply-To: <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> Message-ID: <7142E2F2-6528-4972-8420-3CBA2C407BC7@oracle.com> I presume the reference approach was used to solve the issue of who actually owns/free's the data. > On 5 Jun 2019, at 9:18 am, Dr Paul Dale wrote: > > Shane?s major complaints are about the indirection the OSSL_PARAM structure forces ? for integers and return lengths and the necessity of allocating additional memory in parallel with the OSSL_PARAM. > > The extra indirection was intended to support const arrays of OSSL_PARAM, which turn out to be a rarity because they aren?t thread safe. With most OSSL_PARAM structure being dynamically created, the need for the indirection seems redundant. E.g. could the return length be moved into OSSL_PARAM? I think so. > > Moving integral values into the structure is more difficult because BIGNUMs will always need to be references. Allocating additional memory will still be required. I?ve got three obvious solutions: > > 1. include a void * in the OSSL_PARAM structure that needs to be freed when the structure is destroyed or > 2. have a block of data in the OSSL_PARAM structure that can be used for native types (OSSL_UNION_ALIGN works perfectly for this) or > 3. add a flag field to the OSSL_PARAM to indicate that the referenced value needs to be freed. > > The memory allocation comes to the for when reading e.g. a file and extracting data ? either the reader needs a lot of local variables to hold everything or it has to allocated for each. The file?s data is transient in memory. > > > For the most part, the receiver side APIs seem reasonable. It is the owning side that has the complications. > > I think I might be able come up with some owner side routines that assist here but allowing changes to the params structure would be far easier. > > > I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl functions if not ctrl as well (subject to backward compatibility concerns). > > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > > >> On 4 Jun 2019, at 11:26 pm, Richard Levitte > wrote: >> >> On Tue, 04 Jun 2019 14:57:00 +0200, >> Salz, Rich wrote: >>> >>> >>>> Part of the idea was that this would be a means of communication >>> between application and provider, just like controls are with >>> libcrypto sub-systems. >>> >>> >>> I can probably find the email thread (or maybe it was a GitHub >>> comment on my proposal for params), where you said, quite >>> definitively, that this was *not* a general-purpose mechanism but >>> rather a way to expose the necessary internals for opaque objects >>> like RSA keys. >> >> Either I misunderstood what you said at the time, or you misunderstood >> what I said... there's definitely a disconnect here somewhere. >> >> What I wonder is why it should be exclusively only one of those >> options? >> >> Either way, the OSSL_PARAM is defined publically and openly (i.e. >> non-opaque), and we currently have the following functions in the >> public API: >> >> EVP_MD_CTX_set_params >> EVP_MD_CTX_get_params >> OSSL_PROVIDER_get_params >> >> I fully expect that more will come. I have a branch where I've >> EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if >> EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before >> long (I'm actually rather surprised they haven't already), and I'm >> absolutely sure we will see similar functions for asymmetric >> algorithms. >> >>> What changed your mind? >>> >>> Perhaps not surprisingly, I agree with Shane's assessment and am >>> strongly opposed to the project foisting this on everyone at this >>> time. @DavidBen, your thoughts? >> >> Maybe we're reading differently, I didn't see Shane being opposed to >> parameter passing in this way per se, just the exact form of the >> OSSL_PARAM structure, which is different. >> >> Cheers, >> Richard >> >> -- >> Richard Levitte levitte at openssl.org >> OpenSSL Project http://www.openssl.org/~levitte/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.dale at oracle.com Wed Jun 5 00:50:07 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Wed, 5 Jun 2019 10:50:07 +1000 Subject: OSSL_PARAMs In-Reply-To: <7142E2F2-6528-4972-8420-3CBA2C407BC7@oracle.com> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> <7142E2F2-6528-4972-8420-3CBA2C407BC7@oracle.com> Message-ID: <82A42530-D0D8-44E0-8C3E-2F15A241E246@oracle.com> I thought the references were to allow const arrays of OSSL_PARAM to be viable. A quick check through the code reveals these in test and doc only. There are two instances of OSSL_PARAM arrays being declared in evp, both add the pointed to variable after creation, both only have two elements (the integer and the terminator) and both are stack allocated. I.e. there is currently is no example of the use case for which the indirection is present :( Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 5 Jun 2019, at 10:31 am, SHANE LONTIS wrote: > > I presume the reference approach was used to solve the issue of who actually owns/free's the data. > > >> On 5 Jun 2019, at 9:18 am, Dr Paul Dale > wrote: >> >> Shane?s major complaints are about the indirection the OSSL_PARAM structure forces ? for integers and return lengths and the necessity of allocating additional memory in parallel with the OSSL_PARAM. >> >> The extra indirection was intended to support const arrays of OSSL_PARAM, which turn out to be a rarity because they aren?t thread safe. With most OSSL_PARAM structure being dynamically created, the need for the indirection seems redundant. E.g. could the return length be moved into OSSL_PARAM? I think so. >> >> Moving integral values into the structure is more difficult because BIGNUMs will always need to be references. Allocating additional memory will still be required. I?ve got three obvious solutions: >> >> 1. include a void * in the OSSL_PARAM structure that needs to be freed when the structure is destroyed or >> 2. have a block of data in the OSSL_PARAM structure that can be used for native types (OSSL_UNION_ALIGN works perfectly for this) or >> 3. add a flag field to the OSSL_PARAM to indicate that the referenced value needs to be freed. >> >> The memory allocation comes to the for when reading e.g. a file and extracting data ? either the reader needs a lot of local variables to hold everything or it has to allocated for each. The file?s data is transient in memory. >> >> >> For the most part, the receiver side APIs seem reasonable. It is the owning side that has the complications. >> >> I think I might be able come up with some owner side routines that assist here but allowing changes to the params structure would be far easier. >> >> >> I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl functions if not ctrl as well (subject to backward compatibility concerns). >> >> >> Pauli >> -- >> Dr Paul Dale | Cryptographer | Network Security & Encryption >> Phone +61 7 3031 7217 >> Oracle Australia >> >> >> >>> On 4 Jun 2019, at 11:26 pm, Richard Levitte > wrote: >>> >>> On Tue, 04 Jun 2019 14:57:00 +0200, >>> Salz, Rich wrote: >>>> >>>> >>>>> Part of the idea was that this would be a means of communication >>>> between application and provider, just like controls are with >>>> libcrypto sub-systems. >>>> >>>> >>>> I can probably find the email thread (or maybe it was a GitHub >>>> comment on my proposal for params), where you said, quite >>>> definitively, that this was *not* a general-purpose mechanism but >>>> rather a way to expose the necessary internals for opaque objects >>>> like RSA keys. >>> >>> Either I misunderstood what you said at the time, or you misunderstood >>> what I said... there's definitely a disconnect here somewhere. >>> >>> What I wonder is why it should be exclusively only one of those >>> options? >>> >>> Either way, the OSSL_PARAM is defined publically and openly (i.e. >>> non-opaque), and we currently have the following functions in the >>> public API: >>> >>> EVP_MD_CTX_set_params >>> EVP_MD_CTX_get_params >>> OSSL_PROVIDER_get_params >>> >>> I fully expect that more will come. I have a branch where I've >>> EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if >>> EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before >>> long (I'm actually rather surprised they haven't already), and I'm >>> absolutely sure we will see similar functions for asymmetric >>> algorithms. >>> >>>> What changed your mind? >>>> >>>> Perhaps not surprisingly, I agree with Shane's assessment and am >>>> strongly opposed to the project foisting this on everyone at this >>>> time. @DavidBen, your thoughts? >>> >>> Maybe we're reading differently, I didn't see Shane being opposed to >>> parameter passing in this way per se, just the exact form of the >>> OSSL_PARAM structure, which is different. >>> >>> Cheers, >>> Richard >>> >>> -- >>> Richard Levitte levitte at openssl.org >>> OpenSSL Project http://www.openssl.org/~levitte/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.dale at oracle.com Wed Jun 5 01:11:57 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Wed, 5 Jun 2019 11:11:57 +1000 Subject: OSSL_PARAMs In-Reply-To: <82A42530-D0D8-44E0-8C3E-2F15A241E246@oracle.com> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> <7142E2F2-6528-4972-8420-3CBA2C407BC7@oracle.com> <82A42530-D0D8-44E0-8C3E-2F15A241E246@oracle.com> Message-ID: For a minimally invasive change, the various OSSL_PARAM_construct_ calls could be modified to allocate space for the parameter and its return size if NULL is passed for either. There would need to be some indication of this and a cleanup array call. Done properly, this would permit the same efficiency on the owner?s side but would also simplify OSSL_PARAM array creation for those who aren?t as concerned. The theoretical const array of OSSL_PARAM would still be possible. Is this workable or should something more significantly different be used before things freeze with the 3.0 release? Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 5 Jun 2019, at 10:50 am, Dr Paul Dale wrote: > > I thought the references were to allow const arrays of OSSL_PARAM to be viable. > > A quick check through the code reveals these in test and doc only. There are two instances of OSSL_PARAM arrays being declared in evp, both add the pointed to variable after creation, both only have two elements (the integer and the terminator) and both are stack allocated. I.e. there is currently is no example of the use case for which the indirection is present :( > > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > > >> On 5 Jun 2019, at 10:31 am, SHANE LONTIS > wrote: >> >> I presume the reference approach was used to solve the issue of who actually owns/free's the data. >> >> >>> On 5 Jun 2019, at 9:18 am, Dr Paul Dale > wrote: >>> >>> Shane?s major complaints are about the indirection the OSSL_PARAM structure forces ? for integers and return lengths and the necessity of allocating additional memory in parallel with the OSSL_PARAM. >>> >>> The extra indirection was intended to support const arrays of OSSL_PARAM, which turn out to be a rarity because they aren?t thread safe. With most OSSL_PARAM structure being dynamically created, the need for the indirection seems redundant. E.g. could the return length be moved into OSSL_PARAM? I think so. >>> >>> Moving integral values into the structure is more difficult because BIGNUMs will always need to be references. Allocating additional memory will still be required. I?ve got three obvious solutions: >>> >>> 1. include a void * in the OSSL_PARAM structure that needs to be freed when the structure is destroyed or >>> 2. have a block of data in the OSSL_PARAM structure that can be used for native types (OSSL_UNION_ALIGN works perfectly for this) or >>> 3. add a flag field to the OSSL_PARAM to indicate that the referenced value needs to be freed. >>> >>> The memory allocation comes to the for when reading e.g. a file and extracting data ? either the reader needs a lot of local variables to hold everything or it has to allocated for each. The file?s data is transient in memory. >>> >>> >>> For the most part, the receiver side APIs seem reasonable. It is the owning side that has the complications. >>> >>> I think I might be able come up with some owner side routines that assist here but allowing changes to the params structure would be far easier. >>> >>> >>> I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl functions if not ctrl as well (subject to backward compatibility concerns). >>> >>> >>> Pauli >>> -- >>> Dr Paul Dale | Cryptographer | Network Security & Encryption >>> Phone +61 7 3031 7217 >>> Oracle Australia >>> >>> >>> >>>> On 4 Jun 2019, at 11:26 pm, Richard Levitte > wrote: >>>> >>>> On Tue, 04 Jun 2019 14:57:00 +0200, >>>> Salz, Rich wrote: >>>>> >>>>> >>>>>> Part of the idea was that this would be a means of communication >>>>> between application and provider, just like controls are with >>>>> libcrypto sub-systems. >>>>> >>>>> >>>>> I can probably find the email thread (or maybe it was a GitHub >>>>> comment on my proposal for params), where you said, quite >>>>> definitively, that this was *not* a general-purpose mechanism but >>>>> rather a way to expose the necessary internals for opaque objects >>>>> like RSA keys. >>>> >>>> Either I misunderstood what you said at the time, or you misunderstood >>>> what I said... there's definitely a disconnect here somewhere. >>>> >>>> What I wonder is why it should be exclusively only one of those >>>> options? >>>> >>>> Either way, the OSSL_PARAM is defined publically and openly (i.e. >>>> non-opaque), and we currently have the following functions in the >>>> public API: >>>> >>>> EVP_MD_CTX_set_params >>>> EVP_MD_CTX_get_params >>>> OSSL_PROVIDER_get_params >>>> >>>> I fully expect that more will come. I have a branch where I've >>>> EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if >>>> EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before >>>> long (I'm actually rather surprised they haven't already), and I'm >>>> absolutely sure we will see similar functions for asymmetric >>>> algorithms. >>>> >>>>> What changed your mind? >>>>> >>>>> Perhaps not surprisingly, I agree with Shane's assessment and am >>>>> strongly opposed to the project foisting this on everyone at this >>>>> time. @DavidBen, your thoughts? >>>> >>>> Maybe we're reading differently, I didn't see Shane being opposed to >>>> parameter passing in this way per se, just the exact form of the >>>> OSSL_PARAM structure, which is different. >>>> >>>> Cheers, >>>> Richard >>>> >>>> -- >>>> Richard Levitte levitte at openssl.org >>>> OpenSSL Project http://www.openssl.org/~levitte/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Wed Jun 5 02:34:32 2019 From: levitte at openssl.org (Richard Levitte) Date: Wed, 05 Jun 2019 04:34:32 +0200 Subject: OSSL_PARAMs In-Reply-To: <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> Message-ID: <87k1e0zrnr.wl-levitte@openssl.org> Aside from the discussion below, if there's one thing I would like to change, it the double indirection for the _PTR data types. The data types could still be used to indicate that the value isn't short lived, but could possibly change names to something like OSSL_PARAM_UTF8_CSTRING and OSSL_PARAM_OCTET_CSTRING (C for Constant). On Wed, 05 Jun 2019 01:18:56 +0200, Dr Paul Dale wrote: > Shane?s major complaints are about the indirection the OSSL_PARAM structure forces ? for integers > and return lengths and the necessity of allocating additional memory in parallel with the > OSSL_PARAM. > > The extra indirection was intended to support const arrays of OSSL_PARAM, which turn out to be a > rarity because they aren?t thread safe. The reason why we have this issue is our base C language version choice. C90 doesn't allow this construct: int foo(whatever) { int haha = 0; const OSSL_PARAM params[] = { { 'foo', OSSL_PARAM_INTEGER, &haha, sizeof(haha), NULL }, { NULL, 0, NULL, 0, NULL } }; ... } Because the compiler for that language version isn't allowed to emit code to use '&haha' in an inititializer. Newer C language versions allow this. So while this is an issue for *us*, it isn't necessarily an issue for our users, all depending on what C language version they use. > With most OSSL_PARAM structure being dynamically created, > the need for the indirection seems redundant. E.g. could the return length be moved into > OSSL_PARAM? I think so. The design was not only to be able to have nice compile time initialization, but also to be able to pass the array as 'const OSSL_PARAM *', i.e. an indication to the recipient that the array itself should never be modified (less chance of compromise). Maybe that's overly paranoid, but that was a line of thinking. > Moving integral values into the structure is more difficult because BIGNUMs will always need to be > references. Allocating additional memory will still be required. I?ve got three obvious > solutions: > > 1. include a void * in the OSSL_PARAM structure that needs to be freed when the structure is > destroyed or It's actually perfectly possible to do this today. We already have this pointer, it's called 'data'. > 2. have a block of data in the OSSL_PARAM structure that can be used for native types > (OSSL_UNION_ALIGN works perfectly for this) or My major concern with that, apart from having to modify the OSSL_PARAM items themselves? is that some time in the future, we will want to add another native type that's larger, which means we modify the size of a OSSL_PARAM. It's a public structure, so that can't be treated lightly. Also, with a union of native types, we're losing uniformity on MSB first platforms. Having an exact 1:1 integer size match will be crucial, and that complicates the code quite a bit... not to mention that we have a compatibility problem as soon as one end has a new native type in the union and the other doesn't. (one would imagine that simply using uintmax_t would cover all integer sizes apart from BIGNUM, but the potential size change of that type with newer compilers make such a choice precarious) > 3. add a flag field to the OSSL_PARAM to indicate that the referenced value needs to be freed. By whom? The owner of the array should be in complete control of what's needed already, so should be able to know what needs being deallocated or not. If you're thinking that the receiving side should free certain values, then you need to pass a pointer to the routine to be used to free the value rather than just a flag. > The memory allocation comes to the for when reading e.g. a file and extracting data ? either the > reader needs a lot of local variables to hold everything or it has to allocated for each. The > file?s data is transient in memory. > > For the most part, the receiver side APIs seem reasonable. It is the owning side that has the > complications. > > I think I might be able come up with some owner side routines that assist here but allowing > changes to the params structure would be far easier. > > I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl functions if not ctrl > as well (subject to backward compatibility concerns). > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > On 4 Jun 2019, at 11:26 pm, Richard Levitte wrote: > > On Tue, 04 Jun 2019 14:57:00 +0200, > Salz, Rich wrote: > > Part of the idea was that this would be a means of communication > > between application and provider, just like controls are with > libcrypto sub-systems. > > I can probably find the email thread (or maybe it was a GitHub > comment on my proposal for params), where you said, quite > definitively, that this was *not* a general-purpose mechanism but > rather a way to expose the necessary internals for opaque objects > like RSA keys. > > Either I misunderstood what you said at the time, or you misunderstood > what I said... there's definitely a disconnect here somewhere. > > What I wonder is why it should be exclusively only one of those > options? > > Either way, the OSSL_PARAM is defined publically and openly (i.e. > non-opaque), and we currently have the following functions in the > public API: > > EVP_MD_CTX_set_params > EVP_MD_CTX_get_params > OSSL_PROVIDER_get_params > > I fully expect that more will come. I have a branch where I've > EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if > EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before > long (I'm actually rather surprised they haven't already), and I'm > absolutely sure we will see similar functions for asymmetric > algorithms. > > What changed your mind? > > Perhaps not surprisingly, I agree with Shane's assessment and am > strongly opposed to the project foisting this on everyone at this > time. @DavidBen, your thoughts? > > Maybe we're reading differently, I didn't see Shane being opposed to > parameter passing in this way per se, just the exact form of the > OSSL_PARAM structure, which is different. > > Cheers, > Richard > > -- > Richard Levitte levitte at openssl.org > OpenSSL Project http://www.openssl.org/~levitte/ > > -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From levitte at openssl.org Wed Jun 5 02:40:22 2019 From: levitte at openssl.org (Richard Levitte) Date: Wed, 05 Jun 2019 04:40:22 +0200 Subject: OSSL_PARAMs In-Reply-To: <82A42530-D0D8-44E0-8C3E-2F15A241E246@oracle.com> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> <7142E2F2-6528-4972-8420-3CBA2C407BC7@oracle.com> <82A42530-D0D8-44E0-8C3E-2F15A241E246@oracle.com> Message-ID: <87imtkzre1.wl-levitte@openssl.org> I assume you're talking about the |return_size| indirection here. I think you will see much more of them as soon as the asymmetric algorithms start to show up, as they should naturally contain get_params functionality for the numbers of the key objects. For the moment being, we aren't seeing much of that kind of functionality. Cheers, Richard On Wed, 05 Jun 2019 02:50:07 +0200, Dr Paul Dale wrote: > > > I thought the references were to allow const arrays of OSSL_PARAM to be viable. > > A quick check through the code reveals these in test and doc only. There are two instances of > OSSL_PARAM arrays being declared in evp, both add the pointed to variable after creation, both > only have two elements (the integer and the terminator) and both are stack allocated. I.e. there > is currently is no example of the use case for which the indirection is present :( > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > On 5 Jun 2019, at 10:31 am, SHANE LONTIS wrote: > > I presume the reference approach was used to solve the issue of who actually owns/free's the > data. > > On 5 Jun 2019, at 9:18 am, Dr Paul Dale wrote: > > Shane?s major complaints are about the indirection the OSSL_PARAM structure forces ? for > integers and return lengths and the necessity of allocating additional memory in parallel > with the OSSL_PARAM. > > The extra indirection was intended to support const arrays of OSSL_PARAM, which turn out > to be a rarity because they aren?t thread safe. With most OSSL_PARAM structure being > dynamically created, the need for the indirection seems redundant. E.g. could the return > length be moved into OSSL_PARAM? I think so. > > Moving integral values into the structure is more difficult because BIGNUMs will always > need to be references. Allocating additional memory will still be required. I?ve got > three obvious solutions: > > 1. include a void * in the OSSL_PARAM structure that needs to be freed when the structure > is destroyed or > 2. have a block of data in the OSSL_PARAM structure that can be used for native types > (OSSL_UNION_ALIGN works perfectly for this) or > 3. add a flag field to the OSSL_PARAM to indicate that the referenced value needs to be > freed. > > The memory allocation comes to the for when reading e.g. a file and extracting data ? > either the reader needs a lot of local variables to hold everything or it has to allocated > for each. The file?s data is transient in memory. > > For the most part, the receiver side APIs seem reasonable. It is the owning side that has > the complications. > > I think I might be able come up with some owner side routines that assist here but > allowing changes to the params structure would be far easier. > > I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl functions if > not ctrl as well (subject to backward compatibility concerns). > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > On 4 Jun 2019, at 11:26 pm, Richard Levitte wrote: > > On Tue, 04 Jun 2019 14:57:00 +0200, > Salz, Rich wrote: > > Part of the idea was that this would be a means of communication > > between application and provider, just like controls are with > libcrypto sub-systems. > > I can probably find the email thread (or maybe it was a GitHub > comment on my proposal for params), where you said, quite > definitively, that this was *not* a general-purpose mechanism but > rather a way to expose the necessary internals for opaque objects > like RSA keys. > > Either I misunderstood what you said at the time, or you misunderstood > what I said... there's definitely a disconnect here somewhere. > > What I wonder is why it should be exclusively only one of those > options? > > Either way, the OSSL_PARAM is defined publically and openly (i.e. > non-opaque), and we currently have the following functions in the > public API: > > EVP_MD_CTX_set_params > EVP_MD_CTX_get_params > OSSL_PROVIDER_get_params > > I fully expect that more will come. I have a branch where I've > EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if > EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before > long (I'm actually rather surprised they haven't already), and I'm > absolutely sure we will see similar functions for asymmetric > algorithms. > > What changed your mind? > > Perhaps not surprisingly, I agree with Shane's assessment and am > strongly opposed to the project foisting this on everyone at this > time. @DavidBen, your thoughts? > > Maybe we're reading differently, I didn't see Shane being opposed to > parameter passing in this way per se, just the exact form of the > OSSL_PARAM structure, which is different. > > Cheers, > Richard > > -- > Richard Levitte levitte at openssl.org > OpenSSL Project http://www.openssl.org/~levitte/ > > -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From levitte at openssl.org Wed Jun 5 02:47:56 2019 From: levitte at openssl.org (Richard Levitte) Date: Wed, 05 Jun 2019 04:47:56 +0200 Subject: OSSL_PARAMs In-Reply-To: References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> <7142E2F2-6528-4972-8420-3CBA2C407BC7@oracle.com> <82A42530-D0D8-44E0-8C3E-2F15A241E246@oracle.com> Message-ID: <87h894zr1f.wl-levitte@openssl.org> On Wed, 05 Jun 2019 03:11:57 +0200, Dr Paul Dale wrote: > > > For a minimally invasive change, the various OSSL_PARAM_construct_ calls could be modified to > allocate space for the parameter and its return size if NULL is passed for either. If NULL is passed as return_size, it means the owner isn't interested in the size (this is obviously the case when the OSSL_PARAM array is to be used with a set_params type of call, but may also be true in a get_params type of call if the owner knows the expected size through some other means, such as a parameter being documented to *always* be a 32-bit integer, or for a OSSL_PARAM_UTF8_STRING where the ending NUL byte is always trustable) But you're talking about allocating the whole OSSL_PARAM array on the heap, aren't you? While not structly opposed (even though I think that's wasteful), I think this should be done with a separate set of functions (intuitively, having names with '_alloc_' rather than '_construct_' could be an idea). > There would need to be some indication of this and a cleanup array call. > > Done properly, this would permit the same efficiency on the owner?s side but would also simplify > OSSL_PARAM array creation for those who aren?t as concerned. > The theoretical const array of OSSL_PARAM would still be possible. > > Is this workable or should something more significantly different be used before things freeze > with the 3.0 release? > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > On 5 Jun 2019, at 10:50 am, Dr Paul Dale wrote: > > I thought the references were to allow const arrays of OSSL_PARAM to be viable. > > A quick check through the code reveals these in test and doc only. There are two instances of > OSSL_PARAM arrays being declared in evp, both add the pointed to variable after creation, both > only have two elements (the integer and the terminator) and both are stack allocated. I.e. > there is currently is no example of the use case for which the indirection is present :( > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > On 5 Jun 2019, at 10:31 am, SHANE LONTIS wrote: > > I presume the reference approach was used to solve the issue of who actually owns/free's > the data. > > On 5 Jun 2019, at 9:18 am, Dr Paul Dale wrote: > > Shane?s major complaints are about the indirection the OSSL_PARAM structure forces ? > for integers and return lengths and the necessity of allocating additional memory in > parallel with the OSSL_PARAM. > > The extra indirection was intended to support const arrays of OSSL_PARAM, which turn > out to be a rarity because they aren?t thread safe. With most OSSL_PARAM structure > being dynamically created, the need for the indirection seems redundant. E.g. could > the return length be moved into OSSL_PARAM? I think so. > > Moving integral values into the structure is more difficult because BIGNUMs will > always need to be references. Allocating additional memory will still be required. > I?ve got three obvious solutions: > > 1. include a void * in the OSSL_PARAM structure that needs to be freed when the > structure is destroyed or > 2. have a block of data in the OSSL_PARAM structure that can be used for native types > (OSSL_UNION_ALIGN works perfectly for this) or > 3. add a flag field to the OSSL_PARAM to indicate that the referenced value needs to > be freed. > > The memory allocation comes to the for when reading e.g. a file and extracting data ? > either the reader needs a lot of local variables to hold everything or it has to > allocated for each. The file?s data is transient in memory. > > For the most part, the receiver side APIs seem reasonable. It is the owning side that > has the complications. > > I think I might be able come up with some owner side routines that assist here but > allowing changes to the params structure would be far easier. > > I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl functions > if not ctrl as well (subject to backward compatibility concerns). > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > On 4 Jun 2019, at 11:26 pm, Richard Levitte wrote: > > On Tue, 04 Jun 2019 14:57:00 +0200, > Salz, Rich wrote: > > Part of the idea was that this would be a means of communication > > between application and provider, just like controls are with > libcrypto sub-systems. > > I can probably find the email thread (or maybe it was a GitHub > comment on my proposal for params), where you said, quite > definitively, that this was *not* a general-purpose mechanism but > rather a way to expose the necessary internals for opaque objects > like RSA keys. > > Either I misunderstood what you said at the time, or you misunderstood > what I said... there's definitely a disconnect here somewhere. > > What I wonder is why it should be exclusively only one of those > options? > > Either way, the OSSL_PARAM is defined publically and openly (i.e. > non-opaque), and we currently have the following functions in the > public API: > > EVP_MD_CTX_set_params > EVP_MD_CTX_get_params > OSSL_PROVIDER_get_params > > I fully expect that more will come. I have a branch where I've > EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if > EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before > long (I'm actually rather surprised they haven't already), and I'm > absolutely sure we will see similar functions for asymmetric > algorithms. > > What changed your mind? > > Perhaps not surprisingly, I agree with Shane's assessment and am > strongly opposed to the project foisting this on everyone at this > time. @DavidBen, your thoughts? > > Maybe we're reading differently, I didn't see Shane being opposed to > parameter passing in this way per se, just the exact form of the > OSSL_PARAM structure, which is different. > > Cheers, > Richard > > -- > Richard Levitte levitte at openssl.org > OpenSSL Project http://www.openssl.org/~levitte/ > > -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From shane.lontis at oracle.com Wed Jun 5 02:47:19 2019 From: shane.lontis at oracle.com (SHANE LONTIS) Date: Wed, 5 Jun 2019 12:47:19 +1000 Subject: OSSL_PARAMs In-Reply-To: <87k1e0zrnr.wl-levitte@openssl.org> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> <87k1e0zrnr.wl-levitte@openssl.org> Message-ID: <73FD8437-0939-4A5C-9997-2A1796E5075B@oracle.com> > On 5 Jun 2019, at 12:34 pm, Richard Levitte wrote: > > Aside from the discussion below, if there's one thing I would like to > change, it the double indirection for the _PTR data types. The data > types could still be used to indicate that the value isn't short > lived, but could possibly change names to something like > OSSL_PARAM_UTF8_CSTRING and OSSL_PARAM_OCTET_CSTRING (C for Constant). > > On Wed, 05 Jun 2019 01:18:56 +0200, > Dr Paul Dale wrote: >> Shane?s major complaints are about the indirection the OSSL_PARAM structure forces ? for integers >> and return lengths and the necessity of allocating additional memory in parallel with the >> OSSL_PARAM. >> >> The extra indirection was intended to support const arrays of OSSL_PARAM, which turn out to be a >> rarity because they aren?t thread safe. > > The reason why we have this issue is our base C language version > choice. C90 doesn't allow this construct: > > int foo(whatever) > { > int haha = 0; > const OSSL_PARAM params[] = { > { 'foo', OSSL_PARAM_INTEGER, &haha, sizeof(haha), NULL }, > { NULL, 0, NULL, 0, NULL } > }; > > ... > } > The above code is great in theory, but it looks like in practice we end up dynamically allocating in most cases anyway (via the construct_ methods). And if this is the normal use case then OSSL_PARAMS could be made opaque and only accessed by API?s, then the argument about adding extra types later on should also disappear? > Because the compiler for that language version isn't allowed to emit > code to use '&haha' in an inititializer. Newer C language versions > allow this. > > So while this is an issue for *us*, it isn't necessarily an issue for > our users, all depending on what C language version they use. > >> With most OSSL_PARAM structure being dynamically created, >> the need for the indirection seems redundant. E.g. could the return length be moved into >> OSSL_PARAM? I think so. > > The design was not only to be able to have nice compile time > initialization, but also to be able to pass the array as 'const > OSSL_PARAM *', i.e. an indication to the recipient that the array > itself should never be modified (less chance of compromise). Maybe > that's overly paranoid, but that was a line of thinking. > >> Moving integral values into the structure is more difficult because BIGNUMs will always need to be >> references. Allocating additional memory will still be required. I?ve got three obvious >> solutions: >> >> 1. include a void * in the OSSL_PARAM structure that needs to be freed when the structure is >> destroyed or > > It's actually perfectly possible to do this today. We already have > this pointer, it's called 'data'. > >> 2. have a block of data in the OSSL_PARAM structure that can be used for native types >> (OSSL_UNION_ALIGN works perfectly for this) or > > My major concern with that, apart from having to modify the OSSL_PARAM > items themselves? is that some time in the future, we will want to add > another native type that's larger, which means we modify the size of a > OSSL_PARAM. It's a public structure, so that can't be treated > lightly. > > Also, with a union of native types, we're losing uniformity on MSB > first platforms. Having an exact 1:1 integer size match will be > crucial, and that complicates the code quite a bit... not to mention > that we have a compatibility problem as soon as one end has a new > native type in the union and the other doesn't. > (one would imagine that simply using uintmax_t would cover all integer > sizes apart from BIGNUM, but the potential size change of that type > with newer compilers make such a choice precarious) > >> 3. add a flag field to the OSSL_PARAM to indicate that the referenced value needs to be freed. > > By whom? The owner of the array should be in complete control of > what's needed already, so should be able to know what needs being > deallocated or not. > > If you're thinking that the receiving side should free certain values, > then you need to pass a pointer to the routine to be used to free the > value rather than just a flag. > >> The memory allocation comes to the for when reading e.g. a file and extracting data ? either the >> reader needs a lot of local variables to hold everything or it has to allocated for each. The >> file?s data is transient in memory. >> >> For the most part, the receiver side APIs seem reasonable. It is the owning side that has the >> complications. >> >> I think I might be able come up with some owner side routines that assist here but allowing >> changes to the params structure would be far easier. >> >> I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl functions if not ctrl >> as well (subject to backward compatibility concerns). >> >> Pauli >> -- >> Dr Paul Dale | Cryptographer | Network Security & Encryption >> Phone +61 7 3031 7217 >> Oracle Australia >> >> On 4 Jun 2019, at 11:26 pm, Richard Levitte wrote: >> >> On Tue, 04 Jun 2019 14:57:00 +0200, >> Salz, Rich wrote: >> >> Part of the idea was that this would be a means of communication >> >> between application and provider, just like controls are with >> libcrypto sub-systems. >> >> I can probably find the email thread (or maybe it was a GitHub >> comment on my proposal for params), where you said, quite >> definitively, that this was *not* a general-purpose mechanism but >> rather a way to expose the necessary internals for opaque objects >> like RSA keys. >> >> Either I misunderstood what you said at the time, or you misunderstood >> what I said... there's definitely a disconnect here somewhere. >> >> What I wonder is why it should be exclusively only one of those >> options? >> >> Either way, the OSSL_PARAM is defined publically and openly (i.e. >> non-opaque), and we currently have the following functions in the >> public API: >> >> EVP_MD_CTX_set_params >> EVP_MD_CTX_get_params >> OSSL_PROVIDER_get_params >> >> I fully expect that more will come. I have a branch where I've >> EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if >> EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before >> long (I'm actually rather surprised they haven't already), and I'm >> absolutely sure we will see similar functions for asymmetric >> algorithms. >> >> What changed your mind? >> >> Perhaps not surprisingly, I agree with Shane's assessment and am >> strongly opposed to the project foisting this on everyone at this >> time. @DavidBen, your thoughts? >> >> Maybe we're reading differently, I didn't see Shane being opposed to >> parameter passing in this way per se, just the exact form of the >> OSSL_PARAM structure, which is different. >> >> Cheers, >> Richard >> >> -- >> Richard Levitte levitte at openssl.org >> OpenSSL Project https://urldefense.proofpoint.com/v2/url?u=http-3A__www.openssl.org_-7Elevitte_&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=b1aL1L-m41VGkedIk-9Q7taAEKIshTBwq95Iah07uCk&m=9ytfNGgWmI_VuIgUOtVRqe_gd7wVOdag8ayBWLrTL_Q&s=PH8nRCRnGHZdpfCcpSTpW9mLIgviKCbEw6-5w7cc5i4&e= >> >> > -- > Richard Levitte levitte at openssl.org > OpenSSL Project https://urldefense.proofpoint.com/v2/url?u=http-3A__www.openssl.org_-7Elevitte_&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=b1aL1L-m41VGkedIk-9Q7taAEKIshTBwq95Iah07uCk&m=9ytfNGgWmI_VuIgUOtVRqe_gd7wVOdag8ayBWLrTL_Q&s=PH8nRCRnGHZdpfCcpSTpW9mLIgviKCbEw6-5w7cc5i4&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.dale at oracle.com Wed Jun 5 03:07:24 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Wed, 5 Jun 2019 13:07:24 +1000 Subject: OSSL_PARAMs In-Reply-To: <87k1e0zrnr.wl-levitte@openssl.org> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> <87k1e0zrnr.wl-levitte@openssl.org> Message-ID: Richard wrote: -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > So while this is an issue for *us*, it isn't necessarily an issue for > our users, all depending on what C language version they use. Supporting things *we* can?t use seems a little odd. The alternative construction is about as onerous (the construct calls). On stack arrays won?t provide much performance benefit. Static arrays would but without thread safety. >> With most OSSL_PARAM structure being dynamically created, >> the need for the indirection seems redundant. E.g. could the return length be moved into >> OSSL_PARAM? I think so. > > The design was not only to be able to have nice compile time > initialization, but also to be able to pass the array as 'const > OSSL_PARAM *', i.e. an indication to the recipient that the array > itself should never be modified (less chance of compromise). Maybe > that's overly paranoid, but that was a line of thinking. This is a better reason, not that I think ?const? is all that useful here. An aside: having a const struct that has a pointer to non-const memory isn?t entirely obvious to many. This is a public API, make it as simple as necessary. >> Moving integral values into the structure is more difficult because BIGNUMs will always need to be >> references. Allocating additional memory will still be required. I?ve got three obvious >> solutions: >> >> 1. include a void * in the OSSL_PARAM structure that needs to be freed when the structure is >> destroyed or > > It's actually perfectly possible to do this today. We already have > this pointer, it's called 'data?. And how is a pointer known to be malloced or not? I?m trying to make this easy for users without losing the efficiency that is possible if it is required somewhere. I?m looking at making this kind of code work: OSSL_PARAMS params[10]; int n = 0, max_len; char my_size[20]; scanf(?%d?, &max_len); params[n++] = OSSL_PARAM_construct_utf8_string(?size?, &my_size, sizeof(my_size), NULL); params[n++] = OSSL_PARAM_construct_utf8_string(?name?, NULL, max_len, NULL); params[n++] = OSSL_PARAM_construct_end(); ? OSSL_PARAM_deconstruct_all(params); It is a contrived case but I think it would make using the params easier. >> 2. have a block of data in the OSSL_PARAM structure that can be used for native types >> (OSSL_UNION_ALIGN works perfectly for this) or > > My major concern with that, apart from having to modify the OSSL_PARAM > items themselves? is that some time in the future, we will want to add > another native type that's larger, which means we modify the size of a > OSSL_PARAM. It's a public structure, so that can't be treated > lightly. This is a valid concern. OSSL_UNION_ALIGN isn?t appropriate. Likewise, uintmax_t isn?t binary compatible. > If you're thinking that the receiving side should free certain values, > then you need to pass a pointer to the routine to be used to free the > value rather than just a flag. I agree, this is a bad idea. The API isn?t easy to use at the moment. It is also error prone as soon as something complex is encountered (which we haven?t yet). Shane struggled to figure this out and get it working when trying the KDFs and he is far more experienced than most. I?ll see if I can get some kind of diff or PR together. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.dale at oracle.com Wed Jun 5 03:12:25 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Wed, 5 Jun 2019 13:12:25 +1000 Subject: OSSL_PARAMs In-Reply-To: <87h894zr1f.wl-levitte@openssl.org> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> <7142E2F2-6528-4972-8420-3CBA2C407BC7@oracle.com> <82A42530-D0D8-44E0-8C3E-2F15A241E246@oracle.com> <87h894zr1f.wl-levitte@openssl.org> Message-ID: -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 5 Jun 2019, at 12:47 pm, Richard Levitte wrote: > But you're talking about allocating the whole OSSL_PARAM array on the > heap, aren't you? While not structly opposed (even though I think > that's wasteful), I think this should be done with a separate set of > functions (intuitively, having names with '_alloc_' rather than > '_construct_' could be an idea). Not the whole OSSL_PARAM array, just the data pointed to by (some) of the elements of one. _alloc_ instead of _construct_ makes sense and is better than NULL. I?m fine with `OSSL_PARAM params[20]` in code and filling as many slots as desired. To do entirely heap allocated arrays, we?d want to go via a stack (which is something we shouldn?t require providers to depend on) or use a linked list of some kind. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.dale at oracle.com Wed Jun 5 03:13:39 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Wed, 5 Jun 2019 13:13:39 +1000 Subject: OSSL_PARAMs In-Reply-To: <73FD8437-0939-4A5C-9997-2A1796E5075B@oracle.com> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> <87k1e0zrnr.wl-levitte@openssl.org> <73FD8437-0939-4A5C-9997-2A1796E5075B@oracle.com> Message-ID: <953C196B-6225-443C-9994-5EF9340CFA0C@oracle.com> The OSSL_PARAM structure needs to be visible and not subject to change. Providers shouldn?t necessarily have a dependency on functions from libcrypto. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 5 Jun 2019, at 12:47 pm, SHANE LONTIS wrote: > > > >> On 5 Jun 2019, at 12:34 pm, Richard Levitte > wrote: >> >> Aside from the discussion below, if there's one thing I would like to >> change, it the double indirection for the _PTR data types. The data >> types could still be used to indicate that the value isn't short >> lived, but could possibly change names to something like >> OSSL_PARAM_UTF8_CSTRING and OSSL_PARAM_OCTET_CSTRING (C for Constant). >> >> On Wed, 05 Jun 2019 01:18:56 +0200, >> Dr Paul Dale wrote: >>> Shane?s major complaints are about the indirection the OSSL_PARAM structure forces ? for integers >>> and return lengths and the necessity of allocating additional memory in parallel with the >>> OSSL_PARAM. >>> >>> The extra indirection was intended to support const arrays of OSSL_PARAM, which turn out to be a >>> rarity because they aren?t thread safe. >> >> The reason why we have this issue is our base C language version >> choice. C90 doesn't allow this construct: >> >> int foo(whatever) >> { >> int haha = 0; >> const OSSL_PARAM params[] = { >> { 'foo', OSSL_PARAM_INTEGER, &haha, sizeof(haha), NULL }, >> { NULL, 0, NULL, 0, NULL } >> }; >> >> ... >> } >> > The above code is great in theory, but it looks like in practice we end up dynamically allocating in most cases anyway (via the construct_ methods). > And if this is the normal use case then OSSL_PARAMS could be made opaque and only accessed by API?s, then the argument about adding > extra types later on should also disappear? > > >> Because the compiler for that language version isn't allowed to emit >> code to use '&haha' in an inititializer. Newer C language versions >> allow this. >> >> So while this is an issue for *us*, it isn't necessarily an issue for >> our users, all depending on what C language version they use. >> >>> With most OSSL_PARAM structure being dynamically created, >>> the need for the indirection seems redundant. E.g. could the return length be moved into >>> OSSL_PARAM? I think so. >> >> The design was not only to be able to have nice compile time >> initialization, but also to be able to pass the array as 'const >> OSSL_PARAM *', i.e. an indication to the recipient that the array >> itself should never be modified (less chance of compromise). Maybe >> that's overly paranoid, but that was a line of thinking. >> >>> Moving integral values into the structure is more difficult because BIGNUMs will always need to be >>> references. Allocating additional memory will still be required. I?ve got three obvious >>> solutions: >>> >>> 1. include a void * in the OSSL_PARAM structure that needs to be freed when the structure is >>> destroyed or >> >> It's actually perfectly possible to do this today. We already have >> this pointer, it's called 'data'. >> >>> 2. have a block of data in the OSSL_PARAM structure that can be used for native types >>> (OSSL_UNION_ALIGN works perfectly for this) or >> >> My major concern with that, apart from having to modify the OSSL_PARAM >> items themselves? is that some time in the future, we will want to add >> another native type that's larger, which means we modify the size of a >> OSSL_PARAM. It's a public structure, so that can't be treated >> lightly. >> >> Also, with a union of native types, we're losing uniformity on MSB >> first platforms. Having an exact 1:1 integer size match will be >> crucial, and that complicates the code quite a bit... not to mention >> that we have a compatibility problem as soon as one end has a new >> native type in the union and the other doesn't. >> (one would imagine that simply using uintmax_t would cover all integer >> sizes apart from BIGNUM, but the potential size change of that type >> with newer compilers make such a choice precarious) >> >>> 3. add a flag field to the OSSL_PARAM to indicate that the referenced value needs to be freed. >> >> By whom? The owner of the array should be in complete control of >> what's needed already, so should be able to know what needs being >> deallocated or not. >> >> If you're thinking that the receiving side should free certain values, >> then you need to pass a pointer to the routine to be used to free the >> value rather than just a flag. >> >>> The memory allocation comes to the for when reading e.g. a file and extracting data ? either the >>> reader needs a lot of local variables to hold everything or it has to allocated for each. The >>> file?s data is transient in memory. >>> >>> For the most part, the receiver side APIs seem reasonable. It is the owning side that has the >>> complications. >>> >>> I think I might be able come up with some owner side routines that assist here but allowing >>> changes to the params structure would be far easier. >>> >>> I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl functions if not ctrl >>> as well (subject to backward compatibility concerns). >>> >>> Pauli >>> -- >>> Dr Paul Dale | Cryptographer | Network Security & Encryption >>> Phone +61 7 3031 7217 >>> Oracle Australia >>> >>> On 4 Jun 2019, at 11:26 pm, Richard Levitte > wrote: >>> >>> On Tue, 04 Jun 2019 14:57:00 +0200, >>> Salz, Rich wrote: >>> >>> Part of the idea was that this would be a means of communication >>> >>> between application and provider, just like controls are with >>> libcrypto sub-systems. >>> >>> I can probably find the email thread (or maybe it was a GitHub >>> comment on my proposal for params), where you said, quite >>> definitively, that this was *not* a general-purpose mechanism but >>> rather a way to expose the necessary internals for opaque objects >>> like RSA keys. >>> >>> Either I misunderstood what you said at the time, or you misunderstood >>> what I said... there's definitely a disconnect here somewhere. >>> >>> What I wonder is why it should be exclusively only one of those >>> options? >>> >>> Either way, the OSSL_PARAM is defined publically and openly (i.e. >>> non-opaque), and we currently have the following functions in the >>> public API: >>> >>> EVP_MD_CTX_set_params >>> EVP_MD_CTX_get_params >>> OSSL_PROVIDER_get_params >>> >>> I fully expect that more will come. I have a branch where I've >>> EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if >>> EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before >>> long (I'm actually rather surprised they haven't already), and I'm >>> absolutely sure we will see similar functions for asymmetric >>> algorithms. >>> >>> What changed your mind? >>> >>> Perhaps not surprisingly, I agree with Shane's assessment and am >>> strongly opposed to the project foisting this on everyone at this >>> time. @DavidBen, your thoughts? >>> >>> Maybe we're reading differently, I didn't see Shane being opposed to >>> parameter passing in this way per se, just the exact form of the >>> OSSL_PARAM structure, which is different. >>> >>> Cheers, >>> Richard >>> >>> -- >>> Richard Levitte levitte at openssl.org >>> OpenSSL Project https://urldefense.proofpoint.com/v2/url?u=http-3A__www.openssl.org_-7Elevitte_&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=b1aL1L-m41VGkedIk-9Q7taAEKIshTBwq95Iah07uCk&m=9ytfNGgWmI_VuIgUOtVRqe_gd7wVOdag8ayBWLrTL_Q&s=PH8nRCRnGHZdpfCcpSTpW9mLIgviKCbEw6-5w7cc5i4&e= >>> >>> >> -- >> Richard Levitte levitte at openssl.org >> OpenSSL Project https://urldefense.proofpoint.com/v2/url?u=http-3A__www.openssl.org_-7Elevitte_&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=b1aL1L-m41VGkedIk-9Q7taAEKIshTBwq95Iah07uCk&m=9ytfNGgWmI_VuIgUOtVRqe_gd7wVOdag8ayBWLrTL_Q&s=PH8nRCRnGHZdpfCcpSTpW9mLIgviKCbEw6-5w7cc5i4&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Wed Jun 5 06:28:08 2019 From: levitte at openssl.org (Richard Levitte) Date: Wed, 05 Jun 2019 08:28:08 +0200 Subject: OSSL_PARAMs In-Reply-To: References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> <87k1e0zrnr.wl-levitte@openssl.org> Message-ID: <87ftoozguf.wl-levitte@openssl.org> On Wed, 05 Jun 2019 05:07:24 +0200, Dr Paul Dale wrote: > > > Richard wrote: > > With most OSSL_PARAM structure being dynamically created, > the need for the indirection seems redundant. E.g. could the return length be moved into > OSSL_PARAM? I think so. > > The design was not only to be able to have nice compile time > initialization, but also to be able to pass the array as 'const > OSSL_PARAM *', i.e. an indication to the recipient that the array > itself should never be modified (less chance of compromise). Maybe > that's overly paranoid, but that was a line of thinking. > > This is a better reason, not that I think ?const? is all that useful here. > > An aside: having a const struct that has a pointer to non-const memory isn?t entirely obvious to > many. This is a public API, make it as simple as necessary. Surely, this can be alleviated with a comment or appropriate documentation? > Moving integral values into the structure is more difficult because BIGNUMs will always > need to be > references. Allocating additional memory will still be required. I?ve got three obvious > solutions: > > 1. include a void * in the OSSL_PARAM structure that needs to be freed when the structure > is > destroyed or > > It's actually perfectly possible to do this today. We already have > this pointer, it's called 'data?. > > And how is a pointer known to be malloced or not? I?m trying to make this easy for users without > losing the efficiency that is possible if it is required somewhere. > > I?m looking at making this kind of code work: > > OSSL_PARAMS params[10]; > int n = 0, max_len; > char my_size[20]; > > scanf(?%d?, &max_len); > params[n++] = OSSL_PARAM_construct_utf8_string(?size?, &my_size, sizeof(my_size), NULL); > params[n++] = OSSL_PARAM_construct_utf8_string(?name?, NULL, max_len, NULL); > params[n++] = OSSL_PARAM_construct_end(); > > ? > > OSSL_PARAM_deconstruct_all(params); Well, going along with the idea of having a separate set of functions called something with '_alloc_' instead of '_construct_', I could imagine an interface like this: OSSL_PARAM OSSL_PARAM_alloc_utf8_string(const char *name, void **buf, size_t bufsize, size_t *return_size); That would allow the OSSL_PARAM array owner to keep track of allocated data by passsing in the address to a pointer, and deallocate what needs to be deallocated after the fact. Or in other words: OSSL_PARAMS params[10]; int n = 0, max_len; char my_size[20]; void *namebuf = NULL; scanf(?%d?, &max_len); params[n++] = OSSL_PARAM_construct_utf8_string(?size?, &my_size, sizeof(my_size), NULL); params[n++] = OSSL_PARAM_alloc_utf8_string(?name?, &namebuf, max_len, NULL); params[n++] = OSSL_PARAM_construct_end(); ? OPENSSL_free(namebuf); > It is a contrived case but I think it would make using the params easier. Not at all contrived, I can see uses for it, but then rather for the get_params kind of call. > 2. have a block of data in the OSSL_PARAM structure that can be used for native types > (OSSL_UNION_ALIGN works perfectly for this) or > > My major concern with that, apart from having to modify the OSSL_PARAM > items themselves? is that some time in the future, we will want to add > another native type that's larger, which means we modify the size of a > OSSL_PARAM. It's a public structure, so that can't be treated > lightly. > > This is a valid concern. OSSL_UNION_ALIGN isn?t appropriate. > Likewise, uintmax_t isn?t binary compatible. I'm glad we agree. > If you're thinking that the receiving side should free certain values, > then you need to pass a pointer to the routine to be used to free the > value rather than just a flag. > > I agree, this is a bad idea. > > The API isn?t easy to use at the moment. It is also error prone as soon as something complex is > encountered (which we haven?t yet). > Shane struggled to figure this out and get it working when trying the KDFs and he is far more > experienced than most. I have had some experience with the MACs... > I?ll see if I can get some kind of diff or PR together. -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From rsalz at akamai.com Wed Jun 5 01:17:25 2019 From: rsalz at akamai.com (Salz, Rich) Date: Wed, 5 Jun 2019 01:17:25 +0000 Subject: OSSL_PARAMs In-Reply-To: References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> <7142E2F2-6528-4972-8420-3CBA2C407BC7@oracle.com> <82A42530-D0D8-44E0-8C3E-2F15A241E246@oracle.com> Message-ID: <9560F3FA-74DF-4FC1-AECE-FFA78FFA16CF@akamai.com> * Is this workable or should something more significantly different be used before things freeze with the 3.0 release? Well, since you asked: https://github.com/openssl/openssl/pull/8377 -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Wed Jun 5 12:43:08 2019 From: levitte at openssl.org (Richard Levitte) Date: Wed, 05 Jun 2019 14:43:08 +0200 Subject: OSSL_PARAMs In-Reply-To: <9560F3FA-74DF-4FC1-AECE-FFA78FFA16CF@akamai.com> References: <87muixzqpw.wl-levitte@openssl.org> <5ABCA8A9-84F0-43A6-A832-153CE22058DF@akamai.com> <87lfyhzdkp.wl-levitte@openssl.org> <9919FEC4-975C-42E3-ABFD-33A17096405D@oracle.com> <7142E2F2-6528-4972-8420-3CBA2C407BC7@oracle.com> <82A42530-D0D8-44E0-8C3E-2F15A241E246@oracle.com> <9560F3FA-74DF-4FC1-AECE-FFA78FFA16CF@akamai.com> Message-ID: <87ef48yzhf.wl-levitte@openssl.org> On Wed, 05 Jun 2019 03:17:25 +0200, Salz, Rich wrote: > > > * Is this workable or should something more significantly different be used before things freeze > with the 3.0 release? > > Well, since you asked: https://github.com/openssl/openssl/pull/8377 For the record, I'm opposed to that change. -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From levitte at openssl.org Thu Jun 6 09:17:03 2019 From: levitte at openssl.org (Richard Levitte) Date: Thu, 06 Jun 2019 11:17:03 +0200 Subject: Holiday in Sweden Message-ID: <87d0jrysxc.wl-levitte@openssl.org> Today is Sweden's National Day, which is a holiday. I'm off the rest of the day. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From paul.dale at oracle.com Fri Jun 7 08:03:26 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Fri, 7 Jun 2019 18:03:26 +1000 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: References: <20190607013350.GP67454@straasha.imrryr.org> <20190607015300.GA12323@roeckx.be> <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> Message-ID: <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> Mark, I agree that the project area would be better for this discussion. For those not involved, this revolves around the vote to merge PR#9084 and possible alternate mitigations for the underlying problem. Tim?s explanatory message is: > For the record, I think we need to do a better job of tackling the underlying issue. > Dealing with systems on first boot with no entropy is an issue for a class of systems. > However we should be distinguishing between contexts where we have decent seeding material (from having had reasonable entropy) from those where we haven't. > We also should be able to persist seeding across boots and have recommendations for how vendors should do that. > We should be able to fall back to handling it at the user level (i.e. non-root user, persisted in the file system). > We need knowledge of install state (for FIPS) and that can also act as a first-boot state effectively. > > And this applies equally to getentropy usage - it is only a problem in very limited contexts where blocking actually makes sense to perform. > That the kernel has no additional entropy available is not necessarily an issue - except in a very limited context (first time boot with no other entropy available from previous contexts). Viktor replied: > I just want to make sure we don't lock ourselves in to a single > potentially clumsy solution in the library. Various strategies > may be appropriate depending on the platform, and ultimately the > cooperation of the system administrator to enable the required > mechanisms. > > Potential additional sources for initial entropy on systems > without getrandom(2) might include: > > RDSEED/RDRAND > TPM (or Virtualized TPM which is sometimes better!) > RNG state saved across reboots. > ... > > This requires knowledge of various platforms, and potential > help from the platform vendors. It might, for example, make > sense to delegate initial seeding to systemd on platforms > that use systemd, and work with the systemd maintainers to > provide a set of sensible entropy sources for initial boot. > > Exposing a decent RNG to virtual guests and containers is > would be another area of focus. My suggestion as a fallback would be Stephan M?ller?s CPU Jitter . He?s collected a large corpus of data from many processors and the scheme works relatively quickly. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 7 Jun 2019, at 5:19 pm, Mark J Cox wrote: > > Could we have this more detailed discussion on -project? > > Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From tmraz at redhat.com Fri Jun 7 08:18:32 2019 From: tmraz at redhat.com (Tomas Mraz) Date: Fri, 07 Jun 2019 10:18:32 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> References: <20190607013350.GP67454@straasha.imrryr.org> <20190607015300.GA12323@roeckx.be> <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> Message-ID: <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> On Fri, 2019-06-07 at 18:03 +1000, Dr Paul Dale wrote: > Mark, I agree that the project area would be better for this > discussion. > > For those not involved, this revolves around the vote to merge > PR#9084 and possible alternate mitigations for the underlying > problem. > > > Tim?s explanatory message is: > > > For the record, I think we need to do a better job of tackling the > > underlying issue. > > Dealing with systems on first boot with no entropy is an issue for > > a class of systems. > > However we should be distinguishing between contexts where we have > > decent seeding material (from having had reasonable entropy) from > > those where we haven't. > > We also should be able to persist seeding across boots and have > > recommendations for how vendors should do that. > > We should be able to fall back to handling it at the user level > > (i.e. non-root user, persisted in the file system). > > We need knowledge of install state (for FIPS) and that can also act > > as a first-boot state effectively. > > > > And this applies equally to getentropy usage - it is only a problem > > in very limited contexts where blocking actually makes sense to > > perform. > > That the kernel has no additional entropy available is not > > necessarily an issue - except in a very limited context (first time > > boot with no other entropy available from previous contexts). The previous context can easily provide false entropy - for example due to the booting system being a cloned virtual machine. Or boot image for a small router device which contains such saved entropy which is distributed to millions of devices worldwide. This was the reason why systemd guys rejected the idea to actually seed the kernel rng with such entropy with the call that counts this seed entropy. They just use it as additional seed with no accounting for the entropy it would provide if it was not reused. > > Viktor replied: > > > I just want to make sure we don't lock ourselves in to a single > > potentially clumsy solution in the library. Various strategies > > may be appropriate depending on the platform, and ultimately the > > cooperation of the system administrator to enable the required > > mechanisms. > > > > Potential additional sources for initial entropy on systems > > without getrandom(2) might include: > > > > RDSEED/RDRAND > > TPM (or Virtualized TPM which is sometimes better!) > > RNG state saved across reboots. > > ... > > > > This requires knowledge of various platforms, and potential > > help from the platform vendors. It might, for example, make > > sense to delegate initial seeding to systemd on platforms > > that use systemd, and work with the systemd maintainers to > > provide a set of sensible entropy sources for initial boot. > > > > Exposing a decent RNG to virtual guests and containers is > > would be another area of focus. >From the point of view of distribution maintainer of OpenSSL I would say what we had in 1.1.1 before the introduction of DEVRANDOM_WAIT had no real problems for us. However I can understand that when OpenSSL is built and installed separately by application vendors on some legacy system that they do not have control of they might appreciate that the library itself tries to solve the hard problems related to the entropy gathering for them. So I would just ask for one thing - all this additional stuff should be optional - I do not of course care wheter it is opt-in or opt-out as that is easily handled in our builds. > > My suggestion as a fallback would be Stephan M?ller?s CPU Jitter. > He?s collected a large corpus of data from many processors and the > scheme works relatively quickly. +1 to that. -- Tom?? Mr?z No matter how far down the wrong road you've gone, turn back. Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.] From paul.dale at oracle.com Fri Jun 7 08:37:06 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Fri, 7 Jun 2019 18:37:06 +1000 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> References: <20190607013350.GP67454@straasha.imrryr.org> <20190607015300.GA12323@roeckx.be> <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> Message-ID: <11802C64-B714-4B02-8E1E-EA1676A0390C@oracle.com> Storing entropy has risks ? cloning attacks, reading the file attacks, etc. It is better trying than doing nothing. Counting such stored entropy as additional data is good but many systems have no alternative entropy available. I have had to introduce such storing schemes (with generation during fulfilment) when devices would take well over ten minutes to find enough entropy to generate their initial (long term) key material. There are ways to mitigate the reuse risk ? lock the entropy file, read it, add the contents to the kernel, regenerate the file and finally unlock it. Plus regenerating the contents and storing them on shutdown (if not periodically). A cloned VM would still be vulnerable (at least for a while). Also agreed for an option to disable this. I?d prefer that it is on by default (i.e. secure by default and requiring a decision to not be secure). I?m expecting a somewhat lively discussion about a sensitive topic :) Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 7 Jun 2019, at 6:18 pm, Tomas Mraz wrote: > > On Fri, 2019-06-07 at 18:03 +1000, Dr Paul Dale wrote: >> Mark, I agree that the project area would be better for this >> discussion. >> >> For those not involved, this revolves around the vote to merge >> PR#9084 and possible alternate mitigations for the underlying >> problem. >> >> >> Tim?s explanatory message is: >> >>> For the record, I think we need to do a better job of tackling the >>> underlying issue. >>> Dealing with systems on first boot with no entropy is an issue for >>> a class of systems. >>> However we should be distinguishing between contexts where we have >>> decent seeding material (from having had reasonable entropy) from >>> those where we haven't. >>> We also should be able to persist seeding across boots and have >>> recommendations for how vendors should do that. >>> We should be able to fall back to handling it at the user level >>> (i.e. non-root user, persisted in the file system). >>> We need knowledge of install state (for FIPS) and that can also act >>> as a first-boot state effectively. >>> >>> And this applies equally to getentropy usage - it is only a problem >>> in very limited contexts where blocking actually makes sense to >>> perform. >>> That the kernel has no additional entropy available is not >>> necessarily an issue - except in a very limited context (first time >>> boot with no other entropy available from previous contexts). > > The previous context can easily provide false entropy - for example due > to the booting system being a cloned virtual machine. Or boot image for > a small router device which contains such saved entropy which is > distributed to millions of devices worldwide. > > This was the reason why systemd guys rejected the idea to actually seed > the kernel rng with such entropy with the call that counts this seed > entropy. They just use it as additional seed with no accounting for the > entropy it would provide if it was not reused. > >> >> Viktor replied: >> >>> I just want to make sure we don't lock ourselves in to a single >>> potentially clumsy solution in the library. Various strategies >>> may be appropriate depending on the platform, and ultimately the >>> cooperation of the system administrator to enable the required >>> mechanisms. >>> >>> Potential additional sources for initial entropy on systems >>> without getrandom(2) might include: >>> >>> RDSEED/RDRAND >>> TPM (or Virtualized TPM which is sometimes better!) >>> RNG state saved across reboots. >>> ... >>> >>> This requires knowledge of various platforms, and potential >>> help from the platform vendors. It might, for example, make >>> sense to delegate initial seeding to systemd on platforms >>> that use systemd, and work with the systemd maintainers to >>> provide a set of sensible entropy sources for initial boot. >>> >>> Exposing a decent RNG to virtual guests and containers is >>> would be another area of focus. > > From the point of view of distribution maintainer of OpenSSL I would > say what we had in 1.1.1 before the introduction of DEVRANDOM_WAIT had > no real problems for us. > > However I can understand that when OpenSSL is built and installed > separately by application vendors on some legacy system that they do > not have control of they might appreciate that the library itself tries > to solve the hard problems related to the entropy gathering for them. > > So I would just ask for one thing - all this additional stuff should be > optional - I do not of course care wheter it is opt-in or opt-out as > that is easily handled in our builds. > >> >> My suggestion as a fallback would be Stephan M?ller?s CPU Jitter. >> He?s collected a large corpus of data from many processors and the >> scheme works relatively quickly. > > +1 to that. > > -- > Tom?? Mr?z > No matter how far down the wrong road you've gone, turn back. > Turkish proverb > [You'll know whether the road is wrong if you carefully listen to your > conscience.] -------------- next part -------------- An HTML attachment was scrubbed... URL: From tmraz at redhat.com Fri Jun 7 08:45:59 2019 From: tmraz at redhat.com (Tomas Mraz) Date: Fri, 07 Jun 2019 10:45:59 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> References: <20190607013350.GP67454@straasha.imrryr.org> <20190607015300.GA12323@roeckx.be> <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> Message-ID: On Fri, 2019-06-07 at 10:18 +0200, Tomas Mraz wrote: > On Fri, 2019-06-07 at 18:03 +1000, Dr Paul Dale wrote: > > > > Viktor replied: > > > > > I just want to make sure we don't lock ourselves in to a single > > > potentially clumsy solution in the library. Various strategies > > > may be appropriate depending on the platform, and ultimately the > > > cooperation of the system administrator to enable the required > > > mechanisms. > > > > > > Potential additional sources for initial entropy on systems > > > without getrandom(2) might include: > > > > > > RDSEED/RDRAND > > > TPM (or Virtualized TPM which is sometimes better!) > > > RNG state saved across reboots. > > > ... > > > > > > This requires knowledge of various platforms, and potential > > > help from the platform vendors. It might, for example, make > > > sense to delegate initial seeding to systemd on platforms > > > that use systemd, and work with the systemd maintainers to > > > provide a set of sensible entropy sources for initial boot. > > > > > > Exposing a decent RNG to virtual guests and containers is > > > would be another area of focus. > > From the point of view of distribution maintainer of OpenSSL I would > say what we had in 1.1.1 before the introduction of DEVRANDOM_WAIT > had > no real problems for us. And to clarify myself - we have no problem with the DEVRANDOM_WAIT introduction either as the -DDEVRANDOM=/dev/urandom works nicely for us. -- Tom?? Mr?z No matter how far down the wrong road you've gone, turn back. Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.] From Matthias.St.Pierre at ncp-e.com Fri Jun 7 09:09:45 2019 From: Matthias.St.Pierre at ncp-e.com (Matthias St. Pierre) Date: Fri, 7 Jun 2019 11:09:45 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: References: <20190607013350.GP67454@straasha.imrryr.org> <20190607015300.GA12323@roeckx.be> <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> Message-ID: <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> On 07.06.19 10:45, Tomas Mraz wrote: > >> From the point of view of distribution maintainer of OpenSSL I would >> say what we had in 1.1.1 before the introduction of DEVRANDOM_WAIT >> had >> no real problems for us. > And to clarify myself - we have no problem with the DEVRANDOM_WAIT > introduction either as the -DDEVRANDOM=/dev/urandom works nicely for > us. > Thomas' solution works, but it is not more than a workaround, a hack which is exploiting the fact that the DEVRANDOM_WAIT code is placed inside an `# ifndef DEVRANDOM`. See the discussion on openssl-users: https://mta.openssl.org/pipermail/openssl-users/2019-May/010585.html https://mta.openssl.org/pipermail/openssl-users/2019-May/010593.html https://mta.openssl.org/pipermail/openssl-users/2019-May/010595.html If desired,? I can provide an alternative (competing) pull request which makes the DEVRANDOM_WAIT feature configurable in a proper and reasonable way. The default will be whatever the OMC decides. Matthias From openssl-users at dukhovni.org Fri Jun 7 17:28:30 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 7 Jun 2019 13:28:30 -0400 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> References: <20190607015300.GA12323@roeckx.be> <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> Message-ID: <20190607172830.GS67454@straasha.imrryr.org> On Fri, Jun 07, 2019 at 11:09:45AM +0200, Matthias St. Pierre wrote: > See the discussion on openssl-users: > > https://mta.openssl.org/pipermail/openssl-users/2019-May/010585.html > https://mta.openssl.org/pipermail/openssl-users/2019-May/010593.html > https://mta.openssl.org/pipermail/openssl-users/2019-May/010595.html > > If desired, I can provide an alternative (competing) pull request which > makes the DEVRANDOM_WAIT feature configurable in a proper and > reasonable way. The default will be whatever the OMC decides. I think that having the RNG behaviour capriciously different on different systems based on the whims of whoever built the library for that system is not a good idea. OpenSSL should provide an RNG that does not block "unexpectedly", indefinitely, and unpredictably. Where "unexpectedly", means except possibly early at boot time, but ideally waiting for boot-time entropoy is something that systemd and the like take care of, and application start scripts can just register a dependency on some sort of "entropy" service, whose successful initialization is sufficient to ensure adequately secure non-blocking seeding of applications via one of getentropy(), getrandom(), /dev/urandom... That is, I'd expect most of the work for ensuring adequate entropy to happen outside libcrypto, except for perhaps enabling some additional sources that may be available on various systems. -- Viktor. From kurt at roeckx.be Fri Jun 7 17:42:12 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Fri, 7 Jun 2019 19:42:12 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> References: <20190607013350.GP67454@straasha.imrryr.org> <20190607015300.GA12323@roeckx.be> <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> Message-ID: <20190607174212.GA4547@roeckx.be> On Fri, Jun 07, 2019 at 06:03:26PM +1000, Dr Paul Dale wrote: > > My suggestion as a fallback would be Stephan M?ller?s CPU Jitter . He?s collected a large corpus of data from many processors and the scheme works relatively quickly. I don't think we should be collecting entropy from CPU jitter ourself. If that's something useful, it should be up to the OS to provide it to us. But it seems highly unlikely that that will happen since there seems to be little trust in that it actually provides the entropy. Kurt From kurt at roeckx.be Fri Jun 7 17:52:10 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Fri, 7 Jun 2019 19:52:10 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> References: <20190607013350.GP67454@straasha.imrryr.org> <20190607015300.GA12323@roeckx.be> <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> Message-ID: <20190607175210.GB4547@roeckx.be> On Fri, Jun 07, 2019 at 10:18:32AM +0200, Tomas Mraz wrote: > > From the point of view of distribution maintainer of OpenSSL I would > say what we had in 1.1.1 before the introduction of DEVRANDOM_WAIT had > no real problems for us. Introducing DEVRANDOM_WAIT didn't cause any change for us, because we use getentropy(), and a recent kernel. But even systems that use getentropy() with an older kernel can have a large delay after boot. Kurt From kurt at roeckx.be Fri Jun 7 18:04:55 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Fri, 7 Jun 2019 20:04:55 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <20190607172830.GS67454@straasha.imrryr.org> References: <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> Message-ID: <20190607180455.GC4547@roeckx.be> On Fri, Jun 07, 2019 at 01:28:30PM -0400, Viktor Dukhovni wrote: > > I think that having the RNG behaviour capriciously different on > different systems based on the whims of whoever built the library > for that system is not a good idea. OpenSSL should provide an RNG > that does not block "unexpectedly", indefinitely, and unpredictably. > > Where "unexpectedly", means except possibly early at boot time, but > ideally waiting for boot-time entropoy is something that systemd > and the like take care of, and application start scripts can just > register a dependency on some sort of "entropy" service, whose > successful initialization is sufficient to ensure adequately secure > non-blocking seeding of applications via one of getentropy(), > getrandom(), /dev/urandom... > > That is, I'd expect most of the work for ensuring adequate entropy > to happen outside libcrypto, except for perhaps enabling some > additional sources that may be available on various systems. It seems unlikely that anything related to this will ever change, but we can always ask. The reason I think nothing will change is that the problem is already solved, use getentropy()/getrandom(). The init system would need to create this kind of service, and then all software not using getentropy()/getrandom() would need to depend on that service. It would be eaier to just switch that software to use getentropy()/getrandom(). Changing the init system, means that this will only work for new versions of an OS. But on those new versions we already use getentropy()/getrandom(). What we want to support is people that use an old OS, but run a new version of OpenSSL on it. That is, people that do not use the OS provided OpenSSL version. Kurt From Matthias.St.Pierre at ncp-e.com Fri Jun 7 18:06:02 2019 From: Matthias.St.Pierre at ncp-e.com (Dr. Matthias St. Pierre) Date: Fri, 7 Jun 2019 18:06:02 +0000 Subject: AW: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <20190607175210.GB4547@roeckx.be> References: <20190607013350.GP67454@straasha.imrryr.org> <20190607015300.GA12323@roeckx.be> <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <20190607175210.GB4547@roeckx.be> Message-ID: <2a770b80af954176bb590dbe6b1254b5@Ex13.ncp.local> > Introducing DEVRANDOM_WAIT didn't cause any change for us, because > we use getentropy(), and a recent kernel. But even systems that > use getentropy() with an older kernel can have a large delay after > boot. Yes, but that's the crucial difference IMHO: while getentropy() on blocks once during the early boot phase until its initial seeding completes, the DEVRANDOM_WAIT approach will block several times, depending on how much the other processes drain the /dev/random device. Matthias > -----Urspr?ngliche Nachricht----- > Von: openssl-project Im Auftrag von Kurt Roeckx > Gesendet: Freitag, 7. Juni 2019 19:52 > An: Tomas Mraz > Cc: openssl-project at openssl.org > Betreff: Re: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT > > On Fri, Jun 07, 2019 at 10:18:32AM +0200, Tomas Mraz wrote: > > > > From the point of view of distribution maintainer of OpenSSL I would > > say what we had in 1.1.1 before the introduction of DEVRANDOM_WAIT had > > no real problems for us. > > Introducing DEVRANDOM_WAIT didn't cause any change for us, because > we use getentropy(), and a recent kernel. But even systems that > use getentropy() with an older kernel can have a large delay after > boot. > > > Kurt From Matthias.St.Pierre at ncp-e.com Fri Jun 7 18:11:33 2019 From: Matthias.St.Pierre at ncp-e.com (Dr. Matthias St. Pierre) Date: Fri, 7 Jun 2019 18:11:33 +0000 Subject: AW: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <20190607180455.GC4547@roeckx.be> References: <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> Message-ID: <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> > The reason I think nothing will change is that the problem is > already solved, use getentropy()/getrandom(). I agree completely. > The init system would > need to create this kind of service, and then all software not using > getentropy()/getrandom() would need to depend on that service. It FWIW: systemd already has a service for saving and restoring a random seed. If I understood Tomas correctly, the saved seed is added to the random pool, but without crediting any entropy to it (which sounds reasonable to me). https://www.freedesktop.org/software/systemd/man/systemd-random-seed.service.html Matthias From openssl-users at dukhovni.org Fri Jun 7 18:31:54 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 7 Jun 2019 14:31:54 -0400 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> References: <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> Message-ID: <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> > On Jun 7, 2019, at 2:11 PM, Dr. Matthias St. Pierre wrote: > >> The init system would >> need to create this kind of service, and then all software not using >> getentropy()/getrandom() would need to depend on that service. It > > FWIW: systemd already has a service for saving and restoring a random seed. > If I understood Tomas correctly, the saved seed is added to the random pool, > but without crediting any entropy to it (which sounds reasonable to me). That's a different issue. What I was suggesting was a service that waits for seeding to be ready. So that other services can depend on that service, with that service using various sources to adequately seed the kernel RNG, with configurable additional sources beyond the save file, possibly with a non-zero entropy estimate. Thus, for example, a virtual machine or container might make use of an interface to get a a trusted seed from the host hypervisor/OS. Or a physical host might trust its TPM, ... This is not the sort of thing to bolt into the kernel, but is not unreasonable for systemd and the like. Applications can then use getentropy() and not even block at boot time, if the system collects entropy at boot automatically and without excessive delay. The job of improving the source quality and eliminating avoidable delay is then (correctly IMHO) the responsibility of the platform's init system. As for what to do on older platforms, ... add an entropy gathering service to the system start up configuration, and make applications that need early seed material depend on that service. Perhaps the OpenSSL project can curate some examples of such service configurations/scripts. The simplest might be just DEVRANDOM_WAIT as a service that runs at boot, and only reports "ready" once /dev/random is ready. After that applications can just use /dev/urandom with some confidence of adequate seeding. -- Viktor. From kurt at roeckx.be Fri Jun 7 18:38:17 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Fri, 7 Jun 2019 20:38:17 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <2a770b80af954176bb590dbe6b1254b5@Ex13.ncp.local> References: <20190607015300.GA12323@roeckx.be> <8C2DC914-8974-44E2-AD42-5A9CCC1F8C87@dukhovni.org> <20190607044920.GA15342@roeckx.be> <20190607045446.GQ67454@straasha.imrryr.org> <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <20190607175210.GB4547@roeckx.be> <2a770b80af954176bb590dbe6b1254b5@Ex13.ncp.local> Message-ID: <20190607183817.GA6664@roeckx.be> On Fri, Jun 07, 2019 at 06:06:02PM +0000, Dr. Matthias St. Pierre wrote: > > Introducing DEVRANDOM_WAIT didn't cause any change for us, because > > we use getentropy(), and a recent kernel. But even systems that > > use getentropy() with an older kernel can have a large delay after > > boot. > > Yes, but that's the crucial difference IMHO: while getentropy() on blocks once > during the early boot phase until its initial seeding completes, the DEVRANDOM_WAIT > approach will block several times, depending on how much the other processes drain > the /dev/random device. I agree that the solution is not ideal, but I think it's better than not having it. Kurt From kurt at roeckx.be Fri Jun 7 18:41:01 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Fri, 7 Jun 2019 20:41:01 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> References: <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> Message-ID: <20190607184101.GB6664@roeckx.be> On Fri, Jun 07, 2019 at 02:31:54PM -0400, Viktor Dukhovni wrote: > > That's a different issue. What I was suggesting was a service that > waits for seeding to be ready. So that other services can depend > on that service, with that service using various sources to adequately > seed the kernel RNG, with configurable additional sources beyond the > save file, possibly with a non-zero entropy estimate. Thus, for example, > a virtual machine or container might make use of an interface to get a > a trusted seed from the host hypervisor/OS. Or a physical host might > trust its TPM, ... > > This is not the sort of thing to bolt into the kernel, but is not > unreasonable for systemd and the like. The kernel actually already does this in recent versions, if configured to do it. Kurt From openssl-users at dukhovni.org Fri Jun 7 19:08:24 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 7 Jun 2019 15:08:24 -0400 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <20190607184101.GB6664@roeckx.be> References: <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> Message-ID: <5610616C-E4B8-4295-B622-0268B62A9D31@dukhovni.org> > On Jun 7, 2019, at 2:41 PM, Kurt Roeckx wrote: > >> This is not the sort of thing to bolt into the kernel, but is not >> unreasonable for systemd and the like. > > The kernel actually already does this in recent versions, if > configured to do it. We're talking about what to do with for older kernels, and in cases when the kernel cannot promptly obtain sufficient entropy without external sources. The kernel's job is to mix in entropy from natural activity. Boot-time acquisition of non-trivial entropy by other means falls outside the kernel, and may be needed when the kernel cannot obtain sufficient entropy on its own in a timely manner. -- Viktor. From kurt at roeckx.be Fri Jun 7 19:25:50 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Fri, 7 Jun 2019 21:25:50 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <5610616C-E4B8-4295-B622-0268B62A9D31@dukhovni.org> References: <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> <5610616C-E4B8-4295-B622-0268B62A9D31@dukhovni.org> Message-ID: <20190607192550.GA8153@roeckx.be> On Fri, Jun 07, 2019 at 03:08:24PM -0400, Viktor Dukhovni wrote: > > On Jun 7, 2019, at 2:41 PM, Kurt Roeckx wrote: > > > >> This is not the sort of thing to bolt into the kernel, but is not > >> unreasonable for systemd and the like. > > > > The kernel actually already does this in recent versions, if > > configured to do it. > > We're talking about what to do with for older kernels. For older kernels you install rng-tools that feeds the hwrng in the kernel. Kurt From openssl-users at dukhovni.org Fri Jun 7 19:37:07 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 7 Jun 2019 15:37:07 -0400 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <20190607192550.GA8153@roeckx.be> References: <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> <5610616C-E4B8-4295-B622-0268B62A9D31@dukhovni.org> <20190607192550.GA8153@roeckx.be> Message-ID: <0EB0F2D5-F22E-4F61-8025-2135B8D33730@dukhovni.org> > On Jun 7, 2019, at 3:25 PM, Kurt Roeckx wrote: > > For older kernels you install rng-tools that feeds the hwrng in > the kernel. Which works for me, and is pretty much the point I'm trying to make. Then, read /dev/random once early at boot, and do nothing special libcrypto (safely use /dev/urandom). -- Viktor. From rsalz at akamai.com Fri Jun 7 19:01:30 2019 From: rsalz at akamai.com (Salz, Rich) Date: Fri, 7 Jun 2019 19:01:30 +0000 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <20190607184101.GB6664@roeckx.be> References: <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> Message-ID: <13C1C856-CDF2-4612-83D0-AB657AA95AC6@akamai.com> > The kernel actually already does this in recent versions, if configured to do it. "The" kernel. Which one is that? Which operating system? Modern Linux is fine. Is that all we care about? No issues were raised when the RSA keylength was increased, or MD5 was replaced with SHA1. In fact, that is a very good example; we get many questions about "why can't I decrypt old text" because of this. And here we got what, one posting? 1.1.1c made Solaris (and possibly others) more secure. I would be disappointed if 1.1.1d took that away and tried to rationalize that "it's not my job." *YOU'RE A CRYPTOGRAPHIC LIBRARY* From kurt at roeckx.be Fri Jun 7 22:48:39 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Sat, 8 Jun 2019 00:48:39 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <13C1C856-CDF2-4612-83D0-AB657AA95AC6@akamai.com> References: <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> <13C1C856-CDF2-4612-83D0-AB657AA95AC6@akamai.com> Message-ID: <20190607224839.GA13036@roeckx.be> On Fri, Jun 07, 2019 at 07:01:30PM +0000, Salz, Rich wrote: > > > The kernel actually already does this in recent versions, if > configured to do it. > > "The" kernel. Which one is that? Which operating system? > > Modern Linux is fine. Is that all we care about? This whole discussion has only been about Linux, we only define DEVRANDOM_WAIT on Linux. I think all other OSs have a sane /dev/urandom, but I'm not sure about NetBSD. > 1.1.1c made Solaris (and possibly others) more secure. I would be disappointed if 1.1.1d took that away and tried to rationalize that "it's not my job." *YOU'RE A CRYPTOGRAPHIC LIBRARY* Do you have a reference that Solaris allows reading from /dev/urandom before it's been initialized? Kurt From kurt at roeckx.be Fri Jun 7 22:54:36 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Sat, 8 Jun 2019 00:54:36 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <0EB0F2D5-F22E-4F61-8025-2135B8D33730@dukhovni.org> References: <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> <5610616C-E4B8-4295-B622-0268B62A9D31@dukhovni.org> <20190607192550.GA8153@roeckx.be> <0EB0F2D5-F22E-4F61-8025-2135B8D33730@dukhovni.org> Message-ID: <20190607225435.GB13036@roeckx.be> On Fri, Jun 07, 2019 at 03:37:07PM -0400, Viktor Dukhovni wrote: > > On Jun 7, 2019, at 3:25 PM, Kurt Roeckx wrote: > > > > For older kernels you install rng-tools that feeds the hwrng in > > the kernel. > > Which works for me, and is pretty much the point I'm trying to make. > Then, read /dev/random once early at boot, and do nothing special > libcrypto (safely use /dev/urandom). The only thing rng-tools will actually solve is the starvation issue. No service will depend on it, since they don't have any relationship with it. Nor can you wait for it, it's not because it's started that it has initialized the kernel. I think I've also seen reports that it got started too late, actually after a services that wants to ask the kernel for random numbers. An other solution might be that we enable rdrand/rdseed by default as entropy source, after getentropy() and before /dev/urandom. Kurt From openssl-users at dukhovni.org Fri Jun 7 23:04:57 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 7 Jun 2019 19:04:57 -0400 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <20190607225435.GB13036@roeckx.be> References: <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> <5610616C-E4B8-4295-B622-0268B62A9D31@dukhovni.org> <20190607192550.GA8153@roeckx.be> <0EB0F2D5-F22E-4F61-8025-2135B8D33730@dukhovni.org> <20190607225435.GB13036@roeckx.be> Message-ID: <20190607230457.GU67454@straasha.imrryr.org> On Sat, Jun 08, 2019 at 12:54:36AM +0200, Kurt Roeckx wrote: > On Fri, Jun 07, 2019 at 03:37:07PM -0400, Viktor Dukhovni wrote: > > > On Jun 7, 2019, at 3:25 PM, Kurt Roeckx wrote: > > > > > > For older kernels you install rng-tools that feeds the hwrng in > > > the kernel. > > > > Which works for me, and is pretty much the point I'm trying to make. > > Then, read /dev/random once early at boot, and do nothing special > > libcrypto (safely use /dev/urandom). > > The only thing rng-tools will actually solve is the starvation > issue. No service will depend on it, since they don't have any > relationship with it. Nor can you wait for it, it's not because > it's started that it has initialized the kernel. I think I've also > seen reports that it got started too late, actually after a > services that wants to ask the kernel for random numbers. Then a different service can be developed that does block just once at boot, and tries to obtain entropy from a configurable set of sources (to avoid or reduce unbounded delay, and mix in more independent sources). -- Viktor. From kurt at roeckx.be Fri Jun 7 23:24:45 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Sat, 8 Jun 2019 01:24:45 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <20190607230457.GU67454@straasha.imrryr.org> References: <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> <5610616C-E4B8-4295-B622-0268B62A9D31@dukhovni.org> <20190607192550.GA8153@roeckx.be> <0EB0F2D5-F22E-4F61-8025-2135B8D33730@dukhovni.org> <20190607225435.GB13036@roeckx.be> <20190607230457.GU67454@straasha.imrryr.org> Message-ID: <20190607232444.GA10986@roeckx.be> On Fri, Jun 07, 2019 at 07:04:57PM -0400, Viktor Dukhovni wrote: > On Sat, Jun 08, 2019 at 12:54:36AM +0200, Kurt Roeckx wrote: > > > On Fri, Jun 07, 2019 at 03:37:07PM -0400, Viktor Dukhovni wrote: > > > > On Jun 7, 2019, at 3:25 PM, Kurt Roeckx wrote: > > > > > > > > For older kernels you install rng-tools that feeds the hwrng in > > > > the kernel. > > > > > > Which works for me, and is pretty much the point I'm trying to make. > > > Then, read /dev/random once early at boot, and do nothing special > > > libcrypto (safely use /dev/urandom). > > > > The only thing rng-tools will actually solve is the starvation > > issue. No service will depend on it, since they don't have any > > relationship with it. Nor can you wait for it, it's not because > > it's started that it has initialized the kernel. I think I've also > > seen reports that it got started too late, actually after a > > services that wants to ask the kernel for random numbers. > > Then a different service can be developed that does block just once > at boot, and tries to obtain entropy from a configurable set of > sources (to avoid or reduce unbounded delay, and mix in more > independent sources). That's all very nice, but nobody is going to run that. Kurt From paul.dale at oracle.com Fri Jun 7 23:28:43 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Sat, 8 Jun 2019 09:28:43 +1000 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT Message-ID: This vote has been closed, it passed 5 votes to 2 with no abstentions. Up for discussion is the text of the next vote. I?m proposing this: Topic: The OpenSSL 3.0.0 release will include mitigation for the low entropy on boot and first boot problems. Comment: PR#9084 removed such mitigation due to the negative side effects. I?ll make this formal in a day or so, so if anyone wants to suggest alternative wording, that?s the time line. The vote text is the ?topic? line, the comment is explanatory only. Note: I?m not mentioning the mechanism used, that still needs to be decided on. This is just saying that 3.0.0 *will* have some mechanism. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.dale at oracle.com Fri Jun 7 23:37:09 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Sat, 8 Jun 2019 09:37:09 +1000 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <20190607192550.GA8153@roeckx.be> References: <81BC276C-1966-49F1-8CAB-2017FC245256@oracle.com> <2317cfc0069ae4bdb9330b111ebed09660bb4c3d.camel@redhat.com> <265df95a-96d7-26ae-8880-887be8ec02ca@ncp-e.com> <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> <5610616C-E4B8-4295-B622-0268B62A9D31@dukhovni.org> <20190607192550.GA8153@roeckx.be> Message-ID: <4813BD63-1B55-4E3B-AA1A-E47DD105AF83@oracle.com> The devices where I?ve experienced long start up delays didn?t all have a HRNG. The rng-tools wouldn?t have achieved anything. ColdFire based routers running a 2.0 Linux kernel were the first time I personally saw this one. uClinux still supported 2.0 kernels last I checked ? they are very small and relatively fast. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 8 Jun 2019, at 5:25 am, Kurt Roeckx wrote: > > On Fri, Jun 07, 2019 at 03:08:24PM -0400, Viktor Dukhovni wrote: >>> On Jun 7, 2019, at 2:41 PM, Kurt Roeckx wrote: >>> >>>> This is not the sort of thing to bolt into the kernel, but is not >>>> unreasonable for systemd and the like. >>> >>> The kernel actually already does this in recent versions, if >>> configured to do it. >> >> We're talking about what to do with for older kernels. > > For older kernels you install rng-tools that feeds the hwrng in > the kernel. > > > Kurt > -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Fri Jun 7 23:38:31 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 7 Jun 2019 19:38:31 -0400 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <20190607232444.GA10986@roeckx.be> References: <20190607172830.GS67454@straasha.imrryr.org> <20190607180455.GC4547@roeckx.be> <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> <5610616C-E4B8-4295-B622-0268B62A9D31@dukhovni.org> <20190607192550.GA8153@roeckx.be> <0EB0F2D5-F22E-4F61-8025-2135B8D33730@dukhovni.org> <20190607225435.GB13036@roeckx.be> <20190607230457.GU67454@straasha.imrryr.org> <20190607232444.GA10986@roeckx.be> Message-ID: <19A3CA0A-7D6C-45C7-B2FD-9F5DCCC0EBA4@dukhovni.org> > On Jun 7, 2019, at 7:24 PM, Kurt Roeckx wrote: > > That's all very nice, but nobody is going to run that. They also don't have to upgrade their kernel, or deploy new versions of OpenSSL. If platform release engineers don't deploy core services that ensure reliably CSPRNG seeding, then their platform is less secure at boot. This is their choice. Users can vote with their feet for more secure O/S distributions. Secure CSPRNG seeding is a platform responsibility, OpenSSL then runs secure PRNGs seeded from the platform. There's only so much we can reasonably do. The rest has to happen outside of OpenSSL, as a pre-requisite. And yes, fallback on RDSEED/RDRAND + TPM (real or virtual) + whatever is available, but ideally not in libcrypto, but rather a service that that seeds the system at boot. Those other mechanisms are often either not fully trusted in isolation, not always available, or too expensive at every process start. The logic to identify which are available, and how many are enough, ... is best extracted to run separately at boot, with the library using either getentropy() or read /dev/urandom (older kernels). -- Viktor. From kurt at roeckx.be Sat Jun 8 10:18:56 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Sat, 8 Jun 2019 12:18:56 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <19A3CA0A-7D6C-45C7-B2FD-9F5DCCC0EBA4@dukhovni.org> References: <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> <5610616C-E4B8-4295-B622-0268B62A9D31@dukhovni.org> <20190607192550.GA8153@roeckx.be> <0EB0F2D5-F22E-4F61-8025-2135B8D33730@dukhovni.org> <20190607225435.GB13036@roeckx.be> <20190607230457.GU67454@straasha.imrryr.org> <20190607232444.GA10986@roeckx.be> <19A3CA0A-7D6C-45C7-B2FD-9F5DCCC0EBA4@dukhovni.org> Message-ID: <20190608101854.GB10986@roeckx.be> On Fri, Jun 07, 2019 at 07:38:31PM -0400, Viktor Dukhovni wrote: > > On Jun 7, 2019, at 7:24 PM, Kurt Roeckx wrote: > > > > That's all very nice, but nobody is going to run that. > > They also don't have to upgrade their kernel, or deploy new > versions of OpenSSL. If platform release engineers don't > deploy core services that ensure reliably CSPRNG seeding, > then their platform is less secure at boot. This is their > choice. Users can vote with their feet for more secure > O/S distributions. That all sounds very nice, but when a typical users buys a device they have no idea what OS is going to be on it, or that it's secure or not. They don't actually get to vote with their money. > Secure CSPRNG seeding is a platform responsibility, OpenSSL > then runs secure PRNGs seeded from the platform. There's > only so much we can reasonably do. The rest has to happen > outside of OpenSSL, as a pre-requisite. And the people that make those platforms have no clue and no time to care about that. It works, ship it. If we don't make clear it's broken, it's not going to get fixed. Kurt From openssl-users at dukhovni.org Sat Jun 8 18:24:47 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Sat, 8 Jun 2019 14:24:47 -0400 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: <20190608101854.GB10986@roeckx.be> References: <95bd27b00d274ce7802c790fc910ec6b@Ex13.ncp.local> <840F67D3-A126-49B8-B499-F536406B237F@dukhovni.org> <20190607184101.GB6664@roeckx.be> <5610616C-E4B8-4295-B622-0268B62A9D31@dukhovni.org> <20190607192550.GA8153@roeckx.be> <0EB0F2D5-F22E-4F61-8025-2135B8D33730@dukhovni.org> <20190607225435.GB13036@roeckx.be> <20190607230457.GU67454@straasha.imrryr.org> <20190607232444.GA10986@roeckx.be> <19A3CA0A-7D6C-45C7-B2FD-9F5DCCC0EBA4@dukhovni.org> <20190608101854.GB10986@roeckx.be> Message-ID: <1476A815-DF15-4EBB-B4CB-5807F9E8D78B@dukhovni.org> > On Jun 8, 2019, at 6:18 AM, Kurt Roeckx wrote: > > And the people that make those platforms have no clue and > no time to care about that. It works, ship it. If we don't > make clear it's broken, it's not going to get fixed. Yes, but not at the cost of collateral damage to properly designed working code. I can accept some performance degradation, but not unbounded hangs. Sometimes all we can do is try to educate, and if that fails, shame those who don't follow best practice. Eventually, the cheap unauditable junk will ship with a recent kernel. -- Viktor. From kurt at roeckx.be Sun Jun 9 10:35:52 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Sun, 9 Jun 2019 12:35:52 +0200 Subject: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT In-Reply-To: References: Message-ID: <20190609103552.GA26979@roeckx.be> On Sat, Jun 08, 2019 at 09:28:43AM +1000, Dr Paul Dale wrote: > This vote has been closed, it passed 5 votes to 2 with no abstentions. > > > Up for discussion is the text of the next vote. I?m proposing this: > > Topic: The OpenSSL 3.0.0 release will include mitigation for the low entropy on boot and first boot problems. > Comment: PR#9084 removed such mitigation due to the negative side effects. > > > I?ll make this formal in a day or so, so if anyone wants to suggest alternative wording, that?s the time line. The vote text is the ?topic? line, the comment is explanatory only. > > Note: I?m not mentioning the mechanism used, that still needs to be decided on. This is just saying that 3.0.0 *will* have some mechanism. The only mechanisms I can think of are: - Do something with /dev/random (use it as source, select on it, read a byte from it) - Check for the presence of some file that we require the init system to set up to indicate that /dev/urandom is ready, and wait until it exists. - Don't use /dev/urandom at all Kurt From levitte at openssl.org Wed Jun 12 03:51:44 2019 From: levitte at openssl.org (Richard Levitte) Date: Wed, 12 Jun 2019 05:51:44 +0200 Subject: Removing function names from errors (PR 9058) Message-ID: <878su77967.wl-levitte@openssl.org> Many of us, both past and present, have looked at the error reporting code and wante to change it somehow. There's currently a PR, 9058, which covers one aspect, the function name indicator. A -1 was raised early on for the purpose of starting a discussion, and eventually (possibly?) a vote. A discussion point in that PR is whether it's still interesting to keep information on the system function/callback that was called when we're reporting a system error, i.e. this type of error report: SYSerr(SYS_F_FSTAT, errno); (incidently, there's another PR, 9072, which changes those to 'SYSerr("fstat", errno)') So, the main points of discussion seem to be: - should we remove function indicators in our error reporting? - should we make an exception for system errors (SYSerr())? Cheers, Richard P.S. I'm personally not entirely sure that we need an actual vote, if this discussion shows that we all agree. Perhaps to confirm and make the decision formally solid? -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Wed Jun 12 09:02:25 2019 From: matt at openssl.org (Matt Caswell) Date: Wed, 12 Jun 2019 10:02:25 +0100 Subject: Removing function names from errors (PR 9058) In-Reply-To: <878su77967.wl-levitte@openssl.org> References: <878su77967.wl-levitte@openssl.org> Message-ID: <125134db-3f80-0242-36e4-ddeaa4b0c080@openssl.org> On 12/06/2019 04:51, Richard Levitte wrote: > Many of us, both past and present, have looked at the error reporting > code and wante to change it somehow. There's currently a PR, 9058, > which covers one aspect, the function name indicator. > > A -1 was raised early on for the purpose of starting a discussion, and > eventually (possibly?) a vote. > > A discussion point in that PR is whether it's still interesting to > keep information on the system function/callback that was called when > we're reporting a system error, i.e. this type of error report: > > SYSerr(SYS_F_FSTAT, errno); > > (incidently, there's another PR, 9072, which changes those to > 'SYSerr("fstat", errno)') > > So, the main points of discussion seem to be: > > - should we remove function indicators in our error reporting? > - should we make an exception for system errors (SYSerr())? > > Cheers, > Richard > > P.S. I'm personally not entirely sure that we need an actual vote, if > this discussion shows that we all agree. Perhaps to confirm and make > the decision formally solid? > The big problem with having function codes in the errors is that they are not stable across releases (including patch releases). People can and do write application code that checks for particular types of errors occurring - and those checks can include checking the function code. If the underlying code gets refactored then the functions that are the source of the error can change (even across patch releases). OTOH I do find them quite helpful from a debugging perspective, e.g. when people send in questions along the lines of "I got this error what does it mean/how do I fix it" - although what is actually useful is usually the function name rather than the function code itself. Usually the same information can be figured out if you know the OpenSSL version, filename and line number for the error (which is often but not always also included in the error reports)...very often people neglect to mention the OpenSSL version they are using. In that case knowing the function name can often help to infer it. Another concern is that removing function numbers is a breaking change. But, then again, as I said above these numbers are unstable anyway, so perhaps that's a good thing. I also think we need to reconsider the whole error reporting concept in the light of providers. We cannot and should not expect providers to be using the OpenSSL internal system for generating error codes. So we need to have the ability for providers to be able to register errors on the stack without using OpenSSL built-in error codes. We're getting away with this so far for the default and fips providers because they are "our" providers, so we can ensure all the required function/reason codes are present in libcrypto. That isn't true for third party providers. I suspect if we take away function codes then we will need a vote because it is a breaking change. Matt From openssl at roumenpetrov.info Wed Jun 12 16:39:16 2019 From: openssl at roumenpetrov.info (Roumen Petrov) Date: Wed, 12 Jun 2019 19:39:16 +0300 Subject: Removing function names from errors (PR 9058) In-Reply-To: <878su77967.wl-levitte@openssl.org> References: <878su77967.wl-levitte@openssl.org> Message-ID: <1293090b-c0ef-10a0-a2af-17a99fd6c220@roumenpetrov.info> Richard Levitte wrote: > Many of us, both past and present, have looked at the error reporting > code and wante to change it somehow. There's currently a PR, 9058, > which covers one aspect, the function name indicator. There is a lot of situation where application logs debug information. Adding details for openssl errors is very useful to find reason for failure based only of logs provided by user. Recently an user report to me that it uses application with openssl version, a but based on openssl error information I found that its actual version is a+2 - function code does not exist in (a).? As result is was more easy to find what could be reason for errors. From my point of view current errstr utility is quite useful to find failed function. Error information could be enhanced. I cannot see reasons to strip it down. Regards, Roumen From openssl-users at dukhovni.org Wed Jun 12 22:26:14 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Wed, 12 Jun 2019 18:26:14 -0400 Subject: Removing function names from errors (PR 9058) In-Reply-To: <125134db-3f80-0242-36e4-ddeaa4b0c080@openssl.org> References: <878su77967.wl-levitte@openssl.org> <125134db-3f80-0242-36e4-ddeaa4b0c080@openssl.org> Message-ID: <20190612222614.GG33899@straasha.imrryr.org> On Wed, Jun 12, 2019 at 10:02:25AM +0100, Matt Caswell wrote: > OTOH I do find them quite helpful from a debugging perspective, e.g. when people > send in questions along the lines of "I got this error what does it mean/how do > I fix it" - although what is actually useful is usually the function name rather > than the function code itself. Indeed what's needed is the function name. The numeric code is far less important. On the error consumer side, the idiom I'm familiar with is: while ((err = ERR_get_error_line_data(&file, &line, &data, &flags)) != 0) { ERR_error_string_n(err, buffer, sizeof(buffer)); if (flags & ERR_TXT_STRING) printf("...: %s:%s:%d:%s", buffer, file, line, data); else printf("...: %s:%s:%d", buffer, file, line); } this makes no explicit reference to function numbers, returning the appropriate strings. So any change is likely limited to error producers. On the producer side, my ssl_dane library (used in Exim for example), does depend on the function ordinal API: https://github.com/vdukhovni/ssl_dane/blob/master/danessl.c#L52-L118 https://github.com/vdukhovni/ssl_dane/blob/master/danessl.c#L52-L118 so that would need to change (or be longer supported) if the function ordinals are replaced by strings, or otherwise change. -- Viktor. From paul.dale at oracle.com Thu Jun 13 07:06:16 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Thu, 13 Jun 2019 17:06:16 +1000 Subject: Start up entropy gathering Message-ID: Topic: The OpenSSL 3.0.0 release will include mitigation for the low entropy on boot and first boot problems. Comment: PR#9084 removed such mitigation due to the negative side effects. This vote has passed (two for, four abstain, none against): The discussion now needs to turn to how this can be done. The mitigation is for older Linux kernels only (and perhaps old Unix machines of other flavours). So no Windows, getentropy(3), ?. I have a suggestion. Two kind of. Use a volatile mechanism for communicating between processes ? a file in /tmp, a block of shared memory or whatever. On system boot, the first instance of libcrypto that attempts to read /dev/urandom, first locks the shared resource and reads one byte from /dev/random. It uses the shared resource to indicate that the read was successful and continues seeding from /dev/urandom. Other instances of libcrypto start up, see that the shared resource is properly initialised and go straight to using /dev/urandom. On system shutdown/reboot, the communication mechanism disappears and the entire process will repeat on the next boot. There are some fine details but this is the gist of it. This is more difficult to describe than code: see #9152 . Notes: 1. Reading the byte from /dev/random avoid problems on systems that cannot select(2) on /dev/random. 2. Early start up of /libcrypto will block until entropy is available (and consume eight bits of same). Later starts have the overhead of the locking and resource access. Importantly, no entropy will be drained from the system pool after start up. The second suggestion is broadly similar but requires a file containing entropy that persists across reboots. This alternative requires a more management: the entropy file once read needs to be rewritten immediately (and ideally on shutdown as well). It also introduces a new attack vector against the entropy storage. It also isn?t possible to skip the entropy file read/rewrite sequence because it is impossible to determine if /dev/urandom has actually been seeded. I?ve not attempted to code this, persistent files containing seed material potentially introduce other problems. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Thu Jun 13 01:47:32 2019 From: rsalz at akamai.com (Salz, Rich) Date: Thu, 13 Jun 2019 01:47:32 +0000 Subject: Removing function names from errors (PR 9058) In-Reply-To: <20190612222614.GG33899@straasha.imrryr.org> References: <878su77967.wl-levitte@openssl.org> <125134db-3f80-0242-36e4-ddeaa4b0c080@openssl.org> <20190612222614.GG33899@straasha.imrryr.org> Message-ID: <361CB0EF-3BDB-43E2-98D0-346D7D8D81B1@akamai.com> I think exposing the function internals is a mistake for a couple of reasons: it encourages familiarity with, and dependence on, OpenSSL library internals, and it goes against the spirit of layering, and there is no guarantee that the function code does not change as internal code gets moved around (refactored, removed, etc). We have the source filename and line number available, although this has the some of the same drawbacks as function codes. It's just a little less ugly because C provides that data and we don't wedge it into the error code space. The proper way to handle this, in my experience, is *DO NOT REUSE ERROR CODES.* Each error code appears in exactly one place, and we could eventually build up documentation explaining what they mean, the causes, and how to address this. This means, we don't use ERR_R_MALLOC when trying to create an RSA key, for example, but rather a handful of new errors for ERR_R_RSA_CANT_CREATE_D, ...CANT_CREATE_N, etc. That is a big job, albeit mostly a tedious one. From levitte at openssl.org Thu Jun 13 09:00:06 2019 From: levitte at openssl.org (Richard Levitte) Date: Thu, 13 Jun 2019 11:00:06 +0200 Subject: Removing function names from errors (PR 9058) In-Reply-To: <878su77967.wl-levitte@openssl.org> References: <878su77967.wl-levitte@openssl.org> Message-ID: <877e9p7td5.wl-levitte@openssl.org> The discussion so far has touched on a number of items that aren't necessarily the same thing: - the instability of function codes. - debugging information - display of errors and then, the point that no one talks about but that's the main reason error codes in general exist at all: - an application's need to figure out what went wrong and how to react The current error codes have these items added together: - library code (I'll touch on that further down) - function code (debugging information) - reason code If we look at it from the perspective of the application author, what's most often needed are reliable error/reason codes (to check and to react appropriatly to) and associated reason strings (for display). For the application author, it would normally be completely uninteresting exactly which function the error happened in (think about it, when's the last time you wanted to know exactly which internal function in libc raised the EAGAIN you just got?). For the application author, the interesting bit is usually "what went wrong?" (reason) and "what did I call?" (they already know that). If we look at *our* needs as library writers, then the exact location that raised the error is interesting, of course! But then, is it interesting in the form of codes, or are we rather interested in the text form? I dunno about you, but I don't give a damn about the actual library and function codes, I look at the displayed library names and function names. (and yet, they aren't necessarily enough without knowing the OpenSSL version). Finally, if we look at it from a provider author's perspective, dealing with all these different codes is quite hard. Ideally, a provider author should just need to deal with their own reason codes and associated strings, and that's about it. What I would like to see is a simplified error system that delivers reason codes, and attached extra text information. That text information could have two parts, where one is debugging information (__FILE__, __LINE__, __FUNC__, and possibly exact commit hash generated with 'git archive', see "export-subst" in gitattributes(5)), and the other is the extra data added with the likes of ERR_add_error_data(). We certainly already have the infrastructure to allow this, the modifications needed wouldn't be that big. On the subject of 'openssl errstr', it's abssolutely fine for errors raised within our libraries, but outside of that, it's useless. This is an issue with engines that no one has spoken about but that has been there all along. See, engines, if they use our error reporting system at all, usually allocate a new library code dynamically, by calling ERR_get_next_error_library(). Since the library code may differ from one time to the next, the complete error codes raised will differ, giving 'openssl errstr' an impossible job, even if the engine is loaded. This issue is exactly the same for provider modules. This makes library codes less than useful outside of our libraries. I digress.. the main point is, however, that we need to keep the application author's needs from our needs. That affects how we look at error codes. Cheers, Richard On Wed, 12 Jun 2019 05:51:44 +0200, Richard Levitte wrote: > > Many of us, both past and present, have looked at the error reporting > code and wante to change it somehow. There's currently a PR, 9058, > which covers one aspect, the function name indicator. > > A -1 was raised early on for the purpose of starting a discussion, and > eventually (possibly?) a vote. > > A discussion point in that PR is whether it's still interesting to > keep information on the system function/callback that was called when > we're reporting a system error, i.e. this type of error report: > > SYSerr(SYS_F_FSTAT, errno); > > (incidently, there's another PR, 9072, which changes those to > 'SYSerr("fstat", errno)') > > So, the main points of discussion seem to be: > > - should we remove function indicators in our error reporting? > - should we make an exception for system errors (SYSerr())? > > Cheers, > Richard > > P.S. I'm personally not entirely sure that we need an actual vote, if > this discussion shows that we all agree. Perhaps to confirm and make > the decision formally solid? > > -- > Richard Levitte levitte at openssl.org > OpenSSL Project http://www.openssl.org/~levitte/ > -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Thu Jun 13 09:16:38 2019 From: matt at openssl.org (Matt Caswell) Date: Thu, 13 Jun 2019 10:16:38 +0100 Subject: Removing function names from errors (PR 9058) In-Reply-To: <877e9p7td5.wl-levitte@openssl.org> References: <878su77967.wl-levitte@openssl.org> <877e9p7td5.wl-levitte@openssl.org> Message-ID: <33f929b9-2a30-f1d8-c7bb-b95b005f6f35@openssl.org> I agree with everything Richard just said. I just have some additional thoughts inserted below. On 13/06/2019 10:00, Richard Levitte wrote: > If we look at it from the perspective of the application author, > what's most often needed are reliable error/reason codes (to check and > to react appropriatly to) and associated reason strings (for display). > For the application author, it would normally be completely > uninteresting exactly which function the error happened in (think > about it, when's the last time you wanted to know exactly which > internal function in libc raised the EAGAIN you just got?). For the > application author, the interesting bit is usually "what went wrong?" > (reason) and "what did I call?" (they already know that). > > > If we look at *our* needs as library writers, then the exact location > that raised the error is interesting, of course! But then, is it > interesting in the form of codes, or are we rather interested in the > text form? I dunno about you, but I don't give a damn about the > actual library and function codes, I look at the displayed library > names and function names. (and yet, they aren't necessarily enough > without knowing the OpenSSL version). > > > Finally, if we look at it from a provider author's perspective, > dealing with all these different codes is quite hard. Ideally, a > provider author should just need to deal with their own reason codes > and associated strings, and that's about it. I think you missed another important perspective, i.e. that of the end user. Note that this is slightly different to the needs of the application author. An application author will typically do one of two things if an error occurs: 1) Try and figure out the cause of the error and recover from it, e.g. if I get reason code x then I'll retry the operation with slightly different parameters but if I get y then I'll just ignore this failure and carry on anyway. or 2) If its an error they don't know how to handle then give up and display whatever error details are available to the end user (or log them, or whatever) So the application author is going to want a small number of reason codes that they can handle. If its not in the small set they know about, then give up. The end user OTOH doesn't care about reason codes at all and wants as detailed an explanation of the error (in text form) as is possible. Potentially that might mean different text for every point in the code where we raise an error. Ideally they might want information about the context that led up to the error, i.e. what particular operation was being attempted at the time. I'm thinking that there is some relationship here between the information available from the trace API, and the error API. > What I would like to see is a simplified error system that delivers > reason codes, and attached extra text information. That text > information could have two parts, where one is debugging information > (__FILE__, __LINE__, __FUNC__, and possibly exact commit hash > generated with 'git archive', see "export-subst" in gitattributes(5)), > and the other is the extra data added with the likes of > ERR_add_error_data(). > We certainly already have the infrastructure to allow this, the > modifications needed wouldn't be that big. +1 to this. Matt From tjh at cryptsoft.com Thu Jun 13 09:34:59 2019 From: tjh at cryptsoft.com (Tim Hudson) Date: Thu, 13 Jun 2019 19:34:59 +1000 Subject: Removing function names from errors (PR 9058) In-Reply-To: <361CB0EF-3BDB-43E2-98D0-346D7D8D81B1@akamai.com> References: <878su77967.wl-levitte@openssl.org> <125134db-3f80-0242-36e4-ddeaa4b0c080@openssl.org> <20190612222614.GG33899@straasha.imrryr.org> <361CB0EF-3BDB-43E2-98D0-346D7D8D81B1@akamai.com> Message-ID: On Thu, Jun 13, 2019 at 6:40 PM Salz, Rich wrote: > The proper way to handle this, in my experience, is *DO NOT REUSE ERROR > CODES.* No. This is a path to a rather unacceptable outcome. Taking your example and running forward with it, having an out-of-memory separate error code for every possible context would like to *589 error codes* for just handling out-of-memory in master at a single level. And then on top of that you would need to cascade those errors up the call chain potentially. Each error condition that might require different treatment should have a common code. The concept of out-of-memory (as a simple example) is not context specific (in terms of handling). The concept of unable-to-open-file is also not context specific. What the current error system does is to provide a context for the user when error conditions occur to be able to have some idea as to what happened. It is very much like the java stack trace - and useful for a variety of reasons. The error system had two purposes: 1) allow for handling of an error by the calling function (or any function up the call stack) in a different manner 2) provide the end user with context when things fail (as applications are generally not very good at doing this) Both of these are equally important - but aimed at different contexts (developers, end-users). Neither context should be ignored when considering changes IMHO. Tim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Thu Jun 13 09:50:05 2019 From: levitte at openssl.org (Richard Levitte) Date: Thu, 13 Jun 2019 11:50:05 +0200 Subject: Removing function names from errors (PR 9058) In-Reply-To: <33f929b9-2a30-f1d8-c7bb-b95b005f6f35@openssl.org> References: <878su77967.wl-levitte@openssl.org> <877e9p7td5.wl-levitte@openssl.org> <33f929b9-2a30-f1d8-c7bb-b95b005f6f35@openssl.org> Message-ID: <875zp97r1u.wl-levitte@openssl.org> On Thu, 13 Jun 2019 11:16:38 +0200, Matt Caswell wrote: > > I agree with everything Richard just said. I just have some additional thoughts > inserted below. > > On 13/06/2019 10:00, Richard Levitte wrote: > > If we look at it from the perspective of the application author, > > what's most often needed are reliable error/reason codes (to check and > > to react appropriatly to) and associated reason strings (for display). > > For the application author, it would normally be completely > > uninteresting exactly which function the error happened in (think > > about it, when's the last time you wanted to know exactly which > > internal function in libc raised the EAGAIN you just got?). For the > > application author, the interesting bit is usually "what went wrong?" > > (reason) and "what did I call?" (they already know that). > > > > > > If we look at *our* needs as library writers, then the exact location > > that raised the error is interesting, of course! But then, is it > > interesting in the form of codes, or are we rather interested in the > > text form? I dunno about you, but I don't give a damn about the > > actual library and function codes, I look at the displayed library > > names and function names. (and yet, they aren't necessarily enough > > without knowing the OpenSSL version). > > > > > > Finally, if we look at it from a provider author's perspective, > > dealing with all these different codes is quite hard. Ideally, a > > provider author should just need to deal with their own reason codes > > and associated strings, and that's about it. > > I think you missed another important perspective, i.e. that of the end user. > > Note that this is slightly different to the needs of the application author. An > application author will typically do one of two things if an error occurs: > > 1) Try and figure out the cause of the error and recover from it, e.g. if I get > reason code x then I'll retry the operation with slightly different parameters > but if I get y then I'll just ignore this failure and carry on anyway. > > or > > 2) If its an error they don't know how to handle then give up and display > whatever error details are available to the end user (or log them, or whatever) > > So the application author is going to want a small number of reason codes that > they can handle. If its not in the small set they know about, then give up. > > The end user OTOH doesn't care about reason codes at all and wants as detailed > an explanation of the error (in text form) as is possible. Potentially that > might mean different text for every point in the code where we raise an error. > Ideally they might want information about the context that led up to the error, > i.e. what particular operation was being attempted at the time. I'm thinking > that there is some relationship here between the information available from the > trace API, and the error API. Good point, but note that there is no conflict with what I wrote. The additional information you're talking about is something we currently provide the ERR_add_error_data() function for, and that together with the reason text (derived from the reason code) is the data the end user can reasonably get. It's up to whoever writes the error raising code to provide enough useful information. Incidently, I did mention thinking about a new (better, I hope) error raising function on github, and might as well repeat it here: int ERR_raise_error(int reason, const char *fmt, ...); To allow for debugging information (like I mentioned, some sensible combination of __FILE__, __LINE__, __FUNC__, and possibly git commit id), one might imagine this: int ERR_add_debug_info(const char *file, size_t line, const char *func, const char *commitid); or a combo of both: int ERR_raise_error_debug(int reason, const char *file, size_t line, const char *func, const char *commitid, const char *fmt, ...); Note that this doesn't have to conflict with the current error raising code, all we need to do is to convert whatever input ERR_put_error() gets into the new form. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Thu Jun 13 10:01:46 2019 From: matt at openssl.org (Matt Caswell) Date: Thu, 13 Jun 2019 11:01:46 +0100 Subject: Removing function names from errors (PR 9058) In-Reply-To: <875zp97r1u.wl-levitte@openssl.org> References: <878su77967.wl-levitte@openssl.org> <877e9p7td5.wl-levitte@openssl.org> <33f929b9-2a30-f1d8-c7bb-b95b005f6f35@openssl.org> <875zp97r1u.wl-levitte@openssl.org> Message-ID: On 13/06/2019 10:50, Richard Levitte wrote: > > Good point, but note that there is no conflict with what I wrote. Yes, I realise that. > The > additional information you're talking about is something we currently > provide the ERR_add_error_data() function for, and that together with > the reason text (derived from the reason code) is the data the end > user can reasonably get. It's up to whoever writes the error raising > code to provide enough useful information. Yes, although in practice we don't currently do this (we very rarely add additional explanatory text). Not sure if that is a problem with the API, our coding standards, or our culture. > > Incidently, I did mention thinking about a new (better, I hope) error > raising function on github, and might as well repeat it here: > > int ERR_raise_error(int reason, const char *fmt, ...); > > To allow for debugging information (like I mentioned, some sensible > combination of __FILE__, __LINE__, __FUNC__, and possibly git commit > id), one might imagine this: > > int ERR_add_debug_info(const char *file, size_t line, > const char *func, const char *commitid); > > or a combo of both: > > int ERR_raise_error_debug(int reason, > const char *file, size_t line, > const char *func, const char *commitid, > const char *fmt, ...); > > Note that this doesn't have to conflict with the current error raising > code, all we need to do is to convert whatever input ERR_put_error() > gets into the new form. I think this is quite a good idea. One possible concern is an exponential growth the amount of string data that we have. Not sure what impact that might have on the size of the library. Matt From levitte at openssl.org Thu Jun 13 10:18:57 2019 From: levitte at openssl.org (Richard Levitte) Date: Thu, 13 Jun 2019 12:18:57 +0200 Subject: Removing function names from errors (PR 9058) In-Reply-To: References: <878su77967.wl-levitte@openssl.org> <877e9p7td5.wl-levitte@openssl.org> <33f929b9-2a30-f1d8-c7bb-b95b005f6f35@openssl.org> <875zp97r1u.wl-levitte@openssl.org> Message-ID: <8736kd7ppq.wl-levitte@openssl.org> On Thu, 13 Jun 2019 12:01:46 +0200, Matt Caswell wrote: > > The > > additional information you're talking about is something we currently > > provide the ERR_add_error_data() function for, and that together with > > the reason text (derived from the reason code) is the data the end > > user can reasonably get. It's up to whoever writes the error raising > > code to provide enough useful information. > > Yes, although in practice we don't currently do this (we very rarely add > additional explanatory text). Not sure if that is a problem with the API, our > coding standards, or our culture. This is partly historical... ERR_add_error_data() has been around since the beginning of time, and it looks to me like it was designed in a time where varargs hadn't universally caught on yet (yes, there was a time before varargs, and it's appropriate to call that "the stone age"). In today's coding practices, I personally find ERR_add_error_data() clumsy to deal with, so I seldom use it. Also, being a separate function, it's easy to forget that it's there and useful. That's a reason to think that having all integrated in one function call that includes the flexibility of BIO_printf() probably would encourage producing better information. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From tmraz at redhat.com Thu Jun 13 11:01:02 2019 From: tmraz at redhat.com (Tomas Mraz) Date: Thu, 13 Jun 2019 13:01:02 +0200 Subject: Removing function names from errors (PR 9058) In-Reply-To: References: <878su77967.wl-levitte@openssl.org> <125134db-3f80-0242-36e4-ddeaa4b0c080@openssl.org> <20190612222614.GG33899@straasha.imrryr.org> <361CB0EF-3BDB-43E2-98D0-346D7D8D81B1@akamai.com> Message-ID: <01e629a9e101f416af7c392a80f7018b8f42e80d.camel@redhat.com> On Thu, 2019-06-13 at 19:34 +1000, Tim Hudson wrote: > On Thu, Jun 13, 2019 at 6:40 PM Salz, Rich wrote: > > The proper way to handle this, in my experience, is *DO NOT REUSE > > ERROR CODES.* > > No. This is a path to a rather unacceptable outcome. > Taking your example and running forward with it, having an out-of- > memory separate error code for every possible context would like to > 589 error codes for just handling out-of-memory in master at a single > level. > And then on top of that you would need to cascade those errors up the > call chain potentially. > > Each error condition that might require different treatment should > have a common code. > The concept of out-of-memory (as a simple example) is not context > specific (in terms of handling). > The concept of unable-to-open-file is also not context specific. Exactly, as application writer I do not really care about whether the out of memory condition happened when creating some particular part of RSA key or when allocating memory to store the RSA key der representation or whatever. In case of opening a file, it would be much more useful (by the use of additional error data) to know which file name failed to open. Also the debugging information for the library writers should be attached automatically but of course must be separated from the information useful to the end user. -- Tom?? Mr?z No matter how far down the wrong road you've gone, turn back. Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.] From rsalz at akamai.com Thu Jun 13 16:26:41 2019 From: rsalz at akamai.com (Salz, Rich) Date: Thu, 13 Jun 2019 16:26:41 +0000 Subject: Removing function names from errors (PR 9058) In-Reply-To: <361CB0EF-3BDB-43E2-98D0-346D7D8D81B1@akamai.com> References: <878su77967.wl-levitte@openssl.org> <125134db-3f80-0242-36e4-ddeaa4b0c080@openssl.org> <20190612222614.GG33899@straasha.imrryr.org> <361CB0EF-3BDB-43E2-98D0-346D7D8D81B1@akamai.com> Message-ID: <85962146-A8A6-40AF-B171-CD7F53F71F54@akamai.com> > The proper way to handle this, in my experience, is *DO NOT REUSE ERROR CODES.* Each error code appears in exactly one place, and we could eventually build up documentation explaining what they mean, the causes, and how to address this. This means, we don't use ERR_R_MALLOC when trying to create an RSA key, for example, but rather a handful of new errors for ERR_R_RSA_CANT_CREATE_D, ...CANT_CREATE_N, etc. That is a big job, albeit mostly a tedious one. I got some feedback on- and off-list about this. Most of it was "surely you can't be serious." I am, and stop calling me Shirley. :) Let me add some details. First, recall that OpenSSL has an error stack, and that as errors are "unwound" each function can add its own error code to that stack. This naturally leads to the point where the first entry has the most detailed error, "malloc failed" and the last entry has the most application-oriented error "Could not create RSA key"; along the way are "Could not create d" and "Could not create secure bignum" errors. I hope that makes more sense. HOWEVER, this point (which got the most comments) was a side-note to the main point of my email, which gave some reasons why I think including the function code is a bad idea. Hope this helps. From levitte at openssl.org Thu Jun 13 16:43:04 2019 From: levitte at openssl.org (Richard Levitte) Date: Thu, 13 Jun 2019 18:43:04 +0200 Subject: Removing function names from errors (PR 9058) In-Reply-To: <85962146-A8A6-40AF-B171-CD7F53F71F54@akamai.com> References: <878su77967.wl-levitte@openssl.org> <125134db-3f80-0242-36e4-ddeaa4b0c080@openssl.org> <20190612222614.GG33899@straasha.imrryr.org> <361CB0EF-3BDB-43E2-98D0-346D7D8D81B1@akamai.com> <85962146-A8A6-40AF-B171-CD7F53F71F54@akamai.com> Message-ID: <871rzx77xj.wl-levitte@openssl.org> On Thu, 13 Jun 2019 18:26:41 +0200, Salz, Rich wrote: > > > The proper way to handle this, in my experience, is *DO NOT REUSE ERROR CODES.* Each error code appears in exactly one place, and we could eventually build up documentation explaining what they mean, the causes, and how to address this. This means, we don't use ERR_R_MALLOC when trying to create an RSA key, for example, but rather a handful of new errors for ERR_R_RSA_CANT_CREATE_D, ...CANT_CREATE_N, etc. That is a big job, albeit mostly a tedious one. > > I got some feedback on- and off-list about this. Most of it was > "surely you can't be serious." I am, and stop calling me > Shirley. :) Let me add some details. First, recall that OpenSSL has > an error stack, and that as errors are "unwound" each function can > add its own error code to that stack. This naturally leads to the > point where the first entry has the most detailed error, "malloc > failed" and the last entry has the most application-oriented error > "Could not create RSA key"; along the way are "Could not create d" > and "Could not create secure bignum" errors. I hope that makes more > sense. > > HOWEVER, this point (which got the most comments) was a side-note to > the main point of my email, which gave some reasons why I think > including the function code is a bad idea. So basically, what you're actually saying is that we should add additional errors up the call stack... so if some function called OPENSSL_zalloc(), it should be perfectly OK to raise a ERR_R_MALLOC_FAILURE, but functions above should *add* things like WHATEVER_R_KEY_CREATE_FAILURE, etc etc etc, thereby creating that backtrace that Tim talks about. A point here is that the application may want to find out if there was an allocation error -- at which point it might choose to simply bail -- or some other error that prevented the key to be created (insecure amount of bits, say?) -- at which point it might choose to try and mitigate. The question is what's easier for the application, getting the wanted information as a top error or having to walk to the bottom of the error stack to figure things out. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From rsalz at akamai.com Thu Jun 13 17:13:14 2019 From: rsalz at akamai.com (Salz, Rich) Date: Thu, 13 Jun 2019 17:13:14 +0000 Subject: Removing function names from errors (PR 9058) In-Reply-To: <871rzx77xj.wl-levitte@openssl.org> References: <878su77967.wl-levitte@openssl.org> <125134db-3f80-0242-36e4-ddeaa4b0c080@openssl.org> <20190612222614.GG33899@straasha.imrryr.org> <361CB0EF-3BDB-43E2-98D0-346D7D8D81B1@akamai.com> <85962146-A8A6-40AF-B171-CD7F53F71F54@akamai.com> <871rzx77xj.wl-levitte@openssl.org> Message-ID: <1C6E9BBE-BDF7-413E-9BAE-75F3AB9ABB27@akamai.com> > So basically, what you're actually saying is that we should add additional errors up the call stack... so if some function called OPENSSL_zalloc(), it should be perfectly OK to raise a ERR_R_MALLOC_FAILURE, but functions above should *add* things like WHATEVER_R_KEY_CREATE_FAILURE, etc etc etc, thereby creating that backtrace that Tim talks about. Yes. BUT AGAIN, that's a separate issue to the main thread of function names in errors. Because, as you point out in the second paragraph (which I did not repost), there are still many issues to resolve with that. From openssl at roumenpetrov.info Thu Jun 13 17:50:19 2019 From: openssl at roumenpetrov.info (Roumen Petrov) Date: Thu, 13 Jun 2019 20:50:19 +0300 Subject: Removing function names from errors (PR 9058) In-Reply-To: <20190612222614.GG33899@straasha.imrryr.org> References: <878su77967.wl-levitte@openssl.org> <125134db-3f80-0242-36e4-ddeaa4b0c080@openssl.org> <20190612222614.GG33899@straasha.imrryr.org> Message-ID: <02db4faa-018e-ed18-7e17-b4b42d582e62@roumenpetrov.info> Hi Viktor, Viktor Dukhovni wrote: > On Wed, Jun 12, 2019 at 10:02:25AM +0100, Matt Caswell wrote: > >> OTOH I do find them quite helpful from a debugging perspective, e.g. when people >> send in questions along the lines of "I got this error what does it mean/how do >> I fix it" - although what is actually useful is usually the function name rather >> than the function code itself. > Indeed what's needed is the function name. The numeric code is far > less important. On the error consumer side, the idiom I'm familiar > with is: > > while ((err = ERR_get_error_line_data(&file, &line, &data, &flags)) != 0) { > ERR_error_string_n(err, buffer, sizeof(buffer)); > if (flags & ERR_TXT_STRING) > printf("...: %s:%s:%d:%s", buffer, file, line, data); > else > printf("...: %s:%s:%d", buffer, file, line); > } > > this makes no explicit reference to function numbers, returning the > appropriate strings. So any change is likely limited to error > producers. Actually err encodes library, function and reason. > > On the producer side, my ssl_dane library (used in Exim for example), > does depend on the function ordinal API: > > https://github.com/vdukhovni/ssl_dane/blob/master/danessl.c#L52-L118 > https://github.com/vdukhovni/ssl_dane/blob/master/danessl.c#L52-L118 Why you tables with functions and reasons does not use ERR_PACK ? > so that would need to change (or be longer supported) if the function > ordinals are replaced by strings, or otherwise change. > From mattlindn at gmail.com Thu Jun 13 18:12:01 2019 From: mattlindn at gmail.com (Matthew Lindner) Date: Thu, 13 Jun 2019 11:12:01 -0700 Subject: Removing function names from errors (PR 9058) Message-ID: Hi I'm a developer that writes code that interfaces with OpenSSL. One perspective I think that is missing that I haven't seen is that error codes that point to the exact point in the OpenSSL code that caused an issue are often needed to debug code that interfaces with OpenSSL. Importantly because the documentation is often lacking complete clarity on exactly what any given OpenSSL function does. We (using openssl 1.0.2) from time to time get errors from within OpenSSL that need diving into the OpenSSL code to diagnose. If the errors don't point to the location in the OpenSSL code the error was generated, this makes it all the harder to diagnose issues. Thanks, Matthew From openssl at roumenpetrov.info Thu Jun 13 18:23:05 2019 From: openssl at roumenpetrov.info (Roumen Petrov) Date: Thu, 13 Jun 2019 21:23:05 +0300 Subject: Removing function names from errors (PR 9058) In-Reply-To: <8736kd7ppq.wl-levitte@openssl.org> References: <878su77967.wl-levitte@openssl.org> <877e9p7td5.wl-levitte@openssl.org> <33f929b9-2a30-f1d8-c7bb-b95b005f6f35@openssl.org> <875zp97r1u.wl-levitte@openssl.org> <8736kd7ppq.wl-levitte@openssl.org> Message-ID: Hello, First I did not understand what is wrong to use function or reasons. All of them are subject to particular "library". To parse openssl error code in an application code is bad practice. Richard Levitte wrote: > On Thu, 13 Jun 2019 12:01:46 +0200, > Matt Caswell wrote: >>> The >>> additional information you're talking about is something we currently >>> provide the ERR_add_error_data() function for, and that together with >>> the reason text (derived from the reason code) is the data the end >>> user can reasonably get. It's up to whoever writes the error raising >>> code to provide enough useful information. >> Yes, although in practice we don't currently do this (we very rarely add >> additional explanatory text). Not sure if that is a problem with the API, our >> coding standards, or our culture. > This is partly historical... ERR_add_error_data() has been around > since the beginning of time, and it looks to me like it was designed > in a time where varargs hadn't universally caught on yet (yes, there > was a time before varargs, and it's appropriate to call that "the > stone age"). But developer could? format "extra message" for instance : ??????? NSSerr(NSS_F_RSA_SIGN, NSS_R_UNSUPPORTED_NID); ??????? {/*add extra error message data*/ ??????????? char msgstr[10]; ??????????? BIO_snprintf(msgstr, sizeof(msgstr), "%d", dtype); ??????????? ERR_add_error_data(2, "NID=", msgstr); ??????? } Another sample is when error message is prepared by third party library: ??? ??? SSHLDAPerr(SSHLDAP_F_LDAPSEARCH_ITERATOR, SSHLDAP_R_UNABLE_TO_COUNT_ENTRIES); ?? ... ??????? ret = ldap_parse_result(ld, res, &result, &matcheddn, &errmsg, NULL, NULL, freeit); ?? ... ??? ?????????? if (errmsg) ERR_add_error_data(1, errmsg); > In today's coding practices, I personally find ERR_add_error_data() > clumsy to deal with, so I seldom use it. There is no reason OpenSSL code to use ERR_add_error_data as usually library packs whole functionality. Situation is different when error information if from external sources - IO, CAPI. In my cases it could be from NSS, PKCS#11, LDAP. And so I use this functionality. > Also, being a separate > function, it's easy to forget that it's there and useful. That's a > reason to think that having all integrated in one function call that > includes the flexibility of BIO_printf() probably would encourage > producing better information. > Cheers, > Richard > Roumen From levitte at openssl.org Thu Jun 13 19:31:05 2019 From: levitte at openssl.org (Richard Levitte) Date: Thu, 13 Jun 2019 21:31:05 +0200 Subject: Removing function names from errors (PR 9058) In-Reply-To: References: <878su77967.wl-levitte@openssl.org> <877e9p7td5.wl-levitte@openssl.org> <33f929b9-2a30-f1d8-c7bb-b95b005f6f35@openssl.org> <875zp97r1u.wl-levitte@openssl.org> <8736kd7ppq.wl-levitte@openssl.org> Message-ID: <87zhml5ll2.wl-levitte@openssl.org> On Thu, 13 Jun 2019 20:23:05 +0200, Roumen Petrov wrote: > > Hello, > > First I did not understand what is wrong to use function or > reasons. All of them are subject to particular "library". We didn't say anything against reason codes. As a matter of fact, I do find those useful. Function codes, on the other hand, are unstable and thereby quite useless, at least for figuring out errors. > To parse openssl error code in an application code is bad practice. Is it? Would you say it's bad practice to look at errno codes too? Error *text* is a different matter. > Richard Levitte wrote: > > On Thu, 13 Jun 2019 12:01:46 +0200, > > Matt Caswell wrote: > >>> The > >>> additional information you're talking about is something we currently > >>> provide the ERR_add_error_data() function for, and that together with > >>> the reason text (derived from the reason code) is the data the end > >>> user can reasonably get. It's up to whoever writes the error raising > >>> code to provide enough useful information. > >> Yes, although in practice we don't currently do this (we very rarely add > >> additional explanatory text). Not sure if that is a problem with the API, our > >> coding standards, or our culture. > > This is partly historical... ERR_add_error_data() has been around > > since the beginning of time, and it looks to me like it was designed > > in a time where varargs hadn't universally caught on yet (yes, there > > was a time before varargs, and it's appropriate to call that "the > > stone age"). > > But developer could? format "extra message" for instance : > ??????? NSSerr(NSS_F_RSA_SIGN, NSS_R_UNSUPPORTED_NID); > ??????? {/*add extra error message data*/ > ??????????? char msgstr[10]; > ??????????? BIO_snprintf(msgstr, sizeof(msgstr), "%d", dtype); > ??????????? ERR_add_error_data(2, "NID=", msgstr); > ??????? } I'll counter with a (yet fictitious): ERR_raise_error(NSS_R_UNSUPPORED_NID, "NID=%d", dtype); or if you want to add information that helps debugging: ERR_raise_error_debug(NSS_R_UNSUPPORED_NID, __FILE__, __LINE__, __FUNC__, "$Format:%H", "NID=%d", dtype); (since this is just an idea so far, it's perfectly possible to throw in a library code as well, but like I said already, dynamic library codes have their own issue) I know which I find easier to write. > There is no reason OpenSSL code to use ERR_add_error_data as usually > library packs whole functionality. That's not really true, there are places where OpenSSL code does use ERR_add_error_data(), and for good reason. BIO_lookup and friends add the extra resolver error on error, the CONF code adds information on exactly what value causes a configuration file parsing error, the DSO code adds information on exactly what file it attempted to load (full path if possible), etc etc etc... those are things that can't be part of the error code. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From openssl at roumenpetrov.info Thu Jun 13 20:04:17 2019 From: openssl at roumenpetrov.info (Roumen Petrov) Date: Thu, 13 Jun 2019 23:04:17 +0300 Subject: Removing function names from errors (PR 9058) In-Reply-To: <87zhml5ll2.wl-levitte@openssl.org> References: <878su77967.wl-levitte@openssl.org> <877e9p7td5.wl-levitte@openssl.org> <33f929b9-2a30-f1d8-c7bb-b95b005f6f35@openssl.org> <875zp97r1u.wl-levitte@openssl.org> <8736kd7ppq.wl-levitte@openssl.org> <87zhml5ll2.wl-levitte@openssl.org> Message-ID: <10f578a6-e0a3-e76d-808e-ac2e32e7e40b@roumenpetrov.info> Richard Levitte wrote: > On Thu, 13 Jun 2019 20:23:05 +0200, > Roumen Petrov wrote: >> Hello, >> >> First I did not understand what is wrong to use function or >> reasons. All of them are subject to particular "library". > We didn't say anything against reason codes. As a matter of fact, I > do find those useful. Function codes, on the other hand, are unstable > and thereby quite useless, at least for figuring out errors. I'm not aware of OpenSSL documentation that describes reason codes as stable (fixed). If code is rewritten not only functions are changed. More or less reasons are changed as well. >> To parse openssl error code in an application code is bad practice. > Is it? Would you say it's bad practice to look at errno codes too? No as already in one of my post I wrote that? errsrt utility is very useful. Bad practice is to check that reason code encoded into 'error' code has value of A or B. > > Error *text* is a different matter. > >> Richard Levitte wrote: >>> On Thu, 13 Jun 2019 12:01:46 +0200, >>> Matt Caswell wrote: >>>>> The >>>>> additional information you're talking about is something we currently >>>>> provide the ERR_add_error_data() function for, and that together with >>>>> the reason text (derived from the reason code) is the data the end >>>>> user can reasonably get. It's up to whoever writes the error raising >>>>> code to provide enough useful information. >>>> Yes, although in practice we don't currently do this (we very rarely add >>>> additional explanatory text). Not sure if that is a problem with the API, our >>>> coding standards, or our culture. >>> This is partly historical... ERR_add_error_data() has been around >>> since the beginning of time, and it looks to me like it was designed >>> in a time where varargs hadn't universally caught on yet (yes, there >>> was a time before varargs, and it's appropriate to call that "the >>> stone age"). >> But developer could? format "extra message" for instance : >> ??????? NSSerr(NSS_F_RSA_SIGN, NSS_R_UNSUPPORTED_NID); >> ??????? {/*add extra error message data*/ >> ??????????? char msgstr[10]; >> ??????????? BIO_snprintf(msgstr, sizeof(msgstr), "%d", dtype); >> ??????????? ERR_add_error_data(2, "NID=", msgstr); >> ??????? } > I'll counter with a (yet fictitious): > > ERR_raise_error(NSS_R_UNSUPPORED_NID, "NID=%d", dtype); Reason code could be shared between functions : $ grep CAPI_R_FILE_OPEN_ERROR *.c e_capi.c:??????? CAPIerr(CAPI_F_CAPI_CTRL, CAPI_R_FILE_OPEN_ERROR); e_capi.c:??????? CAPIerr(CAPI_F_CAPI_VTRACE, CAPI_R_FILE_OPEN_ERROR); e_capi_err.c:??? {ERR_PACK(0, 0, CAPI_R_FILE_OPEN_ERROR), "file open error"}, NSS engine has similar case: $ grep NSS_R_MISSING_PVTKEY *.c e_nss_dsa.c:??????? NSSerr(NSS_F_DSA_DO_SIGN, NSS_R_MISSING_PVTKEY); e_nss_ec.c:??????? NSSerr(NSS_F_ECDSA_DO_SIGN, NSS_R_MISSING_PVTKEY); e_nss_err.c:#define NSS_R_MISSING_PVTKEY???????????????????????????????? 133 e_nss_err.c:??? { ERR_REASON(NSS_R_MISSING_PVTKEY)????????????? , "Missing private key" }, e_nss_key.c:??????????? NSSerr(NSS_F_LOAD_KEY, NSS_R_MISSING_PVTKEY); e_nss_rsa.c:??????? NSSerr(NSS_F_RSA_PRIV_DEC, NSS_R_MISSING_PVTKEY); e_nss_rsa.c:??????? NSSerr(NSS_F_RSA_SIGN, NSS_R_MISSING_PVTKEY); > > or if you want to add information that helps debugging: > > ERR_raise_error_debug(NSS_R_UNSUPPORED_NID, > __FILE__, __LINE__, __FUNC__, "$Format:%H", > "NID=%d", dtype); This look like complete different solution. __FILE__ exposes some 'private'? information (build related) and is some cases is not acceptable . __FUNC__ does not look portable? - __func__ vs __FUNCTION__ vs "not defined". Also going into this direction question is why to use "reason code". Functionality similar to errno, sys_errlist? is also outdated. > (since this is just an idea so far, it's perfectly possible to throw > in a library code as well, but like I said already, dynamic library > codes have their own issue) > > I know which I find easier to write. > >> There is no reason OpenSSL code to use ERR_add_error_data as usually >> library packs whole functionality. > That's not really true, there are places where OpenSSL code does use > ERR_add_error_data(), and for good reason. Hmm, you cut my note for externals like IO CAPI ;) > BIO_lookup and friends add > the extra resolver error on error, the CONF code adds information on > exactly what value causes a configuration file parsing error, the DSO > code adds information on exactly what file it attempted to load (full > path if possible), etc etc etc... those are things that can't be part > of the error code. > > Cheers, > Richard > Roumen From kurt at roeckx.be Thu Jun 13 21:38:08 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Thu, 13 Jun 2019 23:38:08 +0200 Subject: Start up entropy gathering In-Reply-To: References: Message-ID: <20190613213807.GA32178@roeckx.be> On Thu, Jun 13, 2019 at 05:06:16PM +1000, Dr Paul Dale wrote: > > The second suggestion is broadly similar but requires a file containing entropy that persists across reboots. This alternative requires a more management: the entropy file once read needs to be rewritten immediately (and ideally on shutdown as well). It also introduces a new attack vector against the entropy storage. It also isn?t possible to skip the entropy file read/rewrite sequence because it is impossible to determine if /dev/urandom has actually been seeded. I?ve not attempted to code this, persistent files containing seed material potentially introduce other problems. This is what init systems have always done. I see no need to also do it. They have a policy not to credit that the entropy from that file, I see no reason why we should override that policy. Kurt From levitte at openssl.org Thu Jun 13 22:08:27 2019 From: levitte at openssl.org (Richard Levitte) Date: Fri, 14 Jun 2019 00:08:27 +0200 Subject: Removing function names from errors (PR 9058) In-Reply-To: <10f578a6-e0a3-e76d-808e-ac2e32e7e40b@roumenpetrov.info> References: <878su77967.wl-levitte@openssl.org> <877e9p7td5.wl-levitte@openssl.org> <33f929b9-2a30-f1d8-c7bb-b95b005f6f35@openssl.org> <875zp97r1u.wl-levitte@openssl.org> <8736kd7ppq.wl-levitte@openssl.org> <87zhml5ll2.wl-levitte@openssl.org> <10f578a6-e0a3-e76d-808e-ac2e32e7e40b@roumenpetrov.info> Message-ID: <87y3255eas.wl-levitte@openssl.org> On Thu, 13 Jun 2019 22:04:17 +0200, Roumen Petrov wrote: > > Richard Levitte wrote: > > On Thu, 13 Jun 2019 20:23:05 +0200, > > Roumen Petrov wrote: > >> Hello, > >> > >> First I did not understand what is wrong to use function or > >> reasons. All of them are subject to particular "library". > > We didn't say anything against reason codes. As a matter of fact, I > > do find those useful. Function codes, on the other hand, are unstable > > and thereby quite useless, at least for figuring out errors. > I'm not aware of OpenSSL documentation that describes reason codes as > stable (fixed). Perhaps, but that's not saying they aren't... The reason codes, specifically, have been fairly stable over time. However, we can get better. Even better, we could get better at documenting them. > If code is rewritten not only functions are changed. More or less > reasons are changed as well. Sometimes, that's true, and especially as new functionality has come up. We could certainly do a better job there by getting better at giving consistent and more well defined reason codes. > >> To parse openssl error code in an application code is bad practice. > > Is it? Would you say it's bad practice to look at errno codes too? > No as already in one of my post I wrote that? errsrt utility is very useful. I'm not talking about display for the moment. > Bad practice is to check that reason code encoded into 'error' code > has value of A or B. Why? Why would it be bad practice to check if the reason for an error is ERR_R_MALLOC_FAILURE, for example? You can make the exact same analogy with errno and checking its values programatically. git grep 'errno *[!=]=' Considering the result of the above command, would you say that we're having bad practice in OpenSSL code? > >> But developer could? format "extra message" for instance : > >> ??????? NSSerr(NSS_F_RSA_SIGN, NSS_R_UNSUPPORTED_NID); > >> ??????? {/*add extra error message data*/ > >> ??????????? char msgstr[10]; > >> ??????????? BIO_snprintf(msgstr, sizeof(msgstr), "%d", dtype); > >> ??????????? ERR_add_error_data(2, "NID=", msgstr); > >> ??????? } > > I'll counter with a (yet fictitious): > > > > ERR_raise_error(NSS_R_UNSUPPORED_NID, "NID=%d", dtype); > Reason code could be shared between functions : Err, yeah? Of course they can! > > or if you want to add information that helps debugging: > > > > ERR_raise_error_debug(NSS_R_UNSUPPORED_NID, > > __FILE__, __LINE__, __FUNC__, "$Format:%H", > > "NID=%d", dtype); > > This look like complete different solution. __FILE__ exposes some > 'private'? information (build related) and is some cases is not > acceptable . Maybe you should look at the WHATEVERerr macros, here's one: # define SYSerr(f,r) ERR_PUT_error(ERR_LIB_SYS,(f),(r),OPENSSL_FILE,OPENSSL_LINE) OPENSSL_FILE and OPENSSL_LINE are macros that are aliases for __FILE__ and __LINE__, if those are available. So we already do expose that "private information". How other applications call ERR_PUT_error() is none of our business. > __FUNC__ does not look portable? - __func__ vs __FUNCTION__ vs "not > defined". It was just an example, to give you an idea... I didn't test it or check it for correctness. > Also going into this direction question is why to use "reason code". It's to allow calling applications to react differently depending on the reason for an error to occur. How else would you have them know? > Functionality similar to errno, sys_errlist? is also outdated. Ok, so what kind of functionality do you want to offer? Remember, we're talking about a language that doesn't have much error handling per se. More advanced languages have an exception system, but even there, the applicatin can catch them and react differently depending on what the error is, so apart from the out-of-band nature of an exception system (and automated unwinding), the difference isn't really that big. > > (since this is just an idea so far, it's perfectly possible to throw > > in a library code as well, but like I said already, dynamic library > > codes have their own issue) > > > > I know which I find easier to write. > > > >> There is no reason OpenSSL code to use ERR_add_error_data as usually > >> library packs whole functionality. > > That's not really true, there are places where OpenSSL code does use > > ERR_add_error_data(), and for good reason. > Hmm, you cut my note for externals like IO CAPI ;) Because I wasn't arguing that. I was arguing that what you were saying about OpenSSL code is incorrect. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From openssl-users at dukhovni.org Fri Jun 14 02:04:24 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Thu, 13 Jun 2019 22:04:24 -0400 Subject: Removing function names from errors (PR 9058) In-Reply-To: <878su77967.wl-levitte@openssl.org> References: <878su77967.wl-levitte@openssl.org> Message-ID: <20190614020424.GM33899@straasha.imrryr.org> On Wed, Jun 12, 2019 at 05:51:44AM +0200, Richard Levitte wrote: > A discussion point in that PR is whether it's still interesting to > keep information on the system function/callback that was called when > we're reporting a system error, i.e. this type of error report: > > SYSerr(SYS_F_FSTAT, errno); > > (incidently, there's another PR, 9072, which changes those to > 'SYSerr("fstat", errno)') > > So, the main points of discussion seem to be: > > - should we remove function indicators in our error reporting? > - should we make an exception for system errors (SYSerr())? Bottom line, my take is that function indicators should stay, but string names would be more convenient all around than numeric codes. -- Viktor. From paul.dale at oracle.com Fri Jun 14 03:41:51 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Fri, 14 Jun 2019 13:41:51 +1000 Subject: Removing function names from errors (PR 9058) In-Reply-To: <20190614020424.GM33899@straasha.imrryr.org> References: <878su77967.wl-levitte@openssl.org> <20190614020424.GM33899@straasha.imrryr.org> Message-ID: <6539AC3A-5BE8-4750-8DFF-2A6B5D0A4A96@oracle.com> I?m behind ditching the function identifier #defines but not their text names. The varargs based error function seems like a good idea too. It would be nicer if the ERR_raise_error function included the file and line information without them needing to be specified. As an exercise for the reader, I?ll skip any implementation details but will note that it is possible without using variadic macros. Pauli Hint: #define ERR_raise_error ERR_raise_error_internal(__FILE__, __LINE__, __FUNC__) int (*)(int, const char *, ...) ERR_raise_error_internal(const char *, int, const char *); -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 14 Jun 2019, at 12:04 pm, Viktor Dukhovni wrote: > > On Wed, Jun 12, 2019 at 05:51:44AM +0200, Richard Levitte wrote: > >> A discussion point in that PR is whether it's still interesting to >> keep information on the system function/callback that was called when >> we're reporting a system error, i.e. this type of error report: >> >> SYSerr(SYS_F_FSTAT, errno); >> >> (incidently, there's another PR, 9072, which changes those to >> 'SYSerr("fstat", errno)') >> >> So, the main points of discussion seem to be: >> >> - should we remove function indicators in our error reporting? >> - should we make an exception for system errors (SYSerr())? > > Bottom line, my take is that function indicators should stay, but > string names would be more convenient all around than numeric codes. > > -- > Viktor. -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Fri Jun 14 05:13:21 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 14 Jun 2019 01:13:21 -0400 Subject: Removing function names from errors (PR 9058) In-Reply-To: <6539AC3A-5BE8-4750-8DFF-2A6B5D0A4A96@oracle.com> References: <878su77967.wl-levitte@openssl.org> <20190614020424.GM33899@straasha.imrryr.org> <6539AC3A-5BE8-4750-8DFF-2A6B5D0A4A96@oracle.com> Message-ID: <20190614051321.GN33899@straasha.imrryr.org> On Fri, Jun 14, 2019 at 01:41:51PM +1000, Dr Paul Dale wrote: > I?m behind ditching the function identifier #defines but not their text names. Good to hear. > #define ERR_raise_error ERR_raise_error_internal(__FILE__, __LINE__, __FUNC__) Well, __FUNC__ is entirely non-standard, and __func__ is C99. Are we ready to abandon C89/C90? If not, then __func__ (and variants) becomes compiler-specific. In test/testutil.h, we have some of the requisite gymnastics: # if !defined(__STDC_VERSION__) || __STDC_VERSION__ < 199901L # if defined(_MSC_VER) # define TEST_CASE_NAME __FUNCTION__ # else # define testutil_stringify_helper(s) #s # define testutil_stringify(s) testutil_stringify_helper(s) # define TEST_CASE_NAME __FILE__ ":" testutil_stringify(__LINE__) # endif /* _MSC_VER */ # else # define TEST_CASE_NAME __func__ # endif /* __STDC_VERSION__ */ While the GCC manual: http://gcc.gnu.org/onlinedocs/gcc-4.8.1/gcc/Function-Names.html suggests: #if __STDC_VERSION__ < 199901L # if __GNUC__ >= 2 # define __func__ __FUNCTION__ # else # define __func__ "" # endif #endif we would also need similar for any other pre-C99 supported compilers. That said, I'm still in favour of function strings, and just the error and library codes as numeric. -- Viktor. From levitte at openssl.org Fri Jun 14 08:47:22 2019 From: levitte at openssl.org (Richard Levitte) Date: Fri, 14 Jun 2019 10:47:22 +0200 Subject: Removing function names from errors (PR 9058) In-Reply-To: <20190614051321.GN33899@straasha.imrryr.org> References: <878su77967.wl-levitte@openssl.org> <20190614020424.GM33899@straasha.imrryr.org> <6539AC3A-5BE8-4750-8DFF-2A6B5D0A4A96@oracle.com> <20190614051321.GN33899@straasha.imrryr.org> Message-ID: <87woho5zad.wl-levitte@openssl.org> On Fri, 14 Jun 2019 07:13:21 +0200, Viktor Dukhovni wrote: > > #define ERR_raise_error ERR_raise_error_internal(__FILE__, __LINE__, __FUNC__) > > Well, __FUNC__ is entirely non-standard, and __func__ is C99. Are > we ready to abandon C89/C90? If not, then __func__ (and variants) > becomes compiler-specific. We've had the argument about C language versions before, and it seems we're currently staying with C90, 'cause we have some prominent users that are stuck with that version. > In test/testutil.h, we have some of the requisite gymnastics: > > # if !defined(__STDC_VERSION__) || __STDC_VERSION__ < 199901L > # if defined(_MSC_VER) > # define TEST_CASE_NAME __FUNCTION__ > # else > # define testutil_stringify_helper(s) #s > # define testutil_stringify(s) testutil_stringify_helper(s) > # define TEST_CASE_NAME __FILE__ ":" testutil_stringify(__LINE__) > # endif /* _MSC_VER */ > # else > # define TEST_CASE_NAME __func__ > # endif /* __STDC_VERSION__ */ > > While the GCC manual: http://gcc.gnu.org/onlinedocs/gcc-4.8.1/gcc/Function-Names.html > suggests: > > #if __STDC_VERSION__ < 199901L > # if __GNUC__ >= 2 > # define __func__ __FUNCTION__ > # else > # define __func__ "" > # endif > #endif > > we would also need similar for any other pre-C99 supported compilers. Yes. We already have our own aliases for __FILE__ and __LINE__ (see include/openssl/opensslconf.h.in), we can simply add an OPENSSL_FUNC that's appropriately defined depending on what the compiler supports. Side note: opensslconf.h.in isn't the best place for this kind of macro, not even for OPENSSL_FILE and OPENSSL_LINE. I'd rather move all that to e_os2.h. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From matt at openssl.org Thu Jun 20 14:39:10 2019 From: matt at openssl.org (Matt Caswell) Date: Thu, 20 Jun 2019 15:39:10 +0100 Subject: punycode licensing Message-ID: <0f19ead7-c5c8-f88a-4e23-0c2bc34eac00@openssl.org> PR 9199 incorporates the C punycode implementation from RFC3492: https://github.com/openssl/openssl/pull/9199 The RFC itself has this section in it: B. Disclaimer and license Regarding this entire document or any portion of it (including the pseudocode and C code), the author makes no guarantees and is not responsible for any damage resulting from its use. The author grants irrevocable permission to anyone to use, modify, and distribute it in any way that does not diminish the rights of anyone else to use, modify, and distribute it, provided that redistributed derivative works do not contain misleading author or version information. Derivative works need not be licensed under similar terms. Which is quite confusing because on the one hand it places a requirement on redistributed derivative works: "provided that redistributed derivative works do not contain misleading author or version information" and then on the other hand states that derivative works are free to licence under different terms: "Derivative works need not be licensed under similar terms" It seems to me that the above gives us the ability to just relicense this under Apache 2 and incorporate it. But I'm not entirely sure. Thoughts? Matt From openssl-users at dukhovni.org Thu Jun 20 16:27:38 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Thu, 20 Jun 2019 12:27:38 -0400 Subject: punycode licensing In-Reply-To: <0f19ead7-c5c8-f88a-4e23-0c2bc34eac00@openssl.org> References: <0f19ead7-c5c8-f88a-4e23-0c2bc34eac00@openssl.org> Message-ID: <20190620162738.GP84864@straasha.imrryr.org> On Thu, Jun 20, 2019 at 03:39:10PM +0100, Matt Caswell wrote: > PR 9199 incorporates the C punycode implementation from RFC3492: > > https://github.com/openssl/openssl/pull/9199 > > The RFC itself has this section in it: > > B. Disclaimer and license > > Regarding this entire document or any portion of it (including the > pseudocode and C code), the author makes no guarantees and is not > responsible for any damage resulting from its use. The author grants > irrevocable permission to anyone to use, modify, and distribute it in > any way that does not diminish the rights of anyone else to use, > modify, and distribute it, provided that redistributed derivative > works do not contain misleading author or version information. > Derivative works need not be licensed under similar terms. > > Which is quite confusing because on the one hand it places a requirement on > redistributed derivative works: > > "provided that redistributed derivative works do not contain misleading author > or version information" > > and then on the other hand states that derivative works are free to licence > under different terms: > > "Derivative works need not be licensed under similar terms" > > It seems to me that the above gives us the ability to just relicense this under > Apache 2 and incorporate it. But I'm not entirely sure. I'd be comfortable with relicensing under Apache, while clearly indicating the provenance of the code, and indicating that the file is also available under the original terms. -- Viktor. From kaduk at mit.edu Fri Jun 21 01:59:45 2019 From: kaduk at mit.edu (Benjamin Kaduk) Date: Thu, 20 Jun 2019 20:59:45 -0500 Subject: punycode licensing In-Reply-To: <20190620162738.GP84864@straasha.imrryr.org> References: <0f19ead7-c5c8-f88a-4e23-0c2bc34eac00@openssl.org> <20190620162738.GP84864@straasha.imrryr.org> Message-ID: <20190621015945.GL52381@kduck.mit.edu> On Thu, Jun 20, 2019 at 12:27:38PM -0400, Viktor Dukhovni wrote: > On Thu, Jun 20, 2019 at 03:39:10PM +0100, Matt Caswell wrote: > > > PR 9199 incorporates the C punycode implementation from RFC3492: > > > > https://github.com/openssl/openssl/pull/9199 > > > > I'd be comfortable with relicensing under Apache, while clearly > indicating the provenance of the code, and indicating that the > file is also available under the original terms. Me, too. -Ben From paul.dale at oracle.com Fri Jun 21 02:12:08 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Fri, 21 Jun 2019 12:12:08 +1000 Subject: punycode licensing In-Reply-To: <20190621015945.GL52381@kduck.mit.edu> References: <0f19ead7-c5c8-f88a-4e23-0c2bc34eac00@openssl.org> <20190620162738.GP84864@straasha.imrryr.org> <20190621015945.GL52381@kduck.mit.edu> Message-ID: <06550780-9407-4F44-AFD9-E0B28C89B9F5@oracle.com> It seems okay from here too. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 21 Jun 2019, at 11:59 am, Benjamin Kaduk wrote: > > On Thu, Jun 20, 2019 at 12:27:38PM -0400, Viktor Dukhovni wrote: >> On Thu, Jun 20, 2019 at 03:39:10PM +0100, Matt Caswell wrote: >> >>> PR 9199 incorporates the C punycode implementation from RFC3492: >>> >>> https://github.com/openssl/openssl/pull/9199 >>> >> >> I'd be comfortable with relicensing under Apache, while clearly >> indicating the provenance of the code, and indicating that the >> file is also available under the original terms. > > Me, too. > > -Ben From tjh at openssl.org Fri Jun 21 06:24:03 2019 From: tjh at openssl.org (Tim Hudson) Date: Fri, 21 Jun 2019 16:24:03 +1000 Subject: punycode licensing In-Reply-To: <06550780-9407-4F44-AFD9-E0B28C89B9F5@oracle.com> References: <0f19ead7-c5c8-f88a-4e23-0c2bc34eac00@openssl.org> <20190620162738.GP84864@straasha.imrryr.org> <20190621015945.GL52381@kduck.mit.edu> <06550780-9407-4F44-AFD9-E0B28C89B9F5@oracle.com> Message-ID: Unfortunately, the issue isn't the compatibility of the license - they do indeed look relatively compatible to me - and the discussion on this thread has so far been about that. However the contributor license agreement requires that the copyright owner grants such permission - it is the fundamental basis of contributor agreements. Both the CCLA and ICLA make that exceedingly clear the contributor (individual or company) is "*the copyright owner or legal entity authorized by the copyright owner*" and the grants in the CLA are not grants that the notice in the RFC provide. In this case, the person who raised the PR is unable to meet those requirements (please do correct me if I am wrong on that) and as such their contribution is unable to be accepted. Tim. On Fri, Jun 21, 2019 at 12:12 PM Dr Paul Dale wrote: > It seems okay from here too. > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > > > > On 21 Jun 2019, at 11:59 am, Benjamin Kaduk wrote: > > > > On Thu, Jun 20, 2019 at 12:27:38PM -0400, Viktor Dukhovni wrote: > >> On Thu, Jun 20, 2019 at 03:39:10PM +0100, Matt Caswell wrote: > >> > >>> PR 9199 incorporates the C punycode implementation from RFC3492: > >>> > >>> https://github.com/openssl/openssl/pull/9199 > >>> > >> > >> I'd be comfortable with relicensing under Apache, while clearly > >> indicating the provenance of the code, and indicating that the > >> file is also available under the original terms. > > > > Me, too. > > > > -Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From beldmit at gmail.com Sat Jun 22 08:00:35 2019 From: beldmit at gmail.com (Dmitry Belyavsky) Date: Sat, 22 Jun 2019 11:00:35 +0300 Subject: punycode licensing In-Reply-To: References: <0f19ead7-c5c8-f88a-4e23-0c2bc34eac00@openssl.org> <20190620162738.GP84864@straasha.imrryr.org> <20190621015945.GL52381@kduck.mit.edu> <06550780-9407-4F44-AFD9-E0B28C89B9F5@oracle.com> Message-ID: Dear Tim, As Appendix B to RFC 3492 says Regarding this entire document or any portion of it (including the pseudocode and C code), the author makes no guarantees and is not responsible for any damage resulting from its use. The author grants irrevocable permission to anyone to use, modify, and distribute it in any way that does not diminish the rights of anyone else to use, modify, and distribute it, provided that redistributed derivative works do not contain misleading author or version information. Derivative works need not be licensed under similar terms. I think the way I used it is permitted. Though the simplest way to clean up this is to ask Adam to sign the CLA... ??, 22 ???? 2019 ?., 10:01 Tim Hudson : > Unfortunately, the issue isn't the compatibility of the license - they do > indeed look relatively compatible to me - and the discussion on this thread > has so far been about that. > However the contributor license agreement requires that the copyright > owner grants such permission - it is the fundamental basis of contributor > agreements. > > Both the CCLA and ICLA make that exceedingly clear the contributor > (individual or company) is "*the copyright owner or legal entity > authorized by the copyright owner*" and the grants in the CLA are not > grants that the notice in the RFC provide. > > In this case, the person who raised the PR is unable to meet those > requirements (please do correct me if I am wrong on that) and as such their > contribution is unable to be accepted. > > Tim. > > > On Fri, Jun 21, 2019 at 12:12 PM Dr Paul Dale > wrote: > >> It seems okay from here too. >> >> Pauli >> -- >> Dr Paul Dale | Cryptographer | Network Security & Encryption >> Phone +61 7 3031 7217 >> Oracle Australia >> >> >> >> > On 21 Jun 2019, at 11:59 am, Benjamin Kaduk wrote: >> > >> > On Thu, Jun 20, 2019 at 12:27:38PM -0400, Viktor Dukhovni wrote: >> >> On Thu, Jun 20, 2019 at 03:39:10PM +0100, Matt Caswell wrote: >> >> >> >>> PR 9199 incorporates the C punycode implementation from RFC3492: >> >>> >> >>> https://github.com/openssl/openssl/pull/9199 >> >>> >> >> >> >> I'd be comfortable with relicensing under Apache, while clearly >> >> indicating the provenance of the code, and indicating that the >> >> file is also available under the original terms. >> > >> > Me, too. >> > >> > -Ben >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.dale at oracle.com Mon Jun 24 01:04:21 2019 From: paul.dale at oracle.com (Dr Paul Dale) Date: Mon, 24 Jun 2019 11:04:21 +1000 Subject: OSSL_PARAM thought Message-ID: <1D323FCB-DEC1-4C37-AE34-5D16ADB2D112@oracle.com> We?re only starting out, so there isn?t any issue yet. I am wondering if instead of terminating out OSSL_PARAM arrays with an empty element, would it make sense to pass a size and the array? I.e. changing this code sequence (from crypto/evp/digest.c): params[i++] = OSSL_PARAM_construct_size_t(OSSL_DIGEST_PARAM_XOFLEN, &size, NULL); params[i++] = OSSL_PARAM_construct_end(); EVP_MD_CTX_set_params(ctx, params); into: params[i++] = OSSL_PARAM_construct_size_t(OSSL_DIGEST_PARAM_XOFLEN, &size, NULL); EVP_MD_CTX_set_params(ctx, i, params); For fixed arrays OSSL_NELEM would be used instead of the counter variable. There are downsides with both approaches of course and neither jumps out as being obviously superior. To me, at least, it looks like we?re going to have a lot of END?s throughout the codebase. Saving one line many times seems like a win. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Thu Jun 20 15:53:36 2019 From: rsalz at akamai.com (Salz, Rich) Date: Thu, 20 Jun 2019 15:53:36 +0000 Subject: punycode licensing In-Reply-To: <0f19ead7-c5c8-f88a-4e23-0c2bc34eac00@openssl.org> References: <0f19ead7-c5c8-f88a-4e23-0c2bc34eac00@openssl.org> Message-ID: You can always contact the RFC author. You should consider asking the OpenSSL counsel. From rsalz at akamai.com Sat Jun 22 16:18:45 2019 From: rsalz at akamai.com (Salz, Rich) Date: Sat, 22 Jun 2019 16:18:45 +0000 Subject: punycode licensing In-Reply-To: References: <0f19ead7-c5c8-f88a-4e23-0c2bc34eac00@openssl.org> <20190620162738.GP84864@straasha.imrryr.org> <20190621015945.GL52381@kduck.mit.edu> <06550780-9407-4F44-AFD9-E0B28C89B9F5@oracle.com> Message-ID: <000DB661-41E4-4D45-9DCF-D33864EE1CEA@akamai.com> * Unfortunately, the issue isn't the compatibility of the license - they do indeed look relatively compatible to me - and the discussion on this thread has so far been about that. * However the contributor license agreement requires that the copyright owner grants such permission - it is the fundamental basis of contributor agreements. Yes, compatibility is important. CLA?s are required only for new code contributed to the project, not for code incorporated from elsewhere. So if it?s compatible, CLA?s are not required. At least, that was the position of the project and its former counsel during the first two years of relicensing. Perhaps the OMC should raise this issue with current counsel if they disagree, but from the public statements on this list, it seems all but one agree. As an aside, I?ve contacted the author and am having a productive exchange. Dimitry has seen the emails. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tshort at akamai.com Mon Jun 24 11:06:03 2019 From: tshort at akamai.com (Short, Todd) Date: Mon, 24 Jun 2019 11:06:03 +0000 Subject: punycode licensing In-Reply-To: <000DB661-41E4-4D45-9DCF-D33864EE1CEA@akamai.com> References: <0f19ead7-c5c8-f88a-4e23-0c2bc34eac00@openssl.org> <20190620162738.GP84864@straasha.imrryr.org> <20190621015945.GL52381@kduck.mit.edu> <06550780-9407-4F44-AFD9-E0B28C89B9F5@oracle.com> <000DB661-41E4-4D45-9DCF-D33864EE1CEA@akamai.com> Message-ID: <2B160E73-7319-4C64-B97F-033400D8454B@akamai.com> This is the second time, that I'm aware of, that the wording of the CLA has prevented a PR from being accepted. While this won't help the first case I'm aware of, perhaps there needs to be an exception/special-case in the CLA for code in RFCs, or other similar publications, where the author is basically saying "you can use this". The motive of the author may very well be done at the point of RFC publication, and has no interest in personally submitting the code to every open source project that may want it. As far as the authors are concerned, putting it into the RFC is sufficient. -- -Todd Short // Sent from my iPhone // "One if by land, two if by sea, three if by the Internet." On Jun 24, 2019, at 5:42 AM, Salz, Rich > wrote: * Unfortunately, the issue isn't the compatibility of the license - they do indeed look relatively compatible to me - and the discussion on this thread has so far been about that. * However the contributor license agreement requires that the copyright owner grants such permission - it is the fundamental basis of contributor agreements. Yes, compatibility is important. CLA?s are required only for new code contributed to the project, not for code incorporated from elsewhere. So if it?s compatible, CLA?s are not required. At least, that was the position of the project and its former counsel during the first two years of relicensing. Perhaps the OMC should raise this issue with current counsel if they disagree, but from the public statements on this list, it seems all but one agree. As an aside, I?ve contacted the author and am having a productive exchange. Dimitry has seen the emails. -------------- next part -------------- An HTML attachment was scrubbed... URL: