[openssl-users] Authentication over ECDHE

[openssl at foocrypt.net]

3 - 6 months

Happy New Year..;)

> On 1 Jan 2019, at 02:43, Richard Levitte <levitte at openssl.org> wrote:
> 
> I'll go ahead and ask, how long do you think such a back door would
> stay unnoticed, let alone survive?  I'm considering the fact that we
> have a lot of people looking at our code, just judging from the issues
> and pull requests raised on github.
> 
> I can't say that I have an actual answer, but it's a question worth
> asking as well, to see if the T.O.L.A. Act is worth a state of panic
> or not.
> 
> Cheers,
> Richard ( who's on vacation and should stop reading these mails )
> 
> In message <11EDC036-87D3-4757-AD33-80C6E608F77F at foocrypt.net> on Tue, 1 Jan 2019 01:27:53 +1100, "openssl at foocrypt.net" <openssl at foocrypt.net> said:
> 
>> Matt et al
>> 
>> 'been reviewed and approved by two current OpenSSL committers (one of whom must
>> be on the OpenSSL Management Committee).’
>> 
>> Due to the recent legislative changes here in Australia around the T.O.L.A. Act, can a change be
>> made to the OpenSSL policy so that the 2 reviewers, don’t reside in Australia, or are Australian
>> citizens ?
>> 
>> ABI/API changes -> breaks -> back door requests….
>> 
>> --
>> 
>> Regards,
>> 
>> Mark A. Lane
>> 
>> Cryptopocalypse NOW 01 04 2016
>> 
>> Volumes 0.0 -> 10.0 Now available through iTunes - iBooks @
>> https://itunes.apple.com/au/author/mark-a.-lane/id1100062966?mt=11
>> 
>> © Mark A. Lane 1980 - 2019, All Rights Reserved.
>> © FooCrypt 1980 - 2019, All Rights Reserved.
>> © FooCrypt, A Tale of Cynical Cyclical Encryption. 1980 - 2019, All Rights Reserved.
>> © Cryptopocalypse 1980 - 2019, All Rights Reserved.
>> 
>> On 1 Jan 2019, at 01:11, Matt Caswell <matt at openssl.org> wrote:
>> 
>> On 31/12/2018 11:36, C.Wehrmeyer wrote:
>> 
>> On 31.12.18 10:12, Richard Levitte wrote:
>> 
>> Yes, it's true, new features are going in. And it's true that it's
>> often more exciting to add new features than to do the janitorial
>> work.
>> 
>> You realised what I have left unspoken thus far, which is this almost
>> obsession-like preference of OSS coders to add new features rather than
>> improving the old, boring codebase. However, there's a reason why it's still
>> called code*base*. It's the base of everything. And adding more and more
>> features to that base is going to make ripping off the band-aid more painful in
>> the long run.
>> 
>> There has been a huge amount of effort put in over the last few years to improve
>> the codebase. Things that immediately spring to mind (and there's probably a
>> whole load more):
>> 
>> - Rewrite of the state machine
>> - libssl record layer refactor
>> - Implementation of the PACKET and WPACKET abstractions in libssl
>> - Rewrite of the rand code
>> - Implementation of the new test harness
>> - Significant effort into developing tests
>> - Implementation of the coding style and reformat of the codebase to meet it
>> - Opaque many of the structures (which I know you don't see as an improvement,
>> but I'll answer that point separately)
>> - Implementation of continuous fuzzing
>> - Significant expansion of the documentation coverage
>> 
>> It is simply not true to claim that we have "an obsession-like preference...to
>> add new features rather than improving the old, boring codebase". None of the
>> above things resulted in or were motivated by user visible features. They were
>> all about improving the codebase.
>> 
>> Also, infrastructure again. I, as a user, don't care if the kernel gets a new
>> feature that makes some black magic happening. What I care about is that the
>> kernel doesn't throw away my writes (which has happened in May of 2018, see):
>> 
>> https://www.postgresql.org/message-id/flat/CAMsr%2BYE5Gs9iPqw2mQ6OHt1aC5Qk5EuBFCyG%2BvzHun1EqMxyQg%40mail.gmail.com#CAMsr+YE5Gs9iPqw2mQ6OHt1aC5Qk5EuBFCyG+vzHun1EqMxyQg@mail.gmail.com)
>> 
>> 
>> Cryptography libs should be equally conservative, considering that cryptography
>> is conservative to begin with. I don't care if TLS 1.3 lets me use new exiting
>> ciphers and handshakes when it unreasonably bogs down my server code.
>> 
>> BUT, you also have to appreciate that stuff is happening around us
>> that affects our focus. TLS 1.3 happened, and rather than having to
>> answer the question "why don't you have TLS 1.3 yet?" (there's a LOT
>> of interest in that version), we decided to add it.
>> 
>> Sure, but didn't Matt just say that there are a lot of volunteers working on
>> that library? The disadvantage here is that quality assurance is barely a thing
>> - however, the *advantage* of this is that OpenSSL does not have to follow
>> commercial interests. If we look at this at face value you could just say "No,
>> people, it's high time we streamline some of the internal aspects of the
>> library, TLS 1.3 will have to wait. You can't wait that long? Well, sorry".
>> 
>> However, your message is clear, we do need to do some cleanup as
>> well. More than that, I agree with you that it's needed (I've
>> screamed out in angst when stumbling upon particularly ugly or
>> misplaced code, so the feeling is shared, more than you might
>> believe).
>> 
>> But what does "cleanup" entail? That's the hot-button question here. I've
>> already made a suggestion, that is to say, getting rid of opaque structures. If
>> that is deemed too insecure (for whatever reasons), export symbols that allow
>> programmers to query the size of structures, and provide two versions of
>> functions: one function expects the caller to pass an object to which the
>> changes are to be made, and the other one allocates the necessary memory
>> dynamically and then calls the first version. Or just don't allocate my object
>> memory dynamically anymore.
>> 
>> That being said, cleanup happens, and documentation happens, in a
>> piecemeal fashion, 'cause that's what most people have capacity for.
>> 
>> So, what you're effectively saying is that I'm the first one who ever asked for
>> SSL object reuse, right? Because if piecemeal work happens on the documentation,
>> and Viktor says that it's possible, then surely no one would have ever answered
>> that question on the mailing list and *not* put it piecemeal-ly in the OpenSSL
>> documentation, right?
>> 
>> Now, here's something else that you need to consider: API/ABI
>> compatibility needs to be preserved.
>> 
>> No it doesn't. We *know* it doesn't. When OpenSSL 1.1 was released it broke all
>> *sorts* of applications out there, because all sorts of applications used struct
>> fields rather than accessors. wget, mutt, neon, python, you name it, you broke it.
>> 
>> API/ABI stability is absolutely required. Every time we make a breaking change
>> it is painful for our users - and more pain is felt the bigger the scale of the
>> break. We simply cannot go around making wholesale breaks on an ongoing basis.
>> If we did so then OpenSSL would be a lot less useful to our users.
>> 
>> This is not to say that we can *never* make breaking changes. Only that when we
>> do so it must be strongly justified and only done relatively infrequently. We
>> made such a decision when we decided to make the structures opaque. It's not a
>> decision we are likely to repeat anytime soon IMO. We are still feeling the pain
>> of that now (and will continue to do so for at least the next year until 1.0.2
>> goes out of support - and probably beyond that).
>> 
>> Which brings me onto why structures were made opaque in the first place. A
>> significant driver for this (probably *the* most important one) was to improve
>> the codebase. I have witnessed first hand the harm that non-opaque structures
>> did to OpenSSL. We will be fixing the fallout from them for years to come.
>> Non-opaque structures combined with the requirements for stable API/ABI means
>> you cannot change anything in those structures. Renaming or deleting structure
>> members constitutes an API break. Even *adding* structure members constitutes an
>> ABI break (due to the changed size of the structure). This means the code
>> ossifies over time and cannot easily be refactored. Much of OpenSSL's internal
>> "quirkiness" results from attempting to work around this restriction.
>> 
>> Things like the state machine refactor and the record layer refactor would not
>> have been possible without opaque structures. In my mind making the structures
>> opaque was one of the best things that ever happened to OpenSSL.
>> 
>> https://breakpoint.cc/openssl-1.1-rebuild-2016-08-26/
>> 
>> So since when do we need to consider API/ABI compatibility? Did we grow up
>> recently?
>> 
>> Or maybe OpenSSL should have switched the language. The point of C is that
>> structures are public. And if I'm going to be honest that approach saved my
>> sorry arse more than a couple times. When zlib choked because it couldn't go
>> past 4 GiBs of data since its fields were uint32_ts, I was able to easily
>> workaround this problem. But what do I know.
>> 
>> If you really want to fiddle with OpenSSL internal structures - feel free. Just
>> include the OpenSSL internal header files and away you go. Just do so in the
>> knowledge that they could be changed at any time, and your code might break. If
>> this isn't a concern to you then - no problem. If it is a concern to you - then
>> actually you *do* care about API/ABI stability after all.
>> 
>> Counting symbols is, however, nothing other than a blunt instrument.
>> Quite a lot of those symbols are convenience macros and functions that
>> have accumulated over time.
>> 
>> You're taking my statement out of context. Counting the symbols wasn't supposed
>> to suggest that there are too *many* of them. I'm in no position to say that,
>> seeing as the original context in which my statement was put is that *I'm not
>> familiar enough with the library*.
>> 
>> What I said was that reading the code is easy. Learning what the library
>> provides is hard, and that you won't learn much just by looking at the symbols
>> because there's so many of them.
>> 
>> But nevertheless, I do hear you call for a remake of the SSL API as
>> well as cleaner internals. The latter is easier, and I'm sure it will
>> happen piecemeal as per usual so as to not break something /
>> inadvertently change a behavior (i.e. break ABI). The former is a
>> fairly massive project, and is more of creating a new API and library
>> rather than a mere cleanup job. That will be a massive effort, and
>> you do have to keep in mind how much time all involved can put into
>> it.
>> 
>> I'm not saying it needs to be done right now. I'm merely suggesting that it
>> might be a good goal post for OpenSSL 2.0.
>> 
>> Turning structures opaque doesn't prevent people from still messing
>> with their internal fields.
>> 
>> True. But it makes for a clear delineation where people are forced to
>> be aware that they are playing with internal stuff, and that it may
>> not be a safe thing to do.
>> 
>> Then why not provide small helper functions for covering the "playing with
>> internal stuff" part? That way it's still controlled, and documented, and
>> unified. You guys must've had some examples to show off in order to justify the
>> process, so surely you know what it is that people do when they use internal
>> stuff. Make functions for those. Don't give them any reason to continue playing
>> with internal stuff.
>> 
>> I don't like code that tries to protect programmers from themselves. I like code
>> that lets good programmers do smart things. And if bad programmers use that
>> freedom to do bad stuff, then doesn't that mean your API simply didn't support
>> this, and they had to make it work somehow else? Again, helper functions.
>> 
>> Uhmmmm.... this is factually incorrect. OpenSSL doesn't use its own
>> memory pooling. We have thin wrappers around the usual malloc() /
>> realloc() / free(), which allows any application to do its own memory
>> pooling.
>> 
>> https://web.archive.org/web/20150207180717/http://article.gmane.org/gmane.os.openbsd.misc/211963
>> 
>> 
>> The BUF_FREELISTS code that this post references was ripped out years ago. This
>> no longer represents the current state of OpenSSL in any supported version.
>> 
>> To conclude, I have a question for you: are you only willing to rant
>> (*), or are you willing to help out in another way?
>> 
>> This is not the question I feel you should ask because we haven't even
>> established if I *could* make contributions to the project, as my mindset
>> appears to be so much more different. Especially the idea of not wanting to
>> break APIs/ABIs is a huge limitation - just looking at SSL_new() made me give up
>> hope here.
>> 
>> Yes - not breaking APIs/ABIs is a huge limitation. BoringSSL is not suitable for
>> general purpose use precisely because of this. The only users BoringSSL cares
>> about whether they break or not are Google users. As soon as you have a library
>> that wants to cater for large numbers of users (which we do) then you have to
>> accept that limitation.
>> 
>> As to whether or not we have established whether you *could* make such
>> contributions - I think you are missing the point. We cannot know whether you
>> are capable or not until you try. It is on the basis of your code that we would
>> make such a judgement. In order for your code to get into OpenSSL it must have
>> been reviewed and approved by two current OpenSSL committers (one of whom must
>> be on the OpenSSL Management Committee). We invite anyone to contribute. In
>> order for this to be a healthy open-source community we *need* those
>> contributions. Only those that make the grade will make it in.
>> 
>> Note - this review process wasn't always the case. Things used to be much more
>> informal in pre-heartbleed days. This is no longer the case.
>> 
>> I'm no cryptography expert, I've made that clear from mail one, and my cleanup
>> jobs would be more widespread than what seems to be deemed acceptable right now.
>> I can read and write scalable C code, otherwise I wouldn't even have tried to
>> reuse that SSL object from the beginning.
>> 
>> So, I ask a question in return: what do you think I *could* be helping with?
>> 
>> Well, you have vocally complained about the state of the documentation. You have
>> the benefit of being a new OpenSSL user. You know what things were confusing or
>> unclear in the documentation. More experienced OpenSSL coders often don't have
>> the perspective - because some things are just "obvious" to them. So help with
>> pull requests to improve the documentation.
>> 
>> Matt
>> --
>> openssl-users mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
>> 
> -- 
> openssl-users mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users