From hkario at redhat.com Fri Sep 1 17:05:51 2017 From: hkario at redhat.com (Hubert Kario) Date: Fri, 01 Sep 2017 19:05:51 +0200 Subject: [openssl-dev] TLS 1.3 non compliance with current draft Message-ID: <2611567.OezTKZKLtv@pintsize.usersys.redhat.com> When openssl sends a second Client Hello message, it modifies it quite extensively, not only client_random is changed but also advertised cipher suites. see https://github.com/openssl/openssl/issues/4292 That makes it non-compliant with the current draft (-21): When a client first connects to a server, it is REQUIRED to send the ClientHello as its first message. The client will also send a ClientHello when the server has responded to its ClientHello with a HelloRetryRequest. In that case, the client *MUST send the same* *ClientHello* (without modification) except: - If a "key_share" extension was supplied in the HelloRetryRequest, replacing the list of shares with a list containing a single KeyShareEntry from the indicated group. - Removing the "early_data" extension (Section 4.2.9) if one was present. Early data is not permitted after HelloRetryRequest. - Including a "cookie" extension if one was provided in the HelloRetryRequest. - Updating the "pre_shared_key" extension if present by recomputing the "obfuscated_ticket_age" and binder values and (optionally) removing any PSKs which are incompatible with the server's indicated cipher suite. -- Regards, Hubert Kario Senior Quality Engineer, QE BaseOS Security team Web: www.cz.redhat.com Red Hat Czech s.r.o., Purky?ova 115, 612 00 Brno, Czech Republic -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part. URL: From David.von.Oheimb at siemens.com Fri Sep 1 21:12:04 2017 From: David.von.Oheimb at siemens.com (David von Oheimb) Date: Fri, 1 Sep 2017 23:12:04 +0200 Subject: [openssl-dev] How to use BIO_do_connect(), blocking and non-blocking with timeout, coping with errors In-Reply-To: References: <69d6c8c3-c3ba-44f5-fad0-4b2294e210d3@siemens.com> Message-ID: On 29.08.2017 16:15, Salz, Rich via openssl-dev wrote: >> Getting the client connect right appears surprisingly messy when one >> needs to cope with all kinds of network error situations including >> domain name resolution issues and temporarily unreachable servers. >> Both indefinitely blocking and non-blocking behavior (i.e., connection >> attempts with and without a timeout) should be supported. > It is a complicated issue and hard to get right for all definitions of right for all applications ? Hmm - on the one hand, good to get confirmation that I did not just miss a simple way of out the maze, ... > A set of API?s that set up all the TLS ?metadata?, and took a connected socket might be a way through the maze. For example: > SSL *SSL_connection(int socket, const char *servername, ?whatever?) ... on the other hand, it's a pity that such a high-level API does not (yet) exist, at least not in OpenSSL. How about adding at least some clearly useful abstractions like the below socket_wait() function, which would reduce code duplication in the OpenSSL crypto lib and apps and help application developers? Maybe other OpenSSL users have specific experience on error and timeout handling for BIO_do_connect() etc. and can comment in more detail on the (approximate) solution, bio_connect(), that I gave below? On 28.08.2017 13:46, David von Oheimb wrote: > Hi all, > > I'm currently enhancing HTTP(S) clients based on OpenSSL in several > flavors, in particular a CMP client, which in turn uses simple HTTP > clients for contacting CRL distribution points or OCSP responders. > > Getting the client connect right appears surprisingly messy when one > needs to cope with all kinds of network error situations including > domain name resolution issues and temporarily unreachable servers. > Both indefinitely blocking and non-blocking behavior (i.e., connection > attempts with and without a timeout) should be supported. > > Since these are pretty general problems I wonder why there there is > rather limited support via generic higher-level OpenSSL or C library > functions, or at least I was unable to find it. Instead, the OpenSSL > apps contain code that calls BIO_do_connect directly (or the equivalent > BIO_do_handshake), in particular query_responder() in apps/ocsp.c. > (The situation is similar for the subsequent exchange of data via the > BIO, optionally with a timeout). > > So I constructed my own abstraction, called bio_connect, which took > quite some effort testing network error situations. Please see below its > code including comments on some strange behavior I experienced and my > workarounds for that. Does this code make sense, or do I miss anything? > > How about adding such a function for instance to crypto/bio/bio_lib.c? > > BTW, my code uses a handy generic helper function, socket_wait, for > waiting for read/write form/to a socket, with a given timeout. Since > several instances of that pretty common code pattern using select() are > spread over the OpenSSL apps (and crypto lib), I suggest adding this > function to the library. Where would be a good place to put it? > > Thanks, > David >> /* returns -1 on error, 0 on timeout, 1 on success */ >> int bio_connect(BIO *bio, int timeout) { >> int blocking; >> time_t max_time; >> int rv; >> >> blocking = timeout == 0; >> max_time = timeout != 0 ? time(NULL) + timeout : 0; >> >> if (!blocking) >> BIO_set_nbio(bio, 1); >> retry: >> rv = BIO_do_connect(bio); >> if (rv <= 0 && (errno == ETIMEDOUT /* in blocking case, >> despite blocking BIO, BIO_do_connect() timed out */ || >> ERR_GET_REASON(ERR_peek_error()) == ETIMEDOUT/* when non-blocking, >> BIO_do_connect() timed out early with rv == -1 and errno == 0 */)) { >> ERR_clear_error(); >> (void)BIO_reset(bio); /* otherwise, blocking next connect() may crash >> and non-blocking next BIO_do_connect() will fail */ >> goto retry; >> } >> if (!blocking && rv <= 0 && BIO_should_retry(bio)) { >> int fd; >> if (BIO_get_fd(bio, &fd) <= 0) >> return -1; >> rv = socket_wait(fd, 1, max_time - time(NULL)); >> if (rv > 0) >> /* for some reason, select() may wrongly have returned success */ >> goto retry; >> } >> return rv; >> } >> /* returns < 0 on error, 0 on timeout, >0 on success */ >> int socket_wait(int fd, int for_read, int timeout) >> { >> fd_set confds; >> struct timeval tv; >> >> if (timeout <= 0) >> return 0; >> >> FD_ZERO(&confds); >> openssl_fdset(fd, &confds); >> tv.tv_usec = 0; >> tv.tv_sec = timeout; >> return select(fd + 1, for_read ? &confds : NULL, >> for_read ? NULL : &confds, NULL, &tv); >> } -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Fri Sep 1 22:32:01 2017 From: rsalz at akamai.com (Salz, Rich) Date: Fri, 1 Sep 2017 22:32:01 +0000 Subject: [openssl-dev] How to use BIO_do_connect(), blocking and non-blocking with timeout, coping with errors In-Reply-To: References: <69d6c8c3-c3ba-44f5-fad0-4b2294e210d3@siemens.com> Message-ID: FWIW, there?s a ?libtls? library from the libre folks that might be worth looking at. If you come up with useful snippets we can start by posting them to the wiki, for example -------------- next part -------------- An HTML attachment was scrubbed... URL: From uri at ll.mit.edu Fri Sep 1 22:42:38 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Fri, 1 Sep 2017 22:42:38 +0000 Subject: [openssl-dev] Fwd: openssl 1-1-0-stable fails References: <6D1CC048-5836-48E6-8C54-BCFC7792EB1C@mit.edu> Message-ID: Begin forwarded > Subject: openssl 1-1-0-stable fails > > OpenSSL_1_1_0-stable current Github > > Test Summary Report > ------------------- > ../test/recipes/80-test_cms.t (Wstat: 256 Tests: 4 Failed: 1) > Failed test: 4 > Non-zero exit status: 1 > Files=95, Tests=561, 165 wallclock secs ( 1.13 usr 0.30 sys + 84.96 cusr 42.27 csys = 128.66 CPU) > Result: FAIL > > Config: > > ./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 enable-tls13downgrade > -- > Uri Blumenthal > uri at mit.edu > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4223 bytes Desc: not available URL: From matt at openssl.org Fri Sep 1 22:48:10 2017 From: matt at openssl.org (Matt Caswell) Date: Fri, 1 Sep 2017 23:48:10 +0100 Subject: [openssl-dev] Fwd: openssl 1-1-0-stable fails In-Reply-To: References: <6D1CC048-5836-48E6-8C54-BCFC7792EB1C@mit.edu> Message-ID: On 01/09/17 23:42, Blumenthal, Uri - 0553 - MITLL wrote: > > > Begin forwarded > >> *Subject:* *openssl 1-1-0-stable fails* >> >> OpenSSL_1_1_0-stable current Github >> >> Test Summary Report >> ------------------- >> ../test/recipes/80-test_cms.t (Wstat: 256 Tests: 4 Failed: 1) >> Failed test: 4 >> Non-zero exit status: 1 >> Files=95, Tests=561, 165 wallclock secs ( 1.13 usr 0.30 sys + 84.96 >> cusr 42.27 csys = 128.66 CPU) >> Result: FAIL >> >> Config: >> >> ./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc >> enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 >> enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 >> enable-tls13downgrade Errr....this config line is using options only available in master (enablet-tls1_3 enable-tls13downgrade). Did you give the right line? Matt From uri at ll.mit.edu Sat Sep 2 01:10:13 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Fri, 1 Sep 2017 21:10:13 -0400 Subject: [openssl-dev] Fwd: openssl 1-1-0-stable fails In-Reply-To: References: <6D1CC048-5836-48E6-8C54-BCFC7792EB1C@mit.edu> Message-ID: <0B435A4B-AB72-4484-8A95-6A6FD750E112@ll.mit.edu> On Sep 1, 2017, at 18:48, Matt Caswell wrote: >>> *Subject:* *openssl 1-1-0-stable fails* >>> >>> OpenSSL_1_1_0-stable current Github >>> >>> Test Summary Report >>> ------------------- >>> ../test/recipes/80-test_cms.t (Wstat: 256 Tests: 4 Failed: 1) >>> Failed test: 4 >>> Non-zero exit status: 1 >>> Files=95, Tests=561, 165 wallclock secs ( 1.13 usr 0.30 sys + 84.96 >>> cusr 42.27 csys = 128.66 CPU) >>> Result: FAIL >>> >>> Config: >>> >>> ./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc >>> enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 >>> enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 >>> enable-tls13downgrade > > Errr....this config line is using options only available in master > (enablet-tls1_3 enable-tls13downgrade). Did you give the right line? Yes, the branch is OpenSSL_1_1_0-stable. Apparently it does not complain seeing these un-realizable parameters. And error is ?genuine?. 1.1 master also fails tests: Test Summary Report ------------------- ../test/recipes/70-test_clienthello.t (Wstat: 256 Tests: 1 Failed: 1) Failed test: 1 Non-zero exit status: 1 Files=136, Tests=1266, 480 wallclock secs ( 2.72 usr 0.55 sys + 226.57 cusr 144.10 csys = 373.94 CPU) Result: FAIL make[1]: *** [_tests] Error 1 make: *** [tests] Error 2 $ Config: ./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc --with-rand-seed=rdcpu enable-aria enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 enable-tls13downgrade As you see, for 1.1.1 master I?m enabling ARIA and setting default entropy source to RDRAND. ;-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From uri at ll.mit.edu Sun Sep 3 01:15:13 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Sun, 3 Sep 2017 01:15:13 +0000 Subject: [openssl-dev] Fwd: openssl 1-1-0-stable fails In-Reply-To: <0B435A4B-AB72-4484-8A95-6A6FD750E112@ll.mit.edu> References: <6D1CC048-5836-48E6-8C54-BCFC7792EB1C@mit.edu> <0B435A4B-AB72-4484-8A95-6A6FD750E112@ll.mit.edu> Message-ID: All my builds include "make distclean" at the start of the process. However when I repeated that cleanup and re-run the build, 1_1_0-stable error disappeared. A fluke?! Regards, Uri Sent from my iPhone > On Sep 1, 2017, at 21:10, Blumenthal, Uri - 0553 - MITLL wrote: > > On Sep 1, 2017, at 18:48, Matt Caswell wrote: >>>> *Subject:* *openssl 1-1-0-stable fails* >>>> >>>> OpenSSL_1_1_0-stable current Github >>>> >>>> Test Summary Report >>>> ------------------- >>>> ../test/recipes/80-test_cms.t (Wstat: 256 Tests: 4 Failed: 1) >>>> Failed test: 4 >>>> Non-zero exit status: 1 >>>> Files=95, Tests=561, 165 wallclock secs ( 1.13 usr 0.30 sys + 84.96 >>>> cusr 42.27 csys = 128.66 CPU) >>>> Result: FAIL >>>> >>>> Config: >>>> >>>> ./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc >>>> enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 >>>> enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 >>>> enable-tls13downgrade >> >> Errr....this config line is using options only available in master >> (enablet-tls1_3 enable-tls13downgrade). Did you give the right line? > > Yes, the branch is OpenSSL_1_1_0-stable. Apparently it does not complain seeing these un-realizable parameters. And error is ?genuine?. > > 1.1 master also fails tests: > > Test Summary Report > ------------------- > ../test/recipes/70-test_clienthello.t (Wstat: 256 Tests: 1 Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > Files=136, Tests=1266, 480 wallclock secs ( 2.72 usr 0.55 sys + 226.57 cusr 144.10 csys = 373.94 CPU) > Result: FAIL > make[1]: *** [_tests] Error 1 > make: *** [tests] Error 2 > $ > > Config: > > ./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc --with-rand-seed=rdcpu enable-aria enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 enable-tls13downgrade > > As you see, for 1.1.1 master I?m enabling ARIA and setting default entropy source to RDRAND. ;-) > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4223 bytes Desc: not available URL: From uri at ll.mit.edu Sun Sep 3 21:18:32 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Sun, 3 Sep 2017 17:18:32 -0400 Subject: [openssl-dev] 1.1.1 master consistently fails (was Re: openssl 1-1-0-stable fails) In-Reply-To: References: <6D1CC048-5836-48E6-8C54-BCFC7792EB1C@mit.edu> <0B435A4B-AB72-4484-8A95-6A6FD750E112@ll.mit.edu> Message-ID: MacOS 10.12.6, Xcode-8.3.3. Current Github master: Test Summary Report ------------------- ../test/recipes/70-test_clienthello.t (Wstat: 256 Tests: 1 Failed: 1) Failed test: 1 Non-zero exit status: 1 Files=136, Tests=1266, 461 wallclock secs ( 2.58 usr 0.48 sys + 221.36 cusr 135.66 csys = 360.08 CPU) Result: FAIL make[1]: *** [_tests] Error 1 make: *** [tests] Error 2 Config & build script (feel free to suggest improvements, BTW): #!/bin/bash -ex make distclean || true # For OpenSSL-1.1.1 master (development branch) ./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc --with-rand-seed=rdcpu enable-aria enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 enable-tls13downgrade # For OpenSSL-1.1.0-stable #./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 enable-tls13downgrade make depend && make clean && make -j 2 all && make test && make install > On Sep 2, 2017, at 21:15, Blumenthal, Uri - 0553 - MITLL wrote: > > All my builds include "make distclean" at the start of the process. However when I repeated that cleanup and re-run the build, 1_1_0-stable error disappeared. A fluke?! > > Regards, > Uri > > Sent from my iPhone >> >> Yes, the branch is OpenSSL_1_1_0-stable. Apparently it does not complain seeing these un-realizable parameters. And error is ?genuine?. >> >> 1.1 master also fails tests: >> >> Test Summary Report >> ------------------- >> ../test/recipes/70-test_clienthello.t (Wstat: 256 Tests: 1 Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> Files=136, Tests=1266, 480 wallclock secs ( 2.72 usr 0.55 sys + 226.57 cusr 144.10 csys = 373.94 CPU) >> Result: FAIL >> make[1]: *** [_tests] Error 1 >> make: *** [tests] Error 2 >> $ >> >> Config: >> >> ./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc --with-rand-seed=rdcpu enable-aria enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 enable-tls13downgrade >> >> As you see, for 1.1.1 master I?m enabling ARIA and setting default entropy source to RDRAND. ;-) >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5734 bytes Desc: not available URL: From Matthias.St.Pierre at ncp-e.com Sun Sep 3 21:53:50 2017 From: Matthias.St.Pierre at ncp-e.com (Dr. Matthias St. Pierre) Date: Sun, 3 Sep 2017 21:53:50 +0000 Subject: [openssl-dev] Plea for a new public OpenSSL RNG API In-Reply-To: References: Message-ID: <65bd8e39a2604d5e91f1c15ca8ffc35e@Ex13.ncp.local> > > The 'RAND_add()/RAND_bytes()' pattern is broken > =============================================== > > In OpenSSL, the classical way for the RNG consumer to add his own randomness is to call 'RAND_add()' before > calling 'RAND_bytes()'. If the new 'RAND_OpenSSL()' method (the "compatibility layer" hiding the public > RAND_DRBG instance) is the default, then this does not work as expected anymore: > > The reason is that a call to 'RAND_add()' adds the provided randomness only to a global buffer > ('rand_bytes'), from which it will be pulled during the next reseed. But no reseed is triggered. So the next > RAND_bytes() call will be unaffected from the RAND_add(), which is not what the consumer expected. (The same > holds for 'RAND_seed()', since 'drbg_seed()' only calls into 'drbg_add()') > > Reseeding of DRBGs occurs only at the following occasions: > > * immediately after a 'fork()' (new) > * if the 'reseed_counter' exceeds the 'reseed_interval' > * if 'RAND_DRBG_generate()' is called requesting 'prediction_resistance' > * 'RAND_DRBG_reseed()' is called explicitely > > *Note:* Currently it looks like the situation is even worse: if 'RAND_add()' is called multiple times before > a reseed occurs, then the result of the previous call is overwritten. I just posted GitHub PR #4328 related to this issue [openssl/openssl] WIP: Fix the RAND_add() reseeding issue (#4328) Matthias St. Pierre -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 4328 bytes Desc: not available URL: From uri at ll.mit.edu Sun Sep 3 21:59:58 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Sun, 3 Sep 2017 17:59:58 -0400 Subject: [openssl-dev] Plea for a new public OpenSSL RNG API In-Reply-To: <65bd8e39a2604d5e91f1c15ca8ffc35e@Ex13.ncp.local> References: <65bd8e39a2604d5e91f1c15ca8ffc35e@Ex13.ncp.local> Message-ID: <13449EA4-0D50-46A4-A7E9-F0600187837B@ll.mit.edu> I like this PR. Thank you! > On Sep 3, 2017, at 17:53, Dr. Matthias St. Pierre wrote: > >> >> The 'RAND_add()/RAND_bytes()' pattern is broken >> =============================================== >> >> In OpenSSL, the classical way for the RNG consumer to add his own randomness is to call 'RAND_add()' before >> calling 'RAND_bytes()'. If the new 'RAND_OpenSSL()' method (the "compatibility layer" hiding the public >> RAND_DRBG instance) is the default, then this does not work as expected anymore: >> >> The reason is that a call to 'RAND_add()' adds the provided randomness only to a global buffer >> ('rand_bytes'), from which it will be pulled during the next reseed. But no reseed is triggered. So the next >> RAND_bytes() call will be unaffected from the RAND_add(), which is not what the consumer expected. (The same >> holds for 'RAND_seed()', since 'drbg_seed()' only calls into 'drbg_add()') >> >> Reseeding of DRBGs occurs only at the following occasions: >> >> * immediately after a 'fork()' (new) >> * if the 'reseed_counter' exceeds the 'reseed_interval' >> * if 'RAND_DRBG_generate()' is called requesting 'prediction_resistance' >> * 'RAND_DRBG_reseed()' is called explicitely >> >> *Note:* Currently it looks like the situation is even worse: if 'RAND_add()' is called multiple times before >> a reseed occurs, then the result of the previous call is overwritten. > > > I just posted GitHub PR #4328 related to this issue > > [openssl/openssl] WIP: Fix the RAND_add() reseeding issue (#4328) > > > Matthias St. Pierre From rsalz at akamai.com Sun Sep 3 22:20:03 2017 From: rsalz at akamai.com (Salz, Rich) Date: Sun, 3 Sep 2017 22:20:03 +0000 Subject: [openssl-dev] 1.1.1 master consistently fails (was Re: openssl 1-1-0-stable fails) In-Reply-To: References: <6D1CC048-5836-48E6-8C54-BCFC7792EB1C@mit.edu> <0B435A4B-AB72-4484-8A95-6A6FD750E112@ll.mit.edu> Message-ID: <866D162A-D859-4FB3-B112-9535FF4E2B6C@akamai.com> ? Config & build script (feel free to suggest improvements, BTW): Perhaps don?t cut/paste green-on-black color? Plaintext is probably sufficient. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Mon Sep 4 06:51:19 2017 From: matt at openssl.org (Matt Caswell) Date: Mon, 4 Sep 2017 07:51:19 +0100 Subject: [openssl-dev] TLS 1.3 non compliance with current draft In-Reply-To: <2611567.OezTKZKLtv@pintsize.usersys.redhat.com> References: <2611567.OezTKZKLtv@pintsize.usersys.redhat.com> Message-ID: On 01/09/17 18:05, Hubert Kario wrote: > When openssl sends a second Client Hello message, it modifies it quite > extensively, not only client_random is changed but also advertised cipher > suites. > > see https://github.com/openssl/openssl/issues/4292 > > That makes it non-compliant with the current draft (-21): Yes, I've seen the github issue on this. I will take a look at this at some point this week. Matt > > When a client first connects to a server, it is REQUIRED to send the > ClientHello as its first message. The client will also send a > ClientHello when the server has responded to its ClientHello with a > HelloRetryRequest. In that case, the client *MUST send the same* > *ClientHello* (without modification) except: > > - If a "key_share" extension was supplied in the HelloRetryRequest, > replacing the list of shares with a list containing a single > KeyShareEntry from the indicated group. > > - Removing the "early_data" extension (Section 4.2.9) if one was > present. Early data is not permitted after HelloRetryRequest. > > - Including a "cookie" extension if one was provided in the > HelloRetryRequest. > > - Updating the "pre_shared_key" extension if present by recomputing > the "obfuscated_ticket_age" and binder values and (optionally) > removing any PSKs which are incompatible with the server's > indicated cipher suite. > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 480 bytes Desc: OpenPGP digital signature URL: From matt at openssl.org Mon Sep 4 07:59:14 2017 From: matt at openssl.org (Matt Caswell) Date: Mon, 4 Sep 2017 08:59:14 +0100 Subject: [openssl-dev] 1.1.1 master consistently fails (was Re: openssl 1-1-0-stable fails) In-Reply-To: References: <6D1CC048-5836-48E6-8C54-BCFC7792EB1C@mit.edu> <0B435A4B-AB72-4484-8A95-6A6FD750E112@ll.mit.edu> Message-ID: On 03/09/17 22:18, Blumenthal, Uri - 0553 - MITLL wrote: > MacOS 10.12.6, Xcode-8.3.3. Current Github master: > > Test Summary Report > ------------------- > ../test/recipes/70-test_clienthello.t (Wstat: 256 Tests: 1 > Failed: 1) > Failed test: 1 > Non-zero exit status: 1 > Files=136, Tests=1266, 461 wallclock secs ( 2.58 usr 0.48 sys + 221.36 > cusr 135.66 csys = 360.08 CPU) > Result: FAIL > make[1]: *** [_tests] Error 1 > make: *** [tests] Error 2 Fix here: https://github.com/openssl/openssl/pull/4331 Matt > > Config & build script (feel free to suggest improvements, BTW): > #!/bin/bash -ex > > make distclean || true > # For OpenSSL-1.1.1 master (development branch) > ./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc > --with-rand-seed=rdcpu enable-aria enable-ec_nistp_64_gcc_128 enable-md2 > enable-rc5 enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 > enable-tls13downgrade > # For OpenSSL-1.1.0-stable > #./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc > enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 enable-weak-ssl-ciphers > enable-zlib-dynamic enable-tls1_3 enable-tls13downgrade > make depend && make clean && make -j 2 all && make test && make install > > >> On Sep 2, 2017, at 21:15, Blumenthal, Uri - 0553 - MITLL >> > wrote: >> >> All my builds include "make distclean" at the start of the process. >> However when I repeated that cleanup and re-run the build, >> 1_1_0-stable error disappeared. A fluke?! >> >> Regards, >> Uri >> >> Sent from my iPhone >>> >>> Yes, the branch is OpenSSL_1_1_0-stable. Apparently it does not >>> complain seeing these un-realizable parameters. And error is ?genuine?. >>> >>> 1.1 master also fails tests: >>> >>> Test Summary Report >>> ------------------- >>> ../test/recipes/70-test_clienthello.t (Wstat: 256 Tests: 1 >>> Failed: 1) >>> Failed test: 1 >>> Non-zero exit status: 1 >>> Files=136, Tests=1266, 480 wallclock secs ( 2.72 usr 0.55 sys + >>> 226.57 cusr 144.10 csys = 373.94 CPU) >>> Result: FAIL >>> make[1]: *** [_tests] Error 1 >>> make: *** [tests] Error 2 >>> $ >>> >>> Config: >>> >>> ./config --prefix=$HOME/openssl-1.1 >>> --openssldir=$HOME/openssl-1.1/etc --with-rand-seed=rdcpu enable-aria >>> enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 >>> enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 >>> enable-tls13downgrade >>> >>> As you see, for 1.1.1 master I?m enabling ARIA and setting default >>> entropy source to RDRAND. ;-) >>> > > > From matt at openssl.org Mon Sep 4 14:24:17 2017 From: matt at openssl.org (Matt Caswell) Date: Mon, 4 Sep 2017 15:24:17 +0100 Subject: [openssl-dev] 1.1.1 master consistently fails (was Re: openssl 1-1-0-stable fails) In-Reply-To: References: <6D1CC048-5836-48E6-8C54-BCFC7792EB1C@mit.edu> <0B435A4B-AB72-4484-8A95-6A6FD750E112@ll.mit.edu> Message-ID: <119a1b78-4aa0-d5a2-5125-e895b5ef2871@openssl.org> On 04/09/17 08:59, Matt Caswell wrote: > > > On 03/09/17 22:18, Blumenthal, Uri - 0553 - MITLL wrote: >> MacOS 10.12.6, Xcode-8.3.3. Current Github master: >> >> Test Summary Report >> ------------------- >> ../test/recipes/70-test_clienthello.t (Wstat: 256 Tests: 1 >> Failed: 1) >> Failed test: 1 >> Non-zero exit status: 1 >> Files=136, Tests=1266, 461 wallclock secs ( 2.58 usr 0.48 sys + 221.36 >> cusr 135.66 csys = 360.08 CPU) >> Result: FAIL >> make[1]: *** [_tests] Error 1 >> make: *** [tests] Error 2 > > Fix here: > > https://github.com/openssl/openssl/pull/4331 This is now merged, so this issue should be fixed. > > > Matt > >> >> Config & build script (feel free to suggest improvements, BTW): >> #!/bin/bash -ex >> >> make distclean || true >> # For OpenSSL-1.1.1 master (development branch) >> ./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc >> --with-rand-seed=rdcpu enable-aria enable-ec_nistp_64_gcc_128 enable-md2 >> enable-rc5 enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 >> enable-tls13downgrade >> # For OpenSSL-1.1.0-stable >> #./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc >> enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 enable-weak-ssl-ciphers >> enable-zlib-dynamic enable-tls1_3 enable-tls13downgrade >> make depend && make clean && make -j 2 all && make test && make install >> >> >>> On Sep 2, 2017, at 21:15, Blumenthal, Uri - 0553 - MITLL >>> > wrote: >>> >>> All my builds include "make distclean" at the start of the process. >>> However when I repeated that cleanup and re-run the build, >>> 1_1_0-stable error disappeared. A fluke?! >>> >>> Regards, >>> Uri >>> >>> Sent from my iPhone >>>> >>>> Yes, the branch is OpenSSL_1_1_0-stable. Apparently it does not >>>> complain seeing these un-realizable parameters. And error is ?genuine?. >>>> >>>> 1.1 master also fails tests: >>>> >>>> Test Summary Report >>>> ------------------- >>>> ../test/recipes/70-test_clienthello.t (Wstat: 256 Tests: 1 >>>> Failed: 1) >>>> Failed test: 1 >>>> Non-zero exit status: 1 >>>> Files=136, Tests=1266, 480 wallclock secs ( 2.72 usr 0.55 sys + >>>> 226.57 cusr 144.10 csys = 373.94 CPU) >>>> Result: FAIL >>>> make[1]: *** [_tests] Error 1 >>>> make: *** [tests] Error 2 >>>> $ >>>> >>>> Config: >>>> >>>> ./config --prefix=$HOME/openssl-1.1 >>>> --openssldir=$HOME/openssl-1.1/etc --with-rand-seed=rdcpu enable-aria >>>> enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 >>>> enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 >>>> enable-tls13downgrade >>>> >>>> As you see, for 1.1.1 master I?m enabling ARIA and setting default >>>> entropy source to RDRAND. ;-) >>>> >> >> >> From uri at ll.mit.edu Mon Sep 4 17:58:26 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Mon, 4 Sep 2017 17:58:26 +0000 Subject: [openssl-dev] 1.1.1 master consistently fails (was Re: openssl 1-1-0-stable fails) In-Reply-To: <119a1b78-4aa0-d5a2-5125-e895b5ef2871@openssl.org> References: <6D1CC048-5836-48E6-8C54-BCFC7792EB1C@mit.edu> <0B435A4B-AB72-4484-8A95-6A6FD750E112@ll.mit.edu> <119a1b78-4aa0-d5a2-5125-e895b5ef2871@openssl.org> Message-ID: <50055EED-8742-4F8A-9D9E-13823E57D01A@ll.mit.edu> Fix confirmed, thank you! Regards, Uri Sent from my iPhone > On Sep 4, 2017, at 10:25, Matt Caswell wrote: > > > >> On 04/09/17 08:59, Matt Caswell wrote: >> >> >>> On 03/09/17 22:18, Blumenthal, Uri - 0553 - MITLL wrote: >>> MacOS 10.12.6, Xcode-8.3.3. Current Github master: >>> >>> Test Summary Report >>> ------------------- >>> ../test/recipes/70-test_clienthello.t (Wstat: 256 Tests: 1 >>> Failed: 1) >>> Failed test: 1 >>> Non-zero exit status: 1 >>> Files=136, Tests=1266, 461 wallclock secs ( 2.58 usr 0.48 sys + 221.36 >>> cusr 135.66 csys = 360.08 CPU) >>> Result: FAIL >>> make[1]: *** [_tests] Error 1 >>> make: *** [tests] Error 2 >> >> Fix here: >> >> https://github.com/openssl/openssl/pull/4331 > > This is now merged, so this issue should be fixed. > > >> >> >> Matt >> >>> >>> Config & build script (feel free to suggest improvements, BTW): >>> #!/bin/bash -ex >>> >>> make distclean || true >>> # For OpenSSL-1.1.1 master (development branch) >>> ./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc >>> --with-rand-seed=rdcpu enable-aria enable-ec_nistp_64_gcc_128 enable-md2 >>> enable-rc5 enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 >>> enable-tls13downgrade >>> # For OpenSSL-1.1.0-stable >>> #./config --prefix=$HOME/openssl-1.1 --openssldir=$HOME/openssl-1.1/etc >>> enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 enable-weak-ssl-ciphers >>> enable-zlib-dynamic enable-tls1_3 enable-tls13downgrade >>> make depend && make clean && make -j 2 all && make test && make install >>> >>> >>>> On Sep 2, 2017, at 21:15, Blumenthal, Uri - 0553 - MITLL >>>> > wrote: >>>> >>>> All my builds include "make distclean" at the start of the process. >>>> However when I repeated that cleanup and re-run the build, >>>> 1_1_0-stable error disappeared. A fluke?! >>>> >>>> Regards, >>>> Uri >>>> >>>> Sent from my iPhone >>>>> >>>>> Yes, the branch is OpenSSL_1_1_0-stable. Apparently it does not >>>>> complain seeing these un-realizable parameters. And error is ?genuine?. >>>>> >>>>> 1.1 master also fails tests: >>>>> >>>>> Test Summary Report >>>>> ------------------- >>>>> ../test/recipes/70-test_clienthello.t (Wstat: 256 Tests: 1 >>>>> Failed: 1) >>>>> Failed test: 1 >>>>> Non-zero exit status: 1 >>>>> Files=136, Tests=1266, 480 wallclock secs ( 2.72 usr 0.55 sys + >>>>> 226.57 cusr 144.10 csys = 373.94 CPU) >>>>> Result: FAIL >>>>> make[1]: *** [_tests] Error 1 >>>>> make: *** [tests] Error 2 >>>>> $ >>>>> >>>>> Config: >>>>> >>>>> ./config --prefix=$HOME/openssl-1.1 >>>>> --openssldir=$HOME/openssl-1.1/etc --with-rand-seed=rdcpu enable-aria >>>>> enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 >>>>> enable-weak-ssl-ciphers enable-zlib-dynamic enable-tls1_3 >>>>> enable-tls13downgrade >>>>> >>>>> As you see, for 1.1.1 master I?m enabling ARIA and setting default >>>>> entropy source to RDRAND. ;-) >>>>> >>> >>> >>> > -- > openssl-dev mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4223 bytes Desc: not available URL: From David.von.Oheimb at siemens.com Tue Sep 5 12:12:23 2017 From: David.von.Oheimb at siemens.com (David von Oheimb) Date: Tue, 5 Sep 2017 14:12:23 +0200 Subject: [openssl-dev] How to use BIO_do_connect(), blocking and non-blocking with timeout, coping with errors In-Reply-To: References: <69d6c8c3-c3ba-44f5-fad0-4b2294e210d3@siemens.com> Message-ID: <687c5c90-0a7c-ab0a-3197-472975acc9af@siemens.com> /[ Further below I quote my first two messages including my original questions and tentative code,// //?since Cc'ing to openssl-users did not work when I tried first. In this way I hope to get further, // //?more detailed responses by people with specific experience on the issues I mentioned,// //?possibly even concrete feedback how to enhance my code or where to find a better solution. ]/ On 09/01/2017 06:32 PM, Salz, Rich via openssl-users wrote: > > FWIW, there?s a ?libtls? library from the libre folks that might be > worth looking at. > This looks very nice. Yet is this of any practical benefit when using OpenSSL? > If you come up with useful snippets we can start by posting them to > the wiki, for example > Which wiki do you mean? I could not find anything related on https://wiki.openssl.org/ Anyway, most people (including me) would not search through wikis for finding useful code, and it would be much more useful if code like the bio_connect() function I mentioned below was readily available within the OpenSSL libraries, in an official high-level API. /[ Since this is worth a topic of its own, I'll write more on this in my next email. ]/ More low-level code that is already used by the crypto lib itself (e.g., using select() in rand_unix.c) would be better packaged into abstractions within that library, for instance the socket_wait() function for waiting on a socket with a timeout that I proposed below. I'd contribute pull requests for those I'm aware of. On 29.08.2017 16:15, Salz, Rich via openssl-dev wrote: >> Getting the client connect right appears surprisingly messy when one >> needs to cope with all kinds of network error situations including >> domain name resolution issues and temporarily unreachable servers. >> Both indefinitely blocking and non-blocking behavior (i.e., connection >> attempts with and without a timeout) should be supported. > It is a complicated issue and hard to get right for all definitions of right for all applications ? Hmm - on the one hand, good to get confirmation that I did not just miss a simple way of out the maze, ... > A set of API?s that set up all the TLS ?metadata?, and took a connected socket might be a way through the maze. For example: > SSL *SSL_connection(int socket, const char *servername, ?whatever?) ... on the other hand, it's a pity that such a high-level API does not (yet) exist, at least not in OpenSSL. How about adding at least some clearly useful abstractions like the below socket_wait() function, which would reduce code duplication in the OpenSSL crypto lib and apps and help application developers? Maybe other OpenSSL users have specific experience on error and timeout handling for BIO_do_connect() etc. and can comment in more detail on the (approximate) solution, bio_connect(), that I gave below? On 28.08.2017 13:46, David von Oheimb wrote: > Hi all, > > I'm currently enhancing HTTP(S) clients based on OpenSSL in several > flavors, in particular a CMP client, which in turn uses simple HTTP > clients for contacting CRL distribution points or OCSP responders. > > Getting the client connect right appears surprisingly messy when one > needs to cope with all kinds of network error situations including > domain name resolution issues and temporarily unreachable servers. > Both indefinitely blocking and non-blocking behavior (i.e., connection > attempts with and without a timeout) should be supported. > > Since these are pretty general problems I wonder why there there is > rather limited support via generic higher-level OpenSSL or C library > functions, or at least I was unable to find it. Instead, the OpenSSL > apps contain code that calls BIO_do_connect directly (or the equivalent > BIO_do_handshake), in particular query_responder() in apps/ocsp.c. > (The situation is similar for the subsequent exchange of data via the > BIO, optionally with a timeout). > > So I constructed my own abstraction, called bio_connect, which took > quite some effort testing network error situations. Please see below its > code including comments on some strange behavior I experienced and my > workarounds for that. Does this code make sense, or do I miss anything? > > How about adding such a function for instance to crypto/bio/bio_lib.c? > > BTW, my code uses a handy generic helper function, socket_wait, for > waiting for read/write form/to a socket, with a given timeout. Since > several instances of that pretty common code pattern using select() are > spread over the OpenSSL apps (and crypto lib), I suggest adding this > function to the library. Where would be a good place to put it? > > Thanks, > David >> /* returns -1 on error, 0 on timeout, 1 on success */ >> int bio_connect(BIO *bio, int timeout) { >> int blocking; >> time_t max_time; >> int rv; >> >> blocking = timeout == 0; >> max_time = timeout != 0 ? time(NULL) + timeout : 0; >> >> if (!blocking) >> BIO_set_nbio(bio, 1); >> retry: >> rv = BIO_do_connect(bio); >> if (rv <= 0 && (errno == ETIMEDOUT /* in blocking case, >> despite blocking BIO, BIO_do_connect() timed out */ || >> ERR_GET_REASON(ERR_peek_error()) == ETIMEDOUT/* when non-blocking, >> BIO_do_connect() timed out early with rv == -1 and errno == 0 */)) { >> ERR_clear_error(); >> (void)BIO_reset(bio); /* otherwise, blocking next connect() may crash >> and non-blocking next BIO_do_connect() will fail */ >> goto retry; >> } >> if (!blocking && rv <= 0 && BIO_should_retry(bio)) { >> int fd; >> if (BIO_get_fd(bio, &fd) <= 0) >> return -1; >> rv = socket_wait(fd, 1, max_time - time(NULL)); >> if (rv > 0) >> /* for some reason, select() may wrongly have returned success */ >> goto retry; >> } >> return rv; >> } >> /* returns < 0 on error, 0 on timeout, >0 on success */ >> int socket_wait(int fd, int for_read, int timeout) >> { >> fd_set confds; >> struct timeval tv; >> >> if (timeout <= 0) >> return 0; >> >> FD_ZERO(&confds); >> openssl_fdset(fd, &confds); >> tv.tv_usec = 0; >> tv.tv_sec = timeout; >> return select(fd + 1, for_read ? &confds : NULL, >> for_read ? NULL : &confds, NULL, &tv); >> } -------------- next part -------------- An HTML attachment was scrubbed... URL: From David.von.Oheimb at siemens.com Tue Sep 5 12:12:37 2017 From: David.von.Oheimb at siemens.com (David von Oheimb) Date: Tue, 5 Sep 2017 14:12:37 +0200 Subject: [openssl-dev] Introduce a TLS application library - a proposal on the overall OpenSSL code structure In-Reply-To: References: Message-ID: Back on 13 May 2016 I had proposed by email to a couple of people including Rich Salz a third library level (on top of crypto and ssl) with more high-level, application-oriented code. His response was: > That is a really interesting idea. Please bring this up on openssl-dev mailing list. Then I posted that by mistake unfortunately not in the right forum but at: https://groups.google.com/forum/#!topic/mailing.openssl.dev/FOL2afc3cb8 I quote my post here for convenience: > So far, the OpenSSL code has essentially a three-level structure with > a hierarchy of two libraries and a command-line application at the top: > > apps/openssl > libssl > libcrypto > > In the apps/ directory there is various generally useful code like > handling crypto-related files and messages, general TLS client/server > and CA functionality, implementing parts of protocols like S/MIME, > CRL, and OCSP, and certainly more to come. > > While this code serves as a model for using the libraries and it can > be used in a limited way by invoking the openssl application binary, > it cannot be re-used directly. Other applications that need similar > functionality need to copy/re-implement and then maintain portions of > that code. > > On the other hand, the libraries contain some code that is actually > too high-level for them, for instance the minimal HTTP client as part > of the crypto library (crypto/ocsp/ocsp_ht.c). > > It would be very helpful to introduce a further level in the hierarchy > consisting of a more application-oriented library: > > apps/openssl > libtlsapps <-- new (with tentative name here) > libssl > libcrypto > > Then all more high-level and application support functionality will go > there. This would make much of the generally useful code that so far > resides in the apps/ folder directly accessible to other > ?applications at the programming level, i.e., in the form of a > library/API, with all the re-usability advantages that this brings. It > would also relieve libcrypto from more application-/high-level topics > like HTTP. > > This library would also form an ideal condensation point for further > high-level uses of TLS that may in the future get integrated with > OpenSSL, like CMP and EST implementations. I recently learned that LibreSSL already/meanwhile has something in this direction: * libtls : a new TLS library, designed to make it easier to write foolproof applications I believe this would be of great benefit also for OpenSSL itself. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Wed Sep 6 22:24:00 2017 From: matt at openssl.org (Matt Caswell) Date: Wed, 6 Sep 2017 23:24:00 +0100 Subject: [openssl-dev] QUIC Message-ID: Issue 4283 (https://github.com/openssl/openssl/issues/4283) has caused me to take a close look at QUIC. This seems to have been getting a *lot* of attention just recently. See the IDs below for details: https://tools.ietf.org/html/draft-ietf-quic-transport-05 https://tools.ietf.org/html/draft-ietf-quic-tls-05 https://tools.ietf.org/html/draft-ietf-quic-recovery-05 For the uninitiated QUIC is a new general-purpose transport protocol built on top of UDP. It provides applications with a secure stream abstraction (like TLS over TCP) with reliable, in-order delivery, as well as the ability to multiplex many streams over a single connection (without head-of-line blocking). It is *very* closely integrated with TLSv1.3. It uses the TLSv1.3 handshake for agreeing various QUIC parameters (via extensions) as well as for agreeing keying material and providing an "early data" capability. The actual packet protection is done by QUIC itself (so it doesn't use TLS application data) using a QUIC ciphersuite that matches the negotiated TLS ciphersuite. Effectively you can think of QUIC as a modernised rival to TLS over TCP. I've spent some time today reading through the IDs. It has become clear to me that in order for OpenSSL to be used to implement QUIC there are a number of new requirements/issues we would need to address: - We need to provide the server half of the TLSv1.3 cookie mechanism. At the moment an OpenSSL client will echo a TLSv1.3 cookie it receives back to the server, but you cannot generate a cookie on the server side. - We need to be able to support *stateless* operation for the ClientHello->HelloRetryRequest exchange. This is very much in the same vein as the stateless way that DTLSv1_listen() works now for DTLS in the ClientHello->HelloVerifyRequest exchange. This is quite a significant requirement. - A QUIC server needs to be able to issue a NewSessionTicket on demand - Ticket PSKs need to be able to have an embedded QUIC layer token (the equivalent of the cookie - but embedded inside the PSK). - We need to extend the "exporter" API to allow early_secret based exports. At the moment you can only export based on the final 1-RTT key. - TLS PSKs are transferable between TLS-TCP and QUIC/TLS-UDP. There are some special rules around ALPN for this that may impact our current logic in this area. - Possibly a QUIC implementation will need to have knowledge of the TLSv1.3 state machine because different TLSv1.3 handshake records need to go into different types of QUIC packets (ClientHello needs to go into "Client Initial" packet, HelloRetryRequest needs to go into a "Server Stateless Retry" packet and everything else goes into "Client Cleartext" or "Server Cleartext" packets). It may be possible for a QUIC implementation to infer the required information without additional APIs, but I'm not sure. - QUIC places size limits on the allowed size of a ClientHello. Possibly we may want some way of failing gracefully if we attempt to exceed that (or maybe we just leave that to the QUIC implementation to detect). I'm going to start working through this list of requirements, but if anyone fancies picking some of it up then let me know. Also, did I miss anything from the above list? Matt From bkaduk at akamai.com Thu Sep 7 16:45:11 2017 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Thu, 7 Sep 2017 11:45:11 -0500 Subject: [openssl-dev] QUIC In-Reply-To: References: Message-ID: On 09/06/2017 05:24 PM, Matt Caswell wrote: > Issue 4283 (https://github.com/openssl/openssl/issues/4283) has caused > me to take a close look at QUIC. This seems to have been getting a *lot* > of attention just recently. See the IDs below for details: Yes, it's generated a lot of excitement and interest at the IETF. > https://tools.ietf.org/html/draft-ietf-quic-transport-05 > https://tools.ietf.org/html/draft-ietf-quic-tls-05 > https://tools.ietf.org/html/draft-ietf-quic-recovery-05 > > For the uninitiated QUIC is a new general-purpose transport protocol > built on top of UDP. It provides applications with a secure stream > abstraction (like TLS over TCP) with reliable, in-order delivery, as > well as the ability to multiplex many streams over a single connection > (without head-of-line blocking). > > It is *very* closely integrated with TLSv1.3. It uses the TLSv1.3 > handshake for agreeing various QUIC parameters (via extensions) as well > as for agreeing keying material and providing an "early data" > capability. The actual packet protection is done by QUIC itself (so it > doesn't use TLS application data) using a QUIC ciphersuite that matches > the negotiated TLS ciphersuite. Effectively you can think of QUIC as a > modernised rival to TLS over TCP. The nature of the QUIC/TLSv1.3 integration is somewhat interesting. QUIC has its origins at Google, and the "Google QUIC" or gQUIC variant is deployed on the public internet even now; since TLS 1.3 was not available then, it uses a separate "quic-crypto" scheme for these purposes. quic-crypto, in turn, helped shape the evolution of TLS 1.3, including the strong desire for 0-RTT functionality. But, as I understand it, the intent is to leave enough hooks that a different crypto layer could be used, including (but not limited to) a subsequent version of TLS. > I've spent some time today reading through the IDs. It has become clear > to me that in order for OpenSSL to be used to implement QUIC there are a > number of new requirements/issues we would need to address: > > - We need to provide the server half of the TLSv1.3 cookie mechanism. At > the moment an OpenSSL client will echo a TLSv1.3 cookie it receives back > to the server, but you cannot generate a cookie on the server side. Yeah, the cookie is pretty clear to the UDP/"stateless" operation. > - We need to be able to support *stateless* operation for the > ClientHello->HelloRetryRequest exchange. This is very much in the same > vein as the stateless way that DTLSv1_listen() works now for DTLS in the > ClientHello->HelloVerifyRequest exchange. This is quite a significant > requirement. The expectation is that the state gets bundled into the cookie, yes. > - A QUIC server needs to be able to issue a NewSessionTicket on demand > > - Ticket PSKs need to be able to have an embedded QUIC layer token (the > equivalent of the cookie - but embedded inside the PSK). I think https://github.com/openssl/openssl/pull/3802 is pretty close, in this space. > - We need to extend the "exporter" API to allow early_secret based > exports. At the moment you can only export based on the final 1-RTT key. It seems in keeping with our existing handling of early data, to at least consider providing a separate API for these early exporter values. > - TLS PSKs are transferable between TLS-TCP and QUIC/TLS-UDP. There are > some special rules around ALPN for this that may impact our current > logic in this area. > > - Possibly a QUIC implementation will need to have knowledge of the > TLSv1.3 state machine because different TLSv1.3 handshake records need > to go into different types of QUIC packets (ClientHello needs to go into > "Client Initial" packet, HelloRetryRequest needs to go into a "Server > Stateless Retry" packet and everything else goes into "Client Cleartext" > or "Server Cleartext" packets). It may be possible for a QUIC > implementation to infer the required information without additional > APIs, but I'm not sure. We do have existing things like the message callback, but I won't try to argue that that's an ideal situation for a QUIC implementor. And the QUIC layer could even parse out the unencrypted records for itself from the output BIO, as silly as that would be. > - QUIC places size limits on the allowed size of a ClientHello. Possibly > we may want some way of failing gracefully if we attempt to exceed that > (or maybe we just leave that to the QUIC implementation to detect). (This is to limit the potential for a DoS amplification attack via spoofed client address, since UDP does not provide the reachability confirmation that TCP's handshake does, for the spectators.) > I'm going to start working through this list of requirements, but if > anyone fancies picking some of it up then let me know. Also, did I miss > anything from the above list? > Nothing sticks out as missing to me, but I've not been following QUIC development as closely as I'd like. -Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From beldmit at gmail.com Sat Sep 9 14:10:32 2017 From: beldmit at gmail.com (Dmitry Belyavsky) Date: Sat, 9 Sep 2017 17:10:32 +0300 Subject: [openssl-dev] X509_cmp_time (possible) bug Message-ID: Hello, The X509_cmp_time function is documented as returning -1 or 1 on success and 0 on error. In fact it returns result of strcmp: int X509_cmp_time(const ASN1_TIME *ctm, time_t *cmp_time) { ... i = strcmp(buff1, buff2); if (i == 0) /* wait a second then return younger */ return -1; else return i; } According to documentation to the strcmp, The strcmp() and strncmp() functions return an integer less than, equal to, or greater than zero if s1 (or the first n bytes thereof) is found, respectively, to be less than, to match, or be greater than s2. It means (and have been met in practice) that X509_cmp_time() returns other values than 1/-1. So it seems reasonable to either update documentation or fix the behavior. Thank you! -- SY, Dmitry Belyavsky -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.hart at hotmail.co.uk Mon Sep 11 11:11:00 2017 From: matt.hart at hotmail.co.uk (Matt Hart) Date: Mon, 11 Sep 2017 04:11:00 -0700 (MST) Subject: [openssl-dev] Openssl and MAC KeyChain tool Integration In-Reply-To: <1499319011379-71393.post@n7.nabble.com> References: <1499319011379-71393.post@n7.nabble.com> Message-ID: <1505128260966-0.post@n7.nabble.com> My understanding is there currently is no OpenSSL engine for the MacOSX chain services. It would be nice to have and I really wish I had time to collaborate on such a project, using Stephen Henson's CAPI work as a template. Currently I would suggest following the same model as OpenSSL for other non-Windows platforms and supply a file based cacert.pem of public certs and a password protected .pfx file for your client cert. -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-Dev-f29372.html From tshort at akamai.com Mon Sep 11 14:16:11 2017 From: tshort at akamai.com (Short, Todd) Date: Mon, 11 Sep 2017 14:16:11 +0000 Subject: [openssl-dev] X509_cmp_time (possible) bug In-Reply-To: References: Message-ID: <827728A8-4490-4EF7-AA43-463DDA2ADB99@akamai.com> Yes, it?s annoying, but it?s historic. I looked into changing this at one point. I recommend using ASN1_TIME_cmp_time_t() (from the master branch) instead, for the results you are expecting. -- -Todd Short // tshort at akamai.com // "One if by land, two if by sea, three if by the Internet." On Sep 9, 2017, at 10:10 AM, Dmitry Belyavsky > wrote: Hello, The X509_cmp_time function is documented as returning -1 or 1 on success and 0 on error. In fact it returns result of strcmp: int X509_cmp_time(const ASN1_TIME *ctm, time_t *cmp_time) { ... i = strcmp(buff1, buff2); if (i == 0) /* wait a second then return younger */ return -1; else return i; } According to documentation to the strcmp, The strcmp() and strncmp() functions return an integer less than, equal to, or greater than zero if s1 (or the first n bytes thereof) is found, respectively, to be less than, to match, or be greater than s2. It means (and have been met in practice) that X509_cmp_time() returns other values than 1/-1. So it seems reasonable to either update documentation or fix the behavior. Thank you! -- SY, Dmitry Belyavsky -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From dkg at fifthhorseman.net Mon Sep 11 14:43:27 2017 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Mon, 11 Sep 2017 10:43:27 -0400 Subject: [openssl-dev] X509_cmp_time (possible) bug In-Reply-To: <827728A8-4490-4EF7-AA43-463DDA2ADB99@akamai.com> References: <827728A8-4490-4EF7-AA43-463DDA2ADB99@akamai.com> Message-ID: <87fubt1ek0.fsf@fifthhorseman.net> On Mon 2017-09-11 14:16:11 +0000, Short, Todd via openssl-dev wrote: > Yes, it?s annoying, but it?s historic. I looked into changing this at one point. I think Dimitry's point was that the documentation doesn't match the implementation because of the flexibility of strcmp's defined return code. However, i think commit 80770da39ebba0101079477611b7ce2f426653c5 ("X509 time: tighten validation per RFC 5280") resolves Dmitry's concerns. --dkg From tshort at akamai.com Mon Sep 11 15:54:04 2017 From: tshort at akamai.com (Short, Todd) Date: Mon, 11 Sep 2017 15:54:04 +0000 Subject: [openssl-dev] X509_cmp_time (possible) bug In-Reply-To: <87fubt1ek0.fsf@fifthhorseman.net> References: <827728A8-4490-4EF7-AA43-463DDA2ADB99@akamai.com> <87fubt1ek0.fsf@fifthhorseman.net> Message-ID: Correct, But if one want?s strcmp()?s behavior (i.e. 0 is equality), ASN1_TIME_cmp_time_t() will work (and was written because X509_cmp_time() couldn?t be changed without breaking other things). -- -Todd Short // tshort at akamai.com // "One if by land, two if by sea, three if by the Internet." On Sep 11, 2017, at 10:43 AM, Daniel Kahn Gillmor > wrote: On Mon 2017-09-11 14:16:11 +0000, Short, Todd via openssl-dev wrote: Yes, it?s annoying, but it?s historic. I looked into changing this at one point. I think Dimitry's point was that the documentation doesn't match the implementation because of the flexibility of strcmp's defined return code. However, i think commit 80770da39ebba0101079477611b7ce2f426653c5 ("X509 time: tighten validation per RFC 5280") resolves Dmitry's concerns. --dkg -------------- next part -------------- An HTML attachment was scrubbed... URL: From zshrdlu at gmail.com Tue Sep 12 00:46:13 2017 From: zshrdlu at gmail.com (Winter Mute) Date: Tue, 12 Sep 2017 03:46:13 +0300 Subject: [openssl-dev] id-kp-OCSPSigning extended key usage Message-ID: Hello, The RFC states that: > OCSP signing delegation SHALL be designated by the inclusion of > id-kp-OCSPSigning in an extended key usage certificate extension > included in the OCSP response signer's certificate. The use of "SHALL" rather than "MUST" indicates that this recommendation can be ignored. How does openssl handle OCSP responses signed by certificates that do not have id-kp-OCSPSigning in the extended key usage certificate extension when the responses are not signed by the issuing CA directly? What informs this decision/policy? Are there any security implications in including or excluding OCSP-sign in the extended key usage extension? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mischa.salle at gmail.com Tue Sep 12 12:26:53 2017 From: mischa.salle at gmail.com (Mischa Salle) Date: Tue, 12 Sep 2017 14:26:53 +0200 Subject: [openssl-dev] id-kp-OCSPSigning extended key usage In-Reply-To: References: Message-ID: Hi, On Tue, Sep 12, 2017 at 2:46 AM, Winter Mute wrote: > Hello, > The RFC states that: > >> OCSP signing delegation SHALL be designated by the inclusion of >> id-kp-OCSPSigning in an extended key usage certificate extension >> included in the OCSP response signer's certificate. > > The use of "SHALL" rather than "MUST" indicates that this recommendation > can be ignored. > SHALL is equivalent to MUST, see RFC2119: .... MUST This word, or the terms "REQUIRED" or "SHALL"... I think you're thinking of SHOULD. Cheers, Mischa How does openssl handle OCSP responses signed by certificates that do not > have id-kp-OCSPSigning in the extended key usage certificate extension > when the responses are not signed by the issuing CA directly? > What informs this decision/policy? > Are there any security implications in including or excluding OCSP-sign in > the extended key usage extension? > > -- > openssl-dev mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zshrdlu at gmail.com Tue Sep 12 12:39:22 2017 From: zshrdlu at gmail.com (Winter Mute) Date: Tue, 12 Sep 2017 15:39:22 +0300 Subject: [openssl-dev] id-kp-OCSPSigning extended key usage In-Reply-To: References: Message-ID: Hi, Thanks for the clarification. Per the spec, then, a certificate designated to sign OCSP responses is required to have the ocsp-sign bit in the key usage extensions set. How does openssl handle cases where this requirement is violated? On Sep 12, 2017 3:27 PM, "Mischa Salle" wrote: > Hi, > > > On Tue, Sep 12, 2017 at 2:46 AM, Winter Mute wrote: > >> Hello, >> The RFC states >> that: >> >>> OCSP signing delegation SHALL be designated by the inclusion of >>> id-kp-OCSPSigning in an extended key usage certificate extension >>> included in the OCSP response signer's certificate. >> >> The use of "SHALL" rather than "MUST" indicates that this recommendation >> can be ignored. >> > > SHALL is equivalent to MUST, see RFC2119: > .... MUST This word, or the terms "REQUIRED" or "SHALL"... > I think you're thinking of SHOULD. > > Cheers, > Mischa > > How does openssl handle OCSP responses signed by certificates that do not >> have id-kp-OCSPSigning in the extended key usage certificate extension >> when the responses are not signed by the issuing CA directly? >> What informs this decision/policy? >> Are there any security implications in including or excluding OCSP-sign >> in the extended key usage extension? >> >> -- >> openssl-dev mailing list >> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev >> >> > > -- > openssl-dev mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Tue Sep 12 12:52:29 2017 From: rsalz at akamai.com (Salz, Rich) Date: Tue, 12 Sep 2017 12:52:29 +0000 Subject: [openssl-dev] id-kp-OCSPSigning extended key usage In-Reply-To: References: Message-ID: ? Thanks for the clarification. Per the spec, then, a certificate designated to sign OCSP responses is required to have the ocsp-sign bit in the key usage extensions set. ? How does openssl handle cases where this requirement is violated? Look at check_delegated() in ocsp/ocsp_vfy.c It returns an error. From Erwann.Abalea at docusign.com Tue Sep 12 11:59:36 2017 From: Erwann.Abalea at docusign.com (Erwann Abalea) Date: Tue, 12 Sep 2017 11:59:36 +0000 Subject: [openssl-dev] id-kp-OCSPSigning extended key usage In-Reply-To: References: Message-ID: Bonjour, SHALL is not equivalent to a SHOULD, but to a MUST. See RFC2119. Cordialement, Erwann Abalea Le 12 sept. 2017 ? 02:46, Winter Mute > a ?crit : Hello, The RFC states that: OCSP signing delegation SHALL be designated by the inclusion of id-kp-OCSPSigning in an extended key usage certificate extension included in the OCSP response signer's certificate. The use of "SHALL" rather than "MUST" indicates that this recommendation can be ignored. How does openssl handle OCSP responses signed by certificates that do not have id-kp-OCSPSigning in the extended key usage certificate extension when the responses are not signed by the issuing CA directly? What informs this decision/policy? Are there any security implications in including or excluding OCSP-sign in the extended key usage extension? -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From qianfan1991 at gmail.com Wed Sep 13 19:24:44 2017 From: qianfan1991 at gmail.com (=?utf-8?B?5LqO5Y2D5biG?=) Date: Wed, 13 Sep 2017 14:24:44 -0500 Subject: [openssl-dev] SSL certificate problem: unable to get local issuer certificate Message-ID: <7B8D6C19-B99A-44C9-B8A7-9C57F9D2D98B@gmail.com> Dear sir or madam, I?m trying to use quantmod package in R to get financial data from Yahoo. It works perfectly on my personal laptops (Mac and Win). But I cannot make it work on my working computer (Win7). Here's the error code: Error in curl::curl_download(cu, tmp, handle = h) : SSL certificate problem: unable to get local issuer certificate I have tried to solve the problems in the following ways: 1. install httr package 2. Delete curl, quantmod and reinstall 3. Update to the latest version of R, RStudio, curl, httr, RCurl and quantmod 4. Install openssl package 5. Set ssl_verifypeer = 0L Unfortunately, none of the above works. I had this problem earlier when I was trying to download a file using R. At that time, I switched to wget and made it work. But now I'm using an existing package and I have no idea how to solve this problem. BTW, fetch data from google finance works but Google only provide ~ 4000 records per request. I think our company has some restrictions on SSL but I'm totally fine in accessing websites start with https:// I have spent a week to solve this problem but I didn't make any progress. Now, I'm wondering is that possible to fix it? Any ideas will be helpful! Thanks to all the contributors, have a great one! Best, Arthur -------------- next part -------------- An HTML attachment was scrubbed... URL: From ted.marynicz at gmail.com Thu Sep 14 10:11:15 2017 From: ted.marynicz at gmail.com (Ted Marynicz) Date: Thu, 14 Sep 2017 11:11:15 +0100 Subject: [openssl-dev] SSL certificate problem: unable to get local issuer certificate In-Reply-To: <7B8D6C19-B99A-44C9-B8A7-9C57F9D2D98B@gmail.com> References: <7B8D6C19-B99A-44C9-B8A7-9C57F9D2D98B@gmail.com> Message-ID: Hi Arthur, did you read the curl docs, especially on TLS? -> https://ec.haxx.se/usingcurl-tls.html Have you run curl from a command line in verbose mode on all machines? What are the results? Can you give the Yahoo URL for others to check? Regards, Ted Marynicz On 13 September 2017 at 20:24, ??? wrote: > Dear sir or madam, > > > > I?m trying to use quantmod package in R to get financial data from Yahoo. > It works perfectly on my personal laptops (Mac and Win). But I cannot make > it work on my working computer (Win7). Here's the error code: > > Error in curl::curl_download(cu, tmp, handle = h) : > > SSL certificate problem: unable to get local issuer certificate > > I have tried to solve the problems in the following ways: > 1. install httr package > 2. Delete curl, quantmod and reinstall > 3. Update to the latest version of R, RStudio, curl, httr, RCurl and > quantmod > 4. Install openssl package > 5. Set ssl_verifypeer = 0L > > Unfortunately, none of the above works. I had this problem earlier when I > was trying to download a file using R. At that time, I switched to wget and > made it work. But now I'm using an existing package and I have no idea how > to solve this problem. BTW, fetch data from google finance works but Google > only provide ~ 4000 records per request. > > I think our company has some restrictions on SSL but I'm totally fine in > accessing websites start with https:// > > I have spent a week to solve this problem but I didn't make any progress. > Now, I'm wondering is that possible to fix it? Any ideas will be helpful! > > Thanks to all the contributors, have a great one! > > Best, > Arthur > > -- > openssl-dev mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From doctor at doctor.nl2k.ab.ca Thu Sep 14 12:09:11 2017 From: doctor at doctor.nl2k.ab.ca (The Doctor) Date: Thu, 14 Sep 2017 06:09:11 -0600 Subject: [openssl-dev] 20170914 snapshots Message-ID: <20170914120911.GA24772@doctor.nl2k.ab.ca> They are missing in action! -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Talk Sense to a fool and he calls you foolish - Euripides From rsalz at akamai.com Thu Sep 14 12:29:38 2017 From: rsalz at akamai.com (Salz, Rich) Date: Thu, 14 Sep 2017 12:29:38 +0000 Subject: [openssl-dev] 20170914 snapshots In-Reply-To: <20170914120911.GA24772@doctor.nl2k.ab.ca> References: <20170914120911.GA24772@doctor.nl2k.ab.ca> Message-ID: <53047E00-A952-4AAD-9F37-5976A0FFF870@akamai.com> We did some system upgrades and they were down during the update time. As I?ve said before, please wait for at least a second day before writing about the snapshots. On 9/14/17, 8:09 AM, "The Doctor" wrote: They are missing in action! From doctor at doctor.nl2k.ab.ca Sat Sep 16 12:11:18 2017 From: doctor at doctor.nl2k.ab.ca (The Doctor) Date: Sat, 16 Sep 2017 06:11:18 -0600 Subject: [openssl-dev] OPenssl 1.1.0 and FIPS Message-ID: <20170916121118.GA74055@doctor.nl2k.ab.ca> I thought it was just me. Tryong to compile Fips into OPEnssl-1.1.0 and I run into cc: warning: argument unused during compilation: '-rdynamic' [-Wunused-command-line-argument] crypto/err/err_all.c:47:69: error: invalid operands to binary expression ('void' and 'int') FIPS_set_error_callbacks(ERR_put_error, ERR_add_error_vdata)==0 || ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ ~ crypto/err/err_all.c:95:33: error: invalid operands to binary expression ('void' and 'int') ERR_load_FIPS_strings() == 0 || ~~~~~~~~~~~~~~~~~~~~~~~ ^ ~ The cc I am using is clang 4.0.0 . A similar error happens in Gcc 6 . PLease fix so that compilers cannot see this. What is happening is that the 2 statements are void but you are assigning an int value. -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Talk Sense to a fool and he calls you foolish - Euripides From rsalz at akamai.com Sat Sep 16 12:56:08 2017 From: rsalz at akamai.com (Salz, Rich) Date: Sat, 16 Sep 2017 12:56:08 +0000 Subject: [openssl-dev] OPenssl 1.1.0 and FIPS In-Reply-To: <20170916121118.GA74055@doctor.nl2k.ab.ca> References: <20170916121118.GA74055@doctor.nl2k.ab.ca> Message-ID: Tryong to compile Fips into OPEnssl-1.1.0 and I run into FIPS is not supported for 1.1.0 From doctor at doctor.nl2k.ab.ca Sat Sep 16 13:26:16 2017 From: doctor at doctor.nl2k.ab.ca (The Doctor) Date: Sat, 16 Sep 2017 07:26:16 -0600 Subject: [openssl-dev] OPenssl 1.1.0 and FIPS In-Reply-To: References: <20170916121118.GA74055@doctor.nl2k.ab.ca> Message-ID: <20170916132616.GA47252@doctor.nl2k.ab.ca> On Sat, Sep 16, 2017 at 12:56:08PM +0000, Salz, Rich via openssl-dev wrote: > > Tryong to compile Fips into OPEnssl-1.1.0 and I run into > > FIPS is not supported for 1.1.0 > jUST A SMALL FIX WILL DO. > -- > openssl-dev mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Talk Sense to a fool and he calls you foolish - Euripides From levitte at openssl.org Sat Sep 16 14:28:39 2017 From: levitte at openssl.org (Richard Levitte) Date: Sat, 16 Sep 2017 16:28:39 +0200 Subject: [openssl-dev] OPenssl 1.1.0 and FIPS In-Reply-To: <20170916132616.GA47252@doctor.nl2k.ab.ca> References: <20170916121118.GA74055@doctor.nl2k.ab.ca> <20170916132616.GA47252@doctor.nl2k.ab.ca> Message-ID: The Doctor skrev: (16 september 2017 15:26:16 CEST) >On Sat, Sep 16, 2017 at 12:56:08PM +0000, Salz, Rich via openssl-dev >wrote: >> >> Tryong to compile Fips into OPEnssl-1.1.0 and I run into >> >> FIPS is not supported for 1.1.0 >> > >jUST A SMALL FIX WILL DO. Really? Are you saying that you made a small fix and then everything worked like a charm? I'm dubious, frankly, because there are just too many changes between OpenSSL 1.0.x (that the FIPS module was made for) and 1.1.0. There should be much more breakage. Cheers Richard -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From rsalz at akamai.com Sat Sep 16 16:00:15 2017 From: rsalz at akamai.com (Salz, Rich) Date: Sat, 16 Sep 2017 16:00:15 +0000 Subject: [openssl-dev] OPenssl 1.1.0 and FIPS In-Reply-To: <20170916132616.GA47252@doctor.nl2k.ab.ca> References: <20170916121118.GA74055@doctor.nl2k.ab.ca> <20170916132616.GA47252@doctor.nl2k.ab.ca> Message-ID: <1478B5E4-CE15-41A8-8A72-B3AC3CF00FDB@akamai.com> > FIPS is not supported for 1.1.0 > > jUST A SMALL FIX WILL DO. No. All of the FIPS supporting code has been pulled out of 1.1.0 Even if you get it to compile, it will fail at link or runtime because of missing functions. From hyc at highlandsun.com Sat Sep 16 21:26:10 2017 From: hyc at highlandsun.com (Howard Chu) Date: Sat, 16 Sep 2017 22:26:10 +0100 Subject: [openssl-dev] libcrypto.pc needs to list libpthread as a dependency Message-ID: <1a218a77-d672-46c0-6232-4b1746920c96@highlandsun.com> In OpenSSL 1.1 on Linux (at least) libcrypto now has a dependency on libpthread but this is not reflected in the pkgconfig file. As a result, tools like CMake fail to detect libcrypto properly when linking against the static library. libpthread should be added to the Libs.private line of the pkgconfig file. For example: https://github.com/monero-project/monero/issues/2402#issuecomment-327514216 -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ From matt at openssl.org Sun Sep 17 07:04:10 2017 From: matt at openssl.org (Matt Caswell) Date: Sun, 17 Sep 2017 08:04:10 +0100 Subject: [openssl-dev] libcrypto.pc needs to list libpthread as a dependency In-Reply-To: <1a218a77-d672-46c0-6232-4b1746920c96@highlandsun.com> References: <1a218a77-d672-46c0-6232-4b1746920c96@highlandsun.com> Message-ID: <20170917080410.9841c397d7ff85b1b66bf125@openssl.org> On Sat, 16 Sep 2017 22:26:10 +0100 Howard Chu via openssl-dev wrote: > In OpenSSL 1.1 on Linux (at least) libcrypto now has a dependency on > libpthread but this is not reflected in the pkgconfig file. As a result, tools > like CMake fail to detect libcrypto properly when linking against the static > library. libpthread should be added to the Libs.private line of the pkgconfig Hmmm - there is no pkgconfig file at all in 1.1.0. It was removed because it was felt this was better added by OS specific packages. If you have one it probably came from the package for your specific distro not the openssl project itself. Matt > file. > > For example: > https://github.com/monero-project/monero/issues/2402#issuecomment-327514216 > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > -- > openssl-dev mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev > -- Matt Caswell From levitte at openssl.org Sun Sep 17 07:56:33 2017 From: levitte at openssl.org (Richard Levitte) Date: Sun, 17 Sep 2017 15:56:33 +0800 Subject: [openssl-dev] libcrypto.pc needs to list libpthread as a dependency In-Reply-To: <20170917080410.9841c397d7ff85b1b66bf125@openssl.org> References: <1a218a77-d672-46c0-6232-4b1746920c96@highlandsun.com> <20170917080410.9841c397d7ff85b1b66bf125@openssl.org> Message-ID: <58D5988A-F3B5-43F7-BE7D-399226F6B153@openssl.org> Matt Caswell skrev: (17 september 2017 15:04:10 GMT+08:00) >On Sat, 16 Sep 2017 22:26:10 +0100 >Howard Chu via openssl-dev wrote: > >> In OpenSSL 1.1 on Linux (at least) libcrypto now has a dependency on >> libpthread but this is not reflected in the pkgconfig file. As a >result, tools >> like CMake fail to detect libcrypto properly when linking against the >static >> library. libpthread should be added to the Libs.private line of the >pkgconfig > >Hmmm - there is no pkgconfig file at all in 1.1.0. It was removed >because it was felt this was better added by OS specific packages. If >you have one it probably came from the package for your specific distro >not the openssl project itself. You have to look in the Makefile template to find it. > >Matt > > > >> file. >> >> For example: >> >https://github.com/monero-project/monero/issues/2402#issuecomment-327514216 >> >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ >> -- >> openssl-dev mailing list >> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev >> -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From hyc at highlandsun.com Sun Sep 17 17:20:19 2017 From: hyc at highlandsun.com (Howard Chu) Date: Sun, 17 Sep 2017 18:20:19 +0100 Subject: [openssl-dev] libcrypto.pc needs to list libpthread as a dependency In-Reply-To: <59BEABE0.6000605@roumenpetrov.info> References: <1a218a77-d672-46c0-6232-4b1746920c96@highlandsun.com> <59BEABE0.6000605@roumenpetrov.info> Message-ID: Roumen Petrov wrote: > Howard Chu via openssl-dev wrote: >> In OpenSSL 1.1 on Linux (at least) libcrypto now has a dependency on >> libpthread but this is not reflected in the pkgconfig file. As a result, >> tools like CMake fail to detect libcrypto properly when linking against the >> static library. libpthread should be added to the Libs.private line of the >> pkgconfig file. >> >> For example: >> https://github.com/monero-project/monero/issues/2402#issuecomment-327514216 > > Problem is that OpenSSL does not add it directly. > Build process does not know libraries added by compiler flags like -pthread. > > Does not look like OpenSSL issue. That sounds like a lame cop-out. Currently the build process already knows that -ldl goes in ${EX_LIBS}. The Makefile is full of system-dependent settings already. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ From kurt at roeckx.be Sun Sep 17 17:24:42 2017 From: kurt at roeckx.be (Kurt Roeckx) Date: Sun, 17 Sep 2017 19:24:42 +0200 Subject: [openssl-dev] libcrypto.pc needs to list libpthread as a dependency In-Reply-To: <20170917080410.9841c397d7ff85b1b66bf125@openssl.org> References: <1a218a77-d672-46c0-6232-4b1746920c96@highlandsun.com> <20170917080410.9841c397d7ff85b1b66bf125@openssl.org> Message-ID: <20170917172442.tm5ough5cin3ihu2@roeckx.be> On Sun, Sep 17, 2017 at 08:04:10AM +0100, Matt Caswell wrote: > On Sat, 16 Sep 2017 22:26:10 +0100 > Howard Chu via openssl-dev wrote: > > > In OpenSSL 1.1 on Linux (at least) libcrypto now has a dependency on > > libpthread but this is not reflected in the pkgconfig file. As a result, tools > > like CMake fail to detect libcrypto properly when linking against the static > > library. libpthread should be added to the Libs.private line of the pkgconfig > > Hmmm - there is no pkgconfig file at all in 1.1.0. It was removed because it was felt this was better added by OS specific packages. If you have one it probably came from the package for your specific distro not the openssl project itself. The .pc files are still in master. Kurt From openssl at roumenpetrov.info Sun Sep 17 17:07:44 2017 From: openssl at roumenpetrov.info (Roumen Petrov) Date: Sun, 17 Sep 2017 20:07:44 +0300 Subject: [openssl-dev] libcrypto.pc needs to list libpthread as a dependency In-Reply-To: <1a218a77-d672-46c0-6232-4b1746920c96@highlandsun.com> References: <1a218a77-d672-46c0-6232-4b1746920c96@highlandsun.com> Message-ID: <59BEABE0.6000605@roumenpetrov.info> Howard Chu via openssl-dev wrote: > In OpenSSL 1.1 on Linux (at least) libcrypto now has a dependency on > libpthread but this is not reflected in the pkgconfig file. As a > result, tools like CMake fail to detect libcrypto properly when > linking against the static library. libpthread should be added to the > Libs.private line of the pkgconfig file. > > For example: > https://github.com/monero-project/monero/issues/2402#issuecomment-327514216 Problem is that OpenSSL does not add it directly. Build process does not know libraries added by compiler flags like -pthread. Does not look like OpenSSL issue. Regards, Roumen From openssl at roumenpetrov.info Sun Sep 17 18:29:57 2017 From: openssl at roumenpetrov.info (Roumen Petrov) Date: Sun, 17 Sep 2017 21:29:57 +0300 Subject: [openssl-dev] libcrypto.pc needs to list libpthread as a dependency In-Reply-To: References: <1a218a77-d672-46c0-6232-4b1746920c96@highlandsun.com> <59BEABE0.6000605@roumenpetrov.info> Message-ID: <59BEBF25.5040701@roumenpetrov.info> Hi Howard, Howard Chu wrote: > Roumen Petrov wrote: >> Howard Chu via openssl-dev wrote: >>> In OpenSSL 1.1 on Linux (at least) libcrypto now has a dependency on >>> libpthread but this is not reflected in the pkgconfig file. As a >>> result, tools like CMake fail to detect libcrypto properly when >>> linking against the static library. libpthread should be added to the >>> Libs.private line of the pkgconfig file. >>> >>> For example: >>> https://github.com/monero-project/monero/issues/2402#issuecomment-327514216 >>> >> >> Problem is that OpenSSL does not add it directly. >> Build process does not know libraries added by compiler flags like >> -pthread. >> >> Does not look like OpenSSL issue. > > That sounds like a lame cop-out. Currently the build process already > knows that -ldl goes in ${EX_LIBS}. The Makefile is full of > system-dependent settings already. > Case is totally different. Library dl is needed by DSO functionality. Thread library is platform dependent . For instance it is not added on android. See gcc spec file (version 4.9+?). Dunno what will be impact if openssl exports C-flag -pthread. Regards Roumen From maheshbhooth at gmail.com Mon Sep 18 06:07:11 2017 From: maheshbhooth at gmail.com (Mahesh Bhoothapuri) Date: Sun, 17 Sep 2017 23:07:11 -0700 Subject: [openssl-dev] TLS 1.3 client hello issue Message-ID: Hi, I am sending a Tls 1.3 client hello, and am seeing an issue with ossl_statem_client_write_transition in statem_clnt.c. /* * Note that immediately before/after a ClientHello we don't know what * version we are going to negotiate yet, so we don't take this branch until * later */ /* * ossl_statem_client_write_transition() works out what handshake state to * move to next when the client is writing messages to be sent to the server. */ WRITE_TRAN ossl_statem_client_write_transition(SSL *s) { if (SSL_IS_TLS13(s)) return ossl_statem_client13_write_transition(s); } And in: /* * ossl_statem_client_write_transition() works out what handshake state to * move to next when the client is writing messages to be sent to the server. */ WRITE_TRAN ossl_statem_client_write_transition(SSL *s) { /* * Note: There are no cases for TLS_ST_BEFORE because we haven't negotiated * TLSv1.3 yet at that point. They are handled by * ossl_statem_client_write_transition(). */ switch (st->hand_state) { default: /* Shouldn't happen */ return WRITE_TRAN_ERROR; } With a TLS 1.3 client hello, using tls 1.3 version, the st->hand_state is TLS_ST_BEFORE and so, the default error is returned. When I added : case TLS_ST_BEFORE: st->hand_state = TLS_ST_CW_CLNT_HELLO; return WRITE_TRAN_CONTINUE; The client hello gets sent out, but I only saw a TLS 1.2 version being sent. Is this a bug? Thanks, Mahesh -------------- next part -------------- An HTML attachment was scrubbed... URL: From bkaduk at akamai.com Mon Sep 18 12:40:15 2017 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Mon, 18 Sep 2017 07:40:15 -0500 Subject: [openssl-dev] TLS 1.3 client hello issue In-Reply-To: References: Message-ID: <34e3afb2-1bf8-1653-bded-ebdfed47fd8f@akamai.com> On 09/18/2017 01:07 AM, Mahesh Bhoothapuri wrote: > > Hi, > > I am sending a Tls 1.3 client hello, and am seeing an issue with > > ossl_statem_client_write_transition in statem_clnt.c. > > > ??? /* > ???? * Note that immediately before/after a ClientHello we don't know what > ???? * version we are going to negotiate yet, so we don't take this > branch until > ???? * later > ???? */ > > /* > ?* ossl_statem_client_write_transition() works out what handshake state to > ?* move to next when the client is writing messages to be sent to the > server. > ?*/ > WRITE_TRAN ossl_statem_client_write_transition(SSL *s) > { > > ??? if (SSL_IS_TLS13(s)) > ??????? return ossl_statem_client13_write_transition(s); > } > > And in: > > > /* > ?* ossl_statem_client_write_transition() works out what handshake state to > ?* move to next when the client is writing messages to be sent to the > server. > ?*/ > WRITE_TRAN ossl_statem_client_write_transition(SSL *s) > { > > ?? /* > ???? * Note: There are no cases for TLS_ST_BEFORE because we haven't > negotiated > ???? * TLSv1.3 yet at that point. They are handled by > ???? * ossl_statem_client_write_transition(). > ???? */ > > ??? switch (st->hand_state) { > ??? default: > ??????? /* Shouldn't happen */ > ??????? return WRITE_TRAN_ERROR; > > } > > With a TLS 1.3 client hello, using tls 1.3 version, the st->hand_state is Sorry, I just want to clarify what you are doing -- are you taking SSL_CTX_new(TLS_method()) and then calling SSL_CTX_set_min_proto_version(ctx, TLS1_3_VERSION) and SSL_CTX_set_max_proto_version(ctx, TLS1_3_VERSION)? I note that there is no version-specific TLSv1_3_method() available, and in any case, it's of questionable wisdom to attempt to force TLS 1.3 only while the specification is still in draft status -- in any case where the client and server implementations are not tightly controlled, negotiation failures seem quite likely. > TLS_ST_BEFORE and so, the default error is returned. > > When I added : > > ??? case TLS_ST_BEFORE: > ??????? st->hand_state = TLS_ST_CW_CLNT_HELLO; > ??????? return WRITE_TRAN_CONTINUE; > The reason there is not currently a case for TLS_ST_BEFORE is that whether or not we're going to be using TLS 1.3 is supposed to be determined on the server as part of version negotiation, so when we're sending a ClientHello, our version is in an indeterminate status -- the general-purpose TLS method must be used at that part of the handshake. > The client hello gets sent out, but I only saw a TLS 1.2 version being > sent. > Is this a bug? The legacy_version field in a TLS 1.3 ClientHello will be 0x0303, matching the historical value for TLS 1.2.? The actual list of versions are conveyed in a "supported_versions" extension, which is what you need to be looking at. -Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From maheshbhooth at gmail.com Mon Sep 18 14:25:18 2017 From: maheshbhooth at gmail.com (Mahesh Bhoothapuri) Date: Mon, 18 Sep 2017 07:25:18 -0700 Subject: [openssl-dev] TLS 1.3 client hello issue In-Reply-To: <34e3afb2-1bf8-1653-bded-ebdfed47fd8f@akamai.com> References: <34e3afb2-1bf8-1653-bded-ebdfed47fd8f@akamai.com> Message-ID: Thanks for responding. Yes, I have done the steps mentioned above. Here are my settings: int min_version = TLS1_3_VERSION, max_version = TLS1_3_VERSION; meth = isClient ? tlsv1_3_client_method() : tlsv1_3_server_method(); //meth = isClient ? TLS_client_method() : TLS_server_method(); /////////////////////////////////////////////////////////// // Create new SSL context using the chosen SSL_METHOD ctx = SSL_CTX_new(meth); if (ctx == NULL) { // throw error } if (SSL_CTX_set_min_proto_version(ctx, min_version) == 0) { // throw error } if (SSL_CTX_set_max_proto_version(ctx, max_version) == 0) { // throw error } // Configure SSL to use the cipher suite specified // TLS1_3_TXT_AES_128_GCM_SHA256 // ./include/openssl/tls1.h:# define TLS1_3_TXT_AES_128_GCM_SHA256 "TLS13-AES-128-GCM-SHA256" int set_cipher; if (! (set_cipher = SSL_CTX_set_cipher_list(ctx, cipherSuite.c_str())) ) { throw (InvalidTestConfiguration("OpenSslApi::OpenSslInitContext", "Failed to set ciphers")); } The set_min_proto/set_max_proto calls succeed. If I want to get the AES_128_GCM_SHA256 Cipher for TLS 1.3 to be used, are these the steps to be used? Should I instead, select also, AES128-GCM-SHA256 a TLS 1.2 cipher in the list, and set the min_proto to TLS 1.2, and max_proto to 1.3 ? I need to avoid hitting the default case below: static WRITE_TRAN ossl_statem_client13_write_transition(SSL *s) { OSSL_STATEM *st = &s->statem; /* * Note: There are no cases for TLS_ST_BEFORE because we haven't negotiated * TLSv1.3 yet at that point. They are handled by * ossl_statem_client_write_transition(). */ switch (st->hand_state) { default: " On Mon, Sep 18, 2017 at 5:40 AM, Benjamin Kaduk wrote: > On 09/18/2017 01:07 AM, Mahesh Bhoothapuri wrote: > > Hi, > > I am sending a Tls 1.3 client hello, and am seeing an issue with > > ossl_statem_client_write_transition in statem_clnt.c. > > > /* > * Note that immediately before/after a ClientHello we don't know what > * version we are going to negotiate yet, so we don't take this branch > until > * later > */ > > /* > * ossl_statem_client_write_transition() works out what handshake state to > * move to next when the client is writing messages to be sent to the > server. > */ > WRITE_TRAN ossl_statem_client_write_transition(SSL *s) > { > > if (SSL_IS_TLS13(s)) > return ossl_statem_client13_write_transition(s); > } > > And in: > > > /* > * ossl_statem_client_write_transition() works out what handshake state to > * move to next when the client is writing messages to be sent to the > server. > */ > WRITE_TRAN ossl_statem_client_write_transition(SSL *s) > { > > /* > * Note: There are no cases for TLS_ST_BEFORE because we haven't > negotiated > * TLSv1.3 yet at that point. They are handled by > * ossl_statem_client_write_transition(). > */ > > switch (st->hand_state) { > default: > /* Shouldn't happen */ > return WRITE_TRAN_ERROR; > > } > > With a TLS 1.3 client hello, using tls 1.3 version, the st->hand_state is > > > Sorry, I just want to clarify what you are doing -- are you taking > SSL_CTX_new(TLS_method()) and then calling SSL_CTX_set_min_proto_version(ctx, > TLS1_3_VERSION) and SSL_CTX_set_max_proto_version(ctx, TLS1_3_VERSION)? > > I note that there is no version-specific TLSv1_3_method() available, and > in any case, it's of questionable wisdom to attempt to force TLS 1.3 only > while the specification is still in draft status -- in any case where the > client and server implementations are not tightly controlled, negotiation > failures seem quite likely. > > TLS_ST_BEFORE and so, the default error is returned. > > When I added : > > case TLS_ST_BEFORE: > st->hand_state = TLS_ST_CW_CLNT_HELLO; > return WRITE_TRAN_CONTINUE; > > > The reason there is not currently a case for TLS_ST_BEFORE is that whether > or not we're going to be using TLS 1.3 is supposed to be determined on the > server as part of version negotiation, so when we're sending a ClientHello, > our version is in an indeterminate status -- the general-purpose TLS method > must be used at that part of the handshake. > > The client hello gets sent out, but I only saw a TLS 1.2 version being > sent. > Is this a bug? > > > The legacy_version field in a TLS 1.3 ClientHello will be 0x0303, matching > the historical value for TLS 1.2. The actual list of versions are conveyed > in a "supported_versions" extension, which is what you need to be looking > at. > > -Ben > -------------- next part -------------- An HTML attachment was scrubbed... URL: From uri at ll.mit.edu Mon Sep 18 14:32:32 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Mon, 18 Sep 2017 14:32:32 +0000 Subject: [openssl-dev] Bug: digest parameter is rejected Message-ID: RSA-OAEP supports different hash functions and MGF. SHA-1 is the default. OpenSSL implementation of OAEP wrongly refuses to set the hash algorithm, preventing one from using SHA-2 family: $ openssl version OpenSSL 1.0.2l? 25 May 2017 $ openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep $ openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep -pkeyopt digest:sha256 parameter setting error 140736155067400:error:06089094:digital envelope routines:EVP_PKEY_CTX_ctrl:invalid operation:pmeth_lib.c:376: $ ~/openssl-1.1/bin/openssl version OpenSSL 1.1.0g-dev? xx XXX xxxx $ ~/openssl-1.1/bin/openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep $ ~/openssl-1.1/bin/openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep -pkeyopt digest:sha256 pkeyutl: Can't set parameter: 140736155067328:error:06089094:digital envelope routines:EVP_PKEY_CTX_ctrl:invalid operation:crypto/evp/pmeth_lib.c:312: $ It seems that OpenSSL tries to enforce the incorrect assumption that digest/hash is only applicable to signature padding, but not to encryption padding? -- Regards, Uri Blumenthal -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5211 bytes Desc: not available URL: From bkaduk at akamai.com Mon Sep 18 14:53:59 2017 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Mon, 18 Sep 2017 09:53:59 -0500 Subject: [openssl-dev] Bug: digest parameter is rejected In-Reply-To: References: Message-ID: On 09/18/2017 09:32 AM, Blumenthal, Uri - 0553 - MITLL wrote: > > RSA-OAEP supports different hash functions and MGF. SHA-1 is the default. > > ? > > OpenSSL implementation of OAEP wrongly refuses to set the hash > algorithm, preventing one from using SHA-2 family: > > You'll probably need to pick up master and its -rsa_mgf1_md argument to pkeyutl. -Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Mon Sep 18 14:56:42 2017 From: matt at openssl.org (Matt Caswell) Date: Mon, 18 Sep 2017 15:56:42 +0100 Subject: [openssl-dev] TLS 1.3 client hello issue In-Reply-To: References: <34e3afb2-1bf8-1653-bded-ebdfed47fd8f@akamai.com> Message-ID: <20170918155642.073dc4a32f5944d4e029430c@openssl.org> Sorry for the slightly slow response on this. I am away this week travelling (https://www.openssl.org/blog/blog/2017/08/28/china/). Some comments inserted below. On Mon, 18 Sep 2017 07:25:18 -0700 Mahesh Bhoothapuri wrote: > Thanks for responding. Yes, I have done the steps mentioned above. Here > are my settings: > > int min_version = TLS1_3_VERSION, max_version = TLS1_3_VERSION; > > meth = isClient ? tlsv1_3_client_method() : tlsv1_3_server_method(); This is *not* correct. These are internal functions that are not exposed via the public header files or exported by the library. You should not be using them at all. > //meth = isClient ? TLS_client_method() : TLS_server_method(); This commented out line is corret. In a private email to me you stated that you are not using enable-tls1_3. You absolutely *must* use that to get TLSv1.3 working. > > /////////////////////////////////////////////////////////// > // Create new SSL context using the chosen SSL_METHOD > ctx = SSL_CTX_new(meth); > if (ctx == NULL) > { > // throw error > } > > if (SSL_CTX_set_min_proto_version(ctx, min_version) == 0) > { > // throw error > } > > if (SSL_CTX_set_max_proto_version(ctx, max_version) == 0) > { > // throw error > } > > // Configure SSL to use the cipher suite specified > // TLS1_3_TXT_AES_128_GCM_SHA256 > // ./include/openssl/tls1.h:# define > TLS1_3_TXT_AES_128_GCM_SHA256 "TLS13-AES-128-GCM-SHA256" > int set_cipher; > if (! (set_cipher = SSL_CTX_set_cipher_list(ctx, cipherSuite.c_str())) ) > { > throw (InvalidTestConfiguration("OpenSslApi::OpenSslInitContext", > "Failed to set ciphers")); > } > > The set_min_proto/set_max_proto calls succeed. > > If I want to get the AES_128_GCM_SHA256 Cipher for TLS 1.3 to be used, are > these the steps to be used? Well I can't see what cipherSuite.c_str() does, but it looks ok. > > Should I instead, select also, AES128-GCM-SHA256 a TLS 1.2 cipher in the > list, and set the min_proto to TLS 1.2, and max_proto to 1.3 ? It depends. If both client and server are guaranteed to support TLSv1.3 then you don't need to support TLSv1.2 ciphersuites as well and you can set min and max to 1.3. If however you want to be able to fallback to TLSv1.2 (or below) then you should set min appropriately (to something less than 1.3). > I need to > avoid hitting the default case below: You are hitting this default case because you are not doing it right. You must not call the internal functions above. Build with enable-tls1_3 and TLS_client_method() and TLS_server_method() will start trying to negotiate TLSv1.3 I believe you would benefit from reading this blog post: https://www.openssl.org/blog/blog/2017/05/04/tlsv1.3/ Matt > > > static WRITE_TRAN ossl_statem_client13_write_transition(SSL *s) > { > OSSL_STATEM *st = &s->statem; > > /* > * Note: There are no cases for TLS_ST_BEFORE because we haven't > negotiated > * TLSv1.3 yet at that point. They are handled by > * ossl_statem_client_write_transition(). > */ > switch (st->hand_state) { > default: > > > > " > > > > > On Mon, Sep 18, 2017 at 5:40 AM, Benjamin Kaduk wrote: > > > On 09/18/2017 01:07 AM, Mahesh Bhoothapuri wrote: > > > > Hi, > > > > I am sending a Tls 1.3 client hello, and am seeing an issue with > > > > ossl_statem_client_write_transition in statem_clnt.c. > > > > > > /* > > * Note that immediately before/after a ClientHello we don't know what > > * version we are going to negotiate yet, so we don't take this branch > > until > > * later > > */ > > > > /* > > * ossl_statem_client_write_transition() works out what handshake state to > > * move to next when the client is writing messages to be sent to the > > server. > > */ > > WRITE_TRAN ossl_statem_client_write_transition(SSL *s) > > { > > > > if (SSL_IS_TLS13(s)) > > return ossl_statem_client13_write_transition(s); > > } > > > > And in: > > > > > > /* > > * ossl_statem_client_write_transition() works out what handshake state to > > * move to next when the client is writing messages to be sent to the > > server. > > */ > > WRITE_TRAN ossl_statem_client_write_transition(SSL *s) > > { > > > > /* > > * Note: There are no cases for TLS_ST_BEFORE because we haven't > > negotiated > > * TLSv1.3 yet at that point. They are handled by > > * ossl_statem_client_write_transition(). > > */ > > > > switch (st->hand_state) { > > default: > > /* Shouldn't happen */ > > return WRITE_TRAN_ERROR; > > > > } > > > > With a TLS 1.3 client hello, using tls 1.3 version, the st->hand_state is > > > > > > Sorry, I just want to clarify what you are doing -- are you taking > > SSL_CTX_new(TLS_method()) and then calling SSL_CTX_set_min_proto_version(ctx, > > TLS1_3_VERSION) and SSL_CTX_set_max_proto_version(ctx, TLS1_3_VERSION)? > > > > I note that there is no version-specific TLSv1_3_method() available, and > > in any case, it's of questionable wisdom to attempt to force TLS 1.3 only > > while the specification is still in draft status -- in any case where the > > client and server implementations are not tightly controlled, negotiation > > failures seem quite likely. > > > > TLS_ST_BEFORE and so, the default error is returned. > > > > When I added : > > > > case TLS_ST_BEFORE: > > st->hand_state = TLS_ST_CW_CLNT_HELLO; > > return WRITE_TRAN_CONTINUE; > > > > > > The reason there is not currently a case for TLS_ST_BEFORE is that whether > > or not we're going to be using TLS 1.3 is supposed to be determined on the > > server as part of version negotiation, so when we're sending a ClientHello, > > our version is in an indeterminate status -- the general-purpose TLS method > > must be used at that part of the handshake. > > > > The client hello gets sent out, but I only saw a TLS 1.2 version being > > sent. > > Is this a bug? > > > > > > The legacy_version field in a TLS 1.3 ClientHello will be 0x0303, matching > > the historical value for TLS 1.2. The actual list of versions are conveyed > > in a "supported_versions" extension, which is what you need to be looking > > at. > > > > -Ben > > -- Matt Caswell From uri at ll.mit.edu Mon Sep 18 15:46:25 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Mon, 18 Sep 2017 15:46:25 +0000 Subject: [openssl-dev] Bug: digest parameter is rejected In-Reply-To: References: Message-ID: <4CE1A669-F96C-4E04-A0C7-229931606E0F@ll.mit.edu> OpenSSL implementation of OAEP wrongly refuses to set the hash algorithm, preventing one from using SHA-2 family: You'll probably need to pick up master and its -rsa_mgf1_md argument to pkeyutl. Thank you ? better with ?-pkeyopt rsa_mgf1_md:sha256?. But still broken ? as it affects only the MGF1 setting, but not the hash setting. I?d say it still needs to allow ?-pkeyutl digest:xxx? parameter. $ ~/openssl-1.1/bin/openssl version OpenSSL 1.1.1-dev? xx XXX xxxx $ ~/openssl-1.1/bin/openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_mgf1_md:sha256 $ yhsm2-tool --decrypt -m RSA-PKCS-OAEP --id 0301 -i t1264.dat.enc2.oaep -o t1264.dat.dec2 --hash-algorithm SHA256 Using slot 0 with a present token (0x0) Logging in to "YubiHSM". Please enter User PIN: Using decrypt algorithm RSA-PKCS-OAEP OAEP parameters: hashAlg=SHA256, mgf=MGF1-SHA256, source_type=0, source_ptr=0x0, source_len=0 error: PKCS11 function C_Decrypt failed: rv = CKR_FUNCTION_FAILED (0x6) Aborting. $ yhsm2-tool --decrypt -m RSA-PKCS-OAEP --id 0301 -i t1264.dat.enc2.oaep -o t1264.dat.dec2 --hash-algorithm SHA-1 --mgf MGF1-SHA256 Using slot 0 with a present token (0x0) Logging in to "YubiHSM". Please enter User PIN: Using decrypt algorithm RSA-PKCS-OAEP OAEP parameters: hashAlg=SHA-1, mgf=MGF1-SHA256, source_type=0, source_ptr=0x0, source_len=0 $ cmp t1264.dat t1264.dat.dec2 $ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5211 bytes Desc: not available URL: From deengert at gmail.com Mon Sep 18 18:50:11 2017 From: deengert at gmail.com (Douglas E Engert) Date: Mon, 18 Sep 2017 13:50:11 -0500 Subject: [openssl-dev] Bug: digest parameter is rejected In-Reply-To: <4CE1A669-F96C-4E04-A0C7-229931606E0F@ll.mit.edu> References: <4CE1A669-F96C-4E04-A0C7-229931606E0F@ll.mit.edu> Message-ID: <16499d81-d87a-1ee1-7b43-4e4585b72ddc@gmail.com> Can you also add -pkeyopt rsa_oaep_md:sah256 See crypto/rsa/rsa_pmeth.c pkey_rsa_ctrl_str for the options. There is also rsa_oaep_label On 9/18/2017 10:46 AM, Blumenthal, Uri - 0553 - MITLL wrote: > OpenSSL implementation of OAEP wrongly refuses to set the hash algorithm, preventing one from using SHA-2 family: > > > You'll probably need to pick up master and its -rsa_mgf1_md argument to pkeyutl. > > *Thank you ? better with ?**-pkeyopt rsa_mgf1_md:sha256**?. But still broken ? as it affects only the MGF1 setting, but _not_ the hash setting. I?d say it still needs to allow ?**-pkeyutl > digest:xxx**? parameter.*** > > $ ~/openssl-1.1/bin/openssl version > > OpenSSL 1.1.1-dev? xx XXX xxxx > > $ ~/openssl-1.1/bin/openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_mgf1_md:sha256 > > $ yhsm2-tool --decrypt -m RSA-PKCS-OAEP --id 0301 -i t1264.dat.enc2.oaep -o t1264.dat.dec2 --hash-algorithm SHA256 > > Using slot 0 with a present token (0x0) > > Logging in to "YubiHSM". > > Please enter User PIN: > > Using decrypt algorithm RSA-PKCS-OAEP > > *OAEP parameters: hashAlg=SHA256, mgf=MGF1-SHA256*, source_type=0, source_ptr=0x0, source_len=0 > > *error*: PKCS11 function C_Decrypt failed: rv = CKR_FUNCTION_FAILED (0x6) > > Aborting. > > $ yhsm2-tool --decrypt -m RSA-PKCS-OAEP --id 0301 -i t1264.dat.enc2.oaep -o t1264.dat.dec2 *--hash-algorithm SHA-1* --mgf MGF1-SHA256 > > Using slot 0 with a present token (0x0) > > Logging in to "YubiHSM". > > Please enter User PIN: > > Using decrypt algorithm RSA-PKCS-OAEP > > *OAEP parameters: hashAlg=SHA-1, mgf=MGF1-SHA256*, source_type=0, source_ptr=0x0, source_len=0 > > $ cmp t1264.dat t1264.dat.dec2 > > $ > > > -- Douglas E. Engert From uri at ll.mit.edu Mon Sep 18 19:00:26 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Mon, 18 Sep 2017 19:00:26 +0000 Subject: [openssl-dev] Bug: digest parameter is rejected In-Reply-To: <16499d81-d87a-1ee1-7b43-4e4585b72ddc@gmail.com> References: <4CE1A669-F96C-4E04-A0C7-229931606E0F@ll.mit.edu> <16499d81-d87a-1ee1-7b43-4e4585b72ddc@gmail.com> Message-ID: On 9/18/17, 14:50, "openssl-dev on behalf of Douglas E Engert" wrote: Can you also add -pkeyopt rsa_oaep_md:sah256 See crypto/rsa/rsa_pmeth.c pkey_rsa_ctrl_str for the options. There is also rsa_oaep_label Thank you!! That saved the day: $ ~/openssl-1.1/bin/openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_mgf1_md:sha256 $ ~/openssl-1.1/bin/openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_mgf1_md:sha256 -pkeyopt rsa_oaep_md:sha256 $ yhsm2-tool --decrypt -m RSA-PKCS-OAEP --id 0301 -i t1264.dat.enc2.oaep -o t1264.dat.dec2 --hash-algorithm SHA256 Using slot 0 with a present token (0x0) Logging in to "YubiHSM". Please enter User PIN: Using decrypt algorithm RSA-PKCS-OAEP OAEP parameters: hashAlg=SHA256, mgf=MGF1-SHA256, source_type=0, source_ptr=0x0, source_len=0 $ cmp t1264.dat t1264.dat.dec2 $ Where can I see the complete list of the options that ?-pkeyopt? supports now? -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5211 bytes Desc: not available URL: From uri at ll.mit.edu Mon Sep 18 19:11:51 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Mon, 18 Sep 2017 19:11:51 +0000 Subject: [openssl-dev] Bug: digest parameter is rejected In-Reply-To: References: <4CE1A669-F96C-4E04-A0C7-229931606E0F@ll.mit.edu> <16499d81-d87a-1ee1-7b43-4e4585b72ddc@gmail.com> Message-ID: <8788D682-969A-4B03-B13F-633C3A20DF42@ll.mit.edu> See crypto/rsa/rsa_pmeth.c pkey_rsa_ctrl_str for the options. There is also rsa_oaep_label Thank you!! That saved the day: . . . . . Where can I see the complete list of the options that ?-pkeyopt? supports now? I missed the crypto/rsa/rsa_pmeth.c pkey_rsa_ctrl_str part. ;-( Apology for not paying attention. The label is supplied as the initial hash, hex-encoded, right? Oh, it would be nice to add ?rsa_oaep_md:digest? and ?rsa_oaep_label:hexstring? to the man page. ;-) -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5211 bytes Desc: not available URL: From maheshbhooth at gmail.com Tue Sep 19 13:05:21 2017 From: maheshbhooth at gmail.com (Mahesh Bhoothapuri) Date: Tue, 19 Sep 2017 06:05:21 -0700 Subject: [openssl-dev] TLS 1.3 client hello issue In-Reply-To: <20170918155642.073dc4a32f5944d4e029430c@openssl.org> References: <34e3afb2-1bf8-1653-bded-ebdfed47fd8f@akamai.com> <20170918155642.073dc4a32f5944d4e029430c@openssl.org> Message-ID: After enabling tls1.3, and uncommenting //meth = isClient ? TLS_client_method() : TLS_server_method(); the handshake does succeed. The client version is still TLS 1.2, but it does contain the cipher suite configured. Confirmed this using the TLS 1.3 version of wireshark. The Server is responding with a Hello Retry request with TLS 1.3. The RFC says that this is caused by a mismatched configuration. The client hello contains the Key Share Extension - with group x22519. Looking into how to configure the groups so that the Server does not need to send a Hello Retry request. Thanks, Mahesh On Mon, Sep 18, 2017 at 7:56 AM, Matt Caswell wrote: > Sorry for the slightly slow response on this. I am away this week > travelling (https://www.openssl.org/blog/blog/2017/08/28/china/). > > Some comments inserted below. > > On Mon, 18 Sep 2017 07:25:18 -0700 > Mahesh Bhoothapuri wrote: > > > Thanks for responding. Yes, I have done the steps mentioned above. > Here > > are my settings: > > > > int min_version = TLS1_3_VERSION, max_version = TLS1_3_VERSION; > > > > meth = isClient ? tlsv1_3_client_method() : tlsv1_3_server_method(); > > This is *not* correct. These are internal functions that are not exposed > via the public header files or exported by the library. You should not be > using them at all. > > > //meth = isClient ? TLS_client_method() : TLS_server_method(); > > This commented out line is corret. > > In a private email to me you stated that you are not using enable-tls1_3. > You absolutely *must* use that to get TLSv1.3 working. > > > > > /////////////////////////////////////////////////////////// > > // Create new SSL context using the chosen SSL_METHOD > > ctx = SSL_CTX_new(meth); > > if (ctx == NULL) > > { > > // throw error > > } > > > > if (SSL_CTX_set_min_proto_version(ctx, min_version) == 0) > > { > > // throw error > > } > > > > if (SSL_CTX_set_max_proto_version(ctx, max_version) == 0) > > { > > // throw error > > } > > > > // Configure SSL to use the cipher suite specified > > // TLS1_3_TXT_AES_128_GCM_SHA256 > > // ./include/openssl/tls1.h:# define > > TLS1_3_TXT_AES_128_GCM_SHA256 > "TLS13-AES-128-GCM-SHA256" > > int set_cipher; > > if (! (set_cipher = SSL_CTX_set_cipher_list(ctx, > cipherSuite.c_str())) ) > > { > > throw (InvalidTestConfiguration("OpenSslApi:: > OpenSslInitContext", > > "Failed to set ciphers")); > > } > > > > The set_min_proto/set_max_proto calls succeed. > > > > If I want to get the AES_128_GCM_SHA256 Cipher for TLS 1.3 to be used, > are > > these the steps to be used? > > Well I can't see what cipherSuite.c_str() does, but it looks ok. > > > > > Should I instead, select also, AES128-GCM-SHA256 a TLS 1.2 cipher in the > > list, and set the min_proto to TLS 1.2, and max_proto to 1.3 ? > > It depends. If both client and server are guaranteed to support TLSv1.3 > then you don't need to support TLSv1.2 ciphersuites as well and you can set > min and max to 1.3. If however you want to be able to fallback to TLSv1.2 > (or below) then you should set min appropriately (to something less than > 1.3). > > > I need to > > avoid hitting the default case below: > > You are hitting this default case because you are not doing it right. You > must not call the internal functions above. Build with enable-tls1_3 and > TLS_client_method() and TLS_server_method() will start trying to negotiate > TLSv1.3 > > I believe you would benefit from reading this blog post: > > https://www.openssl.org/blog/blog/2017/05/04/tlsv1.3/ > > Matt > > > > > > > > > static WRITE_TRAN ossl_statem_client13_write_transition(SSL *s) > > { > > OSSL_STATEM *st = &s->statem; > > > > /* > > * Note: There are no cases for TLS_ST_BEFORE because we haven't > > negotiated > > * TLSv1.3 yet at that point. They are handled by > > * ossl_statem_client_write_transition(). > > */ > > switch (st->hand_state) { > > default: > > > > > > > > " > > > > > > > > > > On Mon, Sep 18, 2017 at 5:40 AM, Benjamin Kaduk > wrote: > > > > > On 09/18/2017 01:07 AM, Mahesh Bhoothapuri wrote: > > > > > > Hi, > > > > > > I am sending a Tls 1.3 client hello, and am seeing an issue with > > > > > > ossl_statem_client_write_transition in statem_clnt.c. > > > > > > > > > /* > > > * Note that immediately before/after a ClientHello we don't know > what > > > * version we are going to negotiate yet, so we don't take this > branch > > > until > > > * later > > > */ > > > > > > /* > > > * ossl_statem_client_write_transition() works out what handshake > state to > > > * move to next when the client is writing messages to be sent to the > > > server. > > > */ > > > WRITE_TRAN ossl_statem_client_write_transition(SSL *s) > > > { > > > > > > if (SSL_IS_TLS13(s)) > > > return ossl_statem_client13_write_transition(s); > > > } > > > > > > And in: > > > > > > > > > /* > > > * ossl_statem_client_write_transition() works out what handshake > state to > > > * move to next when the client is writing messages to be sent to the > > > server. > > > */ > > > WRITE_TRAN ossl_statem_client_write_transition(SSL *s) > > > { > > > > > > /* > > > * Note: There are no cases for TLS_ST_BEFORE because we haven't > > > negotiated > > > * TLSv1.3 yet at that point. They are handled by > > > * ossl_statem_client_write_transition(). > > > */ > > > > > > switch (st->hand_state) { > > > default: > > > /* Shouldn't happen */ > > > return WRITE_TRAN_ERROR; > > > > > > } > > > > > > With a TLS 1.3 client hello, using tls 1.3 version, the st->hand_state > is > > > > > > > > > Sorry, I just want to clarify what you are doing -- are you taking > > > SSL_CTX_new(TLS_method()) and then calling > SSL_CTX_set_min_proto_version(ctx, > > > TLS1_3_VERSION) and SSL_CTX_set_max_proto_version(ctx, > TLS1_3_VERSION)? > > > > > > I note that there is no version-specific TLSv1_3_method() available, > and > > > in any case, it's of questionable wisdom to attempt to force TLS 1.3 > only > > > while the specification is still in draft status -- in any case where > the > > > client and server implementations are not tightly controlled, > negotiation > > > failures seem quite likely. > > > > > > TLS_ST_BEFORE and so, the default error is returned. > > > > > > When I added : > > > > > > case TLS_ST_BEFORE: > > > st->hand_state = TLS_ST_CW_CLNT_HELLO; > > > return WRITE_TRAN_CONTINUE; > > > > > > > > > The reason there is not currently a case for TLS_ST_BEFORE is that > whether > > > or not we're going to be using TLS 1.3 is supposed to be determined on > the > > > server as part of version negotiation, so when we're sending a > ClientHello, > > > our version is in an indeterminate status -- the general-purpose TLS > method > > > must be used at that part of the handshake. > > > > > > The client hello gets sent out, but I only saw a TLS 1.2 version being > > > sent. > > > Is this a bug? > > > > > > > > > The legacy_version field in a TLS 1.3 ClientHello will be 0x0303, > matching > > > the historical value for TLS 1.2. The actual list of versions are > conveyed > > > in a "supported_versions" extension, which is what you need to be > looking > > > at. > > > > > > -Ben > > > > > > -- > Matt Caswell > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Tue Sep 19 15:21:25 2017 From: matt at openssl.org (Matt Caswell) Date: Tue, 19 Sep 2017 16:21:25 +0100 Subject: [openssl-dev] TLS 1.3 client hello issue In-Reply-To: References: <34e3afb2-1bf8-1653-bded-ebdfed47fd8f@akamai.com> <20170918155642.073dc4a32f5944d4e029430c@openssl.org> Message-ID: <20170919162125.bc3f15dd970fe322a042deca@openssl.org> On Tue, 19 Sep 2017 06:05:21 -0700 Mahesh Bhoothapuri wrote: > The client version is still TLS 1.2, but it does contain the cipher suite > configured. This is correct behaviour for a TLS1.3 ClientHello. The client_version field is considered "legacy" and is unused in TLSv1.3 version negotiation and is always set to TLSv1.2. Matt -- Matt Caswell From levitte at openssl.org Thu Sep 21 15:40:52 2017 From: levitte at openssl.org (Richard Levitte) Date: Thu, 21 Sep 2017 17:40:52 +0200 (CEST) Subject: [openssl-dev] libcrypto.pc needs to list libpthread as a dependency In-Reply-To: <59BEBF25.5040701@roumenpetrov.info> References: <59BEABE0.6000605@roumenpetrov.info> <59BEBF25.5040701@roumenpetrov.info> Message-ID: <20170921.174052.989709456698704004.levitte@openssl.org> In message <59BEBF25.5040701 at roumenpetrov.info> on Sun, 17 Sep 2017 21:29:57 +0300, Roumen Petrov said: openssl> Hi Howard, openssl> openssl> Howard Chu wrote: openssl> > Roumen Petrov wrote: openssl> >> Howard Chu via openssl-dev wrote: openssl> >>> In OpenSSL 1.1 on Linux (at least) libcrypto now has a dependency on openssl> >>> libpthread but this is not reflected in the pkgconfig file. As a openssl> >>> result, tools like CMake fail to detect libcrypto properly when openssl> >>> linking against the static library. libpthread should be added to the openssl> >>> Libs.private line of the pkgconfig file. openssl> >>> openssl> >>> For example: openssl> >>> https://github.com/monero-project/monero/issues/2402#issuecomment-327514216 openssl> >>> openssl> >> openssl> >> Problem is that OpenSSL does not add it directly. openssl> >> Build process does not know libraries added by compiler flags like openssl> >> -pthread. openssl> >> openssl> >> Does not look like OpenSSL issue. openssl> > openssl> > That sounds like a lame cop-out. Currently the build process already openssl> > knows that -ldl goes in ${EX_LIBS}. The Makefile is full of openssl> > system-dependent settings already. openssl> > openssl> openssl> Case is totally different. Library dl is needed by DSO functionality. openssl> openssl> Thread library is platform dependent . For instance it is not added on android. See gcc spec file (version 4.9+?). openssl> openssl> Dunno what will be impact if openssl exports C-flag -pthread. -pthread should be part of both compiling and linking, so Howard is perfectly correct, we're not doing this quite right. When -pthread is used, it should also be added to the libcrypto.pc's Libs.private line. I'm currently travelling, but will give this more concrete attention when I've returned, i.e. next week. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From murugesh.pitchaiah at gmail.com Fri Sep 22 06:45:02 2017 From: murugesh.pitchaiah at gmail.com (murugesh pitchaiah) Date: Fri, 22 Sep 2017 12:15:02 +0530 Subject: [openssl-dev] Openssl FIPS ecp 2.0.16: KDF test for TLS and SSH : Segmentation fault In-Reply-To: References: Message-ID: Hi All, I am working on doing the KDF testing. Done with the HMAC and SHA. But when it comes to do the same for TLS and SSH, evidenced that openssl does not support it. As suggested by old threads, added the patch from Cisco: https://github.com/openssl/openssl/pull/368/files Did build. Now i see the 'fips_kdf_ssh' and 'fips_kdf_tls' executables are available in the 'test' directory. But when i try to run the below script, I see segmentation fault: ./openssl-fips-ecp-2.0.16/util/shlib_wrap.sh ./openssl-fips-ecp-2.0.16/test/fips_kdf_ssh < "./KDF135/req/ssh.req" > "./KDF135/rsp/ssh.rsp" Same is the case with TLS. Issue seen while running the test in CentOS. Any help is appreciated. Thanks, Murugesh P. From angus at magsys.co.uk Fri Sep 22 15:09:00 2017 From: angus at magsys.co.uk (Angus Robertson - Magenta Systems Ltd) Date: Fri, 22 Sep 2017 16:09 +0100 (BST) Subject: [openssl-dev] Creating requests and certificates with Subject Alternative Names In-Reply-To: Message-ID: > I'm creating X509 certificate requests and certificates in code, > trying to add X509v3 Subject Alternative Name, with 1.1.0f. > > But if I add a list of four domains, ie: > The certificate seems to ignore some and repeat others: To answer my own question, I was using ASN1_STRING_set0 instead of ASN1_STRING_set and the original ANSI string was a temporary variable, so got lost as a new string was added since it was not copied. But there must be an easier way of adding SANs to certificates than using undocumented GENERAL_NAME APIs. Angus From hyc at highlandsun.com Fri Sep 22 15:51:55 2017 From: hyc at highlandsun.com (Howard Chu) Date: Fri, 22 Sep 2017 16:51:55 +0100 Subject: [openssl-dev] Creating requests and certificates with Subject Alternative Names In-Reply-To: References: Message-ID: <10125168-15f4-c443-2df7-1be899fb828d@highlandsun.com> Angus Robertson - Magenta Systems Ltd wrote: >> I'm creating X509 certificate requests and certificates in code, >> trying to add X509v3 Subject Alternative Name, with 1.1.0f. >> >> But if I add a list of four domains, ie: >> The certificate seems to ignore some and repeat others: > > To answer my own question, I was using ASN1_STRING_set0 instead of > ASN1_STRING_set and the original ANSI string was a temporary variable, > so got lost as a new string was added since it was not copied. > > But there must be an easier way of adding SANs to certificates than > using undocumented GENERAL_NAME APIs. Fyi, here's how we autogenerate certificates in OpenLDAP, with subjectAltNames populated. http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/slapd/overlays/autoca.c;h=5a8ec1b481376df08d4ca7d60bc8fe6d5ad56864;hb=HEAD The corresponding manpage is here http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=doc/man/man5/slapo-autoca.5;h=920c1fe189fc6767b3b8425a985488910b83fadb;hb=HEAD and our test suite script to put it thru its paces is here http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=tests/scripts/test066-autoca;h=05e221b313225f23fe9986003eebcd3ba2be5ce8;hb=HEAD -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ From chengwenping1 at jd.com Mon Sep 25 10:16:28 2017 From: chengwenping1 at jd.com (=?gb2312?B?s8zOxMa9?=) Date: Mon, 25 Sep 2017 10:16:28 +0000 Subject: [openssl-dev] how to static compile ssl engine into openssl Message-ID: <31F771DF13463A429610AEEBF6AFAE820182EBC4@mbx14.360buyAD.local> Hi all, I?m working on accelerating ssl traffic with Intel QAT card, now openssl 1.1.0f is integrated into Nginx, so I need to static compile Intel QAT engine into openssl, and I do not find some useful info about it from Internet, although openssl-1.1.0f/engines/ build.info, it is not applicable from QAT engine from https://github.com/01org/QAT_Engine. Is there a guide line for this case? There is another alternative to do it, just to alone compile openssl and nginx, but it will take effort to deploy it. Any help is appreciated. Nick Cheng -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Tue Sep 26 05:32:06 2017 From: levitte at openssl.org (Richard Levitte) Date: Tue, 26 Sep 2017 07:32:06 +0200 (CEST) Subject: [openssl-dev] how to static compile ssl engine into openssl In-Reply-To: <31F771DF13463A429610AEEBF6AFAE820182EBC4@mbx14.360buyAD.local> References: <31F771DF13463A429610AEEBF6AFAE820182EBC4@mbx14.360buyAD.local> Message-ID: <20170926.073206.1731025271528375213.levitte@openssl.org> In message <31F771DF13463A429610AEEBF6AFAE820182EBC4 at mbx14.360buyAD.local> on Mon, 25 Sep 2017 10:16:28 +0000, ??? said: chengwenping1> I?m working on accelerating ssl traffic with Intel QAT chengwenping1> card, now openssl 1.1.0f is integrated into Nginx, so I chengwenping1> need to static compile Intel QAT engine into openssl, chengwenping1> and I do not find some useful info about it from chengwenping1> Internet, although openssl-1.1.0f/engines/ build.info, chengwenping1> it is not applicable from QAT engine from chengwenping1> https://github.com/01org/QAT_Engine. Is there a guide chengwenping1> line for this case? Unforatunately, there is no such guide that I know of. I just had a look in e_qat.c, and there seems to be support for doing that there (see the sections guarded by OPENSSL_NO_DYNAMIC_ENGINES), but I can't see any way to make use of that in their configuration. If this is what you really want, I suggest you create an issue in the QAT_Engine project... but you probably need to understand that you may not get what you want, and if you do, it's probably going to be an unsupported hack. chengwenping1> There is another alternative to do it, just to alone chengwenping1> compile openssl and nginx, but it will take effort to chengwenping1> deploy it. You mean to have nginx use the shared OpenSSL libraries, which also enables dynamic engines? Yes, that's the usual way to go about these things. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From chengwenping1 at jd.com Tue Sep 26 09:42:42 2017 From: chengwenping1 at jd.com (=?gb2312?B?s8zOxMa9?=) Date: Tue, 26 Sep 2017 09:42:42 +0000 Subject: [openssl-dev] =?gb2312?b?tPC4tDogIGhvdyB0byBzdGF0aWMgY29tcGlsZSBz?= =?gb2312?b?c2wgZW5naW5lIGludG8gb3BlbnNzbA==?= In-Reply-To: <20170926.073206.1731025271528375213.levitte@openssl.org> References: <31F771DF13463A429610AEEBF6AFAE820182EBC4@mbx14.360buyAD.local> <20170926.073206.1731025271528375213.levitte@openssl.org> Message-ID: <31F771DF13463A429610AEEBF6AFAE8201830717@mbx14.360buyAD.local> Hi Richard, Thanks for your response. From your meaning, the QAT engine codes is not applicable for static compile into openssl. Yes, I should keep to run nginx using shared OpenSSL libraries with dynamic QAT engines installed, until QAT engine static compiling is support. Thank, Nick Cheng -----????----- ???: openssl-dev [mailto:openssl-dev-bounces at openssl.org] ?? Richard Levitte ????: 2017?9?26? 13:32 ???: openssl-dev at openssl.org ??: Re: [openssl-dev] how to static compile ssl engine into openssl In message <31F771DF13463A429610AEEBF6AFAE820182EBC4 at mbx14.360buyAD.local> on Mon, 25 Sep 2017 10:16:28 +0000, ??? said: chengwenping1> I?m working on accelerating ssl traffic with Intel QAT chengwenping1> card, now openssl 1.1.0f is integrated into Nginx, so I chengwenping1> need to static compile Intel QAT engine into openssl, and chengwenping1> I do not find some useful info about it from Internet, chengwenping1> although openssl-1.1.0f/engines/ build.info, it is not chengwenping1> applicable from QAT engine from chengwenping1> https://github.com/01org/QAT_Engine. Is there a guide chengwenping1> line for this case? Unforatunately, there is no such guide that I know of. I just had a look in e_qat.c, and there seems to be support for doing that there (see the sections guarded by OPENSSL_NO_DYNAMIC_ENGINES), but I can't see any way to make use of that in their configuration. If this is what you really want, I suggest you create an issue in the QAT_Engine project... but you probably need to understand that you may not get what you want, and if you do, it's probably going to be an unsupported hack. chengwenping1> There is another alternative to do it, just to alone chengwenping1> compile openssl and nginx, but it will take effort to chengwenping1> deploy it. You mean to have nginx use the shared OpenSSL libraries, which also enables dynamic engines? Yes, that's the usual way to go about these things. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev From chengwenping1 at jd.com Tue Sep 26 10:13:39 2017 From: chengwenping1 at jd.com (=?gb2312?B?s8zOxMa9?=) Date: Tue, 26 Sep 2017 10:13:39 +0000 Subject: [openssl-dev] =?gb2312?b?tPC4tDogIGhvdyB0byBzdGF0aWMgY29tcGlsZSBz?= =?gb2312?b?c2wgZW5naW5lIGludG8gb3BlbnNzbA==?= References: <31F771DF13463A429610AEEBF6AFAE820182EBC4@mbx14.360buyAD.local> <20170926.073206.1731025271528375213.levitte@openssl.org> Message-ID: <31F771DF13463A429610AEEBF6AFAE820183179B@mbx14.360buyAD.local> There is some more info. https://github.com/01org/QAT_Engine/issues/9 -----????----- ???: ??? ????: 2017?9?26? 17:43 ???: openssl-dev at openssl.org ??: ??: [openssl-dev] how to static compile ssl engine into openssl Hi Richard, Thanks for your response. From your meaning, the QAT engine codes is not applicable for static compile into openssl. Yes, I should keep to run nginx using shared OpenSSL libraries with dynamic QAT engines installed, until QAT engine static compiling is support. Thank, Nick Cheng -----????----- ???: openssl-dev [mailto:openssl-dev-bounces at openssl.org] ?? Richard Levitte ????: 2017?9?26? 13:32 ???: openssl-dev at openssl.org ??: Re: [openssl-dev] how to static compile ssl engine into openssl In message <31F771DF13463A429610AEEBF6AFAE820182EBC4 at mbx14.360buyAD.local> on Mon, 25 Sep 2017 10:16:28 +0000, ??? said: chengwenping1> I?m working on accelerating ssl traffic with Intel QAT chengwenping1> card, now openssl 1.1.0f is integrated into Nginx, so I chengwenping1> need to static compile Intel QAT engine into openssl, and chengwenping1> I do not find some useful info about it from Internet, chengwenping1> although openssl-1.1.0f/engines/ build.info, it is not chengwenping1> applicable from QAT engine from chengwenping1> https://github.com/01org/QAT_Engine. Is there a guide chengwenping1> line for this case? Unforatunately, there is no such guide that I know of. I just had a look in e_qat.c, and there seems to be support for doing that there (see the sections guarded by OPENSSL_NO_DYNAMIC_ENGINES), but I can't see any way to make use of that in their configuration. If this is what you really want, I suggest you create an issue in the QAT_Engine project... but you probably need to understand that you may not get what you want, and if you do, it's probably going to be an unsupported hack. chengwenping1> There is another alternative to do it, just to alone chengwenping1> compile openssl and nginx, but it will take effort to chengwenping1> deploy it. You mean to have nginx use the shared OpenSSL libraries, which also enables dynamic engines? Yes, that's the usual way to go about these things. Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev From stevenx.linsell at intel.com Tue Sep 26 14:00:40 2017 From: stevenx.linsell at intel.com (Linsell, StevenX) Date: Tue, 26 Sep 2017 14:00:40 +0000 Subject: [openssl-dev] how to static compile ssl engine into openssl Message-ID: On 26/09/2017, Levitte, Richard via openssl-dev wrote: > > chengwenping1> I?m working on accelerating ssl traffic with Intel QAT > chengwenping1> card, now openssl 1.1.0f is integrated into Nginx, so I > chengwenping1> need to static compile Intel QAT engine into openssl, and > chengwenping1> I do not find some useful info about it from Internet, > chengwenping1> although openssl-1.1.0f/engines/ build.info, it is not > chengwenping1> applicable from QAT engine from > chengwenping1> https://github.com/01org/QAT_Engine. Is there a guide > chengwenping1> line for this case? > > Unforatunately, there is no such guide that I know of. I just had a look in > e_qat.c, and there seems to be support for doing that there (see the > sections guarded by OPENSSL_NO_DYNAMIC_ENGINES), but I can't see any > way to make use of that in their configuration. > > If this is what you really want, I suggest you create an issue in the > QAT_Engine project... but you probably need to understand that you may > not get what you want, and if you do, it's probably going to be an > unsupported hack. I can confirm that the Intel Quickassist Technology(QAT) OpenSSL Engine does not support compiling as a static engine against OpenSSL 1.1.0f. As Richard observed there is some legacy code remaining in the engine that would allow it to work as a static engine, but if you wanted to build that way you would need to make modifications to the OpenSSL build system to compile in the engine and then some further code changes for it to use the engine. We purposely left that code in the engine from the previous OpenSSL 1.0.1 engine just in case someone needed a static build but it is untested again OpenSSL 1.1.0. There was a discussion around the feasibility of adding the QAT Engine to the OpenSSL project the other year but it is OpenSSL's direction not to accept new hardware engines into the project as the burden of needing specific hardware and expertise to maintain those engines is too great. Without the engine being part of the main OpenSSL project it is not really feasible to have a static engine as we would need to maintain some sort of OpenSSL patch to make everything work together. Steve Linsell Intel Shannon DCG/CID Software Development Team Stevenx.Linsell at intel.com -------------------------------------------------------------- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From uri at ll.mit.edu Tue Sep 26 20:29:11 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Tue, 26 Sep 2017 20:29:11 +0000 Subject: [openssl-dev] Bug in pkey_rsa_encrypt() and _decrypt() Message-ID: Working on pkcs11 engine, I discovered a bug in crypto/rsa/rsa_pmeth.c in pkey_rsa_encrypt() and pkey_rsa_decrypt(). They cause a crash when called with out==NULL. Normally it should not happen ? but when an engine is called, and it cannot process the padding ? it reverts to the original OpenSSL-provided pkey_rsa_encrypt() or pkey_rsa_decrypt() (as appropriate). OpenSSL pkeyutl makes two calls when the key is not directly available (aka not presented in a disk file), and the first call with out==NULL crashes when RSA_private_decrypt() or RSA_public_encrypt() tries to copy the result to out. The fix should be adding something like ? if (out == NULL) { ? ?????int klen = RSA_size(ctx->pkey->pkey.rsa); ? ?????*outlen = klen; ? ?????return 1; ?} right before the call to RSA_public_encrypt(). P.S. It?s more critical in pkey_rsa_encrypt(), because it?s more likely that the engine would handle the decryption operation completely by itself. -- Regards, Uri Blumenthal -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5211 bytes Desc: not available URL: From kurt at roeckx.be Tue Sep 26 20:30:53 2017 From: kurt at roeckx.be (Kurt Roeckx) Date: Tue, 26 Sep 2017 22:30:53 +0200 Subject: [openssl-dev] how to static compile ssl engine into openssl In-Reply-To: <20170926.073206.1731025271528375213.levitte@openssl.org> References: <31F771DF13463A429610AEEBF6AFAE820182EBC4@mbx14.360buyAD.local> <20170926.073206.1731025271528375213.levitte@openssl.org> Message-ID: <20170926203053.5hlfcbx273lkoeas@roeckx.be> On Tue, Sep 26, 2017 at 07:32:06AM +0200, Richard Levitte wrote: > > You mean to have nginx use the shared OpenSSL libraries, which also > enables dynamic engines? Yes, that's the usual way to go about these > things. Do we support dynamic engines with a static build? Kurt From levitte at openssl.org Tue Sep 26 22:42:58 2017 From: levitte at openssl.org (Richard Levitte) Date: Wed, 27 Sep 2017 00:42:58 +0200 (CEST) Subject: [openssl-dev] how to static compile ssl engine into openssl In-Reply-To: <20170926203053.5hlfcbx273lkoeas@roeckx.be> References: <31F771DF13463A429610AEEBF6AFAE820182EBC4@mbx14.360buyAD.local> <20170926.073206.1731025271528375213.levitte@openssl.org> <20170926203053.5hlfcbx273lkoeas@roeckx.be> Message-ID: <20170927.004258.228081169838331220.levitte@openssl.org> In message <20170926203053.5hlfcbx273lkoeas at roeckx.be> on Tue, 26 Sep 2017 22:30:53 +0200, Kurt Roeckx said: kurt> On Tue, Sep 26, 2017 at 07:32:06AM +0200, Richard Levitte wrote: kurt> > kurt> > You mean to have nginx use the shared OpenSSL libraries, which also kurt> > enables dynamic engines? Yes, that's the usual way to go about these kurt> > things. kurt> kurt> Do we support dynamic engines with a static build? No we don't. no-shared means no-dynamic-engine Cheers, Richard -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From levitte at openssl.org Wed Sep 27 05:01:06 2017 From: levitte at openssl.org (Richard Levitte) Date: Wed, 27 Sep 2017 07:01:06 +0200 (CEST) Subject: [openssl-dev] Bug in pkey_rsa_encrypt() and _decrypt() In-Reply-To: References: Message-ID: <20170927.070106.1596405290941294550.levitte@openssl.org> I think there's some confusion here... OpenSSL's pkeyutl does indeed call something with out==NULL, but it's not calling RSA_private_decrypt() or RSA_public_encrypt() directly, it's calling the EVP_PKEY functions. In *those* functions, there is a check to see if the output argument is NULL and to return the appropriate size in that case. There is no promise that the lower level functions called by the EVP_PKEY interface does the same. As a matter of fact, the manual page for RSA_private_decrypt and RSA_public_encrypt says this: ... RSA_public_encrypt() encrypts the flen bytes at from (usually a session key) using the public key rsa and stores the ciphertext in to. to must point to RSA_size(rsa) bytes of memory. ... RSA_private_decrypt() decrypts the flen bytes at from using the private key rsa and stores the plaintext in to. to must point to a memory section large enough to hold the decrypted data (which is smaller than RSA_size(rsa)). padding is the padding mode that was used to encrypt the data. ... That should make it clear that a NULL output buffer isn't acceptable at that level. Cheers, Richard In message on Tue, 26 Sep 2017 20:29:11 +0000, "Blumenthal, Uri - 0553 - MITLL" said: uri> Working on pkcs11 engine, I discovered a bug in crypto/rsa/rsa_pmeth.c in pkey_rsa_encrypt() and uri> pkey_rsa_decrypt(). uri> uri> They cause a crash when called with out==NULL. Normally it should not happen ? but when an uri> engine is called, and it cannot process the padding ? it reverts to the original OpenSSL-provided uri> pkey_rsa_encrypt() or pkey_rsa_decrypt() (as appropriate). OpenSSL pkeyutl makes two calls when uri> the key is not directly available (aka not presented in a disk file), and the first call with out==NULL uri> crashes when RSA_private_decrypt() or RSA_public_encrypt() tries to copy the result to out. uri> uri> The fix should be adding something like uri> uri> if (out == NULL) { uri> uri> int klen = RSA_size(ctx->pkey->pkey.rsa); uri> uri> *outlen = klen; uri> uri> return 1; uri> uri> } uri> uri> right before the call to RSA_public_encrypt(). uri> uri> P.S. It?s more critical in pkey_rsa_encrypt(), because it?s more likely that the engine would handle uri> the decryption operation completely by itself. uri> uri> -- uri> uri> Regards, uri> uri> Uri Blumenthal uri> From mch.thu at gmail.com Wed Sep 27 14:44:30 2017 From: mch.thu at gmail.com (Ma chunhui) Date: Wed, 27 Sep 2017 22:44:30 +0800 Subject: [openssl-dev] why TLSv1 need two tls1_enc to get decrypted data while TLSv1.1/TLSv1.2 need one in OpenSSL1.1.0f? Message-ID: Hi, I met one problem when using OpenSSL1.1.0f with protocol TLSv1. In brief, when using TLSv1, after server side received encrypted data, and after function tls1_enc finished, the decrypted data is not put in result buffer, after another tls1_enc, the decrypted data is put in result buffer. While TLSv1.1/TLSv1.2 needs only one tls1_enc. The way to reproduce it is quite simple: 1.some preparation: openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes 2.start server: openssl s_server -key key.pem -cert cert.pem -accept 44330 -www it's better to start server with gdb, and set breakpoints at tls1_enc, then continue to run. 3.openssl s_client -connect localhost:44330 -tls1 -debug After the client is started, the server side will stop at breakpoint, do several "c" to make it continue to run to wait for client's messages Then at client side, type a simple "hello" message and press Enter. Then server side will stop at tls1_enc, the input data is same as encrypted data from client side, but after Evp_Cipher and some pad removing, the decrypted data length is 0. After another tls1_enc, the decrypted data "hello" is put in the result buffer. But if client use -tls11 or -tls12, the decrypted "hello" is put in the result buffer after the first tls1_enc. Could anyone explains why the behavior of decryption is different between TLSv1 and TLSv1.1/TLSv1.2? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Andrew.Byrne at dell.com Wed Sep 27 14:32:27 2017 From: Andrew.Byrne at dell.com (Byrne, Andrew) Date: Wed, 27 Sep 2017 14:32:27 +0000 Subject: [openssl-dev] New crypto algorithms in openSSL engine Message-ID: <5B112BCB3892C248BAA2A6A6B9BCED001ED90C89@MX202CL03.corp.emc.com> Hi all, I'm working on testing some lattice based algorithms in openSSL for the establishment of a TLS channel. I've investigated the potential for developing an engine to support this as it would mean I don't need to touch the core openSSL code. However, I've two blind spots which I can't find any answers to... 1. How can I create a new OID/NID for the new algorithms I propose to include in the engine? 2. Is it possible to create a ciphersuite within an engine that could be used to create a TLS connection? Thanks, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From tmraz at redhat.com Wed Sep 27 15:02:06 2017 From: tmraz at redhat.com (Tomas Mraz) Date: Wed, 27 Sep 2017 17:02:06 +0200 Subject: [openssl-dev] Systemwide configurability of OpenSSL Message-ID: <1506524526.13888.1.camel@redhat.com> I would like to restart the discussion about possibilities of system- wide configurability of OpenSSL and particularly libssl. Historically OpenSSL allowed only for configuration of the enabled ciphersuites list if application called appropriate API call. This is now enhanced with the SSL_CONF API and the applications can set thing such as allowed signature algorithms or protocol versions via this API. However libssl currently does not have a way to apply some policy such as using just protocol TLS1.2 or better system-wide with a possibility for sysadmin to configure this via some configuration file. Of course it would still be up to individual application configurations whether they override such policy or not, but it would be useful for sysadmin to be able to set such policy and depend on that setting if he does not modify the settings in individual application configurations. How would openssl maintainers regard a patch that would add loading of a system-wide SSL configuration file on startup and application of it on SSL_CTX initialization (or some other appropriate place)? Is this approach the way to go forward or do you have some better way on mind? Such an effort was initially attempted at: https://github.com/openssl/openssl/pull/192 and https://github.com/openssl/openssl/pull/193 pull requests but given the comments, we are exploring other options to achieve that goal. What do you think could be a better way? Thanks for your comments, -- Tom?? Mr?z Red Hat No matter how far down the wrong road you've gone, turn back. ??????????????????????????????????????????????Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.] * Google and NSA associates, this message is none of your business. * Please leave it alone, and consider whether your actions are * authorized by the contract with Red Hat, or by the US constitution. * If you feel you're being encouraged to disregard the limits built * into them, remember Edward Snowden and Wikileaks. From matt at openssl.org Wed Sep 27 15:12:26 2017 From: matt at openssl.org (Matt Caswell) Date: Wed, 27 Sep 2017 16:12:26 +0100 Subject: [openssl-dev] why TLSv1 need two tls1_enc to get decrypted data while TLSv1.1/TLSv1.2 need one in OpenSSL1.1.0f? In-Reply-To: References: Message-ID: <0890716d-2a3c-a659-f74e-7f2a5a89eca6@openssl.org> On 27/09/17 15:44, Ma chunhui wrote: > Hi,? > > I met one problem when using OpenSSL1.1.0f with protocol TLSv1. > In brief, when using TLSv1, ?after server side received encrypted data, > and after function tls1_enc finished, the decrypted data is not put in > result buffer, after another tls1_enc, the decrypted data is put in > result buffer. While TLSv1.1/TLSv1.2 needs only one tls1_enc. > > > The way to reproduce it is quite simple: > > 1.some preparation: openssl req -x509 -newkey rsa:2048 -keyout key.pem > -out cert.pem -days 365 -nodes > 2.start server: openssl s_server -key key.pem -cert cert.pem -accept > 44330 -www > ? ? it's better to start server with gdb, and set breakpoints at > tls1_enc, then continue to run.? > 3.openssl s_client -connect localhost:44330 -tls1 -debug > > After the client is started, ?the server side will stop at breakpoint, > do several "c" to make it continue to run to wait for client's messages > Then at client side, type a simple "hello" message and press Enter. Then > server side will stop at tls1_enc, the input data is same as encrypted > data from client side, but after Evp_Cipher and some pad removing, the > decrypted data length is 0. After another tls1_enc, the decrypted data > "hello" is put in the result buffer. > > But if client use -tls11 or -tls12, the decrypted "hello" is put in the > result buffer after the first tls1_enc. > > Could anyone explains why the behavior of decryption is different > between TLSv1 and TLSv1.1/TLSv1.2? In TLSv1 and below the CBC IV is the previous record's last ciphertext block. This can enable certain types of attack where an attacker knows the IV that will be used for a record in advance. The problem was fixed in the specification of TLSv1.1 and above where a new IV is used for each record. As a counter measure to this issue OpenSSL (in TLSv1) sends an empty record before each "real" application data record to effectively randomise the IV and make it unpredictable so that an attacker cannot know it in advance. Therefore a TLSv1 OpenSSL client will send two records of application data where a TLSv1.1 or above OpenSSL client will just send one. This results in tls1_enc being called twice on the server side. This behaviour can be switched off by using the option SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS - but since this is considered insecure that would probably be unwise. Matt From steve at openssl.org Wed Sep 27 15:43:35 2017 From: steve at openssl.org (Dr. Stephen Henson) Date: Wed, 27 Sep 2017 15:43:35 +0000 Subject: [openssl-dev] New crypto algorithms in openSSL engine In-Reply-To: <5B112BCB3892C248BAA2A6A6B9BCED001ED90C89@MX202CL03.corp.emc.com> References: <5B112BCB3892C248BAA2A6A6B9BCED001ED90C89@MX202CL03.corp.emc.com> Message-ID: <20170927154335.GA31318@openssl.org> On Wed, Sep 27, 2017, Byrne, Andrew wrote: > Hi all, > > I'm working on testing some lattice based algorithms in openSSL for the establishment of a TLS channel. I've investigated the potential for developing an engine to support this as it would mean I don't need to touch the core openSSL code. However, I've two blind spots which I can't find any answers to... > > 1. How can I create a new OID/NID for the new algorithms I propose to include in the engine? > OBJ_create() is the easiest way. You pass it the oid, short name and long name and get back a NID. > 2. Is it possible to create a ciphersuite within an engine that could be used to create a TLS connection? > No. There is no mechanism to dynamically add ciphersuites to OpenSSL at present. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org From matt at openssl.org Wed Sep 27 15:50:18 2017 From: matt at openssl.org (Matt Caswell) Date: Wed, 27 Sep 2017 16:50:18 +0100 Subject: [openssl-dev] New crypto algorithms in openSSL engine In-Reply-To: <5B112BCB3892C248BAA2A6A6B9BCED001ED90C89@MX202CL03.corp.emc.com> References: <5B112BCB3892C248BAA2A6A6B9BCED001ED90C89@MX202CL03.corp.emc.com> Message-ID: <15184578-9e9e-90e1-e51d-b25d9e8a2d6c@openssl.org> On 27/09/17 15:32, Byrne, Andrew wrote: > I?m working on testing some lattice based algorithms in openSSL for the > establishment of a TLS channel. I?ve investigated the potential for > developing an engine to support this as it would mean I don?t need to > touch the core openSSL code. However, I?ve two blind spots which I can?t > find any answers to? > > 1.?????? How can I create a new OID/NID for the new algorithms I propose > to include in the engine? There are two options: 1) Create a PR for mainline OpenSSL to add a new OID. As its just an OID this is likely to be accepted. This has the advantage that you get a built-in NID, but will obviously only work for versions of OpenSSL after the OID was added. 2) Dynamically create the OID/NID by calling OBJ_create(). This will allocate a new NID at runtime: https://www.openssl.org/docs/man1.1.0/crypto/OBJ_nid2obj.html > > 2.?????? Is it possible to create a ciphersuite within an engine that > could be used to create a TLS connection? No. It is not possible to dynamically add new ciphersuites at runtime. All ciphersuites are built-in. Matt From steve at openssl.org Wed Sep 27 15:54:36 2017 From: steve at openssl.org (Dr. Stephen Henson) Date: Wed, 27 Sep 2017 15:54:36 +0000 Subject: [openssl-dev] Bug in pkey_rsa_encrypt() and _decrypt() In-Reply-To: References: Message-ID: <20170927155436.GB31318@openssl.org> On Tue, Sep 26, 2017, Blumenthal, Uri - 0553 - MITLL wrote: > Working on pkcs11 engine, I discovered a bug in crypto/rsa/rsa_pmeth.c in pkey_rsa_encrypt() and pkey_rsa_decrypt(). > > They cause a crash when called with out==NULL. Normally it should not happen ??? but when an engine is called, and it cannot process the padding ??? it reverts to the original OpenSSL-provided pkey_rsa_encrypt() or pkey_rsa_decrypt() (as appropriate). OpenSSL pkeyutl makes two calls when the key is not directly available (aka not presented in a disk file), and the first call with out==NULL crashes when RSA_private_decrypt() or RSA_public_encrypt() tries to copy the result to out. > The original RSA pkey method has the flag EVP_PKEY_FLAG_AUTOARGLEN set which handles the NULL output automatically so it is not handled in pkey_rsa_*(). The ENGINE should either set this flag itself too or deal with NULL arguments manually if that is not appropriate. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org From uri at ll.mit.edu Wed Sep 27 16:02:10 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Wed, 27 Sep 2017 16:02:10 +0000 Subject: [openssl-dev] Bug in pkey_rsa_encrypt() and _decrypt() In-Reply-To: <20170927155436.GB31318@openssl.org> References: <20170927155436.GB31318@openssl.org> Message-ID: <16C461D5-39E8-4FCF-B941-F98854D799E7@ll.mit.edu> > Working on pkcs11 engine, I discovered a bug in crypto/rsa/rsa_pmeth.c in pkey_rsa_encrypt() and pkey_rsa_decrypt(). > > They cause a crash when called with out==NULL. Normally it should not happen > but when an engine is called, and it cannot process the padding it reverts to the > original OpenSSL-provided pkey_rsa_encrypt() or pkey_rsa_decrypt() (as appropriate). The original RSA pkey method has the flag EVP_PKEY_FLAG_AUTOARGLEN set which handles the NULL output automatically so it is not handled in pkey_rsa_*(). The ENGINE should either set this flag itself too or deal with NULL arguments manually if that is not appropriate. Since hardware tokens I?m dealing with do not perform any public key operations (the engine in this case is used to merely pull and provide the public key to the requestor) I?m somewhat ambivalent about writing engine Encrypt function specifically for handling the NULL argument case. On the one hand, it?s the simplest solution, and it avoids going through OpenSSL modification process.;) On the other hand, it?s not as clean as I?d like. Where would I set this flag ? And would it work when the public key is on the token, and needs to be retrieved via engine? -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5211 bytes Desc: not available URL: From steve at openssl.org Wed Sep 27 16:29:07 2017 From: steve at openssl.org (Dr. Stephen Henson) Date: Wed, 27 Sep 2017 16:29:07 +0000 Subject: [openssl-dev] Bug in pkey_rsa_encrypt() and _decrypt() In-Reply-To: <16C461D5-39E8-4FCF-B941-F98854D799E7@ll.mit.edu> References: <20170927155436.GB31318@openssl.org> <16C461D5-39E8-4FCF-B941-F98854D799E7@ll.mit.edu> Message-ID: <20170927162907.GA32707@openssl.org> On Wed, Sep 27, 2017, Blumenthal, Uri - 0553 - MITLL wrote: > > Working on pkcs11 engine, I discovered a bug in crypto/rsa/rsa_pmeth.c in pkey_rsa_encrypt() and pkey_rsa_decrypt(). > > > > They cause a crash when called with out==NULL. Normally it should not happen > > but when an engine is called, and it cannot process the padding it reverts to the > > original OpenSSL-provided pkey_rsa_encrypt() or pkey_rsa_decrypt() (as appropriate). > > The original RSA pkey method has the flag EVP_PKEY_FLAG_AUTOARGLEN set which > handles the NULL output automatically so it is not handled in pkey_rsa_*(). > > The ENGINE should either set this flag itself too or deal with NULL arguments > manually if that is not appropriate. > > Since hardware tokens I???m dealing with do not perform any public key operations (the engine in this case is used to merely pull and provide the public key to the requestor) I???m somewhat ambivalent about writing engine Encrypt function specifically for handling the NULL argument case. On the one hand, it???s the simplest solution, and it avoids going through OpenSSL modification process.;) On the other hand, it???s not as clean as I???d like. > > Where would I set this flag ? And would it work when the public key is on the token, and needs to be retrieved via engine? It's set when the method is created via EVP_PKEY_meth_new(). If you set it it assumes the public key components are set in the EVP_PKEY and calls EVP_PKEY_size() appropriately to handle the NULL argument and if the supplied buffer is too small. Checkout the M_check_autoarg macro in crypto/evp/pmeth_fn.c to see exactly what it does. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org From steffen at sdaoden.eu Wed Sep 27 22:21:02 2017 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Thu, 28 Sep 2017 00:21:02 +0200 Subject: [openssl-dev] Systemwide configurability of OpenSSL In-Reply-To: <1506524526.13888.1.camel@redhat.com> References: <1506524526.13888.1.camel@redhat.com> Message-ID: <20170927222102.O5IFn%steffen@sdaoden.eu> Hello. Tomas Mraz wrote: |I would like to restart the discussion about possibilities of system- |wide configurability of OpenSSL and particularly libssl. | |Historically OpenSSL allowed only for configuration of the enabled |ciphersuites list if application called appropriate API call. This is |now enhanced with the SSL_CONF API and the applications can set thing |such as allowed signature algorithms or protocol versions via this API. Now is the time to thank the OpenSSL for this improvement which will change the world mid- or long term: thank you! In my opinion it is a tremendous improvement that it is now possible to configure OpenSSL via a central (or not) configuration file if applications adhere to the rules! Oh what a brilliant brain must have hatched this idea. (Just kidding.) The MUA i maintain for example supports this since v14.9.4 released nine days ago, documented as ssl-config-module-USER at HOST, ssl-config-module-HOST, ssl-config-module [Option] If file based application-specific configuration via ssl-config-file is available, announced as ?+ctx-config? by ssl-features, indicating availability of SSL_CTX_config(3), then, it becomes possible to use a central SSL/TLS configuration file for all programs, including steffensmua, e.g.: # Register a configuration section for steffensmua steffensmua = mailx_master # The top configuration section creates a relation # in between dynamic SSL configuration and an actual # program specific configuration section [mailx_master] ssl_conf = mailx_ssl_config # Well that actual program specific configuration section # now can map individual ssl-config-module names to sections, # e.g., ssl-config-module=account_xy [mailx_ssl_config] account_xy = mailx_account_xy account_yz = mailx_account_yz [mailx_account_xy] MinProtocol = TLSv1.2 Curves=P-521 [mailx_account_yz] CipherString = TLSv1.2:!aNULL:!eNULL: MinProtocol = TLSv1.1 Options = Bugs So obviously it took me a while to figure out how this works, the documentation clearly has been written in a hurry by someone who entirely penetrates OpenSSL intellectually, to say the least. Nothing for the highly sensitive ones. My implementation is super simple, then: if((cp = xok_vlook(ssl_config_module, urlp, OXM_ALL)) != NULL){ # ifdef HAVE_XSSL_CTX_CONFIG if(!(a_xssl_state & a_XSSL_S_CONF_LOAD)){ n_err(_("*ssl-config-module*: no *ssl-config-file* " "loaded: %s\n"), n_shexp_quote_cp(cp, FAL0)); goto jleave; }else if(!SSL_CTX_config(ctxp, cp)){ ssl_gen_err(_("*ssl-config-module*: load error for %s, " "section [%s]"), n_uagent, n_shexp_quote_cp(cp, FAL0)); goto jleave; } # else n_err(_("*ssl-config-module*: set but not supported: %s\n"), n_shexp_quote_cp(cp, FAL0)); goto jleave; # endif } |However libssl currently does not have a way to apply some policy such |as using just protocol TLS1.2 or better system-wide with a possibility |for sysadmin to configure this via some configuration file. Of course |it would still be up to individual application configurations whether |they override such policy or not, but it would be useful for sysadmin |to be able to set such policy and depend on that setting if he does not |modify the settings in individual application configurations. | |How would openssl maintainers regard a patch that would add loading of |a system-wide SSL configuration file on startup and application of it Having a global one and especially giving administrators the possibility to provide an outer cramp that cannot be loosened any further, though further restricted, would indeed be good. And that being applied automatically just when SSL library is initialized, without an explicit application-side CONF_modules_load_file(). If i recall correctly that was the original suggestion. And is it actually possible to have a generic "super-section" that is applied even if an application specific one has been chosen? And unfortunately it is not possible to say MinProtocol=Latest, like this users have to be aware, even if they are not. With MinProtocol=Latest they would only have to face this jungle of non-understanding (be honest: Google/DuckDuckGo plus copy-and-paste, isn't it) if something really fails. |on SSL_CTX initialization (or some other appropriate place)? Is this |approach the way to go forward or do you have some better way on mind? | |Such an effort was initially attempted at: |https://github.com/openssl/openssl/pull/192 and |https://github.com/openssl/openssl/pull/193 pull requests but given the |comments, we are exploring other options to achieve that goal. What do |you think could be a better way? | |Thanks for your comments, Always a pleasure. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From murugesh.pitchaiah at gmail.com Thu Sep 28 11:06:33 2017 From: murugesh.pitchaiah at gmail.com (murugesh pitchaiah) Date: Thu, 28 Sep 2017 16:36:33 +0530 Subject: [openssl-dev] FIPS 186-4 Message-ID: Hi All, Any pointers of FIPS 186-4 compliant source code patch for openssl FIPS ? Any help appreciated. Thanks, Murugesh P. From mch.thu at gmail.com Thu Sep 28 13:38:40 2017 From: mch.thu at gmail.com (Ma chunhui) Date: Thu, 28 Sep 2017 21:38:40 +0800 Subject: [openssl-dev] why TLSv1 need two tls1_enc to get decrypted data while TLSv1.1/TLSv1.2 need one in OpenSSL1.1.0f? In-Reply-To: References: Message-ID: Hi, Matt Thanks for your quickly response. And yes, with this option SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS in client side the result can be get from one decryption. But the problem is, sometimes we can't control client's behavior. Maybe the client is openssl s_client, or maybe it's a python script or some other client, and the option is not that safe. Another interesting thing is, if server is using OpenSSL 1.0.2 or1.0.1, the result can be get from just one decryption(one tls1_enc method) with protocol TLSv1, and the client didn't add that option either(In fact, I'm using client OpenSSL1.1.0f). So it seems the process is changed in some version of OpenSSL1.1.0. Could you please explain a bit more on why openSSL 1.1.0f made this change? (I mean, the SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS option is added since 0.9.6d but openSSL1.0.1 and 1.0.2 can get result in one tls1_enc, while OpenSSL1.1.0f needs two tls1_enc) The reason why I'm focus on this is because: I'm using JNI call OpenSSL, my usage is like this(Just like tomcat native ): first, use BIO_write to write data to openssl, and then use SSL_read to read 0 length to trigger decryption, and then use SSL_pending to check how much data there is. If the data length is not put in result buffer in one decryption. then SSL_pending will get a 0 length result. and the whole process needs to be changed ,which is not we want. Thanks. >From: Matt Caswell >To: openssl-dev at openssl.org >Subject: Re: [openssl-dev] why TLSv1 need two tls1_enc to get > decrypted data while TLSv1.1/TLSv1.2 need one in OpenSSL1.1.0f? >Message-ID: <0890716d-2a3c-a659-f74e-7f2a5a89eca6 at openssl.org> >Content-Type: text/plain; charset=utf-8 > > > >On 27/09/17 15:44, Ma chunhui wrote: >> Hi,? >> >> I met one problem when using OpenSSL1.1.0f with protocol TLSv1. >> In brief, when using TLSv1, ?after server side received encrypted data, >> and after function tls1_enc finished, the decrypted data is not put in >> result buffer, after another tls1_enc, the decrypted data is put in >> result buffer. While TLSv1.1/TLSv1.2 needs only one tls1_enc. >> >> >> The way to reproduce it is quite simple: >> >> 1.some preparation: openssl req -x509 -newkey rsa:2048 -keyout key.pem >> -out cert.pem -days 365 -nodes >> 2.start server: openssl s_server -key key.pem -cert cert.pem -accept >> 44330 -www >> ? ? it's better to start server with gdb, and set breakpoints at >> tls1_enc, then continue to run.? >> 3.openssl s_client -connect localhost:44330 -tls1 -debug >> >> After the client is started, ?the server side will stop at breakpoint, >> do several "c" to make it continue to run to wait for client's messages >> Then at client side, type a simple "hello" message and press Enter. Then >> server side will stop at tls1_enc, the input data is same as encrypted >> data from client side, but after Evp_Cipher and some pad removing, the >> decrypted data length is 0. After another tls1_enc, the decrypted data >> "hello" is put in the result buffer. >> >> But if client use -tls11 or -tls12, the decrypted "hello" is put in the >> result buffer after the first tls1_enc. >> >> Could anyone explains why the behavior of decryption is different >> between TLSv1 and TLSv1.1/TLSv1.2? > >In TLSv1 and below the CBC IV is the previous record's last ciphertext >block. This can enable certain types of attack where an attacker knows >the IV that will be used for a record in advance. The problem was fixed >in the specification of TLSv1.1 and above where a new IV is used for >each record. As a counter measure to this issue OpenSSL (in TLSv1) sends >an empty record before each "real" application data record to >effectively randomise the IV and make it unpredictable so that an >attacker cannot know it in advance. > >Therefore a TLSv1 OpenSSL client will send two records of application >data where a TLSv1.1 or above OpenSSL client will just send one. This >results in tls1_enc being called twice on the server side. > >This behaviour can be switched off by using the option >SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS - but since this is considered >insecure that would probably be unwise. > >Matt On Wed, Sep 27, 2017 at 10:44 PM, Ma chunhui wrote: > Hi, > > I met one problem when using OpenSSL1.1.0f with protocol TLSv1. > In brief, when using TLSv1, after server side received encrypted data, > and after function tls1_enc finished, the decrypted data is not put in > result buffer, after another tls1_enc, the decrypted data is put in result > buffer. While TLSv1.1/TLSv1.2 needs only one tls1_enc. > > > The way to reproduce it is quite simple: > > 1.some preparation: openssl req -x509 -newkey rsa:2048 -keyout key.pem > -out cert.pem -days 365 -nodes > 2.start server: openssl s_server -key key.pem -cert cert.pem -accept 44330 > -www > it's better to start server with gdb, and set breakpoints at tls1_enc, > then continue to run. > 3.openssl s_client -connect localhost:44330 -tls1 -debug > > After the client is started, the server side will stop at breakpoint, do > several "c" to make it continue to run to wait for client's messages > Then at client side, type a simple "hello" message and press Enter. Then > server side will stop at tls1_enc, the input data is same as encrypted data > from client side, but after Evp_Cipher and some pad removing, the decrypted > data length is 0. After another tls1_enc, the decrypted data "hello" is put > in the result buffer. > > But if client use -tls11 or -tls12, the decrypted "hello" is put in the > result buffer after the first tls1_enc. > > Could anyone explains why the behavior of decryption is different between > TLSv1 and TLSv1.1/TLSv1.2? > > Thanks. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Thu Sep 28 15:19:08 2017 From: matt at openssl.org (Matt Caswell) Date: Thu, 28 Sep 2017 16:19:08 +0100 Subject: [openssl-dev] why TLSv1 need two tls1_enc to get decrypted data while TLSv1.1/TLSv1.2 need one in OpenSSL1.1.0f? In-Reply-To: References: Message-ID: On 28/09/17 14:38, Ma chunhui wrote: > Hi, Matt > > Thanks for your quickly response.? > > And yes, with this option?SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS in client > side ?the result can be get from one decryption. But the problem is, > sometimes we can't control client's behavior.? Maybe the client is > openssl s_client, or maybe it's a python script or some other client, > and the option is not that safe. > > Another interesting thing is, ?if server is using OpenSSL 1.0.2 or1.0.1, > the result can be get from just one decryption(one tls1_enc method) with > protocol TLSv1, and the client didn't add that option either(In fact, > I'm using client OpenSSL1.1.0f). So it seems the process is changed in > some version of OpenSSL1.1.0. ? > Could you please explain a bit more on why openSSL 1.1.0f made this > change? (I mean, the?SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS option is added > since 0.9.6d but openSSL1.0.1 and 1.0.2 can get result in one tls1_enc, > while OpenSSL1.1.0f needs two tls1_enc) Well I can't replicate that result. If using an OpenSSL 1.0.2 server I still see two calls tls1_enc (with both a 1.1.0 client and a 1.0.2 client). Note - that doesn't necessarily translate to two SSL_read() calls (see below). > The reason why I'm focus on this is because: ?I'm using JNI call > OpenSSL, my usage is like this(Just like tomcat native ): ?first, use > BIO_write to write data to openssl, and then use SSL_read to read 0 > length to trigger decryption, and then use SSL_pending to check how much > data there is. If the data length is not put in result buffer in one > decryption. then SSL_pending will get a 0 length result. and the whole > process needs to be changed ,which is not we want. If OpenSSL reads an empty record then it will immediately try to read the next record if one is available without returning control back to the calling application. Therefore a single SSL_read() call can result in multiple tls1_enc calls. However this is highly dependent on timing. If the empty record and the following non-empty record arrive at the destination slightly separated by time then when OpenSSL reads the first empty record it will attempt to read the next record. This will fail because it has not arrived yet and control will return to the calling application. So sometimes you will have to call SSL_read() twice and sometimes you will have to call it once. This is possibly a reason why you see different behaviour between 1.0.2 and 1.1.0, i.e. because this is very timing sensitive. Basically what you are doing is wrong. You cannot rely on the fact that calling SSL_read() will definitely result in readable data being decrypted. It might do - it might not. Another scenario where this could occur is if a record arrives that is split across multiple TCP packets. You call SSL_read() when the first TCP packet arrives - but because a full record isn't there yet you get no readable application data back. Yet another scenario is if the client attempts a renegotiation: network packets arrive but when decrypted they don't actually contain any application data - just handshake data. All of this is very reliant on timing, how the client behaves (which you cannot control) and how the network behaves. If this was working for you before then it sounds like you've been lucky so far. Matt > > Thanks. > > >>From: Matt Caswell > >>To:?openssl-dev at openssl.org >>Subject: Re: [openssl-dev] why TLSv1 need two tls1_enc to get >> ? ? ? ?decrypted data while TLSv1.1/TLSv1.2 need one in OpenSSL1.1.0f? >>Message-ID: <0890716d-2a3c-a659-f74e-7f2a5a89eca6 at openssl.org > > >>Content-Type: text/plain; charset=utf-8 >> >> >> >>On 27/09/17 15:44, Ma chunhui wrote: >>> Hi,? >>> >>> I met one problem when using OpenSSL1.1.0f with protocol TLSv1. >>> In brief, when using TLSv1, ?after server side received encrypted data, >>> and after function tls1_enc finished, the decrypted data is not put in >>> result buffer, after another tls1_enc, the decrypted data is put in >>> result buffer. While TLSv1.1/TLSv1.2 needs only one tls1_enc. >>> >>> >>> The way to reproduce it is quite simple: >>> >>> 1.some preparation: openssl req -x509 -newkey rsa:2048 -keyout key.pem >>> -out cert.pem -days 365 -nodes >>> 2.start server: openssl s_server -key key.pem -cert cert.pem -accept >>> 44330 -www >>> ? ? it's better to start server with gdb, and set breakpoints at >>> tls1_enc, then continue to run.? >>> 3.openssl s_client -connect localhost:44330 -tls1 -debug >>> >>> After the client is started, ?the server side will stop at breakpoint, >>> do several "c" to make it continue to run to wait for client's messages >>> Then at client side, type a simple "hello" message and press Enter. Then >>> server side will stop at tls1_enc, the input data is same as encrypted >>> data from client side, but after Evp_Cipher and some pad removing, the >>> decrypted data length is 0. After another tls1_enc, the decrypted data >>> "hello" is put in the result buffer. >>> >>> But if client use -tls11 or -tls12, the decrypted "hello" is put in the >>> result buffer after the first tls1_enc. >>> >>> Could anyone explains why the behavior of decryption is different >>> between TLSv1 and TLSv1.1/TLSv1.2? >> >>In TLSv1 and below the CBC IV is the previous record's last ciphertext >>block. This can enable certain types of attack where an attacker knows >>the IV that will be used for a record in advance. The problem was fixed >>in the specification of TLSv1.1 and above where a new IV is used for >>each record. As a counter measure to this issue OpenSSL (in TLSv1) sends >>an empty record before each "real" application data record to >>effectively randomise the IV and make it unpredictable so that an >>attacker cannot know it in advance. >> >>Therefore a TLSv1 OpenSSL client will send two records of application >>data where a TLSv1.1 or above OpenSSL client will just send one. This >>results in tls1_enc being called twice on the server side. >> >>This behaviour can be switched off by using the option >>SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS - but since this is considered >>insecure that would probably be unwise. >> >>Matt > > > On Wed, Sep 27, 2017 at 10:44 PM, Ma chunhui > wrote: > > Hi,? > > I met one problem when using OpenSSL1.1.0f with protocol TLSv1. > In brief, when using TLSv1, ?after server side received encrypted > data, and after function tls1_enc finished, the decrypted data is > not put in result buffer, after another tls1_enc, the decrypted data > is put in result buffer. While TLSv1.1/TLSv1.2 needs only one tls1_enc. > > > The way to reproduce it is quite simple: > > 1.some preparation: openssl req -x509 -newkey rsa:2048 -keyout > key.pem -out cert.pem -days 365 -nodes > 2.start server: openssl s_server -key key.pem -cert cert.pem -accept > 44330 -www > ? ? it's better to start server with gdb, and set breakpoints at > tls1_enc, then continue to run.? > 3.openssl s_client -connect localhost:44330 -tls1 -debug > > After the client is started, ?the server side will stop at > breakpoint, do several "c" to make it continue to run to wait for > client's messages > Then at client side, type a simple "hello" message and press Enter. > Then server side will stop at tls1_enc, the input data is same as > encrypted data from client side, but after Evp_Cipher and some pad > removing, the decrypted data length is 0. After another tls1_enc, > the decrypted data "hello" is put in the result buffer. > > But if client use -tls11 or -tls12, the decrypted "hello" is put in > the result buffer after the first tls1_enc. > > Could anyone explains why the behavior of decryption is different > between TLSv1 and TLSv1.1/TLSv1.2? > > Thanks. > > From uri at ll.mit.edu Fri Sep 29 17:16:19 2017 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Fri, 29 Sep 2017 17:16:19 +0000 Subject: [openssl-dev] Missing EVP_PKEY method to set engine? Message-ID: <03112499-1646-4625-BC93-D90C7315DD10@ll.mit.edu> Apologies in advance for cross-posting ? but I?m not sure which of the two mailing lists this belongs to. A key (say, private key) is loaded from the pkcs11 engine via privkey = ENGINE_load_private_key(engine, ); and this operation succeeds. However the resulting key handle has its engine == NULL. I looked for a method or a macro to explicitly set that value to the pointer to the engine that this key is bound to, but couldn?t find any. I define new methods such as pkcs11_pkey_rsa_decrypt(), and ?try to make OpenSSL aware of them via: ?? ????????EVP_PKEY_METHOD *orig_pmeth = EVP_PKEY_meth_find(EVP_PKEY_RSA); ?? EVP_PKEY_METHOD *pmeth = EVP_PKEY_meth_new(EVP_PKEY_RSA, EVP_PKEY_FLAG_AUTOARGLEN); ?? EVP_PKEY_meth_copy(pmeth, orig_pmeth); ?? EVP_PKEY_meth_get_decrypt(orig_pmeth, &pdecr_init, &pdecr); ?? EVP_PKEY_meth_set_decrypt(pmeth, pdecr_init, pkcs11_pkey_rsa_decrypt); ? And then there?s a function PKCS11_pkey_meths(ENGINE *e, EVP_PKEY_METHOD **pmethods, const int **nids, int nid) that assigns the above pmeth to *pmethods.? Is the above correct/sufficient? Somehow it seems to never invoke pkcs11_pkey_rsa_decrypt()? when it should be used. How can one set the EVP_PKEY->engine field in 1.1+? In ENGINE_set_pkey_meths(engine, pkey_meths) what should pkey_meths() actually be? Is it documented? When does libcrypto use ENGINE->pkey_meths methods instead of ?ENGINE->rsa_meth?? Who/what entity is supposed to invoke ENGINE->pkey_meths to retrieve the RSA methods block? Is it normal if on a key (EVP_PKEY *) loaded by an engine, engine ptr is NULL? And if not ? who/how should set those pointers to the correct value? -- Regards, Uri Blumenthal -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5211 bytes Desc: not available URL: From mch.thu at gmail.com Fri Sep 29 19:04:19 2017 From: mch.thu at gmail.com (Ma chunhui) Date: Sat, 30 Sep 2017 03:04:19 +0800 Subject: [openssl-dev] why TLSv1 need two tls1_enc to get decrypted data while TLSv1.1/TLSv1.2 need one in OpenSSL1.1.0f? In-Reply-To: References: Message-ID: Hi, Matt First, sorry for the mistake I made. In fact, with openSSL1.0.2 as the server, tls1_enc will be called twice in TLSv1, so it's not a timing issue. And besides, we don't rely on SSL_read will definitely result in readable data being decrypted, I'm just saying the most general process, of course we will do some check and deal with every situations, and my server has been working fine at a high concurrency with old version of Openssl for a long time. After some more debugging with my server(not OpenSSL s_server as the server) with different version OpenSSL: 1.0.2l & 1.1.0f, I found the reason why with openSSL1.0.2 my server can pass with TLSv1 while openSSL1.1.0 not. The reason is with OpenSSL1.1.0, after the first decryption, the length of the result buffer is 0, and in the end of method ssl3_get_record, RECORD_LAYER_set_numrpipes is call to set the numrpipes to 1. And then, in second SSL_read call (Note: when we call SSL_read, we pass 0 bytes as the length parameter, so in one SSL_read, only one tls1_enc is called. But we will call SSL_read twice), in ssl3_read_bytes, before ssl3_get_record, the value of numrpipes is checked and is not 0, so it won't call ssl3_get_record to process the second record and hence the second tls1_enc is not called. And I checked the place where this numrpipes will be set to 0 again, but unfortunately, since rr[0].read is always 0, so it won't be set to 0 again. And that's the reason why SSL_read can't get decrypted data. While with openSSL1.0.2,the condition of ssl3_get_record is if(rr->length || xxx), in the second call to SSL_read, the rr->length is 0 in this case, so the second tls1_enc is called and the decrypted data is generated. I don't quite understand of the "numrpipes", could you please explain a bit more about the change of numrpipes? And it looks like there is a bug for me because it seems the numrpipes is not set to 0 correctly. Thanks. On Thu, Sep 28, 2017 at 11:19 PM, Matt Caswell wrote: > > > On 28/09/17 14:38, Ma chunhui wrote: > > Hi, Matt > > > > Thanks for your quickly response. > > > > And yes, with this option SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS in client > > side the result can be get from one decryption. But the problem is, > > sometimes we can't control client's behavior. Maybe the client is > > openssl s_client, or maybe it's a python script or some other client, > > and the option is not that safe. > > > > Another interesting thing is, if server is using OpenSSL 1.0.2 or1.0.1, > > the result can be get from just one decryption(one tls1_enc method) with > > protocol TLSv1, and the client didn't add that option either(In fact, > > I'm using client OpenSSL1.1.0f). So it seems the process is changed in > > some version of OpenSSL1.1.0. > > Could you please explain a bit more on why openSSL 1.1.0f made this > > change? (I mean, the SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS option is added > > since 0.9.6d but openSSL1.0.1 and 1.0.2 can get result in one tls1_enc, > > while OpenSSL1.1.0f needs two tls1_enc) > > Well I can't replicate that result. If using an OpenSSL 1.0.2 server I > still see two calls tls1_enc (with both a 1.1.0 client and a 1.0.2 > client). Note - that doesn't necessarily translate to two SSL_read() > calls (see below). > > > > The reason why I'm focus on this is because: I'm using JNI call > > OpenSSL, my usage is like this(Just like tomcat native ): first, use > > BIO_write to write data to openssl, and then use SSL_read to read 0 > > length to trigger decryption, and then use SSL_pending to check how much > > data there is. If the data length is not put in result buffer in one > > decryption. then SSL_pending will get a 0 length result. and the whole > > process needs to be changed ,which is not we want. > > If OpenSSL reads an empty record then it will immediately try to read > the next record if one is available without returning control back to > the calling application. Therefore a single SSL_read() call can result > in multiple tls1_enc calls. However this is highly dependent on timing. > If the empty record and the following non-empty record arrive at the > destination slightly separated by time then when OpenSSL reads the first > empty record it will attempt to read the next record. This will fail > because it has not arrived yet and control will return to the calling > application. So sometimes you will have to call SSL_read() twice and > sometimes you will have to call it once. This is possibly a reason why > you see different behaviour between 1.0.2 and 1.1.0, i.e. because this > is very timing sensitive. > > Basically what you are doing is wrong. You cannot rely on the fact that > calling SSL_read() will definitely result in readable data being > decrypted. It might do - it might not. Another scenario where this could > occur is if a record arrives that is split across multiple TCP packets. > You call SSL_read() when the first TCP packet arrives - but because a > full record isn't there yet you get no readable application data back. > Yet another scenario is if the client attempts a renegotiation: network > packets arrive but when decrypted they don't actually contain any > application data - just handshake data. > > All of this is very reliant on timing, how the client behaves (which you > cannot control) and how the network behaves. If this was working for you > before then it sounds like you've been lucky so far. > > Matt > > > > > > Thanks. > > > > > >>From: Matt Caswell > > >>To: openssl-dev at openssl.org > >>Subject: Re: [openssl-dev] why TLSv1 need two tls1_enc to get > >> decrypted data while TLSv1.1/TLSv1.2 need one in OpenSSL1.1.0f? > >>Message-ID: <0890716d-2a3c-a659-f74e-7f2a5a89eca6 at openssl.org > > > > >>Content-Type: text/plain; charset=utf-8 > >> > >> > >> > >>On 27/09/17 15:44, Ma chunhui wrote: > >>> Hi,? > >>> > >>> I met one problem when using OpenSSL1.1.0f with protocol TLSv1. > >>> In brief, when using TLSv1, ?after server side received encrypted data, > >>> and after function tls1_enc finished, the decrypted data is not put in > >>> result buffer, after another tls1_enc, the decrypted data is put in > >>> result buffer. While TLSv1.1/TLSv1.2 needs only one tls1_enc. > >>> > >>> > >>> The way to reproduce it is quite simple: > >>> > >>> 1.some preparation: openssl req -x509 -newkey rsa:2048 -keyout key.pem > >>> -out cert.pem -days 365 -nodes > >>> 2.start server: openssl s_server -key key.pem -cert cert.pem -accept > >>> 44330 -www > >>> ? ? it's better to start server with gdb, and set breakpoints at > >>> tls1_enc, then continue to run.? > >>> 3.openssl s_client -connect localhost:44330 -tls1 -debug > >>> > >>> After the client is started, ?the server side will stop at breakpoint, > >>> do several "c" to make it continue to run to wait for client's messages > >>> Then at client side, type a simple "hello" message and press Enter. > Then > >>> server side will stop at tls1_enc, the input data is same as encrypted > >>> data from client side, but after Evp_Cipher and some pad removing, the > >>> decrypted data length is 0. After another tls1_enc, the decrypted data > >>> "hello" is put in the result buffer. > >>> > >>> But if client use -tls11 or -tls12, the decrypted "hello" is put in the > >>> result buffer after the first tls1_enc. > >>> > >>> Could anyone explains why the behavior of decryption is different > >>> between TLSv1 and TLSv1.1/TLSv1.2? > >> > >>In TLSv1 and below the CBC IV is the previous record's last ciphertext > >>block. This can enable certain types of attack where an attacker knows > >>the IV that will be used for a record in advance. The problem was fixed > >>in the specification of TLSv1.1 and above where a new IV is used for > >>each record. As a counter measure to this issue OpenSSL (in TLSv1) sends > >>an empty record before each "real" application data record to > >>effectively randomise the IV and make it unpredictable so that an > >>attacker cannot know it in advance. > >> > >>Therefore a TLSv1 OpenSSL client will send two records of application > >>data where a TLSv1.1 or above OpenSSL client will just send one. This > >>results in tls1_enc being called twice on the server side. > >> > >>This behaviour can be switched off by using the option > >>SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS - but since this is considered > >>insecure that would probably be unwise. > >> > >>Matt > > > > > > On Wed, Sep 27, 2017 at 10:44 PM, Ma chunhui > > wrote: > > > > Hi, > > > > I met one problem when using OpenSSL1.1.0f with protocol TLSv1. > > In brief, when using TLSv1, after server side received encrypted > > data, and after function tls1_enc finished, the decrypted data is > > not put in result buffer, after another tls1_enc, the decrypted data > > is put in result buffer. While TLSv1.1/TLSv1.2 needs only one > tls1_enc. > > > > > > The way to reproduce it is quite simple: > > > > 1.some preparation: openssl req -x509 -newkey rsa:2048 -keyout > > key.pem -out cert.pem -days 365 -nodes > > 2.start server: openssl s_server -key key.pem -cert cert.pem -accept > > 44330 -www > > it's better to start server with gdb, and set breakpoints at > > tls1_enc, then continue to run. > > 3.openssl s_client -connect localhost:44330 -tls1 -debug > > > > After the client is started, the server side will stop at > > breakpoint, do several "c" to make it continue to run to wait for > > client's messages > > Then at client side, type a simple "hello" message and press Enter. > > Then server side will stop at tls1_enc, the input data is same as > > encrypted data from client side, but after Evp_Cipher and some pad > > removing, the decrypted data length is 0. After another tls1_enc, > > the decrypted data "hello" is put in the result buffer. > > > > But if client use -tls11 or -tls12, the decrypted "hello" is put in > > the result buffer after the first tls1_enc. > > > > Could anyone explains why the behavior of decryption is different > > between TLSv1 and TLSv1.1/TLSv1.2? > > > > Thanks. > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at ballinger.io Fri Sep 29 19:25:02 2017 From: chris at ballinger.io (Chris Ballinger) Date: Fri, 29 Sep 2017 12:25:02 -0700 Subject: [openssl-dev] Patch for iOS compilation failure on Xcode 9 / iOS 11 SDK Message-ID: "-fomit-frame-pointer" is no longer allowed for armv7 targets, so I removed it from the iphoneos-cross configure target. I noticed this on openssl-1.0.2l. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- --- Configure.orig 2017-05-25 05:54:38.000000000 -0700 +++ Configure 2017-09-29 12:09:45.000000000 -0700 @@ -652,7 +652,7 @@ "debug-darwin64-x86_64-cc","cc:-arch x86_64 -ggdb -g2 -O0 -DL_ENDIAN -Wall::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:SIXTY_FOUR_BIT_LONG RC4_CHUNK DES_INT DES_UNROLL:".eval{my $asm=$x86_64_asm;$asm=~s/rc4\-[^:]+//;$asm}.":macosx:dlfcn:darwin-shared:-fPIC -fno-common:-arch x86_64 -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib", "debug-darwin-ppc-cc","cc:-DBN_DEBUG -DREF_CHECK -DCONF_DEBUG -DCRYPTO_MDEBUG -DB_ENDIAN -g -Wall -O::-D_REENTRANT:MACOSX::BN_LLONG RC4_CHAR RC4_CHUNK DES_UNROLL BF_PTR:${ppc32_asm}:osx32:dlfcn:darwin-shared:-fPIC:-dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib", # iPhoneOS/iOS -"iphoneos-cross","llvm-gcc:-O3 -isysroot \$(CROSS_TOP)/SDKs/\$(CROSS_SDK) -fomit-frame-pointer -fno-common::-D_REENTRANT:iOS:-Wl,-search_paths_first%:BN_LLONG RC4_CHAR RC4_CHUNK DES_UNROLL BF_PTR:${no_asm}:dlfcn:darwin-shared:-fPIC -fno-common:-dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib", +"iphoneos-cross","llvm-gcc:-O3 -isysroot \$(CROSS_TOP)/SDKs/\$(CROSS_SDK) -fno-common::-D_REENTRANT:iOS:-Wl,-search_paths_first%:BN_LLONG RC4_CHAR RC4_CHUNK DES_UNROLL BF_PTR:${no_asm}:dlfcn:darwin-shared:-fPIC -fno-common:-dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib", ##### A/UX "aux3-gcc","gcc:-O2 -DTERMIO::(unknown):AUX:-lbsd:RC4_CHAR RC4_CHUNK DES_UNROLL BF_PTR:::",