From pablo.platt at gmail.com Fri Nov 1 07:56:57 2019 From: pablo.platt at gmail.com (pablo platt) Date: Fri, 1 Nov 2019 09:56:57 +0200 Subject: Stitched aes-128 and hmac-sha1 (encrypt-then-mac) Message-ID: Hi, Stitching aes-cbc with sha1 can result with x2 performance [1]. Is there support for stitched aes-128-hmac-sha1 encrypt-then-mac? This issue [2] says that only mac-then-encrypt is supported in OpenSSL. Does this implement mac-then-encrypt and relevant [3]? Is it possible to use the same code with just changing the order to achieve encrypt-then-mac? How can I compile the Perl file to be used from a C program? [1] https://software.intel.com/en-us/articles/improving-openssl-performance [2] https://github.com/openssl/openssl/issues/8785 [3] https://github.com/openssl/openssl/blob/master/crypto/aes/asm/aesni-sha1-x86_64.pl Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Fri Nov 1 11:32:36 2019 From: matt at openssl.org (Matt Caswell) Date: Fri, 1 Nov 2019 11:32:36 +0000 Subject: Stitched aes-128 and hmac-sha1 (encrypt-then-mac) In-Reply-To: References: Message-ID: On 01/11/2019 07:56, pablo platt wrote: > Hi, > > Stitching aes-cbc with sha1 can result with x2 performance [1]. > Is there support for stitched aes-128-hmac-sha1 encrypt-then-mac? This > issue [2] says that only mac-then-encrypt is supported in OpenSSL. The issue is correct. Only mac-then-encrypt is supported. Furthermore these stitched ciphers are specifically targeted at use by libssl and are designed for use in SSL/TLS only. They are not general purpose ciphers and should not be used directly unless you *really* know what you are doing. Note that more modern TLS ciphersuites use AEAD modes such as GCM or CCM so that mac-then-encrypt vs encrypt-then-mac and "stitched" ciphers are irrelevant anyway. > > Does this implement mac-then-encrypt and relevant [3]? [3] is the aesni assembler implementation used behind the EVP_aes_128_cbc_hmac_sha1() and EVP_aes_256_cbc_hmac_sha1() ciphers, i.e. all the same comments I made above apply here. It's mac-then-encrypt, and specifically targeted for use in SSL/TLS by libssl. It's not intended for general purpose use. The documentation says this about these ciphers: "EVP_aes_128_cbc_hmac_sha1(), EVP_aes_256_cbc_hmac_sha1() Authenticated encryption with AES in CBC mode using SHA-1 as HMAC, with keys of 128 and 256 bits length respectively. The authentication tag is 160 bits long. WARNING: this is not intended for usage outside of TLS and requires calling of some undocumented ctrl functions. These ciphers do not conform to the EVP AEAD interface." https://www.openssl.org/docs/man1.1.1/man3/EVP_aes_128_cbc_hmac_sha1.html > Is it possible to use the same code with just changing the order to > achieve encrypt-then-mac? No. > How can I compile the Perl file to be used from a C program? This is an internal file not intended for use outside of OpenSSL and not intended to be compiled separately. You might be able to extract it - but if so, you're on your own. Matt From pablo.platt at gmail.com Fri Nov 1 11:59:31 2019 From: pablo.platt at gmail.com (pablo platt) Date: Fri, 1 Nov 2019 13:59:31 +0200 Subject: Stitched aes-128 and hmac-sha1 (encrypt-then-mac) In-Reply-To: References: Message-ID: Thank you for the explanation. The use case is a WebRTC server (SFU) that encrypts and authenticate SRTP packets. Encryption is a major part of CPU load on SFU servers. Reducing it by 50% will have a large impact. Is it planned to add aes-128-hmac-sha1 encrypt-then-mac? On Fri, Nov 1, 2019 at 1:32 PM Matt Caswell wrote: > > > On 01/11/2019 07:56, pablo platt wrote: > > Hi, > > > > Stitching aes-cbc with sha1 can result with x2 performance [1]. > > Is there support for stitched aes-128-hmac-sha1 encrypt-then-mac? This > > issue [2] says that only mac-then-encrypt is supported in OpenSSL. > > The issue is correct. Only mac-then-encrypt is supported. Furthermore > these stitched ciphers are specifically targeted at use by libssl and > are designed for use in SSL/TLS only. They are not general purpose > ciphers and should not be used directly unless you *really* know what > you are doing. > > Note that more modern TLS ciphersuites use AEAD modes such as GCM or CCM > so that mac-then-encrypt vs encrypt-then-mac and "stitched" ciphers are > irrelevant anyway. > > > > > Does this implement mac-then-encrypt and relevant [3]? > > [3] is the aesni assembler implementation used behind the > EVP_aes_128_cbc_hmac_sha1() and EVP_aes_256_cbc_hmac_sha1() ciphers, > i.e. all the same comments I made above apply here. It's > mac-then-encrypt, and specifically targeted for use in SSL/TLS by > libssl. It's not intended for general purpose use. > > The documentation says this about these ciphers: > > "EVP_aes_128_cbc_hmac_sha1(), > EVP_aes_256_cbc_hmac_sha1() > > Authenticated encryption with AES in CBC mode using SHA-1 as HMAC, with > keys of 128 and 256 bits length respectively. The authentication tag is > 160 bits long. > > WARNING: this is not intended for usage outside of TLS and requires > calling of some undocumented ctrl functions. These ciphers do not > conform to the EVP AEAD interface." > > https://www.openssl.org/docs/man1.1.1/man3/EVP_aes_128_cbc_hmac_sha1.html > > > > > Is it possible to use the same code with just changing the order to > > achieve encrypt-then-mac? > > No. > > > How can I compile the Perl file to be used from a C program? > > This is an internal file not intended for use outside of OpenSSL and not > intended to be compiled separately. You might be able to extract it - > but if so, you're on your own. > > > Matt > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Fri Nov 1 12:01:53 2019 From: matt at openssl.org (Matt Caswell) Date: Fri, 1 Nov 2019 12:01:53 +0000 Subject: Stitched aes-128 and hmac-sha1 (encrypt-then-mac) In-Reply-To: References: Message-ID: <9df89f12-d974-7df6-3f79-4f27f679cb64@openssl.org> On 01/11/2019 11:59, pablo platt wrote: > Thank you for the explanation. > > The use case is a WebRTC server (SFU) that encrypts and authenticate > SRTP packets. > Encryption is a major part of CPU load on SFU servers. Reducing it by > 50% will have a large impact. > > Is it planned to add aes-128-hmac-sha1 encrypt-then-mac? There are no current plans. You might investigate the impact of using AEAD ciphers instead. Matt > > On Fri, Nov 1, 2019 at 1:32 PM Matt Caswell > wrote: > > > > On 01/11/2019 07:56, pablo platt wrote: > > Hi, > > > > Stitching aes-cbc with sha1 can result with x2 performance [1]. > > Is there support for stitched aes-128-hmac-sha1 encrypt-then-mac? This > > issue [2] says that only mac-then-encrypt is supported in OpenSSL. > > The issue is correct. Only mac-then-encrypt is supported. Furthermore > these stitched ciphers are specifically targeted at use by libssl and > are designed for use in SSL/TLS only. They are not general purpose > ciphers and should not be used directly unless you *really* know what > you are doing. > > Note that more modern TLS ciphersuites use AEAD modes such as GCM or CCM > so that mac-then-encrypt vs encrypt-then-mac and "stitched" ciphers are > irrelevant anyway. > > > > > Does this implement mac-then-encrypt and relevant [3]? > > [3] is the aesni assembler implementation used behind the > EVP_aes_128_cbc_hmac_sha1() and EVP_aes_256_cbc_hmac_sha1() ciphers, > i.e. all the same comments I made above apply here. It's > mac-then-encrypt, and specifically targeted for use in SSL/TLS by > libssl. It's not intended for general purpose use. > > The documentation says this about these ciphers: > > "EVP_aes_128_cbc_hmac_sha1(), > EVP_aes_256_cbc_hmac_sha1() > > Authenticated encryption with AES in CBC mode using SHA-1 as HMAC, with > keys of 128 and 256 bits length respectively. The authentication tag is > 160 bits long. > > WARNING: this is not intended for usage outside of TLS and requires > calling of some undocumented ctrl functions. These ciphers do not > conform to the EVP AEAD interface." > > https://www.openssl.org/docs/man1.1.1/man3/EVP_aes_128_cbc_hmac_sha1.html > > > > > Is it possible to use the same code with just changing the order to > > achieve encrypt-then-mac? > > No. > > > How can I compile the Perl file to be used from a C program? > > This is an internal file not intended for use outside of OpenSSL and not > intended to be compiled separately. You might be able to extract it - > but if so, you're on your own. > > > Matt > From pablo.platt at gmail.com Fri Nov 1 12:05:09 2019 From: pablo.platt at gmail.com (pablo platt) Date: Fri, 1 Nov 2019 14:05:09 +0200 Subject: Stitched aes-128 and hmac-sha1 (encrypt-then-mac) In-Reply-To: <9df89f12-d974-7df6-3f79-4f27f679cb64@openssl.org> References: <9df89f12-d974-7df6-3f79-4f27f679cb64@openssl.org> Message-ID: AES-GCM will be supported in WebRTC in the future. It has great performance and I think better security. The only downside is that packets will be 6 bytes larger and it'll take few months/years most browsers support it. Thanks On Fri, Nov 1, 2019 at 2:01 PM Matt Caswell wrote: > > > On 01/11/2019 11:59, pablo platt wrote: > > Thank you for the explanation. > > > > The use case is a WebRTC server (SFU) that encrypts and authenticate > > SRTP packets. > > Encryption is a major part of CPU load on SFU servers. Reducing it by > > 50% will have a large impact. > > > > Is it planned to add aes-128-hmac-sha1 encrypt-then-mac? > > There are no current plans. You might investigate the impact of using > AEAD ciphers instead. > > Matt > > > > > On Fri, Nov 1, 2019 at 1:32 PM Matt Caswell > > wrote: > > > > > > > > On 01/11/2019 07:56, pablo platt wrote: > > > Hi, > > > > > > Stitching aes-cbc with sha1 can result with x2 performance [1]. > > > Is there support for stitched aes-128-hmac-sha1 encrypt-then-mac? > This > > > issue [2] says that only mac-then-encrypt is supported in OpenSSL. > > > > The issue is correct. Only mac-then-encrypt is supported. Furthermore > > these stitched ciphers are specifically targeted at use by libssl and > > are designed for use in SSL/TLS only. They are not general purpose > > ciphers and should not be used directly unless you *really* know what > > you are doing. > > > > Note that more modern TLS ciphersuites use AEAD modes such as GCM or > CCM > > so that mac-then-encrypt vs encrypt-then-mac and "stitched" ciphers > are > > irrelevant anyway. > > > > > > > > Does this implement mac-then-encrypt and relevant [3]? > > > > [3] is the aesni assembler implementation used behind the > > EVP_aes_128_cbc_hmac_sha1() and EVP_aes_256_cbc_hmac_sha1() ciphers, > > i.e. all the same comments I made above apply here. It's > > mac-then-encrypt, and specifically targeted for use in SSL/TLS by > > libssl. It's not intended for general purpose use. > > > > The documentation says this about these ciphers: > > > > "EVP_aes_128_cbc_hmac_sha1(), > > EVP_aes_256_cbc_hmac_sha1() > > > > Authenticated encryption with AES in CBC mode using SHA-1 as HMAC, > with > > keys of 128 and 256 bits length respectively. The authentication tag > is > > 160 bits long. > > > > WARNING: this is not intended for usage outside of TLS and requires > > calling of some undocumented ctrl functions. These ciphers do not > > conform to the EVP AEAD interface." > > > > > https://www.openssl.org/docs/man1.1.1/man3/EVP_aes_128_cbc_hmac_sha1.html > > > > > > > > > Is it possible to use the same code with just changing the order to > > > achieve encrypt-then-mac? > > > > No. > > > > > How can I compile the Perl file to be used from a C program? > > > > This is an internal file not intended for use outside of OpenSSL and > not > > intended to be compiled separately. You might be able to extract it - > > but if so, you're on your own. > > > > > > Matt > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nagalakshmi.j at altran.com Fri Nov 1 14:19:19 2019 From: nagalakshmi.j at altran.com (Nagalakshmi V J) Date: Fri, 1 Nov 2019 14:19:19 +0000 Subject: OpenSSL compilation errors in Windows In-Reply-To: <92db124c-3cf0-0474-dc88-a5663fa5561e@openssl.org> References: <2739d542-13a2-4778-47fd-c45b45492f37@openssl.org> <66049c61-3b52-3be0-b568-9a36f2a8b166@openssl.org> <92db124c-3cf0-0474-dc88-a5663fa5561e@openssl.org> Message-ID: Hi Matt, Thanks for your help. I am able to proceed now. Thanks and regards, Nagalakshmi -----Original Message----- From: Matt Caswell Sent: Wednesday, October 30, 2019 7:55 PM To: Nagalakshmi V J ; openssl-users at openssl.org Subject: Re: OpenSSL compilation errors in Windows ** This mail has been sent from an external source ** On 29/10/2019 11:55, Nagalakshmi V J wrote: > Hi Matt, > > Thank you so much for your response. Those mentioned APIs resolved my > errors. > > For the below code, > > return SSL_get_session(pConnection) != NULL && > pConnection->session->session_id_length != 0; > > Any reference for accessing session_id_length? > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.openssl.org_d > ocs_man1.1.0_man3_SSL-5FCTX-5Fset-5Fgenerate-5Fsession-5Fid.html&d=DwI > D-g&c=cxWN2QSDopt5SklNfbjIjg&r=zbjUR56YPF3jaTRTjX4KZlHM9-LmYAuR5atSqEG > OnpA&m=mgmrDa8wrs1zaAUL-PLOcRGKsCoFwXg9ZmrJMt56Yso&s=GW6E7NE-6ODy28APY > gBz7MYCKAuXh9wULiPQjZ-AMR0&e= You should use SSL_SESSION_get_id() to get hold of the length: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.openssl.org_docs_man1.1.1_man3_SSL-5FSESSION-5Fget-5Fid.html&d=DwID-g&c=cxWN2QSDopt5SklNfbjIjg&r=zbjUR56YPF3jaTRTjX4KZlHM9-LmYAuR5atSqEGOnpA&m=mgmrDa8wrs1zaAUL-PLOcRGKsCoFwXg9ZmrJMt56Yso&s=rGqb0VAIAgD_dzrh6Cpv2AyI6wzAaog-HYn_OY_0mMU&e= Matt > > Not sure if I can use the above link. > > > /Thanks & Regards,/ > /Nagalakshmi V J/ > ---------------------------------------------------------------------- > -- > *From:* Matt Caswell > *Sent:* 29 October 2019 10:47 > *To:* Nagalakshmi V J ; > openssl-users at openssl.org > *Subject:* Re: OpenSSL compilation errors in Windows > > ** This mail has been sent from an external source ** > > > On 29/10/2019 10:34, Nagalakshmi V J wrote: >> >> tls1_PRF(s->ctx->md5,s->ctx->sha1,buf,(int)(p-buf), >> >> pGenerator->master_secret,sizeof(pGenerator->master_secret), >> >> km,tmp,num); > > It seems your code is replicating parts of libssl - which seems like a > strange (and possibly dangerous) thing to do! > >> Struct ssl_ctx_st { >> >> ... >> >> constEVP_MD *md5; /* For SSLv3/TLSv1 'ssl3-md5' */ >> >> constEVP_MD *sha1; /* For SSLv3/TLSv1 'ssl3->sha1' */ >> >> ... >> >> } > > You really don't need to access these things. They're just cached > references to the value returned by EVP_get_digestbyname("ssl3-md5") > and EVP_get_digestbyname("ssl3-sha1"). So you can call those functions > directly anyway. > > Matt > > ===================================================== > Please refer to https://northamerica.altran.com/email-disclaimer > for important disclosures regarding this electronic communication. > ===================================================== ===================================================== Please refer to https://northamerica.altran.com/email-disclaimer for important disclosures regarding this electronic communication. ===================================================== From openssl at jordan.maileater.net Fri Nov 1 17:34:57 2019 From: openssl at jordan.maileater.net (Jordan Brown) Date: Fri, 1 Nov 2019 17:34:57 +0000 Subject: Digest algorithms for Ruby In-Reply-To: <167EC78D-8587-4CE2-9B24-DB454B8E7490@dukhovni.org> References: <167EC78D-8587-4CE2-9B24-DB454B8E7490@dukhovni.org> Message-ID: <0101016e2808d331-1d8d1851-59d4-4a90-9796-39a08c388228-000000@us-west-2.amazonses.com> On 10/31/2019 7:35 AM, Viktor Dukhovni wrote: > My advice would be to avoid specific support for any *particular* > digest algorithm. Instead, provide bindings to: > - EVP_get_digestbyname(), > - EVP_MD_CTX_create(3), > - EVP_DigestInit_ex(3), > - EVP_DigestUpdate(3), > - EVP_DigestFinal_ex(3), > - EVP_MD_CTX_destroy(3) > > which can they use *any* available digest algorithm (by name). > That avoids having *your* software be dependent on the digest algorithms, but it does so by exporting the dependency out to your caller. The bottom line for somebody trying to maintain compatibility is that when you remove some algorithm X, there's always a risk that something in the stack - be it software or user configuration - explicitly depends on X and so will fail on upgrade. -- Jordan Brown, Oracle ZFS Storage Appliance, Oracle Solaris -------------- next part -------------- An HTML attachment was scrubbed... URL: From cauldwell.thomas at gmail.com Sat Nov 2 21:28:02 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Sat, 2 Nov 2019 22:28:02 +0100 Subject: Force the use of engine in config file Message-ID: I have a config file, "/etc/ssl/openssl.cnf". This config file gives the details of an engine to use for random number generation. I know that this config file is well-formed because I have confirmed that it uses my engine when I try to use the "openssl" utility at the command line to generate a random number. I have been able to determine though that some other programs which link with "libssl.so" are NOT using my engine. Since I already know that my config file is well-formed, I can only conclude that these other programs are initialising the OpenSSL library as follows: OPENSSL_noconfig(); So in order to make sure that every program that links with "libssl.so" actually uses my engine, I think I need to go into the OpenSSL source code and replace the OPENSSL_noconfig function like this: int OPENSSL_config(void) { return OPENSSL_config(); } Can anyone think of any other ideas to ensure that "libssl.so" always uses the engine specified in the config file? From cauldwell.thomas at gmail.com Sat Nov 2 21:39:33 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Sat, 2 Nov 2019 22:39:33 +0100 Subject: Force the use of engine in config file In-Reply-To: References: Message-ID: >> int OPENSSL_config(void) >> { >> return OPENSSL_config(); >> } That first line should be: int OPENSSL_noconfig(void) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Sat Nov 2 22:19:55 2019 From: rsalz at akamai.com (Salz, Rich) Date: Sat, 2 Nov 2019 22:19:55 +0000 Subject: Force the use of engine in config file In-Reply-To: References: Message-ID: <90AE72A7-50EE-4CC2-BC1E-2211DA5CA097@akamai.com> If you are changing openssl, why not just change the init function to load your engine and abort/exit/fail if it doesn?t load? -------------- next part -------------- An HTML attachment was scrubbed... URL: From cauldwell.thomas at gmail.com Sat Nov 2 23:21:24 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Sun, 3 Nov 2019 00:21:24 +0100 Subject: Force the use of engine in config file In-Reply-To: <90AE72A7-50EE-4CC2-BC1E-2211DA5CA097@akamai.com> References: <90AE72A7-50EE-4CC2-BC1E-2211DA5CA097@akamai.com> Message-ID: Since I already have a well-formed config file, I think it would be a minimalistic change to hijack the "OPENSSL_noconfig" function (instead of changing the code for Init). But your idea could work too. Even if I do implement your idea though, I will still remove the random number generation routines in drbg_lib.c, as there should not be any software psudeorandomness generator on my embedded device. On Saturday, November 2, 2019, Salz, Rich wrote: > If you are changing openssl, why not just change the init function to load > your engine and abort/exit/fail if it doesn?t load? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aijazbaig1 at gmail.com Mon Nov 4 06:29:36 2019 From: aijazbaig1 at gmail.com (Aijaz Baig) Date: Mon, 4 Nov 2019 11:59:36 +0530 Subject: static linking libssl and libcrypto Message-ID: I am trying to build a shared library that internally links openssl and crypto libraries statically so I can use it in a production environment. To that end I am using the following Makefile APPBASE=/home/AB/Documents/APP/APP_2.17.0 OPENSSL1.0.2p_INSTALL_LOC=/home/AB/Documents/APP/OpenSSL-1.0.2p-installation CC=gcc CFLAGS= -Wall -g -O -fPIC RM= rm -f .PHONY: all clean src=$(wildcard *Generic/*.c *Linux/*.c) $(info source=$(src)) #we use the custom compiled openssl version #and NOT the one available on the system #INC=-I/usr/include/openssl INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include/openssl INC+=$(foreach d,$(incdir),-I$d) $(info includes=$(INC)) LIB=-L$(OPENSSL1.0.2p_INSTALL_LOC)/lib #LIB=-llibssl -llibcrypto LIB+=-lssl -lcrypto $(info links=$(LIB)) #LIB+=-L/usr/lib/ obj=$(src:.c=.o) all: libAPP.so clean: $(RM) *.o *.so $(shell find $(APPBASE) -type f -iname "*.o" -exec rm -rf {} \;) .c.o: ${CC} -static ${CFLAGS} $(INC) -c $< $(LIB) -o $@ libAPP.so: $(obj) $(LINK.c) -shared $^ -o $@ As mentioned here ( https://stackoverflow.com/questions/18185618/how-to-use-static-linking-with-openssl-in-c-c/25811538#25811538), I've changed the linker flags to: -lcrypto -lz -ldl -static-libgcc but it doesn't seem to change the size of the generated so file. On checking for references to SSL in this so file, I see there are a total of 87 entries nm libAPP.so | grep -i "ssl" | wc -l 87 whereas listing *only* the global symbols from libssl.a tells me it has 1113 globally defined symbols. nm -g ../OpenSSL-1.0.2p-installation/lib/libssl.a | grep -i "ssl" | wc -l 1113 I do not know if libSSL got indeed linked statically or not. Could someone please shed some light on it? -- Best Regards, Aijaz Baig -------------- next part -------------- An HTML attachment was scrubbed... URL: From brice at famille-andre.be Mon Nov 4 11:29:02 2019 From: brice at famille-andre.be (=?UTF-8?B?QnJpY2UgQW5kcsOp?=) Date: Mon, 4 Nov 2019 12:29:02 +0100 Subject: static linking libssl and libcrypto In-Reply-To: References: Message-ID: Hello, It's not an open-ssl issue, but more a compiler specific one. With info you provided, I cannot tell you what you get as results, but two points that may help: 1. regarding the 87 ssl symbols : when you link with a library, only the useful symbols are imported. So, if the code in you libAPP library only uses some sparse functions of libSSL, it's normal you only have corresponding symbols in your final image. I don't know what you plan to do, but note that statically linking your dll with open-ssl will prevent this dll from needing the openssl dynamic library. But it will not prevent your main program to require the open-ssl library to run properly if some part of it is dynamically linked with open-ssl ! 2. depending on how you compiled your libssl.a, it can be either a static library containing the full openssl binary code, or a static library that just makes the "link" between you code and the ssl dynamic library. In the second case, even if you properly statically link with this lib, you will still need the dll to execute your program. Regards, Brice Le lun. 4 nov. 2019 ? 07:28, Aijaz Baig a ?crit : > I am trying to build a shared library that internally links openssl and > crypto libraries statically so I can use it in a production environment. To > that end I am using the following Makefile > > APPBASE=/home/AB/Documents/APP/APP_2.17.0 > OPENSSL1.0.2p_INSTALL_LOC=/home/AB/Documents/APP/OpenSSL-1.0.2p-installation > CC=gcc > CFLAGS= -Wall -g -O -fPIC > RM= rm -f > .PHONY: all clean > > src=$(wildcard *Generic/*.c *Linux/*.c) > $(info source=$(src)) > > #we use the custom compiled openssl version > #and NOT the one available on the system > #INC=-I/usr/include/openssl > INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include/openssl > INC+=$(foreach d,$(incdir),-I$d) > $(info includes=$(INC)) > > LIB=-L$(OPENSSL1.0.2p_INSTALL_LOC)/lib > #LIB=-llibssl -llibcrypto > LIB+=-lssl -lcrypto > $(info links=$(LIB)) > #LIB+=-L/usr/lib/ > > obj=$(src:.c=.o) > all: libAPP.so > clean: > $(RM) *.o *.so > $(shell find $(APPBASE) -type f -iname "*.o" -exec rm -rf {} \;) > > .c.o: > ${CC} -static ${CFLAGS} $(INC) -c $< $(LIB) -o $@ > > libAPP.so: $(obj) > $(LINK.c) -shared $^ -o $@ > > As mentioned here ( > https://stackoverflow.com/questions/18185618/how-to-use-static-linking-with-openssl-in-c-c/25811538#25811538), > I've changed the linker flags to: > -lcrypto -lz -ldl -static-libgcc > > but it doesn't seem to change the size of the generated so file. On > checking for references to SSL in this so file, I see there are a total of > 87 entries > > nm libAPP.so | grep -i "ssl" | wc -l > 87 > > whereas listing *only* the global symbols from libssl.a tells me it has > 1113 globally defined symbols. > nm -g ../OpenSSL-1.0.2p-installation/lib/libssl.a | grep -i "ssl" | wc -l > 1113 > > I do not know if libSSL got indeed linked statically or not. Could someone > please shed some light on it? > > -- > > Best Regards, > Aijaz Baig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cauldwell.thomas at gmail.com Mon Nov 4 12:49:41 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Mon, 4 Nov 2019 12:49:41 -0000 (UTC) Subject: Force the use of engine in config file References: Message-ID: Okay first I'll show the changes that I've made to the source code and build setup for "libopenssl". I have added two compiler flags: OPENSSL_NO_RDRAND, OPENSSL_LOAD_CONFIG Not that the following compiler flag is NOT set: OPENSSL_NO_AUTOLOAD_CONFIG And here are the source code changes: (1) File: ssl_init.c Purpose of Alteration: Clear the option flag bit for not loadind conf Alteration: In the function "OPENSSL_init_ssl", insert the following line at the beginning of the function: opts &= ~(uint64_t)OPENSSL_INIT_NO_LOAD_CONFIG; /* Clear the bit for not loading TPM2 engine */ (2) File: drbg_lib.c Purpose of Alteration: Make a log of all uses of the built-in generator Alteration: Rename the function definition "drbg_bytes" to "drbg_bytes_REAL", and then append the following to the end of the file: #include static int drbg_bytes(unsigned char *out, int count) { int const retval = drbg_bytes_REAL(out, count); /* I renamed the real function */ int const fd_lock = open("/tmp/locker_for_randomness_log", O_CREAT); flock(fd_lock, LOCK_EX); { FILE *const pfile = fopen("/var/log/bad_randomness.log", "a"); if ( NULL != pfile ) { time_t ltime; struct tm result; char stime[32]; ltime = time(NULL); localtime_r(<ime, &result); asctime_r(&result, stime); stime[ strlen(stime) - 1 ] = ' '; /* Get rid of newline char at the end */ fprintf(pfile, "%s - - - %u bytes\n", stime, (unsigned) count); fclose(pfile); } } flock(fd_lock, LOCK_UN); return retval; } I have reconfigured and rebuilt "libopenssl", and so I boot up my device and then I run the following command: tail -F /var/log/bad_randomness.log This file shouldn't exist if the built-in generator is never used -- but some how, some way, even with all the changes I've made above, at least one of the running processes that links with "libssl.so" is NOT using the engine I specify in the config file "/etc/ssl/openssl.cnf". Looking at the output from the 'tail' command above, it's requesting 16 bytes of random data every 6 seconds. Here's the repeated line: Mon Nov 04 12:41:06 2019 - - - 16 bytes Here's how I get a list of all the procesess currently using "libssl.so": grep libssl /proc/*/maps | cut -d ':' -f 1 | cut -d '/' -f 3 | uniq | xargs -n1 -i ls -l /proc/{}/exe And there's the output I'm getting: lrwxrwxrwx 1 root root 0 Feb 16 02:54 /proc/1622/exe -> /usr/sbin/lighttpd lrwxrwxrwx 1 root root 0 Feb 16 02:54 /proc/1681/exe -> /opt/prodanko/bin/callar_plugin So this means that one of these two progams is some how managing to load up the 'libopenssl' library and get it to use its internal random number generator. I wonder if this is being achieved with explicit library calls to functions such as "OPENSSL_add_all_algorithms_noconf"? I suppose I could also add a stack trace to my log file to try figure out which process is requesting those 16 bytes every 6 seconds. And idead on what to try next? From thomas_floodeenjr at mentor.com Mon Nov 4 13:01:30 2019 From: thomas_floodeenjr at mentor.com (Floodeenjr, Thomas) Date: Mon, 4 Nov 2019 13:01:30 +0000 Subject: static linking libssl and libcrypto In-Reply-To: References: Message-ID: To check if you are linked statically or dynamically, what does ldd tell you? (ldd libAPP.so) Your library should not have a dependency on libssl.so or libcrypto.so if you are linked statically. -Tom From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of Aijaz Baig Sent: Sunday, November 3, 2019 11:30 PM To: openssl-users at openssl.org Subject: static linking libssl and libcrypto I am trying to build a shared library that internally links openssl and crypto libraries statically so I can use it in a production environment. To that end I am using the following Makefile APPBASE=/home/AB/Documents/APP/APP_2.17.0 OPENSSL1.0.2p_INSTALL_LOC=/home/AB/Documents/APP/OpenSSL-1.0.2p-installation CC=gcc CFLAGS= -Wall -g -O -fPIC RM= rm -f .PHONY: all clean src=$(wildcard *Generic/*.c *Linux/*.c) $(info source=$(src)) #we use the custom compiled openssl version #and NOT the one available on the system #INC=-I/usr/include/openssl INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include/openssl INC+=$(foreach d,$(incdir),-I$d) $(info includes=$(INC)) LIB=-L$(OPENSSL1.0.2p_INSTALL_LOC)/lib #LIB=-llibssl -llibcrypto LIB+=-lssl -lcrypto $(info links=$(LIB)) #LIB+=-L/usr/lib/ obj=$(src:.c=.o) all: libAPP.so clean: $(RM) *.o *.so $(shell find $(APPBASE) -type f -iname "*.o" -exec rm -rf {} \;) .c.o: ${CC} -static ${CFLAGS} $(INC) -c $< $(LIB) -o $@ libAPP.so: $(obj) $(LINK.c) -shared $^ -o $@ As mentioned here (https://stackoverflow.com/questions/18185618/how-to-use-static-linking-with-openssl-in-c-c/25811538#25811538), I've changed the linker flags to: -lcrypto -lz -ldl -static-libgcc but it doesn't seem to change the size of the generated so file. On checking for references to SSL in this so file, I see there are a total of 87 entries nm libAPP.so | grep -i "ssl" | wc -l 87 whereas listing only the global symbols from libssl.a tells me it has 1113 globally defined symbols. nm -g ../OpenSSL-1.0.2p-installation/lib/libssl.a | grep -i "ssl" | wc -l 1113 I do not know if libSSL got indeed linked statically or not. Could someone please shed some light on it? -- Best Regards, Aijaz Baig -------------- next part -------------- An HTML attachment was scrubbed... URL: From cauldwell.thomas at gmail.com Mon Nov 4 13:59:39 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Mon, 4 Nov 2019 13:59:39 -0000 (UTC) Subject: Force the use of engine in config file References: Message-ID: > Okay first I'll show the changes that I've made to the source code and > build setup for "libopenssl". I added one more change, I added to the beginning of the function "OPENSSL_init_crypto" these two lines: opts &= ~(uint64_t)OPENSSL_INIT_NO_LOAD_CONFIG; opts |= OPENSSL_INIT_LOAD_CONFIG; I think that this might finally have made **every** process load the libssl.so library and use the engine I specify. Unfortunately though, when my device boots up, it doesn't get past this point: random: crng init done random: 7 urandom warning(s) missed due to ratelimiting It freezes at that point. . . I don't know what it's doing (if anything) in the background. The machine isn't **totally** frozen though, because when I plug in a USB stick, I get: usb 1-6: new high-speed USB device number 6 using xhci_hcd From jqian at tibco.com Mon Nov 4 22:34:36 2019 From: jqian at tibco.com (Jason Qian) Date: Mon, 4 Nov 2019 17:34:36 -0500 Subject: Help on Diffie Hellman key exchange Message-ID: Hi We have an application that does the Diffie Hellman key exchange (OpenSSL/1.1.0f). It works fine, but under heavy loaded conditions, sometimes an invalide secret been generated and other side couldn't decrypt the data (the secret seems offset by one). The client side is c++ and the server side is java. DH_compute_key(secretKey, bnY, m_DH); Someone in the openssl group also talks about a similar issue, but not sure if have a solution. Thanks for your help, Jason -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen.farrell at cs.tcd.ie Tue Nov 5 04:23:47 2019 From: stephen.farrell at cs.tcd.ie (Stephen Farrell) Date: Tue, 5 Nov 2019 04:23:47 +0000 Subject: compile error with tracing on in CMP code Message-ID: <905d9532-7c34-c123-3503-d340c3c7b1b2@cs.tcd.ie> Hiya, I just cloned from the tip and after: $ ./config enable-ssl-trace enable-trace --debug I get a compile error (see below). I didn't see an immediately obvious fix, but (for me) just setting "-no-cmp" in addition to the above seems to work fine, so I've stopped delving into it for now. Cheers, S. ... gcc -I. -Iinclude -Iapps/include -I. -Iinclude -Iapps/include -pthread -m64 -Wa,--noexecstack -Wall -O0 -g -MMD -MF test/cmp_status_test-bin-cmp_testlib.d.tmp -MT test/cmp_status_test-bin-cmp_testlib.o -c -o test/cmp_status_test-bin-cmp_testlib.o test/cmp_testlib.c In file included from include/openssl/cmp_util.h:19, from include/openssl/cmp.h:20, from test/cmp_testlib.h:15, from test/cmp_ctx_test.c:12: test/cmp_ctx_test.c: In function 'execute_cmp_ctx_log_cb_test': include/openssl/cmp_util.h:35:30: error: expected ')' before 'OPENSSL_MSTR' 35 | OPENSSL_MSTR(OPENSSL_LINE) ":" OSSL_CMP_LOG_PREFIX ... lots more errors ... -------------- next part -------------- A non-text attachment was scrubbed... Name: 0x5AB2FAF17B172BEA.asc Type: application/pgp-keys Size: 10715 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From heyhgl at gmail.com Tue Nov 5 06:52:47 2019 From: heyhgl at gmail.com (Glen Huang) Date: Tue, 5 Nov 2019 14:52:47 +0800 Subject: How to convert ed25519 private key from der to pem? Message-ID: <7C13DC4F-3E87-4EEE-AF08-DE98C4E7340C@gmail.com> To convert RSA private key from der to pem, I can use the rsa subcommand, but there is no ed25519, and ec doesn?t seem to work for ed25519. I wonder if it?s possible to convert ed25519 private key from der to pem with openssl? What?s the proper way? From heyhgl at gmail.com Tue Nov 5 07:13:20 2019 From: heyhgl at gmail.com (Glen Huang) Date: Tue, 5 Nov 2019 15:13:20 +0800 Subject: How to convert ed25519 private key from der to pem? In-Reply-To: <7C13DC4F-3E87-4EEE-AF08-DE98C4E7340C@gmail.com> References: <7C13DC4F-3E87-4EEE-AF08-DE98C4E7340C@gmail.com> Message-ID: <26F770D7-DED3-46E4-BFFA-E103B95174B1@gmail.com> NVM, just found out it was asked sometime ago: https://www.mail-archive.com/openssl-users at openssl.org/msg86720.html > On Nov 5, 2019, at 2:52 PM, Glen Huang wrote: > > To convert RSA private key from der to pem, I can use the rsa subcommand, but there is no ed25519, and ec doesn?t seem to work for ed25519. > > I wonder if it?s possible to convert ed25519 private key from der to pem with openssl? What?s the proper way? From aijazbaig1 at gmail.com Tue Nov 5 17:22:37 2019 From: aijazbaig1 at gmail.com (Aijaz Baig) Date: Tue, 5 Nov 2019 22:52:37 +0530 Subject: static linking libssl and libcrypto In-Reply-To: References: Message-ID: Thank you for the information. I will address your points here: 1. I was not aware of the fact that only those symbols that have been used get imported when linking a library statically. So that very well could be the case. I didn't get what you mentioned about the static linking preventing the program from requiring libSSL.so. I mean the way I am linking my library should be of no concern to the source code right? Or so I think. 2. when I downloaded and compiled the openssl library (from source), I followed the INSTALL read me. All it resulted was libssl.a and libcrypto.a. I didn't find any file name libSSL.so. So how will this static library (archive) have references to libSSL.so on the system?? I am kind of confused here now. Keen to hear from you, Aijaz On Mon, Nov 4, 2019 at 4:59 PM Brice Andr? wrote: > Hello, > > It's not an open-ssl issue, but more a compiler specific one. > > With info you provided, I cannot tell you what you get as results, but two > points that may help: > > 1. regarding the 87 ssl symbols : when you link with a library, only > the useful symbols are imported. So, if the code in you libAPP library only > uses some sparse functions of libSSL, it's normal you only have > corresponding symbols in your final image. I don't know what you plan to > do, but note that statically linking your dll with open-ssl will prevent > this dll from needing the openssl dynamic library. But it will not prevent > your main program to require the open-ssl library to run properly if some > part of it is dynamically linked with open-ssl ! > 2. depending on how you compiled your libssl.a, it can be either a > static library containing the full openssl binary code, or a static library > that just makes the "link" between you code and the ssl dynamic library. In > the second case, even if you properly statically link with this lib, you > will still need the dll to execute your program. > > Regards, > Brice > > > Le lun. 4 nov. 2019 ? 07:28, Aijaz Baig a ?crit : > >> I am trying to build a shared library that internally links openssl and >> crypto libraries statically so I can use it in a production environment. To >> that end I am using the following Makefile >> >> APPBASE=/home/AB/Documents/APP/APP_2.17.0 >> OPENSSL1.0.2p_INSTALL_LOC=/home/AB/Documents/APP/OpenSSL-1.0.2p-installation >> CC=gcc >> CFLAGS= -Wall -g -O -fPIC >> RM= rm -f >> .PHONY: all clean >> >> src=$(wildcard *Generic/*.c *Linux/*.c) >> $(info source=$(src)) >> >> #we use the custom compiled openssl version >> #and NOT the one available on the system >> #INC=-I/usr/include/openssl >> INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include/openssl >> INC+=$(foreach d,$(incdir),-I$d) >> $(info includes=$(INC)) >> >> LIB=-L$(OPENSSL1.0.2p_INSTALL_LOC)/lib >> #LIB=-llibssl -llibcrypto >> LIB+=-lssl -lcrypto >> $(info links=$(LIB)) >> #LIB+=-L/usr/lib/ >> >> obj=$(src:.c=.o) >> all: libAPP.so >> clean: >> $(RM) *.o *.so >> $(shell find $(APPBASE) -type f -iname "*.o" -exec rm -rf {} \;) >> >> .c.o: >> ${CC} -static ${CFLAGS} $(INC) -c $< $(LIB) -o $@ >> >> libAPP.so: $(obj) >> $(LINK.c) -shared $^ -o $@ >> >> As mentioned here ( >> https://stackoverflow.com/questions/18185618/how-to-use-static-linking-with-openssl-in-c-c/25811538#25811538), >> I've changed the linker flags to: >> -lcrypto -lz -ldl -static-libgcc >> >> but it doesn't seem to change the size of the generated so file. On >> checking for references to SSL in this so file, I see there are a total of >> 87 entries >> >> nm libAPP.so | grep -i "ssl" | wc -l >> 87 >> >> whereas listing *only* the global symbols from libssl.a tells me it has >> 1113 globally defined symbols. >> nm -g ../OpenSSL-1.0.2p-installation/lib/libssl.a | grep -i "ssl" | wc -l >> 1113 >> >> I do not know if libSSL got indeed linked statically or not. Could >> someone please shed some light on it? >> >> -- >> >> Best Regards, >> Aijaz Baig >> > -- Best Regards, Aijaz Baig -------------- next part -------------- An HTML attachment was scrubbed... URL: From aijazbaig1 at gmail.com Tue Nov 5 17:24:06 2019 From: aijazbaig1 at gmail.com (Aijaz Baig) Date: Tue, 5 Nov 2019 22:54:06 +0530 Subject: static linking libssl and libcrypto In-Reply-To: References: Message-ID: ldd libAPP.so does NOT have any reference to libSSL.so or for that matter any SSL library. Which is why I had to use nm to check for SSL related symbols On Mon, Nov 4, 2019 at 6:31 PM Floodeenjr, Thomas < thomas_floodeenjr at mentor.com> wrote: > To check if you are linked statically or dynamically, what does ldd tell > you? (ldd libAPP.so) Your library should not have a dependency on > libssl.so or libcrypto.so if you are linked statically. > > > > -Tom > > > > *From:* openssl-users [mailto:openssl-users-bounces at openssl.org] *On > Behalf Of *Aijaz Baig > *Sent:* Sunday, November 3, 2019 11:30 PM > *To:* openssl-users at openssl.org > *Subject:* static linking libssl and libcrypto > > > > I am trying to build a shared library that internally links openssl and > crypto libraries statically so I can use it in a production environment. To > that end I am using the following Makefile > > APPBASE=/home/AB/Documents/APP/APP_2.17.0 > > OPENSSL1.0.2p_INSTALL_LOC=/home/AB/Documents/APP/OpenSSL-1.0.2p-installation > > CC=gcc > > CFLAGS= -Wall -g -O -fPIC > > RM= rm -f > > .PHONY: all clean > > > > src=$(wildcard *Generic/*.c *Linux/*.c) > > $(info source=$(src)) > > > > #we use the custom compiled openssl version > > #and NOT the one available on the system > > #INC=-I/usr/include/openssl > > INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include/openssl > > INC+=$(foreach d,$(incdir),-I$d) > > $(info includes=$(INC)) > > > > LIB=-L$(OPENSSL1.0.2p_INSTALL_LOC)/lib > > #LIB=-llibssl -llibcrypto > > LIB+=-lssl -lcrypto > > $(info links=$(LIB)) > > #LIB+=-L/usr/lib/ > > > > obj=$(src:.c=.o) > > all: libAPP.so > > clean: > > $(RM) *.o *.so > > $(shell find $(APPBASE) -type f -iname "*.o" -exec rm -rf {} \;) > > > > .c.o: > > ${CC} -static ${CFLAGS} $(INC) -c $< $(LIB) -o $@ > > > > libAPP.so: $(obj) > > $(LINK.c) -shared $^ -o $@ > > As mentioned here ( > https://stackoverflow.com/questions/18185618/how-to-use-static-linking-with-openssl-in-c-c/25811538#25811538), > I've changed the linker flags to: > > -lcrypto -lz -ldl -static-libgcc > > > > but it doesn't seem to change the size of the generated so file. On > checking for references to SSL in this so file, I see there are a total of > 87 entries > > > > nm libAPP.so | grep -i "ssl" | wc -l > > 87 > > > > whereas listing *only* the global symbols from libssl.a tells me it has > 1113 globally defined symbols. > > nm -g ../OpenSSL-1.0.2p-installation/lib/libssl.a | grep -i "ssl" | wc -l > > 1113 > > > > I do not know if libSSL got indeed linked statically or not. Could someone > please shed some light on it? > > > > -- > > > Best Regards, > > Aijaz Baig > -- Best Regards, Aijaz Baig -------------- next part -------------- An HTML attachment was scrubbed... URL: From brice at famille-andre.be Wed Nov 6 08:11:15 2019 From: brice at famille-andre.be (=?UTF-8?B?QnJpY2UgQW5kcsOp?=) Date: Wed, 6 Nov 2019 09:11:15 +0100 Subject: static linking libssl and libcrypto In-Reply-To: References: Message-ID: Then it means you properly compiled SSL in static. Regarding my first point, what I mean is that : if you statically link your libApp.so with ssl library, it will no need any dynamic library dependencies, which is probably what you want. But your libApp.so will be loaded by a program. If this program also uses SSL (direct access, not through your libApp.so library), and performs a dynamic link, you will still need the ssl.so to run your application, even if your dll is statically linked with it. Regards, Brice Le mar. 5 nov. 2019 ? 18:24, Aijaz Baig a ?crit : > ldd libAPP.so does NOT have any reference to libSSL.so or for that matter > any SSL library. Which is why I had to use nm to check for SSL related > symbols > > On Mon, Nov 4, 2019 at 6:31 PM Floodeenjr, Thomas < > thomas_floodeenjr at mentor.com> wrote: > >> To check if you are linked statically or dynamically, what does ldd tell >> you? (ldd libAPP.so) Your library should not have a dependency on >> libssl.so or libcrypto.so if you are linked statically. >> >> >> >> -Tom >> >> >> >> *From:* openssl-users [mailto:openssl-users-bounces at openssl.org] *On >> Behalf Of *Aijaz Baig >> *Sent:* Sunday, November 3, 2019 11:30 PM >> *To:* openssl-users at openssl.org >> *Subject:* static linking libssl and libcrypto >> >> >> >> I am trying to build a shared library that internally links openssl and >> crypto libraries statically so I can use it in a production environment. To >> that end I am using the following Makefile >> >> APPBASE=/home/AB/Documents/APP/APP_2.17.0 >> >> OPENSSL1.0.2p_INSTALL_LOC=/home/AB/Documents/APP/OpenSSL-1.0.2p-installation >> >> CC=gcc >> >> CFLAGS= -Wall -g -O -fPIC >> >> RM= rm -f >> >> .PHONY: all clean >> >> >> >> src=$(wildcard *Generic/*.c *Linux/*.c) >> >> $(info source=$(src)) >> >> >> >> #we use the custom compiled openssl version >> >> #and NOT the one available on the system >> >> #INC=-I/usr/include/openssl >> >> INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include/openssl >> >> INC+=$(foreach d,$(incdir),-I$d) >> >> $(info includes=$(INC)) >> >> >> >> LIB=-L$(OPENSSL1.0.2p_INSTALL_LOC)/lib >> >> #LIB=-llibssl -llibcrypto >> >> LIB+=-lssl -lcrypto >> >> $(info links=$(LIB)) >> >> #LIB+=-L/usr/lib/ >> >> >> >> obj=$(src:.c=.o) >> >> all: libAPP.so >> >> clean: >> >> $(RM) *.o *.so >> >> $(shell find $(APPBASE) -type f -iname "*.o" -exec rm -rf {} \;) >> >> >> >> .c.o: >> >> ${CC} -static ${CFLAGS} $(INC) -c $< $(LIB) -o $@ >> >> >> >> libAPP.so: $(obj) >> >> $(LINK.c) -shared $^ -o $@ >> >> As mentioned here ( >> https://stackoverflow.com/questions/18185618/how-to-use-static-linking-with-openssl-in-c-c/25811538#25811538), >> I've changed the linker flags to: >> >> -lcrypto -lz -ldl -static-libgcc >> >> >> >> but it doesn't seem to change the size of the generated so file. On >> checking for references to SSL in this so file, I see there are a total of >> 87 entries >> >> >> >> nm libAPP.so | grep -i "ssl" | wc -l >> >> 87 >> >> >> >> whereas listing *only* the global symbols from libssl.a tells me it has >> 1113 globally defined symbols. >> >> nm -g ../OpenSSL-1.0.2p-installation/lib/libssl.a | grep -i "ssl" | wc -l >> >> 1113 >> >> >> >> I do not know if libSSL got indeed linked statically or not. Could >> someone please shed some light on it? >> >> >> >> -- >> >> >> Best Regards, >> >> Aijaz Baig >> > > > -- > > Best Regards, > Aijaz Baig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jb-openssl at wisemo.com Wed Nov 6 08:38:20 2019 From: jb-openssl at wisemo.com (Jakob Bohm) Date: Wed, 6 Nov 2019 09:38:20 +0100 Subject: static linking libssl and libcrypto In-Reply-To: References: Message-ID: <30055b09-ccc0-1d15-76fa-b606ff0a656f@wisemo.com> Regarding #1: Using libSSL.a instead of libSSL.so should avoid using libSSL.so by definition.? Otherwise something went seriously wrong with the linking.? Same for any other library. On 05/11/2019 18:22, Aijaz Baig wrote: > Thank you for the information. > > I will address your points here: > 1. I was not aware of the fact that only those symbols that have been > used get imported when linking a library statically. So that very well > could be the case. I didn't get what you mentioned about the static > linking preventing the program from requiring libSSL.so. I mean the > way I am linking my library should be of no concern to the source code > right? Or so I think. > > 2. when I downloaded and compiled the openssl library (from source), I > followed the INSTALL read me. All it resulted was libssl.a and > libcrypto.a. I didn't find any file name libSSL.so. So how will this > static library (archive) have references to libSSL.so on the system?? > I am kind of confused here now. > > > On Mon, Nov 4, 2019 at 4:59 PM Brice Andr? > wrote: > > Hello, > > It's not an open-ssl issue, but more a compiler specific one. > > With info you provided, I cannot tell you what you get as results, > but two points that may help: > > 1. regarding the 87 ssl symbols : when you link with a library, > only the useful symbols are imported. So, if the code in you > libAPP library only uses some sparse functions of libSSL, it's > normal you only have corresponding symbols in your final > image. I don't know what you plan to do, but note that > statically linking your dll with open-ssl will prevent this > dll from needing the openssl dynamic library. But it will not > prevent your main program to require the open-ssl library to > run properly if some part of it is dynamically linked with > open-ssl ! > 2. depending on how you compiled your libssl.a, it can be either > a static library containing the full openssl binary code, or a > static library that just makes the "link" between you code and > the ssl dynamic library. In the second case, even if you > properly statically link with this lib, you will still need > the dll to execute your program. > > Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 S?borg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded From brice at famille-andre.be Wed Nov 6 08:57:17 2019 From: brice at famille-andre.be (=?UTF-8?B?QnJpY2UgQW5kcsOp?=) Date: Wed, 6 Nov 2019 09:57:17 +0100 Subject: static linking libssl and libcrypto In-Reply-To: <30055b09-ccc0-1d15-76fa-b606ff0a656f@wisemo.com> References: <30055b09-ccc0-1d15-76fa-b606ff0a656f@wisemo.com> Message-ID: Sorry, but you are wrong : using libSSL.a in your libApp.so library does not prevent you from using libSSL.so elsewhere in your program. In such case, your program would still need libSSL.so Le mer. 6 nov. 2019 ? 09:38, Jakob Bohm via openssl-users < openssl-users at openssl.org> a ?crit : > Regarding #1: Using libSSL.a instead of libSSL.so should avoid using > libSSL.so by definition. Otherwise something went seriously wrong > with the linking. Same for any other library. > > On 05/11/2019 18:22, Aijaz Baig wrote: > > Thank you for the information. > > > > I will address your points here: > > 1. I was not aware of the fact that only those symbols that have been > > used get imported when linking a library statically. So that very well > > could be the case. I didn't get what you mentioned about the static > > linking preventing the program from requiring libSSL.so. I mean the > > way I am linking my library should be of no concern to the source code > > right? Or so I think. > > > > 2. when I downloaded and compiled the openssl library (from source), I > > followed the INSTALL read me. All it resulted was libssl.a and > > libcrypto.a. I didn't find any file name libSSL.so. So how will this > > static library (archive) have references to libSSL.so on the system?? > > I am kind of confused here now. > > > > > > On Mon, Nov 4, 2019 at 4:59 PM Brice Andr? > > wrote: > > > > Hello, > > > > It's not an open-ssl issue, but more a compiler specific one. > > > > With info you provided, I cannot tell you what you get as results, > > but two points that may help: > > > > 1. regarding the 87 ssl symbols : when you link with a library, > > only the useful symbols are imported. So, if the code in you > > libAPP library only uses some sparse functions of libSSL, it's > > normal you only have corresponding symbols in your final > > image. I don't know what you plan to do, but note that > > statically linking your dll with open-ssl will prevent this > > dll from needing the openssl dynamic library. But it will not > > prevent your main program to require the open-ssl library to > > run properly if some part of it is dynamically linked with > > open-ssl ! > > 2. depending on how you compiled your libssl.a, it can be either > > a static library containing the full openssl binary code, or a > > static library that just makes the "link" between you code and > > the ssl dynamic library. In the second case, even if you > > properly statically link with this lib, you will still need > > the dll to execute your program. > > > > > > Enjoy > > Jakob > -- > Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com > Transformervej 29, 2860 S?borg, Denmark. Direct +45 31 13 16 10 > This public discussion message is non-binding and may contain errors. > WiseMo - Remote Service Management for PCs, Phones and Embedded > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cauldwell.thomas at gmail.com Wed Nov 6 10:18:21 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Wed, 6 Nov 2019 10:18:21 -0000 (UTC) Subject: ssh-keygen freezes if you force use of engine Message-ID: I have edited the source code for OpenSSL in the init functions so that the config file always gets loaded. My config file specifies a custom engine, so this means that every process that links with "libssl.so" or "libcrypto.so" will use my custom engine. The problem I'm having is that my device won't boot up. It gets as far as running "ssh-keygen" and then it freezes. In order to troubleshoot this, I recompiled "ssh-keygen" with the flag "- rdynamic" so that I can see function names in a stack trace. Next I enabled core dumps with "ulimit -c unlimited". Then I ran "ssh-keygen", and it froze as expected, but this time I pressed "Ctrl + \", and so it produced a core dump file. I used GDB on the core file and saw the following backtrace: Core was generated by `/home/service/ssh-keygen'. Program terminated with signal SIGQUIT, Quit. #0 0x00007f656e35e3c7 in __pthread_once_slow () from /lib/libpthread.so.0 (gdb) bt #0 0x00007f656e35e3c7 in __pthread_once_slow () from /lib/libpthread.so.0 #1 0x00007f656f730fbe in CRYPTO_THREAD_run_once () from ./libcrypto.so.1.1 #2 0x00007f656f6d5c80 in OPENSSL_init_crypto () from ./libcrypto.so.1.1 #3 0x00007f656f66163b in openssl_config_int () from ./libcrypto.so.1.1 #4 0x00007f656f6d56cf in ossl_init_config_ossl_ () from ./libcrypto.so.1.1 #5 0x00007f656e35e407 in __pthread_once_slow () from /lib/libpthread.so.0 #6 0x00007f656f730fbe in CRYPTO_THREAD_run_once () from ./libcrypto.so.1.1 #7 0x00007f656f6d5c80 in OPENSSL_init_crypto () from ./libcrypto.so.1.1 #8 0x000055a8d125708b in main () Can anyone see what might be going on here? From aijazbaig1 at gmail.com Wed Nov 6 10:28:01 2019 From: aijazbaig1 at gmail.com (Aijaz Baig) Date: Wed, 6 Nov 2019 15:58:01 +0530 Subject: static linking libssl and libcrypto In-Reply-To: References: Message-ID: Ok. Got it. Thanks On Wed, Nov 6, 2019 at 1:41 PM Brice Andr? wrote: > Then it means you properly compiled SSL in static. > > Regarding my first point, what I mean is that : if you statically link > your libApp.so with ssl library, it will no need any dynamic library > dependencies, which is probably what you want. But your libApp.so will be > loaded by a program. If this program also uses SSL (direct access, not > through your libApp.so library), and performs a dynamic link, you will > still need the ssl.so to run your application, even if your dll is > statically linked with it. > > Regards, > Brice > > > Le mar. 5 nov. 2019 ? 18:24, Aijaz Baig a ?crit : > >> ldd libAPP.so does NOT have any reference to libSSL.so or for that matter >> any SSL library. Which is why I had to use nm to check for SSL related >> symbols >> >> On Mon, Nov 4, 2019 at 6:31 PM Floodeenjr, Thomas < >> thomas_floodeenjr at mentor.com> wrote: >> >>> To check if you are linked statically or dynamically, what does ldd tell >>> you? (ldd libAPP.so) Your library should not have a dependency on >>> libssl.so or libcrypto.so if you are linked statically. >>> >>> >>> >>> -Tom >>> >>> >>> >>> *From:* openssl-users [mailto:openssl-users-bounces at openssl.org] *On >>> Behalf Of *Aijaz Baig >>> *Sent:* Sunday, November 3, 2019 11:30 PM >>> *To:* openssl-users at openssl.org >>> *Subject:* static linking libssl and libcrypto >>> >>> >>> >>> I am trying to build a shared library that internally links openssl and >>> crypto libraries statically so I can use it in a production environment. To >>> that end I am using the following Makefile >>> >>> APPBASE=/home/AB/Documents/APP/APP_2.17.0 >>> >>> OPENSSL1.0.2p_INSTALL_LOC=/home/AB/Documents/APP/OpenSSL-1.0.2p-installation >>> >>> CC=gcc >>> >>> CFLAGS= -Wall -g -O -fPIC >>> >>> RM= rm -f >>> >>> .PHONY: all clean >>> >>> >>> >>> src=$(wildcard *Generic/*.c *Linux/*.c) >>> >>> $(info source=$(src)) >>> >>> >>> >>> #we use the custom compiled openssl version >>> >>> #and NOT the one available on the system >>> >>> #INC=-I/usr/include/openssl >>> >>> INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include/openssl >>> >>> INC+=$(foreach d,$(incdir),-I$d) >>> >>> $(info includes=$(INC)) >>> >>> >>> >>> LIB=-L$(OPENSSL1.0.2p_INSTALL_LOC)/lib >>> >>> #LIB=-llibssl -llibcrypto >>> >>> LIB+=-lssl -lcrypto >>> >>> $(info links=$(LIB)) >>> >>> #LIB+=-L/usr/lib/ >>> >>> >>> >>> obj=$(src:.c=.o) >>> >>> all: libAPP.so >>> >>> clean: >>> >>> $(RM) *.o *.so >>> >>> $(shell find $(APPBASE) -type f -iname "*.o" -exec rm -rf {} \;) >>> >>> >>> >>> .c.o: >>> >>> ${CC} -static ${CFLAGS} $(INC) -c $< $(LIB) -o $@ >>> >>> >>> >>> libAPP.so: $(obj) >>> >>> $(LINK.c) -shared $^ -o $@ >>> >>> As mentioned here ( >>> https://stackoverflow.com/questions/18185618/how-to-use-static-linking-with-openssl-in-c-c/25811538#25811538), >>> I've changed the linker flags to: >>> >>> -lcrypto -lz -ldl -static-libgcc >>> >>> >>> >>> but it doesn't seem to change the size of the generated so file. On >>> checking for references to SSL in this so file, I see there are a total of >>> 87 entries >>> >>> >>> >>> nm libAPP.so | grep -i "ssl" | wc -l >>> >>> 87 >>> >>> >>> >>> whereas listing *only* the global symbols from libssl.a tells me it has >>> 1113 globally defined symbols. >>> >>> nm -g ../OpenSSL-1.0.2p-installation/lib/libssl.a | grep -i "ssl" | wc >>> -l >>> >>> 1113 >>> >>> >>> >>> I do not know if libSSL got indeed linked statically or not. Could >>> someone please shed some light on it? >>> >>> >>> >>> -- >>> >>> >>> Best Regards, >>> >>> Aijaz Baig >>> >> >> >> -- >> >> Best Regards, >> Aijaz Baig >> > -- Best Regards, Aijaz Baig -------------- next part -------------- An HTML attachment was scrubbed... URL: From cauldwell.thomas at gmail.com Wed Nov 6 11:03:45 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Wed, 6 Nov 2019 11:03:45 -0000 (UTC) Subject: ssh-keygen freezes if you force use of engine References: Message-ID: > I used GDB on the core file and saw the following backtrace: I think I'm actually getting a stack overflow from "openssl_config_int" calling "OPENSSL_init_crypto" because of the recursion. From beldmit at gmail.com Wed Nov 6 12:17:58 2019 From: beldmit at gmail.com (Dmitry Belyavsky) Date: Wed, 6 Nov 2019 15:17:58 +0300 Subject: ssh-keygen freezes if you force use of engine In-Reply-To: References: Message-ID: Could you please reproduce this behavior without your patches? On Wed, Nov 6, 2019 at 2:04 PM Frederick Gotham wrote: > > > > I used GDB on the core file and saw the following backtrace: > > > I think I'm actually getting a stack overflow from "openssl_config_int" > calling "OPENSSL_init_crypto" because of the recursion. > > -- SY, Dmitry Belyavsky -------------- next part -------------- An HTML attachment was scrubbed... URL: From cauldwell.thomas at gmail.com Wed Nov 6 12:35:16 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Wed, 6 Nov 2019 12:35:16 -0000 (UTC) Subject: ssh-keygen freezes if you force use of engine References: Message-ID: Dmitry Belyavsky wrote in news:CADqLbz+6WbCpday_1Gq-Zmst1y-17Hbc1S9OvYtE4=b=NJpW5w at mail.gmail.com: > Could you please reproduce this behavior without your patches? > > On Wed, Nov 6, 2019 at 2:04 PM Frederick Gotham > wrote: > >> >> >> > I used GDB on the core file and saw the following backtrace: >> >> >> I think I'm actually getting a stack overflow from "openssl_config_int" >> calling "OPENSSL_init_crypto" because of the recursion. >> >> > My temporary solution is to link "ssh-keygen" statically with a normal version of "libcrypto.a", and then have every other program link dynamically with my altered version of "libcrypto.so". I'll at least see if that works. From cauldwell.thomas at gmail.com Wed Nov 6 12:52:18 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Wed, 6 Nov 2019 12:52:18 -0000 (UTC) Subject: ssh-keygen freezes if you force use of engine References: Message-ID: > My temporary solution is to link "ssh-keygen" statically with a normal > version of "libcrypto.a", and then have every other program link > dynamically with my altered version of "libcrypto.so". > > I'll at least see if that works. Yeah the problem is **definitely** that the function "OPENSSL_init_crypto" is calling itself recursively. I might have to put in a thread-duration variable to keep track of the stack depth, and only tell it to load the config file on the first call. I also tried playing around with OPENSSL_SSL_CLIENT_ENGINE_AUTO at compile time, but I can't seem to get it to work. Here's my latest stack trace when I link statically with libcrypto.a: OPENSSL_init_crypto, with the following backtrace: ================== FUNCTION CALL STACK BACKTRACE ================== /usr/lib/libcrypto.so.1.1(OPENSSL_init_crypto+0xb3) [0x7fce24d6ca6a] /usr/lib/libcrypto.so.1.1(+0x143126) [0x7fce24d6a126] /lib/libpthread.so.0(+0xf407) [0x7fce2460c407] /usr/lib/libcrypto.so.1.1(CRYPTO_THREAD_run_once+0x9) [0x7fce24dc7fbe] /usr/lib/libcrypto.so.1.1(+0x14317c) [0x7fce24d6a17c] /usr/lib/libcrypto.so.1.1(CRYPTO_new_ex_data+0x1b) [0x7fce24d6a4d3] /usr/lib/libcrypto.so.1.1(BIO_new+0x73) [0x7fce24cc8d37] /usr/lib/libcrypto.so.1.1(BIO_new_fp+0x16) [0x7fce24ccca06] openssl() [0x41fe6e] /lib/libc.so.6(__libc_start_main+0xe7) [0x7fce2426ae77] openssl() [0x42019a] =================================================================== OPENSSL_init_crypto, with the following backtrace: ================== FUNCTION CALL STACK BACKTRACE ================== /usr/lib/libcrypto.so.1.1(OPENSSL_init_crypto+0xb3) [0x7fce24d6ca6a] /usr/lib/libcrypto.so.1.1(+0xd163b) [0x7fce24cf863b] /usr/lib/libcrypto.so.1.1(+0x1456cf) [0x7fce24d6c6cf] /lib/libpthread.so.0(+0xf407) [0x7fce2460c407] /usr/lib/libcrypto.so.1.1(CRYPTO_THREAD_run_once+0x9) [0x7fce24dc7fbe] /usr/lib/libcrypto.so.1.1(OPENSSL_init_crypto+0x2c9) [0x7fce24d6cc80] /usr/lib/libcrypto.so.1.1(+0x143126) [0x7fce24d6a126] /lib/libpthread.so.0(+0xf407) [0x7fce2460c407] /usr/lib/libcrypto.so.1.1(CRYPTO_THREAD_run_once+0x9) [0x7fce24dc7fbe] /usr/lib/libcrypto.so.1.1(+0x14317c) [0x7fce24d6a17c] /usr/lib/libcrypto.so.1.1(CRYPTO_new_ex_data+0x1b) [0x7fce24d6a4d3] /usr/lib/libcrypto.so.1.1(BIO_new+0x73) [0x7fce24cc8d37] /usr/lib/libcrypto.so.1.1(BIO_new_fp+0x16) [0x7fce24ccca06] openssl() [0x41fe6e] /lib/libc.so.6(__libc_start_main+0xe7) [0x7fce2426ae77] openssl() [0x42019a] From cauldwell.thomas at gmail.com Wed Nov 6 16:08:09 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Wed, 6 Nov 2019 16:08:09 -0000 (UTC) Subject: Latest idea to force usage of custom engine Message-ID: I have abandoned my old idea of trying to get libOpenSSL to always read my engine info from the config file (/etc/ssl/openssl.cnf). Instead I'm going to try to manually load my own engine in the source code for libOpenSSL. I have created a new function in "rand_lib.c" as follows: static void Do_Whatever_Needs_Done_For_TPM2(void) { CRYPTO_THREAD_write_lock(rand_meth_lock); { /* Initialize the dynamic engine loader */ ENGINE_load_dynamic(); ENGINE *tpm2_engine; /* If the first fails, try the second one */ (tpm2_engine = ENGINE_by_id("tpm2tss")) || (tpm2_engine = ENGINE_by_id("libtpm2tss")); if ( NULL == tpm2_engine ) { assert( 0 == "Do_Whatever_Needs_Done_For_TPM2: tpm2 _engine == NULL" ); abort(); } int init_res = ENGINE_init(tpm2_engine); if ( !init_res ) { assert( 0 == "Do_Whatever_Needs_Done_For_TPM2: init_res == 0" ); abort(); } RAND_METHOD const *const p_rm = ENGINE_get_RAND(tpm2_engine); if ( NULL == p_rm ) { assert( 0 == "Do_Whatever_Needs_Done_For_TPM2: p_rm == NULL" ); abort(); } funct_ref = tpm2_engine; default_RAND_meth = p_rm; rand_meth = *default_RAND_meth; /* Even set the Drbg func pointers */ } CRYPTO_THREAD_unlock(rand_meth_lock); } And then the next thing I've done is added the following code to the start of "RAND_get_rand_method": const RAND_METHOD *RAND_get_rand_method(void) { static int first_time_for_entire_process = 1; if ( first_time_for_entire_process ) { first_time_for_entire_process = 0; Do_Whatever_Needs_Done_For_TPM2(); } /* The rest of the function goes here */ } I'm testing it now but it still seems that something isn't right. I'm going to keep at this until every process (including ssh-keygen) is using my custom engine for random numbers. From openssl at jordan.maileater.net Wed Nov 6 16:34:04 2019 From: openssl at jordan.maileater.net (Jordan Brown) Date: Wed, 6 Nov 2019 16:34:04 +0000 Subject: ssh-keygen freezes if you force use of engine In-Reply-To: References: Message-ID: <0101016e4190e072-0ae2e77d-6c2b-4779-8942-38a82cfbc35d-000000@us-west-2.amazonses.com> On 11/6/2019 3:03 AM, Frederick Gotham wrote: >> I used GDB on the core file and saw the following backtrace: > > I think I'm actually getting a stack overflow from "openssl_config_int" > calling "OPENSSL_init_crypto" because of the recursion. No, a stack overflow would be much deeper. > #0 0x00007f656e35e3c7 in __pthread_once_slow () from /lib/libpthread.so.0 > (gdb) bt > #0 0x00007f656e35e3c7 in __pthread_once_slow () from /lib/libpthread.so.0 > #1 0x00007f656f730fbe in CRYPTO_THREAD_run_once () from ./libcrypto.so.1.1 > #2 0x00007f656f6d5c80 in OPENSSL_init_crypto () from ./libcrypto.so.1.1 > #3 0x00007f656f66163b in openssl_config_int () from ./libcrypto.so.1.1 > #4 0x00007f656f6d56cf in ossl_init_config_ossl_ () from ./libcrypto.so.1.1 > #5 0x00007f656e35e407 in __pthread_once_slow () from /lib/libpthread.so.0 > #6 0x00007f656f730fbe in CRYPTO_THREAD_run_once () from ./libcrypto.so.1.1 > #7 0x00007f656f6d5c80 in OPENSSL_init_crypto () from ./libcrypto.so.1.1 > #8 0x000055a8d125708b in main () Note that __pthread_once_slow is in the stack twice, called from OPENSSL_init_crypto and CRYPTO_THREAD_run_once. "once" functions ensure that they call their function argument exactly once, even if they are called multiple times in parallel while their function is running.? They do that by locking a mutex around the execution of the function.? The second call attempted to lock the mutex to wait for the first call to complete, and deadlocked. -- Jordan Brown, Oracle ZFS Storage Appliance, Oracle Solaris -------------- next part -------------- An HTML attachment was scrubbed... URL: From cauldwell.thomas at gmail.com Thu Nov 7 07:42:33 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Thu, 7 Nov 2019 07:42:33 -0000 (UTC) Subject: ssh-keygen freezes if you force use of engine References: <0101016e4190e072-0ae2e77d-6c2b-4779-8942-38a82cfbc35d-000000@us-west-2.amazonses.com> Message-ID: Jordan Brown wrote: > Note that __pthread_once_slow is in the stack twice, called from > OPENSSL_init_crypto and CRYPTO_THREAD_run_once. > > "once" functions ensure that they call their function argument exactly > once, even if they are called multiple times in parallel while their > function is running.?? They do that by locking a mutex around the > execution of the function.?? The second call attempted to lock the mutex > to wait for the first call to complete, and deadlocked. I still don't know what to do about this. For them moment I'm continuing with my idea in the thread "Latest idea to force usage of custom engine". From cauldwell.thomas at gmail.com Thu Nov 7 08:44:18 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Thu, 7 Nov 2019 08:44:18 -0000 (UTC) Subject: Latest idea to force usage of custom engine References: Message-ID: > I'm going to keep at this until every process (including ssh-keygen) is > using my custom engine for random numbers. Now every process that links with 'libcrypto.so' is segfaulting in libpthread-2.28.so. Here's the error: openssl[1744]: segfault at 18 ip 00007f1798d44930 sp 00007ffcf46bb1b8 error 4 in libpthread-2.28.so[7f1798d38000+1a000] Here's the totality of the changes I've made. . . I added the following to "rand_lib.c": /* =================== The fields in RAND_METHOD ====================== int (*seed) (const void *buf, int num); int (*bytes) (unsigned char *buf, int num); void (*cleanup) (void); int (*add) (const void *buf, int num, double randomness); int (*pseudorand) (unsigned char *buf, int num); int (*status) (void); ==================================================================== */ #include static int Supplemented_seed(const void *buf, int num) { (void)buf; (void) num; return 1; } static void Supplemented_cleanup(void) { /* Do Nothing */ return; } static int Supplemented_add(const void *buf, int num, double randomness) { (void)buf; (void)num; (void)randomness; return 1; } static int Supplemented_status(void) { /* Do Nothing */ return 1; } static CRYPTO_ONCE crypto_once_tpm2_init = CRYPTO_ONCE_STATIC_INIT; DEFINE_RUN_ONCE_STATIC(Do_Whatever_Needs_Done_For_TPM2) { /* Initialize the dynamic engine loader */ ENGINE_load_dynamic(); ENGINE *tpm2_engine; /* If the first fails, try the second one */ (tpm2_engine = ENGINE_by_id("tpm2tss")) || (tpm2_engine = ENGINE_by_id("libtpm2tss")); if ( NULL == tpm2_engine ) { assert( 0 == "Do_Whatever_Needs_Done_For_TPM2: NULL == tpm2 _engine" ); abort(); } int const init_res = ENGINE_init(tpm2_engine); if ( 0 == init_res ) { assert( 0 == "Do_Whatever_Needs_Done_For_TPM2: 0 == init_res" ); abort(); } RAND_METHOD const *const p_rm = ENGINE_get_RAND(tpm2_engine); if ( NULL == p_rm ) { assert( 0 == "Do_Whatever_Needs_Done_For_TPM2: NULL == p_rm" ); abort(); } static RAND_METHOD funcptrs = { Supplemented_seed, /* seed() */ 0, /* bytes() */ Supplemented_cleanup, /* cleanup() */ Supplemented_add, /* add() */ 0, /* pseudorand() */ Supplemented_status /* status() */ }; CRYPTO_THREAD_write_lock(rand_meth_lock); funcptrs.bytes = p_rm->bytes; funcptrs.pseudorand = p_rm->pseudorand; funct_ref = tpm2_engine; default_RAND_meth = &funcptrs; rand_meth = funcptrs; /* Even set the Drbg func pointers */ CRYPTO_THREAD_unlock(rand_meth_lock); return 0; } const RAND_METHOD *RAND_get_rand_method(void) { RUN_ONCE(&crypto_once_tpm2_init, Do_Whatever_Needs_Done_For_TPM2); From cauldwell.thomas at gmail.com Thu Nov 7 09:44:18 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Thu, 7 Nov 2019 09:44:18 -0000 (UTC) Subject: Latest idea to force usage of custom engine References: Message-ID: Frederick wrote: > > CRYPTO_THREAD_write_lock(rand_meth_lock); > CRYPTO_THREAD_write_lock(rand_engine_lock); This is what's causing the segfault in libpthread.so I can only imagine that the same thread is trying to re-lock a single-lock mutex. So my code should be something like: int already_locked_meth = is_already_locked(rand_meth_lock); int already_locked_engine = is_already_locked(rand_engine_lock); if (!already_locked_meth) CRYPTO_THREAD_write_lock(rand_meth_lock); if (!already_locked_engine) CRYPTO_THREAD_write_lock(rand_engine_lock); /* DO STUFF */ if (!already_locked_meth) CRYPTO_THREAD_unlock(rand_meth_lock); if (!already_locked_engine) CRYPTO_THREAD_unlock(rand_engine_lock); From cauldwell.thomas at gmail.com Thu Nov 7 13:39:03 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Thu, 7 Nov 2019 13:39:03 -0000 (UTC) Subject: SOLVED: Force use of engine all the time Message-ID: I know I've called this "solved", but I've achieved my objective about 98% so there's only a little bit left to go. My changes to the source code for LibOpenSSL are confined to the file "rand_lib.c". Firstly, I've added the following code: #include #include #include static int Dummy__seed(const void *buf, int num) { (void)buf; (void)num; return 1; } static void Dummy__cleanup(void) { /* Do Nothing */ return; } static int Dummy__add(const void *buf, int num, double randomness) { (void)buf; (void)num; (void)randomness; return 1; } static int Dummy__status(void) { /* Do Nothing */ return 1; } static void Do_Whatever_Needs_Done_For_TPM2(void) { CRYPTO_THREAD_write_lock(rand_meth_lock); CRYPTO_THREAD_write_lock(rand_engine_lock); /* Initialize the dynamic engine loader */ ENGINE_load_dynamic(); ENGINE *tpm2_engine; /* If the first fails, try the second one */ (tpm2_engine = ENGINE_by_id("tpm2tss")) || (tpm2_engine = ENGINE_by_id("libtpm2tss")); if ( NULL == tpm2_engine ) { assert( 0 == "Do_Whatever_Needs_Done_For_TPM2: NULL == ENGINE_by_id" ); abort(); } # define TPM2TSS_SET_TCTI (ENGINE_CMD_BASE + 1) if ( 0 == ENGINE_ctrl(tpm2_engine, TPM2TSS_SET_TCTI, 0, "device", NULL) ) { assert( 0 == "Do_Whatever_Needs_Done_For_TPM2: 0 == ENGINE_ctrl(tcti:device)" ); abort(); } int const init_res = ENGINE_init(tpm2_engine); if ( 0 == init_res ) { assert( 0 == "Do_Whatever_Needs_Done_For_TPM2: 0 == ENGINE_init" ); abort(); } RAND_METHOD const *const p_rm = ENGINE_get_RAND(tpm2_engine); if ( NULL == p_rm ) { assert( 0 == "Do_Whatever_Needs_Done_For_TPM2: NULL == ENGINE_get_RAND" ); abort(); } static RAND_METHOD funcptrs = { Dummy__seed, /* seed() */ 0, /* bytes() */ Dummy__cleanup, /* cleanup() */ Dummy__add, /* add() */ 0, /* pseudorand() */ Dummy__status /* status() */ }; funcptrs.bytes = p_rm->bytes; //rand_meth.bytes; funcptrs.pseudorand = p_rm->bytes; //rand_meth.pseudorand; funct_ref = tpm2_engine; default_RAND_meth = &funcptrs; rand_meth = funcptrs; /* Even set the Drbg func pointers */ /* DON'T DO THIS - - - ENGINE_set_default(tpm2_engine, ENGINE_METHOD_RAND); */ CRYPTO_THREAD_unlock(rand_meth_lock); CRYPTO_THREAD_unlock(rand_engine_lock); } And then I add one line to the funciton "do_rand_init": Do_Whatever_Needs_Done_For_TPM2(); So my device starts up just fine, and the first thing I notice is that it takes a little longer to boot up (maybe because it takes a long longer to get random numbers from hardware?). So, when I do the following at the command line: openssl rand -hex 128 It sometimes gives me back 128 bytes immediately, although sometimes it gives this: ERROR:tcti:src/tss2-tcti/tcti-device.c:319:Tss2_Tcti_Device_Init() Failed to open device file /dev/tpm0: Device or resource busy WARNING:esys:src/tss2-esys/esys_tcti_default.c:83:tcti_from_init() TCTI init for function 0x7f6528b376f6 failed with a000a WARNING:esys:src/tss2-esys/esys_tcti_default.c:113:tcti_from_info() Could not initialize TCTI named: tcti-device ERROR:esys:src/tss2-esys/esys_tcti_default.c:150:tcti_from_file() Could not initialize TCTI file: libtss2-tcti-default.so WARNING:esys:src/tss2-esys/esys_tcti_default.c:137:tcti_from_file() Could not load TCTI file: libtss2-tcti-tabrmd.so In order to try get around this problem of more than one thread (or process) acessing /dev/tpm0 simultaneously, I added a global named mutex to my code, and while it does improve things, it doesn't irradicate the problem completely. I don't think any code other than "libcrypto.so" is using the TPM2, and so I don't know why I'm getting "device or resource busy" (considering I'm using a global named mutex)". I see that there's also a TPM2 tcti called "abrmd", and I'm not sure if this purpose is exactly what it's for. Any ideas? From cauldwell.thomas at gmail.com Thu Nov 7 15:39:11 2019 From: cauldwell.thomas at gmail.com (Frederick Gotham) Date: Thu, 7 Nov 2019 15:39:11 -0000 (UTC) Subject: SOLVED: Force use of engine all the time References: Message-ID: Frederick Gotham wrote: > I don't think any code other than "libcrypto.so" is using the TPM2, and so > I don't know why I'm getting "device or resource busy" (considering I'm > using a global named mutex)". I see that there's also a TPM2 tcti called > "abrmd", and I'm not sure if this purpose is exactly what it's for. I tweaked my global mutex a bit and now it's working fine. From kgoldman at us.ibm.com Thu Nov 7 16:04:20 2019 From: kgoldman at us.ibm.com (Kenneth Goldman) Date: Thu, 7 Nov 2019 11:04:20 -0500 Subject: SOLVED: Force use of engine all the time In-Reply-To: References: Message-ID: From: Frederick Gotham To: openssl-users at openssl.org Date: 11/07/2019 10:39 AM Subject: [EXTERNAL] Re: SOLVED: Force use of engine all the time Sent by: "openssl-users" Frederick Gotham wrote: > I don't think any code other than "libcrypto.so" is using the TPM2, and so > I don't know why I'm getting "device or resource busy" (considering I'm > using a global named mutex)". I see that there's also a TPM2 tcti called > "abrmd", and I'm not sure if this purpose is exactly what it's for. Just FYI, abrmd was a TPM 2.0 user space resource manager, similar to the TPM 1.2 tcsd. Recent Linux kernels have a kernel driver resource manager at /dev/tpmrm0 that removes the need for a user space daemon. /dev/tpmrm0 supports multiple opens and does the swapping and scheduling. /dev/tpm0 is the original single open entry point, and would not normally be used. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: From openssl at jordan.maileater.net Thu Nov 7 18:20:53 2019 From: openssl at jordan.maileater.net (Jordan Brown) Date: Thu, 7 Nov 2019 18:20:53 +0000 Subject: ssh-keygen freezes if you force use of engine In-Reply-To: References: <0101016e4190e072-0ae2e77d-6c2b-4779-8942-38a82cfbc35d-000000@us-west-2.amazonses.com> Message-ID: <0101016e47190970-0ea8cb39-c64a-41f7-bcac-08946a2e5331-000000@us-west-2.amazonses.com> On 11/6/2019 11:42 PM, Frederick Gotham wrote: > Jordan Brown wrote: > >> Note that __pthread_once_slow is in the stack twice, called from >> OPENSSL_init_crypto and CRYPTO_THREAD_run_once. >> >> "once" functions ensure that they call their function argument exactly >> once, even if they are called multiple times in parallel while their >> function is running.? They do that by locking a mutex around the >> execution of the function.? The second call attempted to lock the mutex >> to wait for the first call to complete, and deadlocked. > > > I still don't know what to do about this. > > For them moment I'm continuing with my idea in the thread "Latest idea to > force usage of custom engine". Ensure that you don't end up calling the initialization functions recursively. I'm not really in a position to help in any detail, either in terms of expertise or time. -- Jordan Brown, Oracle ZFS Storage Appliance, Oracle Solaris -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Thu Nov 7 22:37:05 2019 From: matt at openssl.org (Matt Caswell) Date: Thu, 7 Nov 2019 22:37:05 +0000 Subject: OpenSSL Blog Post Message-ID: <25f6721c-5e60-b392-2a73-0a4d682746de@openssl.org> Please take a look at my blog post that gives an update on OpenSSL 3.0 development, FIPS and 1.0.2 EOL: https://www.openssl.org/blog/blog/2019/11/07/3.0-update/ Matt From shivakumar2696 at gmail.com Sun Nov 10 14:47:51 2019 From: shivakumar2696 at gmail.com (shiva kumar) Date: Sun, 10 Nov 2019 20:17:51 +0530 Subject: dsaparam error OpenSSL 1.1.1d Message-ID: Hi, I'm using OpenSSL 1.1.1d, when I tried executing the dsaparam command in command line as shown below it takes long time and I won't get output, it just keeps on processing. Is this an error? what is causing the problem? *openssl dsaparam 128 -rand file* *openssl dsaparam -rand file* -- *With Best Regards* *Shivakumar S* -------------- next part -------------- An HTML attachment was scrubbed... URL: From shivakumar2696 at gmail.com Mon Nov 11 05:29:52 2019 From: shivakumar2696 at gmail.com (shiva kumar) Date: Mon, 11 Nov 2019 10:59:52 +0530 Subject: dsaparam error OpenSSL 1.1.1d In-Reply-To: References: Message-ID: Hi, can anyone please help me to solve this issue? *openssl dsaparam 128 -rand file * is taking long time processing the command and not producing any output. On Sun, Nov 10, 2019 at 8:17 PM shiva kumar wrote: > Hi, > I'm using OpenSSL 1.1.1d, when I tried executing the dsaparam command in > command line as shown below it takes long time and I won't get output, it > just keeps on processing. Is this an error? what is causing the problem? > > *openssl dsaparam 128 -rand file* > > *openssl dsaparam -rand file* > > > -- > *With Best Regards* > *Shivakumar S* > -- *With Best Regards* *Shivakumar S* -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Mon Nov 11 05:42:59 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Mon, 11 Nov 2019 00:42:59 -0500 Subject: dsaparam error OpenSSL 1.1.1d In-Reply-To: References: Message-ID: <20191111054259.GR34850@straasha.imrryr.org> On Mon, Nov 11, 2019 at 10:59:52AM +0530, shiva kumar wrote: > *openssl dsaparam 128 -rand file * > is taking long time processing the command and not producing any output. It is waiting for *input* (to decode already generated parameters). If you want to generate a key you need to provide the "-genkey" option. And 128 bit primes are way too short for DSA. You should generally use 2048 bits, and definitely avoid anything less than 1024 bits. The below example generates a 1280-bit key, which is about as small as I'd be generally willing to go for short to medium-term keys. $ openssl dsaparam -genkey 1280 Generating DSA parameters, 1280 bit long prime This could take some time ... -----BEGIN DSA PARAMETERS----- ... -----END DSA PARAMETERS----- -----BEGIN DSA PRIVATE KEY----- ... -----END DSA PRIVATE KEY----- -- Viktor. From rsalz at akamai.com Mon Nov 11 13:09:42 2019 From: rsalz at akamai.com (Salz, Rich) Date: Mon, 11 Nov 2019 13:09:42 +0000 Subject: dsaparam error OpenSSL 1.1.1d In-Reply-To: References: Message-ID: <5CA71CA2-98B8-4251-AA4E-6E117973CD94@akamai.com> * openssl dsaparam 128 -rand file Why are you using the -rand option? Unless this is a special platform, don?t do that. * is taking long time processing the command and not producing any output. What is your hardware and software? Can you run it under a debugger, and interrupt it sometimes to see where it is? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsalz at akamai.com Mon Nov 11 13:31:22 2019 From: rsalz at akamai.com (Salz, Rich) Date: Mon, 11 Nov 2019 13:31:22 +0000 Subject: dsaparam error OpenSSL 1.1.1d In-Reply-To: <5CA71CA2-98B8-4251-AA4E-6E117973CD94@akamai.com> References: <5CA71CA2-98B8-4251-AA4E-6E117973CD94@akamai.com> Message-ID: <8937AD40-FDE2-4548-9DCE-79DA55FDD5C6@akamai.com> The question about -rand option is valid, but Viktor?s post is right and the rest of my post is wrong :( From: openssl-users Reply-To: Rich Salz Date: Monday, November 11, 2019 at 8:10 AM To: shiva kumar , openssl-users Subject: Re: dsaparam error OpenSSL 1.1.1d * openssl dsaparam 128 -rand file Why are you using the -rand option? Unless this is a special platform, don?t do that. * is taking long time processing the command and not producing any output. What is your hardware and software? Can you run it under a debugger, and interrupt it sometimes to see where it is? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ratheesh.ksz at gmail.com Mon Nov 11 13:37:55 2019 From: ratheesh.ksz at gmail.com (ratheesh kannoth) Date: Mon, 11 Nov 2019 19:07:55 +0530 Subject: aesni_cbc_encrypt() Message-ID: Hi, I am trying to pass explicit IV during TLS 1.2 encryption. I copied explicit IV to "ivec" before below function call. But IV on the encrypted output is totally different. Can any one help ? void aesni_cbc_encrypt(const unsigned char *in, unsigned char *out, size_t length, const AES_KEY *key, unsigned char *ivec, int enc); -Ratheesh From stephen.farrell at cs.tcd.ie Mon Nov 11 14:53:07 2019 From: stephen.farrell at cs.tcd.ie (Stephen Farrell) Date: Mon, 11 Nov 2019 14:53:07 +0000 Subject: valgrind complaining about s_client (maybe assembler code?) Message-ID: <779218ce-6278-43b4-0c89-0de00300e246@cs.tcd.ie> Hiya, I'm seeing some errors from valgrind when running s_client from a clean build from the tip. (Details of that below.) In another build, (for ESNI), when I do a GCM encrypt and then read the tag, it looks like the error is coming from some sha256 assembler code: ==27027== Uninitialised value was created by a stack allocation ==27027== at 0x4B0ED63: sha256_block_data_order_avx2 (sha256-x86_64.s:4192) Building either (clean or my ESNI fork) with "no-asm" works without valgrind complaining, as do other debug builds, but it seems like once optimisation is turned on, these errors occur. They don't however, seem to affect correct operation of TLS though (in either build). On a 32-bit system the ESNI build also seems fine with or without optimisation. Details below for a clean clone from github. The full valgrind/s_client output with stdout & stderr can be found at [1]. I manually added a "-g" to the Makefile (leaving on "-O3" as well), and the equivalent output is at [2] and seems to show that valgrind sees the error around some GCM tag handling code again. The equivalent output when built with "no-asm" is at [3] and has no valgrind errors. Any ideas? Thanks, S. [1] https://down.dsg.cs.tcd.ie/misc/vgerrs.txt [2] https://down.dsg.cs.tcd.ie/misc/vgerrs-sym.txt [3] https://down.dsg.cs.tcd.ie/misc/vgnoasm.txt My system: Machine: Dell XPS13 OS: Ubuntu 19.10 up to date CPU: Intel? Core? i7-10510U CPU @ 1.80GHz ? 8 The build is using gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008 The first error seen for the clean build from the tip is: ==19663== Conditional jump or move depends on uninitialised value(s) ==19663== at 0x4B6F962: gcm_stream_final (in /home/stephen/code/openssl-clean-upstream/libcrypto.so.3) ==19663== by 0x4A7BE35: EVP_DecryptFinal_ex (in /home/stephen/code/openssl-clean-upstream/libcrypto.so.3) ==19663== by 0x4899256: tls13_enc (in /home/stephen/code/openssl-clean-upstream/libssl.so.3) ==19663== by 0x4897AED: ssl3_get_record (in /home/stephen/code/openssl-clean-upstream/libssl.so.3) ==19663== by 0x4894D27: ssl3_read_bytes (in /home/stephen/code/openssl-clean-upstream/libssl.so.3) ==19663== by 0x48AE320: tls_get_message_header (in /home/stephen/code/openssl-clean-upstream/libssl.so.3) ==19663== by 0x48A44FC: state_machine.part.0 (in /home/stephen/code/openssl-clean-upstream/libssl.so.3) ==19663== by 0x48942B7: ssl3_write_bytes (in /home/stephen/code/openssl-clean-upstream/libssl.so.3) ==19663== by 0x487B868: ssl_write_internal (in /home/stephen/code/openssl-clean-upstream/libssl.so.3) ==19663== by 0x487BA96: SSL_write (in /home/stephen/code/openssl-clean-upstream/libssl.so.3) ==19663== by 0x172E5A: s_client_main (in /home/stephen/code/openssl-clean-upstream/apps/openssl) ==19663== by 0x160105: do_cmd (in /home/stephen/code/openssl-clean-upstream/apps/openssl) The commands I used to build and generate the errors: $ cd $HOME/code $ git clone https://github.com/openssl/openssl.git openssl-clean-upstream $ cd openssl-clean-upstream $ ./config ... stuff ... $ make -j8 ... stuff ... $ export LD_LIBRARY_PATH=$HOME/code/openssl-clean-upstream $ echo -e "GET /" | valgrind ./apps/openssl s_client -msg -debug -CApath /etc/ssl/certs/ -no_ssl3 -no_tls1 -no_tls1_1 -no_tls1_2 -connect www.cloudflare.com:443 -servername www.cloudflare.com >vgerrs.txt 2>&1 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0x5AB2FAF17B172BEA.asc Type: application/pgp-keys Size: 10715 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From wnichols at tibco.com Mon Nov 11 15:42:21 2019 From: wnichols at tibco.com (Wendell Nichols) Date: Mon, 11 Nov 2019 08:42:21 -0700 Subject: Problems porting Openssl 1.1.1d to zos. Message-ID: <9f376311-0608-c59b-cdf4-ff92a8b26cff@tibco.com> Is there anyone on this group with experience with ebcdic platforms, specifically zOS?? I have built 1.1.1d on zOS and connections to my server work for firefox 60 but not newer versions.? I don't know exactly where the cut off is or what they changed but current versions get an HMAC error.? I strongly suspect that it is keying the hmac with some combination of inputs that include an improperly translated text string, but I don't know for sure.? Its quite hard to track down when you don't have a debugger. The error message: > An error occurred during a connection to cafe.na.tibco.com:1802. SSL > received a record with an incorrect Message Authentication Code. Error > code: SSL_ERROR_BAD_MAC_READ If anyone can suggest an approach to figuring this out I'd be grateful. Wendell Nichols From Matthias.St.Pierre at ncp-e.com Mon Nov 11 16:13:12 2019 From: Matthias.St.Pierre at ncp-e.com (Matthias St. Pierre) Date: Mon, 11 Nov 2019 17:13:12 +0100 Subject: Problems porting Openssl 1.1.1d to zos. In-Reply-To: <9f376311-0608-c59b-cdf4-ff92a8b26cff@tibco.com> References: <9f376311-0608-c59b-cdf4-ff92a8b26cff@tibco.com> Message-ID: <7c5ff7da-adc9-d1e0-7895-c706f10f83cd@ncp-e.com> On 11.11.19 16:42, Wendell Nichols via openssl-users wrote: > Is there anyone on this group with experience with ebcdic platforms, specifically zOS?? I have built 1.1.1d on zOS and connections to my server work for firefox 60 but not newer versions.? I don't know exactly where the cut off is or what they changed but current versions get an HMAC error.? I strongly suspect that it is keying the hmac with some combination of inputs that include an improperly translated text string, but I don't know for sure.? Its quite hard to track down when you don't have a debugger. > > The error message: > >> An error occurred during a connection to cafe.na.tibco.com:1802. SSL received a record with an incorrect Message Authentication Code. Error code: SSL_ERROR_BAD_MAC_READ > > If anyone can suggest an approach to figuring this out I'd be grateful. > > > Wendell Nichols > Incidentally, I just merged a pull request that fixed a misspelled EBCDIC string to master and 1.1.1. https://github.com/openssl/openssl/pull/10396#issuecomment-552506972 But I have no idea whether it is related to your problem. Nevertheless, you might want to retry with the current tip of the OpenSSL_1_1_1-stable branch. Regards, Matthias From Michael.Wojcik at microfocus.com Mon Nov 11 16:50:32 2019 From: Michael.Wojcik at microfocus.com (Michael Wojcik) Date: Mon, 11 Nov 2019 16:50:32 +0000 Subject: valgrind complaining about s_client (maybe assembler code?) In-Reply-To: <779218ce-6278-43b4-0c89-0de00300e246@cs.tcd.ie> References: <779218ce-6278-43b4-0c89-0de00300e246@cs.tcd.ie> Message-ID: > From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of > Stephen Farrell > Sent: Monday, November 11, 2019 09:53 > > I'm seeing some errors from valgrind when running s_client > from a clean build from the tip. (Details of that below.) Have you reviewed past discussions about valgrind and OpenSSL? https://duckduckgo.com/html?q=site%3Amta.openssl.org%20openssl-users%20valgrind In general, memory-use checkers such as valgrind's memcheck are going to be suspicious of some aspects of OpenSSL, which makes some assumptions about memory that it knows are suitable, but which cannot generally be verified by typical static or dynamic analysis. -- Michael Wojcik Distinguished Engineer, Micro Focus From stephen.farrell at cs.tcd.ie Mon Nov 11 17:14:15 2019 From: stephen.farrell at cs.tcd.ie (Stephen Farrell) Date: Mon, 11 Nov 2019 17:14:15 +0000 Subject: valgrind complaining about s_client (maybe assembler code?) In-Reply-To: References: <779218ce-6278-43b4-0c89-0de00300e246@cs.tcd.ie> Message-ID: Hiya, On 11/11/2019 16:50, Michael Wojcik wrote: >> From: openssl-users [mailto:openssl-users-bounces at openssl.org] On >> Behalf Of Stephen Farrell Sent: Monday, November 11, 2019 09:53 >> >> I'm seeing some errors from valgrind when running s_client from a >> clean build from the tip. (Details of that below.) > > Have you reviewed past discussions about valgrind and OpenSSL? I wouldn't say I reviewed it all in detail no, but yep, I did spend a while checking and didn't find the same issue. (Apologies if I missed it, which is always possible;-) > https://duckduckgo.com/html?q=site%3Amta.openssl.org%20openssl-users%20valgrind > > In general, memory-use checkers such as valgrind's memcheck are > going to be suspicious of some aspects of OpenSSL, which makes some > assumptions about memory that it knows are suitable, but which cannot > generally be verified by typical static or dynamic analysis. Sure. OTOH, I didn't see these errors until just now when I rebased with upstream so they seem to be new, (in the last couple of months), or I guess could be down to some novelty with my CPU/compiler, given I'm using a brand new laptop and Ubuntu 19.10;-) FWIW, valgrind hasn't been giving me false positives over the last year or so, and has been v. useful in finding leaks, so it'd be nice if there were a fix or better workaround than "no-asm" (but since I do have "no-asm", and don't need go-faster-stripes, this isn't urgent for me). Cheers, S. > > -- Michael Wojcik Distinguished Engineer, Micro Focus > > -------------- next part -------------- A non-text attachment was scrubbed... Name: 0x5AB2FAF17B172BEA.asc Type: application/pgp-keys Size: 10715 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From pneumiller at directstream.com Mon Nov 11 19:32:22 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Mon, 11 Nov 2019 12:32:22 -0700 (MST) Subject: Removing Extensions from Client Hello Header Message-ID: <1573500742320-0.post@n7.nabble.com> I am speaking TLS 1.3 with openssl to a hardware device that I can't change. I need the client hello header to only support certain extensions, yet I see no way in the SSL API to remove the default extensions in the TLS 1.3 client hello. Can I clear them all and just add the ones I want? What am I missing? Do I have to modify the SSL code to do this? It seems like there should be an orthodox way to do this. ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From bkaduk at akamai.com Mon Nov 11 19:43:40 2019 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Mon, 11 Nov 2019 11:43:40 -0800 Subject: Removing Extensions from Client Hello Header In-Reply-To: <1573500742320-0.post@n7.nabble.com> References: <1573500742320-0.post@n7.nabble.com> Message-ID: <20191111194340.GF20609@akamai.com> On Mon, Nov 11, 2019 at 12:32:22PM -0700, Phil Neumiller wrote: > I am speaking TLS 1.3 with openssl to a hardware device that I can't change. > I need the client hello header to only support certain extensions, yet I > see no way in the SSL API to remove the default extensions in the TLS 1.3 > client hello. Can I clear them all and just add the ones I want? What am I > missing? Do I have to modify the SSL code to do this? It seems like there > should be an orthodox way to do this. You have to disable them one by one; see SSL_CTX_set_options(3) and (e.g.) SSL_OP_NO_EXTENDED_MASTER_SECRET. -Ben From matt at openssl.org Mon Nov 11 19:51:00 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 11 Nov 2019 19:51:00 +0000 Subject: Removing Extensions from Client Hello Header In-Reply-To: <20191111194340.GF20609@akamai.com> References: <1573500742320-0.post@n7.nabble.com> <20191111194340.GF20609@akamai.com> Message-ID: <02b3ae0e-df74-520c-457f-6d3e9f62c770@openssl.org> On 11/11/2019 19:43, Benjamin Kaduk via openssl-users wrote: > On Mon, Nov 11, 2019 at 12:32:22PM -0700, Phil Neumiller wrote: >> I am speaking TLS 1.3 with openssl to a hardware device that I can't change. >> I need the client hello header to only support certain extensions, yet I Any compliant implementation should ignore extensions it doesn't understand so why do you need to do this? >> see no way in the SSL API to remove the default extensions in the TLS 1.3 >> client hello. Can I clear them all and just add the ones I want? What am I >> missing? Do I have to modify the SSL code to do this? It seems like there >> should be an orthodox way to do this. > > You have to disable them one by one; see SSL_CTX_set_options(3) and (e.g.) > SSL_OP_NO_EXTENDED_MASTER_SECRET. Only certain headers can be disabled in this way. Many of the extensions present in a TLSv1.3 ClientHello are necessary for proper functioning of the protocol. Which extensions did you actually want to disable? Matt From pneumiller at directstream.com Mon Nov 11 20:51:39 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Mon, 11 Nov 2019 13:51:39 -0700 (MST) Subject: Removing Extensions from Client Hello Header In-Reply-To: <02b3ae0e-df74-520c-457f-6d3e9f62c770@openssl.org> References: <1573500742320-0.post@n7.nabble.com> <20191111194340.GF20609@akamai.com> <02b3ae0e-df74-520c-457f-6d3e9f62c770@openssl.org> Message-ID: <1573505499403-0.post@n7.nabble.com> By doing the following in my code: I was able to get the Client Hello Extensions down to. Handshake Protocol: Client Hello Handshake Type: Client Hello (1) Length: 365 Version: TLS 1.2 (0x0303) Random: 19ff8a9231e83985887f5e45f2c9b243f0ccaa955beb1f03? Session ID Length: 32 Session ID: ebcab15bff6e5abfc14588298b45a56f74963eda97645992? Cipher Suites Length: 8 Cipher Suites (4 suites) Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302) Cipher Suite: TLS_CHACHA20_POLY1305_SHA256 (0x1303) Cipher Suite: TLS_AES_128_GCM_SHA256 (0x1301) Cipher Suite: TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00ff) Compression Methods Length: 1 Compression Methods (1 method) Compression Method: null (0) Extensions Length: 284 Extension: ec_point_formats (len=4) Type: ec_point_formats (11) Length: 4 EC point formats Length: 3 Elliptic curves point formats (3) EC point format: uncompressed (0) EC point format: ansiX962_compressed_prime (1) EC point format: ansiX962_compressed_char2 (2) Extension: supported_groups (len=8) Type: supported_groups (10) Length: 8 Supported Groups List Length: 6 Supported Groups (3 groups) Supported Group: secp521r1 (0x0019) Supported Group: secp384r1 (0x0018) Supported Group: secp256r1 (0x0017) Extension: session_ticket (len=0) Type: session_ticket (35) Length: 0 Data (0 bytes) Extension: encrypt_then_mac (len=0) Type: encrypt_then_mac (22) Length: 0 Extension: extended_master_secret (len=0) Type: extended_master_secret (23) Length: 0 Extension: signature_algorithms (len=30) Type: signature_algorithms (13) Length: 30 Signature Hash Algorithms Length: 28 Signature Hash Algorithms (14 algorithms) Signature Algorithm: ecdsa_secp256r1_sha256 (0x0403) Signature Hash Algorithm Hash: SHA256 (4) Signature Hash Algorithm Signature: ECDSA (3) Signature Algorithm: ecdsa_secp384r1_sha384 (0x0503) Signature Hash Algorithm Hash: SHA384 (5) Signature Hash Algorithm Signature: ECDSA (3) Signature Algorithm: ecdsa_secp521r1_sha512 (0x0603) Signature Hash Algorithm Hash: SHA512 (6) Signature Hash Algorithm Signature: ECDSA (3) Signature Algorithm: ed25519 (0x0807) Signature Hash Algorithm Hash: Unknown (8) Signature Hash Algorithm Signature: Unknown (7) Signature Algorithm: ed448 (0x0808) Signature Hash Algorithm Hash: Unknown (8) Signature Hash Algorithm Signature: Unknown (8) Signature Algorithm: rsa_pss_pss_sha256 (0x0809) Signature Hash Algorithm Hash: Unknown (8) Signature Hash Algorithm Signature: Unknown (9) Signature Algorithm: rsa_pss_pss_sha384 (0x080a) Signature Hash Algorithm Hash: Unknown (8) Signature Hash Algorithm Signature: Unknown (10) Signature Algorithm: rsa_pss_pss_sha512 (0x080b) Signature Hash Algorithm Hash: Unknown (8) Signature Hash Algorithm Signature: Unknown (11) Signature Algorithm: rsa_pss_rsae_sha256 (0x0804) Signature Hash Algorithm Hash: Unknown (8) Signature Hash Algorithm Signature: Unknown (4) Signature Algorithm: rsa_pss_rsae_sha384 (0x0805) Signature Hash Algorithm Hash: Unknown (8) Signature Hash Algorithm Signature: Unknown (5) Signature Algorithm: rsa_pss_rsae_sha512 (0x0806) Signature Hash Algorithm Hash: Unknown (8) Signature Hash Algorithm Signature: Unknown (6) Signature Algorithm: rsa_pkcs1_sha256 (0x0401) Signature Hash Algorithm Hash: SHA256 (4) Signature Hash Algorithm Signature: RSA (1) Signature Algorithm: rsa_pkcs1_sha384 (0x0501) Signature Hash Algorithm Hash: SHA384 (5) Signature Hash Algorithm Signature: RSA (1) Signature Algorithm: rsa_pkcs1_sha512 (0x0601) Signature Hash Algorithm Hash: SHA512 (6) Signature Hash Algorithm Signature: RSA (1) Extension: supported_versions (len=3) Type: supported_versions (43) Length: 3 Supported Versions length: 2 Supported Version: TLS 1.3 (0x0304) Extension: psk_key_exchange_modes (len=2) Type: psk_key_exchange_modes (45) Length: 2 PSK Key Exchange Modes Length: 1 PSK Key Exchange Mode: PSK with (EC)DHE key establishment (psk_dhe_ke) (1) Extension: key_share (len=139) Type: key_share (51) Length: 139 Key Share extension Client Key Share Length: 137 Key Share Entry: Group: secp521r1, Key Exchange length: 133 Group: secp521r1 (25) Key Exchange Length: 133 Key Exchange: 040044c7b3890387abc775e036f375acf9247ffad580a078? Extension: pre_shared_key (len=58) Type: pre_shared_key (41) Length: 58 Pre-Shared Key extension Identities Length: 21 PSK Identity (length: 15) Identity Length: 15 Identity: 436c69656e745f6964656e74697479 Obfuscated Ticket Age: 0 PSK Binders length: 33 PSK Binders Is this the minimal standard compliant set of extensions? ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From pneumiller at directstream.com Mon Nov 11 20:56:36 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Mon, 11 Nov 2019 13:56:36 -0700 (MST) Subject: Removing Extensions from Client Hello Header In-Reply-To: <1573505499403-0.post@n7.nabble.com> References: <1573500742320-0.post@n7.nabble.com> <20191111194340.GF20609@akamai.com> <02b3ae0e-df74-520c-457f-6d3e9f62c770@openssl.org> <1573505499403-0.post@n7.nabble.com> Message-ID: <1573505796368-0.post@n7.nabble.com> Code: SSL_CTX_set_options(ctx, !SSL_OP_ALL); ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From pneumiller at directstream.com Mon Nov 11 21:09:39 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Mon, 11 Nov 2019 14:09:39 -0700 (MST) Subject: Removing Extensions from Client Hello Header In-Reply-To: <02b3ae0e-df74-520c-457f-6d3e9f62c770@openssl.org> References: <1573500742320-0.post@n7.nabble.com> <20191111194340.GF20609@akamai.com> <02b3ae0e-df74-520c-457f-6d3e9f62c770@openssl.org> Message-ID: <1573506579313-0.post@n7.nabble.com> The hardware wants to see a client hello like the following: Handshake Protocol: Client Hello Handshake Type: Client Hello (1) Length: 253 Version: TLS 1.2 (0x0303) Random: 000000000000000100000002000000040000000900000012? GMT Unix Time: Dec 31, 1969 17:00:00.000000000 MST Random Bytes: 000000010000000200000004000000090000001200000024? Session ID Length: 0 Cipher Suites Length: 2 Cipher Suites (1 suite) Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302) Compression Methods Length: 1 Compression Methods (1 method) Compression Method: null (0) Extensions Length: 210 Extension: supported_groups (len=4) Type: supported_groups (10) Length: 4 Supported Groups List Length: 2 Supported Groups (1 group) Supported Group: x25519 (0x001d) Extension: signature_algorithms (len=4) Type: signature_algorithms (13) Length: 4 Signature Hash Algorithms Length: 2 Signature Hash Algorithms (1 algorithm) Signature Algorithm: ecdsa_secp256r1_sha256 (0x0403) Signature Hash Algorithm Hash: SHA256 (4) Signature Hash Algorithm Signature: ECDSA (3) Extension: key_share (len=38) Type: key_share (51) Length: 38 Key Share extension Client Key Share Length: 36 Key Share Entry: Group: x25519, Key Exchange length: 32 Group: x25519 (29) Key Exchange Length: 32 Key Exchange: 000000920000012400000249000004920000092400001249? Extension: psk_key_exchange_modes (len=2) Type: psk_key_exchange_modes (45) Length: 2 PSK Key Exchange Modes Length: 1 PSK Key Exchange Mode: PSK with (EC)DHE key establishment (psk_dhe_ke) (1) Extension: supported_versions (len=3) Type: supported_versions (43) Length: 3 Supported Versions length: 2 Supported Version: TLS 1.3 (0x0304) Extension: heartbeat (len=1) Type: heartbeat (15) Length: 1 Mode: Peer not allowed to send requests (2) Extension: pre_shared_key (len=130) Type: pre_shared_key (41) Length: 130 Pre-Shared Key extension Identities Length: 28 PSK Identity (length: 8) Identity Length: 8 Identity: 0000924900012492 Obfuscated Ticket Age: 0 PSK Identity (length: 8) Identity Length: 8 Identity: 0000000000000000 Obfuscated Ticket Age: 0 PSK Binders length: 98 PSK Binders ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From Michael.Wojcik at microfocus.com Mon Nov 11 22:12:38 2019 From: Michael.Wojcik at microfocus.com (Michael Wojcik) Date: Mon, 11 Nov 2019 22:12:38 +0000 Subject: Removing Extensions from Client Hello Header In-Reply-To: <1573505796368-0.post@n7.nabble.com> References: <1573500742320-0.post@n7.nabble.com> <20191111194340.GF20609@akamai.com> <02b3ae0e-df74-520c-457f-6d3e9f62c770@openssl.org> <1573505499403-0.post@n7.nabble.com> <1573505796368-0.post@n7.nabble.com> Message-ID: -----Original Message----- > From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of > Phil Neumiller > Sent: Monday, November 11, 2019 15:57 > > Code: SSL_CTX_set_options(ctx, !SSL_OP_ALL); That's just a verbose way of saying SSL_CTX_set_options(ctx, 0). Perhaps you meant SSL_CTX_set_options(ctx, ~SSL_OP_ALL)? I certainly wouldn't recommend that - it would enable a host of options which aren't included in SSL_OP_ALL, and which you very likely shouldn't be enabling. (And also some you perhaps should, such as SSL_OP_SINGLE_ECDH_USE, though I don't remember offhand if that affects TLSv1.3.) SSL_OP_ALL is defined as "various bug workarounds that should be rather harmless". I don't believe its use is appropriate here. As with any implementation of any protocol, there are limits to OpenSSL's ability to deal with noncompliant peers. This may be a case where you have to customize your OpenSSL build in order to get it to connect to your apparently-non-compliant server. -- Michael Wojcik Distinguished Engineer, Micro Focus From matt at openssl.org Mon Nov 11 22:16:02 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 11 Nov 2019 22:16:02 +0000 Subject: Removing Extensions from Client Hello Header In-Reply-To: References: <1573500742320-0.post@n7.nabble.com> <20191111194340.GF20609@akamai.com> <02b3ae0e-df74-520c-457f-6d3e9f62c770@openssl.org> <1573505499403-0.post@n7.nabble.com> <1573505796368-0.post@n7.nabble.com> Message-ID: On 11/11/2019 22:12, Michael Wojcik wrote: > -----Original Message----- >> From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of >> Phil Neumiller >> Sent: Monday, November 11, 2019 15:57 >> >> Code: SSL_CTX_set_options(ctx, !SSL_OP_ALL); > > That's just a verbose way of saying SSL_CTX_set_options(ctx, 0). > > Perhaps you meant SSL_CTX_set_options(ctx, ~SSL_OP_ALL)? I certainly wouldn't recommend that - it would enable a host of options which aren't included in SSL_OP_ALL, and which you very likely shouldn't be enabling. (And also some you perhaps should, such as SSL_OP_SINGLE_ECDH_USE, though I don't remember offhand if that affects TLSv1.3.) There is no need to enable SSL_OP_SINGLE_ECDH_USE. In fact that option does nothing: /* Removed from OpenSSL 1.1.0. Was 0x00080000L */ # define SSL_OP_SINGLE_ECDH_USE 0x0 Matt From matt at openssl.org Mon Nov 11 22:17:14 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 11 Nov 2019 22:17:14 +0000 Subject: Removing Extensions from Client Hello Header In-Reply-To: <1573506579313-0.post@n7.nabble.com> References: <1573500742320-0.post@n7.nabble.com> <20191111194340.GF20609@akamai.com> <02b3ae0e-df74-520c-457f-6d3e9f62c770@openssl.org> <1573506579313-0.post@n7.nabble.com> Message-ID: <04110410-9e59-be7d-fa1c-51bf7fd83349@openssl.org> On 11/11/2019 21:09, Phil Neumiller wrote: > The hardware wants to see a client hello like the following: By this do you imply that if you give it additional extensions it fails? That is a highly non-compliant implementation!! Matt From matt at openssl.org Mon Nov 11 22:33:36 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 11 Nov 2019 22:33:36 +0000 Subject: Removing Extensions from Client Hello Header In-Reply-To: <1573505499403-0.post@n7.nabble.com> References: <1573500742320-0.post@n7.nabble.com> <20191111194340.GF20609@akamai.com> <02b3ae0e-df74-520c-457f-6d3e9f62c770@openssl.org> <1573505499403-0.post@n7.nabble.com> Message-ID: <65ef232c-64a2-4327-9c3e-9cf022a6aa2b@openssl.org> On 11/11/2019 20:51, Phil Neumiller wrote: > Extension: ec_point_formats (len=4) > Type: ec_point_formats (11) > Length: 4 > EC point formats Length: 3 > Elliptic curves point formats (3) > EC point format: uncompressed (0) > EC point format: ansiX962_compressed_prime (1) > EC point format: ansiX962_compressed_char2 (2) > Extension: session_ticket (len=0) > Type: session_ticket (35) > Length: 0 > Data (0 bytes) > Extension: encrypt_then_mac (len=0) > Type: encrypt_then_mac (22) > Length: 0 > Extension: extended_master_secret (len=0) > Type: extended_master_secret (23) > Length: 0 You don't need these four for TLSv1.3 SSL_OP_NO_TICKET will turn off session_ticket. SSL_OP_NO_ENCRYPT_THEN_MAC will turn off encrypt_then_mac. SSL_OP_NO_EXTENDED_MASTER_SECRET will turn off extended_master_secret. Don't switch off encrypt-then-mac or extended-master-secret unless you *really* need to. They don't do anything in TLSv1.3 but if you ever ended up negotiating TLSv1.2 by mistake for some reason then switching these things off has security consequences. I think the only way to get rid of ec_point_formats would be to disable EC from being used completely. But, you need EC to be enabled in order use TLSv1.3 (at least in 1.1.1 - in master its different). So I don't think you can get rid of this extension. But I'd really look at why your hardware is failing when these extensions are present. Is it intolerant of one particular extension? If so I'd just disable that one. Matt From paherbst at gmail.com Mon Nov 11 23:38:39 2019 From: paherbst at gmail.com (Patrick Herbst) Date: Mon, 11 Nov 2019 18:38:39 -0500 Subject: Resetting DTLS server In-Reply-To: References: Message-ID: If i setup a DTLS server, the client can connect once and send messages find. but if the client restarts and tries to send data, the server hangs on SSL_read. I'm assuming the server does not like a clienthello message when it is expecting application data. How can the server be made to recover and re-handshake with the restarted client? From mcr at sandelman.ca Tue Nov 12 07:53:27 2019 From: mcr at sandelman.ca (Michael Richardson) Date: Tue, 12 Nov 2019 15:53:27 +0800 Subject: Resetting DTLS server In-Reply-To: References: Message-ID: On 2019-11-12 7:38 a.m., Patrick Herbst wrote: > If i setup a DTLS server, the client can connect once and send > messages find. but if the client restarts and tries to send data, the > server hangs on SSL_read. How are you handling the sockets on the server? If you are creating a new 5-tuple [bind/connect] socket on the server for each client, and the client then reuses it's socket, then it's trying to speak the old instance on the server.? > I'm assuming the server does not like a clienthello message when it is > expecting application data. > > How can the server be made to recover and re-handshake with the > restarted client? Close the UDP socket on the client and open a new one to get a new source port. Does that work?? I'm not terribly happy with this solution, but it does match what TCP would do. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: OpenPGP digital signature URL: From tmraz at redhat.com Tue Nov 12 08:14:32 2019 From: tmraz at redhat.com (Tomas Mraz) Date: Tue, 12 Nov 2019 09:14:32 +0100 Subject: Help on Diffie Hellman key exchange In-Reply-To: References: Message-ID: On Mon, 2019-11-04 at 17:34 -0500, Jason Qian via openssl-users wrote: > Hi > > We have an application that does the Diffie Hellman key exchange > (OpenSSL/1.1.0f). > It works fine, but under heavy loaded conditions, sometimes an > invalide secret been generated and other side couldn't decrypt the > data (the secret seems offset by one). > > The client side is c++ and the server side is java. > > DH_compute_key(secretKey, bnY, m_DH); > > Someone in the openssl group also talks about a similar issue, but > not sure if have a solution. Could it be a padding issue? I.E. use DH_compute_key_padded() instead. -- Tom?? Mr?z No matter how far down the wrong road you've gone, turn back. Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.] From Matthias.St.Pierre at ncp-e.com Tue Nov 12 12:45:30 2019 From: Matthias.St.Pierre at ncp-e.com (Matthias St. Pierre) Date: Tue, 12 Nov 2019 13:45:30 +0100 Subject: Problems porting Openssl 1.1.1d to zos. In-Reply-To: <7c5ff7da-adc9-d1e0-7895-c706f10f83cd@ncp-e.com> References: <9f376311-0608-c59b-cdf4-ff92a8b26cff@tibco.com> <7c5ff7da-adc9-d1e0-7895-c706f10f83cd@ncp-e.com> Message-ID: <3b178721-54da-0fac-0974-c86fe33708e6@ncp-e.com> Please see also GitHub issue #4154, in particular https://github.com/openssl/openssl/issues/4154#issuecomment-552838141 From psteuer9 at gmail.com Tue Nov 12 12:53:29 2019 From: psteuer9 at gmail.com (Patrick Steuer) Date: Tue, 12 Nov 2019 13:53:29 +0100 Subject: Problems porting Openssl 1.1.1d to zos. In-Reply-To: <9f376311-0608-c59b-cdf4-ff92a8b26cff@tibco.com> References: <9f376311-0608-c59b-cdf4-ff92a8b26cff@tibco.com> Message-ID: <195ee925-2fda-7b1f-324d-1625a20821e6@gmail.com> > An error occurred during a connection to cafe.na.tibco.com:1802. SSL > received a record with an incorrect Message Authentication Code. Error > code: SSL_ERROR_BAD_MAC_READ In case this error occurs with a chacha-poly cipher suite, the following PR probably has a fix: https://github.com/openssl/openssl/pull/10417 Patrick From paherbst at gmail.com Tue Nov 12 13:30:53 2019 From: paherbst at gmail.com (Patrick Herbst) Date: Tue, 12 Nov 2019 08:30:53 -0500 Subject: Resetting DTLS server In-Reply-To: References: Message-ID: On Tue, Nov 12, 2019 at 3:00 AM Michael Richardson wrote: > On 2019-11-12 7:38 a.m., Patrick Herbst wrote: > > If i setup a DTLS server, the client can connect once and send > > messages find. but if the client restarts and tries to send data, the > > server hangs on SSL_read. > > How are you handling the sockets on the server? > If you are creating a new 5-tuple [bind/connect] socket on the server > for each client, and the client then reuses it's socket, then it's > trying to speak the old instance on the server. > > I'm assuming the server does not like a clienthello message when it is > > expecting application data. > > > > How can the server be made to recover and re-handshake with the > > restarted client? > > Close the UDP socket on the client and open a new one to get a new > source port. > Does that work? I'm not terribly happy with this solution, but it does > match what TCP would do. > In general, here is what i do (assuming only 1 client for proof of concept, and skipping some mundane steps) also assuming the client is using the same addr/port, so "connect" would not make a difference. s=socket(AF_INET, SOCK_DGRAM, 0); bind(s, &serverAddr, sizeof(serverAddr)); ssl=SSL_new(ctx); bio=BIO_new_dgram(s, BIO_NOCLOSE); SSL_accept(ssl); while (1) { select(FD_SETSIZE, fds, NULL, NULL, NULL); if (FD_ISSET(s)) { n=SSL_read(ssl, buffer, sizeof(buffer)); if (n>0) { printf("rx: %s\n", buffer); } else { printf("bad things\n"); } } } What happens is form the Server standpoint, it doesn't know when a client restarts. When the client does restart, the server blocks on SSL_read while the internals of the library keep reading packets until it gets app data... so it ignores another clienthello, but doesn't notify the server of that condition. am i missing something? is this worth fixing in the library? is this intended behavior? From mcr at sandelman.ca Tue Nov 12 14:05:38 2019 From: mcr at sandelman.ca (Michael Richardson) Date: Tue, 12 Nov 2019 22:05:38 +0800 Subject: Resetting DTLS server In-Reply-To: References: Message-ID: On 2019-11-12 9:30 p.m., Patrick Herbst wrote: > On Tue, Nov 12, 2019 at 3:00 AM Michael Richardson wrote: >> Close the UDP socket on the client and open a new one to get a new >> source port. >> Does that work? I'm not terribly happy with this solution, but it does >> match what TCP would do. >> > In general, here is what i do (assuming only 1 client for proof of > concept, and skipping some mundane steps) > also assuming the client is using the same addr/port, so "connect" > would not make a difference. so you are showing me your server code, correct, and this is for DTLS, right? Do you call DTLSv1_accept()? You don't seem to be creating a new socket anywhere, or calling connect() on this socket. I'm not sure I understand your comment above about connect would not be a difference. If your DGRAM socket is not connected, how can you send packets back?? It would be nice if DTLS code would store the origin of every packet and demux it into multiple SSL*, but it doesn't work that way. > > s=socket(AF_INET, SOCK_DGRAM, 0); > bind(s, &serverAddr, sizeof(serverAddr)); > ssl=SSL_new(ctx); > bio=BIO_new_dgram(s, BIO_NOCLOSE); > SSL_accept(ssl); > > while (1) { > select(FD_SETSIZE, fds, NULL, NULL, NULL); > if (FD_ISSET(s)) { > n=SSL_read(ssl, buffer, sizeof(buffer)); > if (n>0) { > printf("rx: %s\n", buffer); > } else { > printf("bad things\n"); > } > } > } > > What happens is form the Server standpoint, it doesn't know when a > client restarts. When the client does restart, the server blocks on > SSL_read while the internals of the library keep reading packets until > it gets app data... so it ignores another clienthello, but doesn't > notify the server of that condition. > > am i missing something? is this worth fixing in the library? is this > intended behavior? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: OpenPGP digital signature URL: From paherbst at gmail.com Tue Nov 12 14:30:39 2019 From: paherbst at gmail.com (Patrick Herbst) Date: Tue, 12 Nov 2019 09:30:39 -0500 Subject: Resetting DTLS server In-Reply-To: References: Message-ID: On Tue, Nov 12, 2019 at 9:07 AM Michael Richardson wrote: > > so you are showing me your server code, correct, and this is for DTLS, > right? > Do you call DTLSv1_accept()? Yes, DTLS. There is no DTLSv1_accept. SSL_accept should work because it is based on 'method' and underlying BIO. I left some steps out of my example code (i was just hand typing it one the fly, not copy/paste). > > You don't seem to be creating a new socket anywhere, or calling > connect() on this socket. > I'm not sure I understand your comment above about connect would not be > a difference. > If your DGRAM socket is not connected, how can you send packets back? > It would be nice > if DTLS code would store the origin of every packet and demux it into > multiple SSL*, but it doesn't work that way. I'm not creating a new socket because it is UDP, and i'm assuming only one client. If you use a BIO_new_dgram, then you dont need to "connect" the UDP socket, the dgram BIO will keep track of the client's addr. So because of this behavior, "connect" doesn't change anything. I have called "connect" on the sockets in other tests, but it gives the exact same result. SSL_accept waits for a 'clienthello', which the underlying dgram BIO will store the client's addr, so that when SSL_accept writes the response via the BIO, it'll get sent to the proper address. My tests show this working just fine the first time the client connects; the server handshakes and can read messages. Even if i were the "connect" the socket to the clients addr, the client comes up with the same addr/port combination, so the server's "connected" UDP socket will continue reading mesgs from the client. BUT it'll get stuck in SSL_read when the client restarts because SSL_read is not expecting a "clienthello", and the library continues to try to read more packets. Here is a more correct version of the code s=socket(AF_INET, SOCK_DGRAM, 0); bind(s, &serverAddr, sizeof(serverAddr)); ssl=SSL_new(ctx); bio=BIO_new_dgram(s, BIO_NOCLOSE); SSL_set_bio(ssl, bio, bio); SSL_accept(ssl); // at this point the client is authenticated and handshake is complete. ssl's underlying BIO has the clients addr. while (1) { FD_ZERO(&fds); FD_SET(s, &fds); select(FD_SETSIZE, fds, NULL, NULL, NULL); if (FD_ISSET(s)) { n=SSL_read(ssl, buffer, sizeof(buffer)); if (n>0) { printf("rx: %s\n", buffer); } else { printf("bad things\n"); } } } > > am i missing something? is this worth fixing in the library? is this > > intended behavior? From beldmit at gmail.com Tue Nov 12 15:46:58 2019 From: beldmit at gmail.com (Dmitry Belyavsky) Date: Tue, 12 Nov 2019 18:46:58 +0300 Subject: EVP_CIPHER_CTX_FLAG_WRAP_ALLOW Message-ID: Hello, I'm trying to implement the new Russian GOST CMS specification. It uses the key wrap algorithm described here: https://tools.ietf.org/html/draft-smyshlyaev-tls12-gost-suites-06#section-8.2 I've implemented the algorithm as a cipher with the EVP_CIPH_WRAP_MODE flag. It seems to me that the only way to avoid clearing the EVP_CIPHER_CTX_FLAG_WRAP_ALLOW flag in the EVP_CipherInit function is providing the ctrl function in the corresponding EVP_CIPHER object because the EVP_CipherInit function resets the passed EVP_CIPHER_CTX object. The EVP_CipherInit_ex does not reset the EVP_CIPHER_CTX object and theEVP_CIPHER_CTX_FLAG_WRAP_ALLOW stays untouched, so the behavior seems a bit controversial (and undocumented, at least for the 1.1.1 branch). Is this difference a desired one or an accidental one? Should it be documented or fixed? -- SY, Dmitry Belyavsky -------------- next part -------------- An HTML attachment was scrubbed... URL: From jqian at tibco.com Tue Nov 12 15:50:13 2019 From: jqian at tibco.com (Jason Qian) Date: Tue, 12 Nov 2019 10:50:13 -0500 Subject: Help on Diffie Hellman key exchange In-Reply-To: References: Message-ID: Thanks Tomas, I will try that. On Tue, Nov 12, 2019 at 3:14 AM Tomas Mraz wrote: > On Mon, 2019-11-04 at 17:34 -0500, Jason Qian via openssl-users wrote: > > Hi > > > > We have an application that does the Diffie Hellman key exchange > > (OpenSSL/1.1.0f). > > It works fine, but under heavy loaded conditions, sometimes an > > invalide secret been generated and other side couldn't decrypt the > > data (the secret seems offset by one). > > > > The client side is c++ and the server side is java. > > > > DH_compute_key(secretKey, bnY, m_DH); > > > > Someone in the openssl group also talks about a similar issue, but > > not sure if have a solution. > > Could it be a padding issue? I.E. use DH_compute_key_padded() instead. > > -- > Tom?? Mr?z > No matter how far down the wrong road you've gone, turn back. > Turkish proverb > [You'll know whether the road is wrong if you carefully listen to your > conscience.] > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pneumiller at directstream.com Tue Nov 12 20:13:49 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Tue, 12 Nov 2019 13:13:49 -0700 (MST) Subject: Removing Extensions from Client Hello Header In-Reply-To: <20191111194340.GF20609@akamai.com> References: <1573500742320-0.post@n7.nabble.com> <20191111194340.GF20609@akamai.com> Message-ID: <1573589629446-0.post@n7.nabble.com> Thanks for all the useful device. I was able to get the server to accept this client hello message. TLSv1.3 Record Layer: Handshake Protocol: Client Hello Content Type: Handshake (22) Version: TLS 1.2 (0x0303) Length: 257 Handshake Protocol: Client Hello Handshake Type: Client Hello (1) Length: 253 Version: TLS 1.2 (0x0303) Random: 000000000000000100000002000000040000000900000012? Session ID Length: 0 Cipher Suites Length: 2 Cipher Suites (1 suite) Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302) Compression Methods Length: 1 Compression Methods (1 method) Compression Method: null (0) Extensions Length: 210 Extension: supported_groups (len=4) Type: supported_groups (10) Length: 4 Supported Groups List Length: 2 Supported Groups (1 group) Supported Group: x25519 (0x001d) Extension: signature_algorithms (len=4) Type: signature_algorithms (13) Length: 4 Signature Hash Algorithms Length: 2 Signature Hash Algorithms (1 algorithm) Signature Algorithm: rsa_pss_rsae_sha512 (0x0806) Signature Hash Algorithm Hash: Unknown (8) Signature Hash Algorithm Signature: Unknown (6) Extension: key_share (len=38) Type: key_share (51) Length: 38 Key Share extension Client Key Share Length: 36 Key Share Entry: Group: x25519, Key Exchange length: 32 Group: x25519 (29) Key Exchange Length: 32 Key Exchange: 000000920000012400000249000004920000092400001249? Extension: psk_key_exchange_modes (len=2) Type: psk_key_exchange_modes (45) Length: 2 PSK Key Exchange Modes Length: 1 PSK Key Exchange Mode: PSK with (EC)DHE key establishment (psk_dhe_ke) (1) Extension: supported_versions (len=3) Type: supported_versions (43) Length: 3 Supported Versions length: 2 Supported Version: TLS 1.3 (0x0304) Extension: heartbeat (len=1) Type: heartbeat (15) Length: 1 Mode: Peer not allowed to send requests (2) Extension: pre_shared_key (len=130) Type: pre_shared_key (41) Length: 130 Pre-Shared Key extension Identities Length: 28 PSK Identity (length: 8) Identity Length: 8 Identity: 0000924900012492 Obfuscated Ticket Age: 0 PSK Identity (length: 8) Identity Length: 8 Identity: 0000000000000000 Obfuscated Ticket Age: 0 PSK Binders length: 98 PSK Binders So just one signature algorithm. Now the response I got from the OpenSSL TLS server is this server hello. TLSv1.3 Record Layer: Handshake Protocol: Server Hello Content Type: Handshake (22) Version: TLS 1.2 (0x0303) Length: 90 Handshake Protocol: Server Hello Handshake Type: Server Hello (2) Length: 86 Version: TLS 1.2 (0x0303) Random: 7f9801c0f94da77d9d2c100cba7ff587bec25bca39defd81? Session ID Length: 0 Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302) Compression Method: null (0) Extensions Length: 46 Extension: supported_versions (len=2) Type: supported_versions (43) Length: 2 Supported Version: TLS 1.3 (0x0304) Extension: key_share (len=36) Type: key_share (51) Length: 36 Key Share extension Key Share Entry: Group: x25519, Key Exchange length: 32 Group: x25519 (29) Key Exchange Length: 32 Key Exchange: ab6c1e5e5a83cdeee70487c509bd0810668a32fa2402f7d7? Now to try the actual hardware.... At least openssl TLS 1.3 is OK with just 1 signature algorithm for my special case of external out of band PSK. ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From bkaduk at akamai.com Tue Nov 12 20:22:51 2019 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Tue, 12 Nov 2019 12:22:51 -0800 Subject: Removing Extensions from Client Hello Header In-Reply-To: <1573589629446-0.post@n7.nabble.com> References: <1573500742320-0.post@n7.nabble.com> <20191111194340.GF20609@akamai.com> <1573589629446-0.post@n7.nabble.com> Message-ID: <20191112202251.GK20609@akamai.com> On Tue, Nov 12, 2019 at 01:13:49PM -0700, Phil Neumiller wrote: > Thanks for all the useful device. I was able to get the server to accept > this client hello message. If you're willing/able to share, it can be useful for us to know what products are buggy in that they don't implement extensions in a proper, extensible, manner and need to have the ClientHello extensions adjusted like this. If we have a list of "likely suspects" it can make diagnosing future connection issues easier. Thanks, Ben From pneumiller at directstream.com Tue Nov 12 22:08:19 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Tue, 12 Nov 2019 15:08:19 -0700 (MST) Subject: Removing Extensions from Client Hello Header In-Reply-To: <1573500742320-0.post@n7.nabble.com> References: <1573500742320-0.post@n7.nabble.com> Message-ID: <1573596499127-0.post@n7.nabble.com> I find the comment below about TLS 1.3 troubling. static int test_set_sigalgs(int idx) { SSL_CTX *cctx = NULL, *sctx = NULL; SSL *clientssl = NULL, *serverssl = NULL; int testresult = 0; const sigalgs_list *curr; int testctx; /* Should never happen */ if (!TEST_size_t_le((size_t)idx, OSSL_NELEM(testsigalgs) * 2)) return 0; testctx = ((size_t)idx < OSSL_NELEM(testsigalgs)); curr = testctx ? &testsigalgs[idx] : &testsigalgs[idx - OSSL_NELEM(testsigalgs)]; if (!TEST_true(create_ssl_ctx_pair(TLS_server_method(), TLS_client_method(), TLS1_VERSION, 0, &sctx, &cctx, cert, privkey))) return 0; * /* * TODO(TLS1.3): These APIs cannot set TLSv1.3 sig algs so we just test it * for TLSv1.2 for now until we add a new API. */* SSL_CTX_set_max_proto_version(cctx, TLS1_2_VERSION); if (testctx) { int ret; if (curr->list != NULL) ret = SSL_CTX_set1_sigalgs(cctx, curr->list, curr->listlen); else ret = SSL_CTX_set1_sigalgs_list(cctx, curr->liststr); if (!ret) { ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From bkaduk at akamai.com Tue Nov 12 22:25:18 2019 From: bkaduk at akamai.com (Benjamin Kaduk) Date: Tue, 12 Nov 2019 14:25:18 -0800 Subject: Removing Extensions from Client Hello Header In-Reply-To: <1573596499127-0.post@n7.nabble.com> References: <1573500742320-0.post@n7.nabble.com> <1573596499127-0.post@n7.nabble.com> Message-ID: <20191112222518.GL20609@akamai.com> On Tue, Nov 12, 2019 at 03:08:19PM -0700, Phil Neumiller wrote: > I find the comment below about TLS 1.3 troubling. [...] > * /* > * TODO(TLS1.3): These APIs cannot set TLSv1.3 sig algs so we just test > it > * for TLSv1.2 for now until we add a new API. > */* > SSL_CTX_set_max_proto_version(cctx, TLS1_2_VERSION); > > if (testctx) { > int ret; > > if (curr->list != NULL) > ret = SSL_CTX_set1_sigalgs(cctx, curr->list, curr->listlen); > else > ret = SSL_CTX_set1_sigalgs_list(cctx, curr->liststr); I don't. >From SSL_CTX_set1_sigalgs.pod: % The TLS 1.3 signature scheme names (such as "rsa_pss_pss_sha256") can also % be used with the B<_list> forms of the API. The TLS 1.3 schemes don't decompose into SIG+HASH, so this is just a constraint inherent to the old API, not a bug. -Ben From pneumiller at directstream.com Tue Nov 12 22:53:59 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Tue, 12 Nov 2019 15:53:59 -0700 (MST) Subject: Using PSKs with openssl app. Message-ID: <1573599239803-0.post@n7.nabble.com> H, This is my method for using external PSKs with the openssl tool. Does this appear correct? The application darta seems to be exchanged and if I change a PSK it will fail. I *think* this is correct... Server side: PSK=b2c9b9f57ef2fbbba8b624070b301d7f278f1b39c352d5fa849f85a3e7a3f77b openssl s_server -accept 8400 -tls1_3 -nocert -psk $PSK -ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256 Client side: PSK=b2c9b9f57ef2fbbba8b624070b301d7f278f1b39c352d5fa849f85a3e7a3f77b openssl s_client -connect 127.0.0.1:8400 -tls1_3 -psk $PSK -tlsextdebug Here are the hello messages that are exchanged: TLSv1.3 Record Layer: Handshake Protocol: Client Hello Content Type: Handshake (22) Version: TLS 1.0 (0x0301) Length: 282 Handshake Protocol: Client Hello Handshake Type: Client Hello (1) Length: 278 Version: TLS 1.2 (0x0303) Random: d9cd1e44a462699f2a2f794a7fb3dd129b183d3c22183bab? Session ID Length: 32 Session ID: 5525acf9be6afd90e7a7853405157bc21cda45bd708a65f9? Cipher Suites Length: 8 Cipher Suites (4 suites) Compression Methods Length: 1 Compression Methods (1 method) Extensions Length: 197 Extension: ec_point_formats (len=4) Type: ec_point_formats (11) Length: 4 EC point formats Length: 3 Elliptic curves point formats (3) Extension: supported_groups (len=22) Type: supported_groups (10) Length: 22 Supported Groups List Length: 20 Supported Groups (10 groups) Extension: session_ticket (len=0) Type: session_ticket (35) Length: 0 Data (0 bytes) Extension: encrypt_then_mac (len=0) Type: encrypt_then_mac (22) Length: 0 Extension: extended_master_secret (len=0) Type: extended_master_secret (23) Length: 0 Extension: signature_algorithms (len=30) Type: signature_algorithms (13) Length: 30 Signature Hash Algorithms Length: 28 Signature Hash Algorithms (14 algorithms) Extension: supported_versions (len=3) Type: supported_versions (43) Length: 3 Supported Versions length: 2 Supported Version: TLS 1.3 (0x0304) Extension: psk_key_exchange_modes (len=2) Type: psk_key_exchange_modes (45) Length: 2 PSK Key Exchange Modes Length: 1 PSK Key Exchange Mode: PSK with (EC)DHE key establishment (psk_dhe_ke) (1) Extension: key_share (len=38) Type: key_share (51) Length: 38 Key Share extension Client Key Share Length: 36 Key Share Entry: Group: x25519, Key Exchange length: 32 Group: x25519 (29) Key Exchange Length: 32 Key Exchange: eb7a84e24c88e64c0032bbdba0485281702c7929d72d1417? Extension: pre_shared_key (len=58) Type: pre_shared_key (41) Length: 58 Pre-Shared Key extension Identities Length: 21 PSK Identity (length: 15) PSK Binders length: 33 PSK Binders TLSv1.3 Record Layer: Handshake Protocol: Server Hello Content Type: Handshake (22) Version: TLS 1.2 (0x0303) Length: 128 Handshake Protocol: Server Hello Handshake Type: Server Hello (2) Length: 124 Version: TLS 1.2 (0x0303) Random: 4b491c81e70b2ded5bb9d922009b9d8579f9c4415f067f9b? Session ID Length: 32 Session ID: 5525acf9be6afd90e7a7853405157bc21cda45bd708a65f9? Cipher Suite: TLS_CHACHA20_POLY1305_SHA256 (0x1303) Compression Method: null (0) Extensions Length: 52 Extension: supported_versions (len=2) Type: supported_versions (43) Length: 2 Supported Version: TLS 1.3 (0x0304) Extension: key_share (len=36) Type: key_share (51) Length: 36 Key Share extension Key Share Entry: Group: x25519, Key Exchange length: 32 Group: x25519 (29) Key Exchange Length: 32 Key Exchange: 33f67b055f03bb7ce049dc4cb338569d015acc5911f3c55f? Extension: pre_shared_key (len=2) Type: pre_shared_key (41) Length: 2 Pre-Shared Key extension Selected Identity: 0 Here is the client output: ? scripts git:(working) ? ./client CONNECTED(00000003) TLS server extension "supported versions" (id=43), len=2 0000 - 03 04 .. TLS server extension "key share" (id=51), len=36 0000 - 00 1d 00 20 cd c7 59 0b-f3 98 90 e0 34 bc 01 32 ... ..Y.....4..2 0010 - ed 86 cd 9c 9e e4 89 be-fe 3a 57 d0 68 c7 e5 5f .........:W.h.._ 0020 - fc c1 f5 2f .../ TLS server extension "psk" (id=41), len=2 0000 - 00 00 .. Can't use SSL_get_servername --- no peer certificate available --- No client certificate CA names sent Server Temp Key: X25519, 253 bits --- SSL handshake has read 225 bytes and written 351 bytes Verification: OK --- Reused, TLSv1.3, Cipher is TLS_CHACHA20_POLY1305_SHA256 Secure Renegotiation IS NOT supported No ALPN negotiated Early data was not sent Verify return code: 0 (ok) --- --- Post-Handshake New Session Ticket arrived: SSL-Session: Protocol : TLSv1.3 Cipher : TLS_CHACHA20_POLY1305_SHA256 Session-ID: CA31612F1DF0EC3BCF9CB77641FBB9C9E52DDD60E87DDB213D33B5A80B8AB1CD Session-ID-ctx: Resumption PSK: 9BB195D4013A7B45176BD1B0BA04B9EF782E03F678A5373B68C659D24C06DCD7 PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 304 (seconds) TLS session ticket: 0000 - b2 b7 8d 84 0b 3c d7 9f-35 d1 2a a3 0a 1b 64 1f .....<..5.*...d. 0010 - ba 0c b3 83 5e 3c 8b 83-3c 2a e3 f8 63 7b d7 0b ....^<..<*..c{.. 0020 - 18 40 db 63 1e f7 df f4-2d 95 42 b8 08 be 47 2a . at .c....-.B...G* 0030 - 75 5c 1f df 5f 0c ea 54-ec 9b e6 20 1c 74 d9 20 u\.._..T... .t. 0040 - a9 5c af 29 5f 8a cf 12-03 7c ef 4a b8 3f fe 04 .\.)_....|.J.?.. 0050 - 49 cc 6d eb 18 3b c8 86-0b b9 ba 41 83 2d f8 da I.m..;.....A.-.. 0060 - 0d 16 68 f9 7e d9 e6 69-e2 6e e5 77 2e 9c 0a 1a ..h.~..i.n.w.... 0070 - a4 3f b0 9d f4 f2 f4 67-13 22 b6 ac 94 0a dc b5 .?.....g."...... 0080 - cf 0f b8 39 cb 64 00 42-6f 8f 03 b2 be c9 3b 13 ...9.d.Bo.....;. 0090 - a7 a0 de e7 0c 29 d5 0e-2e 2d be 5e a4 a7 37 00 .....)...-.^..7. 00a0 - 00 4e c5 a8 e5 dd 31 ad-20 27 c9 b1 cd 57 ec c1 .N....1. '...W.. 00b0 - b3 35 05 9b 2f ee 12 54-f7 2e 2f 65 d0 d5 5e d9 .5../..T../e..^. Start Time: 1573598575 Timeout : 304 (sec) Verify return code: 1 (unspecified certificate verification error) Extended master secret: no Max Early Data: 0 --- read R BLOCK 213 ? scripts git:(working) ? ./server2 Using default temp DH parameters ACCEPT -----BEGIN SSL SESSION PARAMETERS----- MHICAQECAgMEBAITAwQgq58EYhoHgoCQ2c5Vu6JK/6a4jSyMsKtSOaQkgy5Of/0E IHEPU755SzYf7LVKFCel24+y2MYbjtZtJ/3ftEuPWyM3oQYCBF3LNRmiBAICATCk BgQEAQAAAKUDAgEBrgYCBAGzBnI= -----END SSL SESSION PARAMETERS----- Shared ciphers:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256 Supported Elliptic Groups: X25519:P-256:X448:P-521:P-384:ffdhe2048:ffdhe3072:ffdhe4096:ffdhe6144:ffdhe8192 Shared Elliptic groups: X25519:P-256:X448:P-521:P-384:ffdhe2048:ffdhe3072:ffdhe4096:ffdhe6144:ffdhe8192 CIPHER is TLS_CHACHA20_POLY1305_SHA256 Reused session-id Secure Renegotiation IS supported ERROR shutting down SSL CONNECTION CLOSED ERROR C0:D5:15:08:01:00:00:00:error:SSL routines::binder does not verify:ssl/statem/extensions.c:1614: shutting down SSL CONNECTION CLOSED ERROR C0:D5:15:08:01:00:00:00:error:SSL routines::binder does not verify:ssl/statem/extensions.c:1614: shutting down SSL CONNECTION CLOSED -----BEGIN SSL SESSION PARAMETERS----- MHICAQECAgMEBAITAwQgGCCjChaAp/rv2yYw7BCn3x6AZy5JZocHzEhop5K0K3EE IJuxldQBOntFF2vRsLoEue94LgP2eKU3O2jGWdJMBtzXoQYCBF3LNW+iBAICATCk BgQEAQAAAKUDAgEBrgYCBDTrhfY= -----END SSL SESSION PARAMETERS----- Shared ciphers:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256 Supported Elliptic Groups: X25519:P-256:X448:P-521:P-384:ffdhe2048:ffdhe3072:ffdhe4096:ffdhe6144:ffdhe8192 Shared Elliptic groups: X25519:P-256:X448:P-521:P-384:ffdhe2048:ffdhe3072:ffdhe4096:ffdhe6144:ffdhe8192 CIPHER is TLS_CHACHA20_POLY1305_SHA256 Reused session-id Secure Renegotiation IS supported 213 ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From aijazbaig1 at gmail.com Wed Nov 13 06:45:26 2019 From: aijazbaig1 at gmail.com (Aijaz Baig) Date: Wed, 13 Nov 2019 12:15:26 +0530 Subject: Static linking libssl.a and libcrypto.a on Linux x64 fails Message-ID: So I posted this question over at stackoverflow ( https://stackoverflow.com/questions/58771714/compiling-c-and-c-with-single-makefile) but the gist of it is as follows: I am trying to statically link libssl.a and libcrypto.a into a static library of my own which I will be using in an application (Linux). So I first create that library (let's call it libAPP.a) and then I use that library in an application. So first things first: 1. when I checkout the contents of that library (nm libAPP.a | less) , almost all (I haven't confirmed every single one but it sure looks that way) SSL related symbols (ex. BIO_new) are left undefined in libAPP.a. Why is that? 2. perhaps this is related to question one above, but when I use libAPP.a in my application, I get a whole bunch of 'undefined reference to' errors for the SSL symbols; they disappear when I pass libSSL.a and libCrypto.a to the linker. may be I need a refresher course on libraries but what is the point of linking statically when I still have to keep those libraries? 3. Even after supplying all these libraries, my linker was complaining about a whole bunch of symbols like dlopen, dlclose etc so after some googling, I supplied -ldl and -lz to the linker. So those linker errors related to dlopen etc went but now it complains that it cannot find libz on the system when it is clearly present. Can anyone please elaborate on the steps need for a clean static linking of libssl.a into an application? Is it even doable on Linux considering such tight coupling of libc everywhere?? If not what are the alternatives?? -- Best Regards, Aijaz Baig -------------- next part -------------- An HTML attachment was scrubbed... URL: From mcr at sandelman.ca Wed Nov 13 09:15:04 2019 From: mcr at sandelman.ca (Michael Richardson) Date: Wed, 13 Nov 2019 17:15:04 +0800 Subject: Resetting DTLS server In-Reply-To: References: Message-ID: On 2019-11-12 10:30 p.m., Patrick Herbst wrote: > > I'm not creating a new socket because it is UDP, and i'm assuming only > one client. If you use a BIO_new_dgram, then you dont need to You assumed one client, and you got one client.? What's the problem :-) > Even if i were the "connect" the socket to the clients addr, the > client comes up with the same addr/port combination, so the server's > "connected" UDP socket will continue reading mesgs from the client. I think that it won't work with OpenSSL for more than one DTLS session to occur on the same 5-tuple pair. Whether or not that is correct behaviour according to a specification, I suspect is open to debate. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: OpenPGP digital signature URL: From numamaheswari at pulsesecure.net Wed Nov 13 09:22:06 2019 From: numamaheswari at pulsesecure.net (Umamaheswari Nagarajan) Date: Wed, 13 Nov 2019 09:22:06 +0000 Subject: upgrading the openSSL version from 1.0.2j to 1.1.1c - api changes Message-ID: Hi, We are upgrading the openSSL version from 1.0.2j to 1.1.1c in our product. The following apis seems to be unavailable or modified in 1.1.1c, ENGINE_cleanup RAND_cleanup CRYPTO_lock RSA_PKCS1_SSLeay DH_get_ex_new_index DSA_get_ex_new_index RSA_get_ex_new_index Please advise on the replacement api's that can to be used. Thanks Uma N -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Wed Nov 13 13:02:57 2019 From: matt at openssl.org (Matt Caswell) Date: Wed, 13 Nov 2019 13:02:57 +0000 Subject: upgrading the openSSL version from 1.0.2j to 1.1.1c - api changes In-Reply-To: References: Message-ID: On 13/11/2019 09:22, Umamaheswari Nagarajan wrote: > The following apis seems to be unavailable or modified in 1.1.1c, > > ENGINE_cleanup This still exists, but was converted from a function to a macro that does nothing. You no longer need to call it. OpenSSL 1.1.0+ cleans itself up automatically. > RAND_cleanup As above. > CRYPTO_lock This function was used to lock an internal OpenSSL lock. These locks are not exposed any more. You will need to rewrite your code to not need this function. There are some new lock related functions that exist that are essentially thin wrappers around the platform threading library that is in use. Depending on what you need it for they might suffice: CRYPTO_RWLOCK *CRYPTO_THREAD_lock_new(void); int CRYPTO_THREAD_read_lock(CRYPTO_RWLOCK *lock); int CRYPTO_THREAD_write_lock(CRYPTO_RWLOCK *lock); int CRYPTO_THREAD_unlock(CRYPTO_RWLOCK *lock); void CRYPTO_THREAD_lock_free(CRYPTO_RWLOCK *lock); https://www.openssl.org/docs/man1.1.1/man3/CRYPTO_THREAD_lock_new.html > > RSA_PKCS1_SSLeay Now called RSA_PKCS1_OpenSSL(). > DH_get_ex_new_index> DSA_get_ex_new_index > RSA_get_ex_new_index All of these still exist but were just converted from functions to macros. Matt From Michael.Wojcik at microfocus.com Wed Nov 13 14:23:22 2019 From: Michael.Wojcik at microfocus.com (Michael Wojcik) Date: Wed, 13 Nov 2019 14:23:22 +0000 Subject: Static linking libssl.a and libcrypto.a on Linux x64 fails In-Reply-To: References: Message-ID: > From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of Aijaz Baig > Sent: Wednesday, November 13, 2019 01:45 > I am trying to statically link libssl.a and libcrypto.a into a static library of my own > which I will be using in an application (Linux). You can't link anything into a Linux static library, technically. ELF static libraries, like the older UNIX static libraries they're descended from, are just collections of object files, possibly with some additional metadata. (In BSD 4.x, for example, libraries often had an index member added using the ranlib utility, so that the linker didn't have to search the entire library for each symbol.) On some platforms, where objects can be relinked, the constituent object files produced by compiling source files are sometimes combined into a single large object. This is most often seen on AIX, which uses IBM's XCOFF object format (an enhanced COFF); XCOFF supports relinking objects, so you can bundle objects up this way and save some time in symbol resolution when you link against the library later. But even on AIX this is commonly seen with dynamic libraries and relatively rare for static ones. Normally the linker isn't even involved in creating a static library. You compile sources to objects, and then use ar(1) to create the static library. The makefile you posted to StackOverflow doesn't include this step, so it's hard to tell what exactly you're doing. But in any case, linking a static library against another static library is essentially a no-op. What you *can* do, if you don't want to have to list your library and the OpenSSL libraries when linking your application, is combine multiple static libraries into a single one - provided the object names don't conflict. This is straightforward: $ mkdir tmp; cd tmp $ ar x /path/to/libssl.a $ ar x /path/to/libcrypto.a $ cp /path/to/your/objects/*.o . $ ar c ../your-library.a *.o $ cd .. $ rm -rf tmp (Untested, but see the ar manpage if you run into issues.) That should create a single archive library containing all the objects from the three input libraries. Again, it relies on there being no filename clashes among the objects; if there are, you'll have to rename some of them. -- Michael Wojcik Distinguished Engineer, Micro Focus From hkario at redhat.com Wed Nov 13 14:48:35 2019 From: hkario at redhat.com (Hubert Kario) Date: Wed, 13 Nov 2019 15:48:35 +0100 Subject: Removing Extensions from Client Hello Header In-Reply-To: <20191112202251.GK20609@akamai.com> References: <1573500742320-0.post@n7.nabble.com> <20191111194340.GF20609@akamai.com> <1573589629446-0.post@n7.nabble.com> <20191112202251.GK20609@akamai.com> Message-ID: <27da9122-259b-4f9a-a132-601226343d8d@redhat.com> On Tuesday, 12 November 2019 21:22:51 CET, Benjamin Kaduk via openssl-users wrote: > On Tue, Nov 12, 2019 at 01:13:49PM -0700, Phil Neumiller wrote: >> Thanks for all the useful device. I was able to get the server to accept >> this client hello message. > > If you're willing/able to share, it can be useful for us to > know what products > are buggy in that they don't implement extensions in a proper, > extensible, manner > and need to have the ClientHello extensions adjusted like this. > If we have a > list of "likely suspects" it can make diagnosing future connection issues > easier. contributing a fingerprint to https://github.com/WestpointLtd/tls_prober would also be really welcome, for the same reasons -- Regards, Hubert Kario Senior Quality Engineer, QE BaseOS Security team Web: www.cz.redhat.com Red Hat Czech s.r.o., Purky?ova 115, 612 00 Brno, Czech Republic From jb-openssl at wisemo.com Wed Nov 13 16:08:29 2019 From: jb-openssl at wisemo.com (Jakob Bohm) Date: Wed, 13 Nov 2019 17:08:29 +0100 Subject: Static linking libssl.a and libcrypto.a on Linux x64 fails In-Reply-To: References: Message-ID: On 13/11/2019 15:23, Michael Wojcik wrote: >> From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of Aijaz Baig >> Sent: Wednesday, November 13, 2019 01:45 >> I am trying to statically link libssl.a and libcrypto.a into a static library of my own >> which I will be using in an application (Linux). > You can't link anything into a Linux static library, technically. > > ELF static libraries, like the older UNIX static libraries they're descended from, are just collections of object files, possibly with some additional metadata. (In BSD 4.x, for example, libraries often had an index member added using the ranlib utility, so that the linker didn't have to search the entire library for each symbol.) Actually, that is also the format and mechanism with Microsoft Win32 tools, they just use the DOS-like file name "foo.lib" instead of "libfoo.a" to maintain makefile compatibility with their older Intel OMF-based toolchains. The object files inside the archive are in COFF format, as they seem to have used Unix tools to bring up the initial "NT" operating system internally back before the initial 1993 release. > On some platforms, where objects can be relinked, the constituent object files produced by compiling source files are sometimes combined into a single large object. This is most often seen on AIX, which uses IBM's XCOFF object format (an enhanced COFF); XCOFF supports relinking objects, so you can bundle objects up this way and save some time in symbol resolution when you link against the library later. But even on AIX this is commonly seen with dynamic libraries and relatively rare for static ones. > > Normally the linker isn't even involved in creating a static library. You compile sources to objects, and then use ar(1) to create the static library. The makefile you posted to StackOverflow doesn't include this step, so it's hard to tell what exactly you're doing. > > But in any case, linking a static library against another static library is essentially a no-op. > > What you *can* do, if you don't want to have to list your library and the OpenSSL libraries when linking your application, is combine multiple static libraries into a single one - provided the object names don't conflict. This is straightforward: > > $ mkdir tmp; cd tmp > $ ar x /path/to/libssl.a > $ ar x /path/to/libcrypto.a > $ cp /path/to/your/objects/*.o . > $ ar c ../your-library.a *.o > $ cd .. > $ rm -rf tmp > > (Untested, but see the ar manpage if you run into issues.) > > That should create a single archive library containing all the objects from the three input libraries. Again, it relies on there being no filename clashes among the objects; if there are, you'll have to rename some of them. Note: I seem to recall from a long time ago that GNU ar can combine static libraries directly (without all those temporary file names). In BinUtils 2.25 this was apparently done by invoking ar in "MRI compatibility mode" and using the script command "ADDLIB" inside the provided MRI-style linker script.? For more details see the "ar scripts" part of the full GNU BinUtils TexInfo manual. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 S?borg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded From jqian at tibco.com Wed Nov 13 16:11:30 2019 From: jqian at tibco.com (Jason Qian) Date: Wed, 13 Nov 2019 11:11:30 -0500 Subject: Help on Diffie Hellman key exchange In-Reply-To: References: Message-ID: Hi Tomas, Using DH_compute_key_padded() seems fixed the problem. I have one more question regarding a similar issue but this time is about AES key generation. I think the problem is related to RAND_seed or RAND_bytes (someone also mentioned on another thread). RAND_seed(seed, ::strlen(seed)); RAND_bytes(buf, keySize / 8); What other method do you suggest to use ? Thanks Jason On Tue, Nov 12, 2019 at 10:50 AM Jason Qian wrote: > Thanks Tomas, > > I will try that. > > On Tue, Nov 12, 2019 at 3:14 AM Tomas Mraz wrote: > >> On Mon, 2019-11-04 at 17:34 -0500, Jason Qian via openssl-users wrote: >> > Hi >> > >> > We have an application that does the Diffie Hellman key exchange >> > (OpenSSL/1.1.0f). >> > It works fine, but under heavy loaded conditions, sometimes an >> > invalide secret been generated and other side couldn't decrypt the >> > data (the secret seems offset by one). >> > >> > The client side is c++ and the server side is java. >> > >> > DH_compute_key(secretKey, bnY, m_DH); >> > >> > Someone in the openssl group also talks about a similar issue, but >> > not sure if have a solution. >> >> Could it be a padding issue? I.E. use DH_compute_key_padded() instead. >> >> -- >> Tom?? Mr?z >> No matter how far down the wrong road you've gone, turn back. >> Turkish proverb >> [You'll know whether the road is wrong if you carefully listen to your >> conscience.] >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From tmraz at redhat.com Wed Nov 13 16:27:26 2019 From: tmraz at redhat.com (Tomas Mraz) Date: Wed, 13 Nov 2019 17:27:26 +0100 Subject: Help on Diffie Hellman key exchange In-Reply-To: References: Message-ID: On Wed, 2019-11-13 at 11:11 -0500, Jason Qian wrote: > Hi Tomas, > > Using DH_compute_key_padded() seems fixed the problem. > > I have one more question regarding a similar issue but this time is > about AES key generation. > > I think the problem is related to RAND_seed or RAND_bytes (someone > also mentioned on another thread). > > RAND_seed(seed, ::strlen(seed)); > RAND_bytes(buf, keySize / 8); > I do not understand what is the problem you have. But nevertheless - you should not need to call RAND_seed() unless you are running the code on some very special platform where no method of automatical seeding of the OpenSSL RNG is available. Your RAND_bytes() call should be fine to produce an AES key of bit length keySize. > > > > > > On Tue, Nov 12, 2019 at 10:50 AM Jason Qian wrote: > > Thanks Tomas, > > > > I will try that. > > > > On Tue, Nov 12, 2019 at 3:14 AM Tomas Mraz > > wrote: > > > On Mon, 2019-11-04 at 17:34 -0500, Jason Qian via openssl-users > > > wrote: > > > > Hi > > > > > > > > We have an application that does the Diffie Hellman key > > > exchange > > > > (OpenSSL/1.1.0f). > > > > It works fine, but under heavy loaded conditions, sometimes > > > an > > > > invalide secret been generated and other side couldn't decrypt > > > the > > > > data (the secret seems offset by one). > > > > > > > > The client side is c++ and the server side is java. > > > > > > > > DH_compute_key(secretKey, bnY, m_DH); > > > > > > > > Someone in the openssl group also talks about a similar > > > issue, but > > > > not sure if have a solution. > > > > > > Could it be a padding issue? I.E. use DH_compute_key_padded() > > > instead. > > > -- Tom?? Mr?z No matter how far down the wrong road you've gone, turn back. Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.] From rsalz at akamai.com Wed Nov 13 17:11:43 2019 From: rsalz at akamai.com (Salz, Rich) Date: Wed, 13 Nov 2019 17:11:43 +0000 Subject: Help on Diffie Hellman key exchange In-Reply-To: References: Message-ID: > RAND_seed(seed, ::strlen(seed)); > RAND_bytes(buf, keySize / 8); I don?t know where you are getting the seed, but it is typically binary data, not a C string. If you are using 1.1.0 or later, you do not need to seed things. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jetson23 at hotmail.com Wed Nov 13 17:20:01 2019 From: jetson23 at hotmail.com (Jason Schultz) Date: Wed, 13 Nov 2019 17:20:01 +0000 Subject: sk_X509_OBJECT_num() Message-ID: Hello- I am updating my Linux application from using OpenSSL 1.0.2 to 1.1.1 in preparation for OpenSSL 3.0 (and of course the EOL of 1.0.2). I'm confused about the function in the subject line as well as other, related sk_X509_* functions. My code has always used these functions, and currently my code compiles and runs successfully against 1.1.1. I was sort of doing an audit of my code, evaluating the API calls that have changed vs not changed when I noticed these functions. I searched for them in the 1.1.1 source. They don't exist, except where called in x509_lu.c. In the 1.0.2 code base, they are called in the same file, as well as are defined in a header, /include/openssl/safestack.h. My question is, how are those symbols in my application being resolved since they are no longer found in the safestack.h header file? My system previously had OpenSSL 1.0.2 installed when I installed 1.1.1, but I don't think I have any old headers around that are being found when I compile and link. But for some reason this works. They obviously work within the OpenSSL 1.1.1 code also. I'm thinking I could be missing something basic about the compile/link process that explains this. Any ideas? Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Wed Nov 13 17:22:04 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Wed, 13 Nov 2019 12:22:04 -0500 Subject: Help on Diffie Hellman key exchange In-Reply-To: References: Message-ID: <6FD96060-94A7-4780-A0EE-E209C1AC2D78@dukhovni.org> > On Nov 12, 2019, at 3:14 AM, Tomas Mraz wrote: > > Could it be a padding issue? I.E. use DH_compute_key_padded() instead. Do we have an open issue to document DH_compute_key_padded(3)? It should be documented right next to DH_compute_key(3), with some words to suggest that the caller needs to know whether the peer expects a key octet string that is fixed width (same as "p") or stripped of inessential leading zero octets. -- Viktor. From jqian at tibco.com Wed Nov 13 17:23:37 2019 From: jqian at tibco.com (Jason Qian) Date: Wed, 13 Nov 2019 12:23:37 -0500 Subject: Help on Diffie Hellman key exchange In-Reply-To: References: Message-ID: Thanks Rich and Tomas, Here is the code for creating the key (openssl-0.9.8h) int AESCipher::createKey(unsigned char *buf, int keySize) { char seed[256]; ::sprintf(seed, "%ldXXX_XXX_H__xxxxx_xxxx_xxx_xxxxx_xxxxxxx__INCLUDED_", MiscUtils::generateId()); RAND_seed(seed, ::strlen(seed)); RAND_bytes(buf, keySize / 8); return keySize / 8; } For using 1.1.0, we only need to call RAND_bytes() ? Jason On Wed, Nov 13, 2019 at 12:11 PM Salz, Rich wrote: > *>* RAND_seed(seed, ::strlen(seed)); > > RAND_bytes(buf, keySize / 8); > > > > I don?t know where you are getting the seed, but it is typically binary > data, not a C string. > > > > If you are using 1.1.0 or later, you do not need to seed things. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Wed Nov 13 17:32:11 2019 From: matt at openssl.org (Matt Caswell) Date: Wed, 13 Nov 2019 17:32:11 +0000 Subject: sk_X509_OBJECT_num() In-Reply-To: References: Message-ID: <71b16d64-47e5-14a5-35f5-8fcbb41571d7@openssl.org> On 13/11/2019 17:20, Jason Schultz wrote: > Hello- > > I am updating my Linux application from using OpenSSL 1.0.2 to 1.1.1 in > preparation for OpenSSL 3.0 (and of course the EOL of 1.0.2). I'm > confused about the function in the subject line as well as other, > related sk_X509_* functions.? > > My code has always used these functions, and currently my code compiles > and runs successfully against 1.1.1. I was sort of doing an audit of my > code, evaluating the API calls that have changed vs not changed when I > noticed these functions. I searched for them in the 1.1.1 source. They > don't exist, except where called in x509_lu.c. In the 1.0.2 code base, > they are called in the same file, as well as are defined in a header, > /include/openssl/safestack.h. > > My question is, how are those symbols in my application being resolved > since they are no longer found in the safestack.h header file?? > > My system previously had OpenSSL 1.0.2 installed when I installed 1.1.1, > but I don't think I have any old headers around that are being found > when I compile and link. But for some reason this works. They obviously > work within the OpenSSL 1.1.1 code also.? > > I'm thinking I could be missing something basic about the compile/link > process that explains this.?Any ideas?? > In the header file openssl/x509_vfy.h you will see this line: DEFINE_STACK_OF(X509_OBJECT) That macro is defined inside safestack.h and generates all the sk_*_*() functions (as inline functions). Matt > Thanks in advance. From rsalz at akamai.com Wed Nov 13 17:34:14 2019 From: rsalz at akamai.com (Salz, Rich) Date: Wed, 13 Nov 2019 17:34:14 +0000 Subject: Help on Diffie Hellman key exchange In-Reply-To: References: Message-ID: >For using 1.1.0, we only need to call RAND_bytes() ? Yes. But do check the return value of RAND_bytes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Wed Nov 13 17:35:10 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Wed, 13 Nov 2019 12:35:10 -0500 Subject: Help on Diffie Hellman key exchange In-Reply-To: References: Message-ID: <20191113173510.GY34850@straasha.imrryr.org> On Wed, Nov 13, 2019 at 12:23:37PM -0500, Jason Qian via openssl-users wrote: > Here is the code for creating the key (openssl-0.9.8h) Is this is a new question? It seems to no longer be related to DH key agreement. > int AESCipher::createKey(unsigned char *buf, int keySize) { > char seed[256]; > ::sprintf(seed, "%ldXXX_XXX_H__xxxxx_xxxx_xxx_xxxxx_xxxxxxx__INCLUDED_", > MiscUtils::generateId()); > RAND_seed(seed, ::strlen(seed)); > > RAND_bytes(buf, keySize / 8); > return keySize / 8; > } > > For using 1.1.0, we only need to call RAND_bytes() ? If the application running this code has no other sources of entropy, and the above is the only "random" data stirred into the PRNG, then you may be generating predictable AES keys in your 0.9.8h code. It is likely that MiscUtils::generateId() does not generate cryptographically secure random numbers, and even if it did, the output is at most 64 bits (%ld), which is not long enough for an AES key. -- Viktor. From dcoombs at carillon.ca Wed Nov 13 17:30:10 2019 From: dcoombs at carillon.ca (Dave Coombs) Date: Wed, 13 Nov 2019 12:30:10 -0500 Subject: sk_X509_OBJECT_num() In-Reply-To: References: Message-ID: Hi, They're macros, defined in SKM_DEFINE_STACK_OF() in safestack.h. If you DEFINE_STACK_OF(Foo), you'll automatically end up with a sk_Foo_num() macro. Cheers, -Dave > On Nov 13, 2019, at 12:20, Jason Schultz wrote: > > Hello- > > I am updating my Linux application from using OpenSSL 1.0.2 to 1.1.1 in preparation for OpenSSL 3.0 (and of course the EOL of 1.0.2). I'm confused about the function in the subject line as well as other, related sk_X509_* functions. > > My code has always used these functions, and currently my code compiles and runs successfully against 1.1.1. I was sort of doing an audit of my code, evaluating the API calls that have changed vs not changed when I noticed these functions. I searched for them in the 1.1.1 source. They don't exist, except where called in x509_lu.c. In the 1.0.2 code base, they are called in the same file, as well as are defined in a header, /include/openssl/safestack.h. > > My question is, how are those symbols in my application being resolved since they are no longer found in the safestack.h header file? > > My system previously had OpenSSL 1.0.2 installed when I installed 1.1.1, but I don't think I have any old headers around that are being found when I compile and link. But for some reason this works. They obviously work within the OpenSSL 1.1.1 code also. > > I'm thinking I could be missing something basic about the compile/link process that explains this. Any ideas? > > Thanks in advance. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 9617 bytes Desc: not available URL: From jqian at tibco.com Wed Nov 13 17:41:43 2019 From: jqian at tibco.com (Jason Qian) Date: Wed, 13 Nov 2019 12:41:43 -0500 Subject: Help on Diffie Hellman key exchange In-Reply-To: References: Message-ID: Thanks Rich, On Wed, Nov 13, 2019 at 12:34 PM Salz, Rich wrote: > *>*For using 1.1.0, we only need to call RAND_bytes() ? > > > > Yes. But do check the return value of RAND_bytes. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From scp.stjohn at gmail.com Wed Nov 13 18:47:31 2019 From: scp.stjohn at gmail.com (SP) Date: Wed, 13 Nov 2019 13:47:31 -0500 Subject: sshd fails to start - undefined symbol: EVP_KDF_ctrl Message-ID: Following an OS upgrade of the server I have been unable to start sshd service. On this server some software is upgraded from the OS packages whereas others are manually built. Openssl is manually built from source. After the upgrade of the OS the error message I get when starting sshd is sshd: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b The version I was running was openssl.1.1.1a I then replaced it with openssl.1.1.1d hoping it would resolve the problem. It did not. I would be grateful for any pointers as to how I can resolve this. From openssl-users at dukhovni.org Wed Nov 13 19:08:33 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Wed, 13 Nov 2019 14:08:33 -0500 Subject: sshd fails to start - undefined symbol: EVP_KDF_ctrl In-Reply-To: References: Message-ID: <20191113190833.GD34850@straasha.imrryr.org> On Wed, Nov 13, 2019 at 01:47:31PM -0500, SP wrote: > Following an OS upgrade of the server I have been unable to start sshd > service. On this server some software is upgraded from the OS packages > whereas others are manually built. Openssl is manually built from source. > After the upgrade of the OS the error message I get when starting sshd is > sshd: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b The version I > was running was openssl.1.1.1a I then replaced it with openssl.1.1.1d hoping > it would resolve the problem. It did not. I would be grateful for any > pointers as to how I can resolve this. OpenSSL from openssl.org (upstream) has never had EVP_KDF_ctrl in the OpenSSL 1.1.1 release branch. That function briefly appeared in the 3.0.0 development (master) branch, but has since been replaced. Whoever built your SSH packages used a custom verson of OpenSSL. You really SHOULD NOT replace system packages (like OpenSSL) with your own builds, unless you're willing and able to ensure ABI compatibility with the vendor build. It is safer to install any custom copy of OpenSSL in a non-default location (I use /opt/openssl/1.1), and give the shared libraries in such copies non-default SONAMEs so as to avoid conflict. -- Viktor. From aijazbaig1 at gmail.com Thu Nov 14 00:58:30 2019 From: aijazbaig1 at gmail.com (Aijaz Baig) Date: Thu, 14 Nov 2019 06:28:30 +0530 Subject: Static linking libssl.a and libcrypto.a on Linux x64 fails In-Reply-To: References: Message-ID: Thank you for the suggestion. Will try that. Regarding the static library, the term 'linking' I used was more tongue in cheek but nonetheless. However my current concern here is meeting libSSL and libCrypto's dependencies on host libraries on Linux platform. For instance, when I talked about 'linking' errors with respect to symbols like 'dlopen', so as I mentioned, I had to specify '-ldl' and '-lz' to the gcc linker that suggests a dynamic dependency on these platform libraries. I was trying to understand the components that would be needed to package the whole shebang into an archive which I can later 'just run' on a similar Linux system that has been completely stripped down for purposes of size and speed Is there a way to do that? On Wed, Nov 13, 2019 at 8:04 PM Michael Wojcik < Michael.Wojcik at microfocus.com> wrote: > > From: openssl-users [mailto:openssl-users-bounces at openssl.org] On > Behalf Of Aijaz Baig > > Sent: Wednesday, November 13, 2019 01:45 > > > I am trying to statically link libssl.a and libcrypto.a into a static > library of my own > > which I will be using in an application (Linux). > > You can't link anything into a Linux static library, technically. > > ELF static libraries, like the older UNIX static libraries they're > descended from, are just collections of object files, possibly with some > additional metadata. (In BSD 4.x, for example, libraries often had an index > member added using the ranlib utility, so that the linker didn't have to > search the entire library for each symbol.) > > On some platforms, where objects can be relinked, the constituent object > files produced by compiling source files are sometimes combined into a > single large object. This is most often seen on AIX, which uses IBM's XCOFF > object format (an enhanced COFF); XCOFF supports relinking objects, so you > can bundle objects up this way and save some time in symbol resolution when > you link against the library later. But even on AIX this is commonly seen > with dynamic libraries and relatively rare for static ones. > > Normally the linker isn't even involved in creating a static library. You > compile sources to objects, and then use ar(1) to create the static > library. The makefile you posted to StackOverflow doesn't include this > step, so it's hard to tell what exactly you're doing. > > But in any case, linking a static library against another static library > is essentially a no-op. > > What you *can* do, if you don't want to have to list your library and the > OpenSSL libraries when linking your application, is combine multiple static > libraries into a single one - provided the object names don't conflict. > This is straightforward: > > $ mkdir tmp; cd tmp > $ ar x /path/to/libssl.a > $ ar x /path/to/libcrypto.a > $ cp /path/to/your/objects/*.o . > $ ar c ../your-library.a *.o > $ cd .. > $ rm -rf tmp > > (Untested, but see the ar manpage if you run into issues.) > > That should create a single archive library containing all the objects > from the three input libraries. Again, it relies on there being no filename > clashes among the objects; if there are, you'll have to rename some of them. > > -- > Michael Wojcik > Distinguished Engineer, Micro Focus > > > > -- Best Regards, Aijaz Baig -------------- next part -------------- An HTML attachment was scrubbed... URL: From aijazbaig1 at gmail.com Thu Nov 14 04:27:56 2019 From: aijazbaig1 at gmail.com (Aijaz Baig) Date: Thu, 14 Nov 2019 09:57:56 +0530 Subject: Static linking libssl.a and libcrypto.a on Linux x64 fails In-Reply-To: References: Message-ID: > Actually, that is also the format and mechanism with Microsoft Win32 tools, they just use the DOS-like file name "foo.lib" instead of "libfoo.a" to maintain makefile compatibility with their older Intel OMF-based toolchains. The object files inside > the archive are in COFF format, as they seem to have used Unix tools to bring up the initial "NT" operating system internally back before the initial 1993 release. On some platforms, where objects can be relinked, the constituent object files > produced by compiling source files are sometimes combined into a single large object. This is most often seen on AIX, which uses IBM's XCOFF object format (an enhanced COFF); XCOFF supports relinking objects, so you can bundle > objects up this way and save some time in symbol resolution when you link against the library later. But even on AIX this is commonly seen with dynamic libraries and relatively rare for static ones. As I mentioned earlier in my reply to Micheal, I would want to create a single archive that is a self contained 'package' (if you will) that alone would suffice for any application that requires openssl functionality. I've created certain wrappers over the original SSL APIs and the object files that contain this functionality (let's call this collectively as libapp.a) would be 'bundled' along with libssl.a and libcrypto.a. So as mike suggested, I'll combine the whole into a new shiny 'libappssl.a' and all that the application has to be is to pass this to the platform linker (which BTW is posix compliant) and that's all he requires. > Normally the linker isn't even involved in creating a static library. You compile sources to objects, and then use ar(1) to create the static library. The makefile you posted to StackOverflow doesn't include this step, so it's hard to tell what exactly you're doing. That StackOverflow question contains yet another link that points to another question (which I had asked about creating the library). The Makefile for that library contains this at the final linking (or archiving) stage: libAPP.a: $(obj) @ar rcs $@ $^ #$(LINK.c) -shared $^ -o $@ I was creating a shared library earlier but now I am creating a static archive from the object files. > Note: I seem to recall from a long time ago that GNU ar can combine static libraries directly (without all those temporary file names). > In BinUtils 2.25 this was apparently done by invoking ar in "MRI compatibility mode" and using the script command "ADDLIB" inside the provided MRI-style linker script. > For more details see the "ar scripts" part of the full GNU BinUtils TexInfo manual. > Enjoy > Jakob Well, I'll take a look however Mike's method is pretty easy enough to follow. However, what I am concerned with is, which other libraries do I need to archive (except libc which seems to be obvious I assume) to allow libssl and libcrypto to run on a stripped down system that resembles Linux (aka is POSIX compliant)? Regards, Aijaz On Thu, Nov 14, 2019 at 6:28 AM Aijaz Baig wrote: > > Thank you for the suggestion. Will try that. > > Regarding the static library, the term 'linking' I used was more tongue in cheek but nonetheless. However my current concern here is meeting libSSL and libCrypto's dependencies on host libraries on Linux platform. For instance, when I talked about 'linking' errors with respect to symbols like 'dlopen', so as I mentioned, I had to specify '-ldl' and '-lz' to the gcc linker that suggests a dynamic dependency on these platform libraries. > > I was trying to understand the components that would be needed to package the whole shebang into an archive which I can later 'just run' on a similar Linux system that has been completely stripped down for purposes of size and speed > > Is there a way to do that? > > On Wed, Nov 13, 2019 at 8:04 PM Michael Wojcik wrote: >> >> > From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of Aijaz Baig >> > Sent: Wednesday, November 13, 2019 01:45 >> >> > I am trying to statically link libssl.a and libcrypto.a into a static library of my own >> > which I will be using in an application (Linux). >> >> You can't link anything into a Linux static library, technically. >> >> ELF static libraries, like the older UNIX static libraries they're descended from, are just collections of object files, possibly with some additional metadata. (In BSD 4.x, for example, libraries often had an index member added using the ranlib utility, so that the linker didn't have to search the entire library for each symbol.) >> >> On some platforms, where objects can be relinked, the constituent object files produced by compiling source files are sometimes combined into a single large object. This is most often seen on AIX, which uses IBM's XCOFF object format (an enhanced COFF); XCOFF supports relinking objects, so you can bundle objects up this way and save some time in symbol resolution when you link against the library later. But even on AIX this is commonly seen with dynamic libraries and relatively rare for static ones. >> >> Normally the linker isn't even involved in creating a static library. You compile sources to objects, and then use ar(1) to create the static library. The makefile you posted to StackOverflow doesn't include this step, so it's hard to tell what exactly you're doing. >> >> But in any case, linking a static library against another static library is essentially a no-op. >> >> What you *can* do, if you don't want to have to list your library and the OpenSSL libraries when linking your application, is combine multiple static libraries into a single one - provided the object names don't conflict. This is straightforward: >> >> $ mkdir tmp; cd tmp >> $ ar x /path/to/libssl.a >> $ ar x /path/to/libcrypto.a >> $ cp /path/to/your/objects/*.o . >> $ ar c ../your-library.a *.o >> $ cd .. >> $ rm -rf tmp >> >> (Untested, but see the ar manpage if you run into issues.) >> >> That should create a single archive library containing all the objects from the three input libraries. Again, it relies on there being no filename clashes among the objects; if there are, you'll have to rename some of them. >> >> -- >> Michael Wojcik >> Distinguished Engineer, Micro Focus >> >> >> > > > -- > > Best Regards, > Aijaz Baig -- Best Regards, Aijaz Baig From joshelson at gmail.com Thu Nov 14 05:42:28 2019 From: joshelson at gmail.com (josh) Date: Wed, 13 Nov 2019 22:42:28 -0700 (MST) Subject: Crash in OpenSSL v1.0.1 from dtls1_do_write OPENSSL_assert(len == (unsigned int)ret); In-Reply-To: References: <3a41866e-07d0-d78d-e9b4-10dc5f63fdfd@openssl.org> Message-ID: <1573710148682-0.post@n7.nabble.com> For anyone running across this, we also were affected by this issue and worked with the Asterisk community to produce the patch referenced here: https://issues.asterisk.org/jira/browse/ASTERISK-28576 This will return nonzero values to prevent the hard assert, for anyone that is using packaged versions of OpenSSL that may be difficult to change. -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From jetson23 at hotmail.com Thu Nov 14 13:45:51 2019 From: jetson23 at hotmail.com (Jason Schultz) Date: Thu, 14 Nov 2019 13:45:51 +0000 Subject: sk_X509_OBJECT_num() In-Reply-To: References: , Message-ID: That makes sense. Thanks to everyone for the responses. Jason ________________________________ From: Dave Coombs Sent: Wednesday, November 13, 2019 5:30 PM To: Jason Schultz Cc: openssl-users at openssl.org Subject: Re: sk_X509_OBJECT_num() Hi, They're macros, defined in SKM_DEFINE_STACK_OF() in safestack.h. If you DEFINE_STACK_OF(Foo), you'll automatically end up with a sk_Foo_num() macro. Cheers, -Dave > On Nov 13, 2019, at 12:20, Jason Schultz wrote: > > Hello- > > I am updating my Linux application from using OpenSSL 1.0.2 to 1.1.1 in preparation for OpenSSL 3.0 (and of course the EOL of 1.0.2). I'm confused about the function in the subject line as well as other, related sk_X509_* functions. > > My code has always used these functions, and currently my code compiles and runs successfully against 1.1.1. I was sort of doing an audit of my code, evaluating the API calls that have changed vs not changed when I noticed these functions. I searched for them in the 1.1.1 source. They don't exist, except where called in x509_lu.c. In the 1.0.2 code base, they are called in the same file, as well as are defined in a header, /include/openssl/safestack.h. > > My question is, how are those symbols in my application being resolved since they are no longer found in the safestack.h header file? > > My system previously had OpenSSL 1.0.2 installed when I installed 1.1.1, but I don't think I have any old headers around that are being found when I compile and link. But for some reason this works. They obviously work within the OpenSSL 1.1.1 code also. > > I'm thinking I could be missing something basic about the compile/link process that explains this. Any ideas? > > Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pneumiller at directstream.com Thu Nov 14 17:46:41 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Thu, 14 Nov 2019 10:46:41 -0700 (MST) Subject: Why can't I force a specific cipher with the openssl app with TLS 1.3? Message-ID: <1573753601529-0.post@n7.nabble.com> Here is my server script is: PSK=63ef2024b1 openssl s_server -accept 4433 -tls1_3 -nocert -psk $PSK -ciphersuites TLS_AES_256_GCM_SHA384 Here is the client: PSK=63ef2024b1 openssl s_client -tls1_3 -psk $PSK -connect :4433 -ciphersuites TLS_AES_256_GCM_SHA384 And here is the error: Using default temp DH parameters ACCEPT ERROR C0:65:9F:08:01:00:00:00:error:SSL routines::no suitable signature algorithm:ssl/t1_lib.c:2810: shutting down SSL CONNECTION CLOSED So why can't I force the usage of this cipher? Why does it complain about signature algorithms when I didn't specify any? ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From matt at openssl.org Thu Nov 14 19:56:36 2019 From: matt at openssl.org (Matt Caswell) Date: Thu, 14 Nov 2019 19:56:36 +0000 Subject: Why can't I force a specific cipher with the openssl app with TLS 1.3? In-Reply-To: <1573753601529-0.post@n7.nabble.com> References: <1573753601529-0.post@n7.nabble.com> Message-ID: On 14/11/2019 17:46, Phil Neumiller wrote: > Here is my server script is: > > PSK=63ef2024b1 > openssl s_server -accept 4433 -tls1_3 -nocert -psk $PSK -ciphersuites > TLS_AES_256_GCM_SHA384 > > Here is the client: > > PSK=63ef2024b1 > openssl s_client -tls1_3 -psk $PSK -connect :4433 -ciphersuites > TLS_AES_256_GCM_SHA384 > > And here is the error: > > Using default temp DH parameters > ACCEPT > ERROR > C0:65:9F:08:01:00:00:00:error:SSL routines::no suitable signature > algorithm:ssl/t1_lib.c:2810: > shutting down SSL > CONNECTION CLOSED > > So why can't I force the usage of this cipher? Why does it complain about > signature algorithms when I didn't specify any? All TLSv1.3 PSKs must have an associated hash algorithm. The "-psk" arguument is the old TLSv1.2 way of doing PSKs. This works perfectly fine in TLSv1.3 too but uses a default hash. Which is, as defined by the TLSv1.3 spec, SHA-256. Because your chosen ciphersuite uses SHA-384 it is not compatible with a SHA-256 PSK and therefore the PSK attempt fails. Given that the attempt to use a PSK has failed OpenSSL then tries to fallback to a full handshake. One of the first steps in that is to select which certificate to use. It compares the set of signature algorithms supplied by the client to see which ones are compatible with the set of certificates configured for use on the server. In your case you have specified "-nocert" so, because there are no certificates, there are no suitable signature algorithms that match them. In order to get this to work you need to do one of two things: 1) Change your ciphersuite to one that uses SHA-256 or 2) Use the "-psksession" argument instead of "-psk". That will require you to have an SSL_SESSION object serialised into a file to read. That SSL_SESSION needs to be set up as described on this page: https://www.openssl.org/docs/man1.1.1/man3/SSL_set_psk_use_session_callback.html Matt From pneumiller at directstream.com Thu Nov 14 22:30:43 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Thu, 14 Nov 2019 15:30:43 -0700 (MST) Subject: Why can't I force a specific cipher with the openssl app with TLS 1.3? In-Reply-To: References: <1573753601529-0.post@n7.nabble.com> Message-ID: <1573770643888-0.post@n7.nabble.com> Hi Matt, That works fine for 256 as you mentioned. I trying to speak to a piece of hardware that has one supported cipher, i.e. TLS_AES_256_GCM_SHA384. I tried the naive approach of PSK=63ef2024b1 openssl s_server -accept 4433 -tls1_3 -nocert -psk $PSK -sigalgs RSA+SHA384 -ciphersuites TLS_AES_256_GCM_SHA384 And the server starts up as it does with ECDSA+SHA384. However, PSK=63ef2024b1 openssl s_client -tls1_3 -psk $PSK -connect :4433 -sigalgs RSA+SHA384 -ciphersuites TLS_AES_256_GCM_SHA384 Fails with invalid signature algorithm - which from your post I'm interpreting as I need a session file. The link you mentioned in your post only describes the problem from the call back or API perspective and I was really hoping to get this to work with something like: openssl s_server -session_file fname ... But when I follow that link it doesn't describe how to create the file. I seem to be misinterpreting something. Thanks, Phil ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From kristabrackin at yahoo.com Fri Nov 15 00:00:13 2019 From: kristabrackin at yahoo.com (Krista Brackin) Date: Fri, 15 Nov 2019 00:00:13 +0000 (UTC) Subject: Outbound FTP java errors References: <203472297.327464.1573776013222.ref@mail.yahoo.com> Message-ID: <203472297.327464.1573776013222@mail.yahoo.com> I am a novice...so any help please.... Below is the stack trace on the outbound file but I cannot make any setting change work....thoughts on what could be blocked me from transferring the file? log attached.? thank you!Krista -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: FTP_log.txt URL: From beldmit at gmail.com Fri Nov 15 06:30:15 2019 From: beldmit at gmail.com (Dmitry Belyavsky) Date: Fri, 15 Nov 2019 09:30:15 +0300 Subject: Outbound FTP java errors In-Reply-To: <203472297.327464.1573776013222@mail.yahoo.com> References: <203472297.327464.1573776013222.ref@mail.yahoo.com> <203472297.327464.1573776013222@mail.yahoo.com> Message-ID: I think it has nothing to do with OpenSSL. On Fri, Nov 15, 2019 at 3:00 AM Krista Brackin via openssl-users < openssl-users at openssl.org> wrote: > I am a novice...so any help please.... > > Below is the stack trace on the outbound file but I cannot make any > setting change work....thoughts on what could be blocked me from > transferring the file? log attached. > > thank you! > Krista > -- SY, Dmitry Belyavsky -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Fri Nov 15 09:25:10 2019 From: matt at openssl.org (Matt Caswell) Date: Fri, 15 Nov 2019 09:25:10 +0000 Subject: Why can't I force a specific cipher with the openssl app with TLS 1.3? In-Reply-To: <1573770643888-0.post@n7.nabble.com> References: <1573753601529-0.post@n7.nabble.com> <1573770643888-0.post@n7.nabble.com> Message-ID: <01a18709-21cb-1ad4-8452-0f7d422b1818@openssl.org> On 14/11/2019 22:30, Phil Neumiller wrote: > Hi Matt, > > That works fine for 256 as you mentioned. I trying to speak to a piece of > hardware that has one supported cipher, i.e. TLS_AES_256_GCM_SHA384. I > tried the naive approach of > > PSK=63ef2024b1 > openssl s_server -accept 4433 -tls1_3 -nocert -psk $PSK -sigalgs RSA+SHA384 > -ciphersuites TLS_AES_256_GCM_SHA384 > > And the server starts up as it does with ECDSA+SHA384. However, > > PSK=63ef2024b1 > openssl s_client -tls1_3 -psk $PSK -connect :4433 -sigalgs RSA+SHA384 > -ciphersuites TLS_AES_256_GCM_SHA384 > > Fails with invalid signature algorithm - which from your post I'm > interpreting as I need a session file. The link you mentioned in your post > only describes the problem from the call back or API perspective and I was > really hoping to get this to work with something like: > > openssl s_server -session_file fname ... > > But when I follow that link it doesn't describe how to create the file. I > seem to be misinterpreting something. Both s_client and s_server have the option of taking a psk session file on the command line using the "-psk_session" option (sorry I think I referred to this as "-psksession" in my previous email - without the underscore) - but you need to have a session file already in existence in order for that to work. It might be nice if we added a new option "-pskmd" or similar which enabled you to specify the md from the command line without having to have a session file first. However that isn't currently possible. Ideally you would create your session file in the manner described on the man page I linked to. However that does require some C programming. There is a "hack" you can use to generate a session file from the command line. However this will generate a new random PSK so if you have a specific value already that you need to use then it won't help. I also wouldn't necessarily recommend this approach for production use - but for testing purposes it should be fine. You'll need a server certificate and server key in order to generate the file (any certificate including self-signed will be just fine). Run an s_server instance like this, specifying your cert/key: $ openssl s_server -ciphersuites TLS_AES_256_GCM_SHA384 -cert server.pem -key server.pem Now connect via s_client: $ openssl s_client -ciphersuites TLS_AES_256_GCM_SHA384 -sess_out psksession.pem This will generate you a session file called psksession.pem compatible with the TLS_AES_256_GCM_SHA384 ciphersuite. Now you should be able to use it as a PSK: $ openssl s_server -psk_session psksession.pem -nocert -ciphersuites TLS_AES_256_GCM_SHA384 $ openssl s_client -ciphersuites TLS_AES_256_GCM_SHA384 -psk_session psksession.pem However if you need to use a a specific PSK value then, at the moment, the only real choice is to programmatically create the session file. Matt From openssl-users at dukhovni.org Fri Nov 15 09:43:26 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 15 Nov 2019 04:43:26 -0500 Subject: Why can't I force a specific cipher with the openssl app with TLS 1.3? In-Reply-To: <01a18709-21cb-1ad4-8452-0f7d422b1818@openssl.org> References: <1573753601529-0.post@n7.nabble.com> <1573770643888-0.post@n7.nabble.com> <01a18709-21cb-1ad4-8452-0f7d422b1818@openssl.org> Message-ID: <7BEF31DC-76F3-409B-B398-D0663CF92DEF@dukhovni.org> > On Nov 15, 2019, at 4:25 AM, Matt Caswell wrote: > > It might be nice if we added a new option "-pskmd" or similar which > enabled you to specify the md from the command line without having to > have a session file first. However that isn't currently possible. With a saved session there may actually be enough key material to arrive at non-trivial security. As it stands, the OP wrote: > PSK=63ef2024b1 > openssl s_client -tls1_3 -psk $PSK -connect :4433 -ciphersuites TLS_AES_256_GCM_SHA384 That 40-bit PSK does not provide much security. I would hope that "in real life" (simple tests aside) the PSKs will have non-trivial entropy. -- Viktor. From meikkr at gmail.com Fri Nov 15 11:18:39 2019 From: meikkr at gmail.com (Meik Kreyenkoetter) Date: Fri, 15 Nov 2019 12:18:39 +0100 Subject: CMS with ECC Keys is incompatibel to Windows CMS / Outlook Message-ID: <23BF76CE-437E-406F-9416-1EF1F34BA2D9@gmail.com> Hello, when generating a CMS with OpenSSL 1.1.1d or OpenSSL 1.0.2g using only ECC Keys, Windows 10 is unable to decrypt the CMS. All Passwords for keys is "test". Encrypting: openssl cms -encrypt -outform PEM -recip bob.pem -in Test.eml -out opensslencrypted.cms -aes256 -aes128-wrap Decryption on Windows 10 (with installed Keys in Store): Unprotect-CmsMessage -Path .\opensslencrypted.cms Unprotect-CmsMessage : Die Daten sind unzul?ssig. In Zeile:1 Zeichen:1 + Unprotect-CmsMessage -Path .\opensslencrypted.cms + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [Unprotect-CmsMessage], CryptographicException + FullyQualifiedErrorId : System.Security.Cryptography.CryptographicException,Microsoft.PowerShell.Commands.Unprot ectCmsMessageCommand The file outlookencrypted.cms contains a CMS with ECC keys generated on Windows 10. It's decryptable by Windows and OpenSSL. Inspecting the Windows and Openssl generated CMS, they both look ok. The only difference if have seen in CMS -print output is parameter absent in openssl generated and NULL in Windows generated: OpenSSL, openssl cms -in opensslencrypted.cms -cmsout -print -inform PEM: recipientInfos: d.kari: version: 3 d.originatorKey: algorithm: algorithm: id-ecPublicKey (1.2.840.10045.2.1) parameter: publicKey: (0 unused bits) Windows generated, openssl cms -in outlookencrypted.cms -cmsout -print -inform PEM: recipientInfos: d.kari: version: 3 d.originatorKey: algorithm: algorithm: id-ecPublicKey (1.2.840.10045.2.1) parameter: NULL publicKey: (0 unused bits) I have changed the OpenSSL sources to include "parameter: NULL" in CMS generation, but that makes no difference. The CMS with changed sources is decryptable by OpenSSL, but not on Windows: openssl cms -decrypt -in opensslencrypted_changed_sources.cms -inform PEM -recip bob.pem I have attached all keys and output. Anything i am missing here? Meik -------------- next part -------------- A non-text attachment was scrubbed... Name: opensslencrypted_changed_sources.cms Type: application/octet-stream Size: 693 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: outlookencrypted.cms Type: application/octet-stream Size: 3906 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: opensslencrypted.cms Type: application/octet-stream Size: 693 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cacert.crt Type: application/x-x509-ca-cert Size: 940 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bob at external.com.p12 Type: application/x-pkcs12 Size: 1399 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bob.pem Type: application/x-x509-ca-cert Size: 2052 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bob.cer Type: application/x-x509-ca-cert Size: 1074 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: alice at internal.com.p12 Type: application/x-pkcs12 Size: 1419 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: alice.pem Type: application/x-x509-ca-cert Size: 2074 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: alice.cer Type: application/x-x509-ca-cert Size: 1086 bytes Desc: not available URL: -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: no subject Date: no date Size: 123 URL: -------------- next part -------------- From meikkr at gmail.com Fri Nov 15 16:11:20 2019 From: meikkr at gmail.com (Meik Kreyenkoetter) Date: Fri, 15 Nov 2019 17:11:20 +0100 Subject: CMS with ECC Keys is incompatibel to Windows CMS / Outlook In-Reply-To: <23BF76CE-437E-406F-9416-1EF1F34BA2D9@gmail.com> References: <23BF76CE-437E-406F-9416-1EF1F34BA2D9@gmail.com> Message-ID: Hello again, maybe i have found the difference in the CMSes generated by OpenSSL and Windows. This is the keyEncryptionAlgorithm in kari generated on Windows: keyEncryptionAlgorithm: algorithm: dhSinglePass-stdDH-sha1kdf-scheme (1.3.133.16.840.63.0.2) parameter: SEQUENCE: 0:d=0 hl=2 l= 13 cons: SEQUENCE 2:d=1 hl=2 l= 9 prim: OBJECT :id-aes256-wrap 13:d=1 hl=2 l= 0 prim: NULL recipientEncryptedKeys: This is the keyEncryptionAlgorithm in kari generated with OpenSSL: keyEncryptionAlgorithm: algorithm: dhSinglePass-stdDH-sha1kdf-scheme (1.3.133.16.840.63.0.2) parameter: SEQUENCE: 0:d=0 hl=2 l= 11 cons: SEQUENCE 2:d=1 hl=2 l= 9 prim: OBJECT :id-aes256-wrap recipientEncryptedKeys: As one can see, there is a NULL at the end of the parameter sequence generated on Windows. CMS output from BouncyCaste is like OpenSSL: keyEncryptionAlgorithm: algorithm: dhSinglePass-stdDH-sha1kdf-scheme (1.3.133.16.840.63.0.2) parameter: SEQUENCE: 0:d=0 hl=2 l= 11 cons: SEQUENCE 2:d=1 hl=2 l= 9 prim: OBJECT :id-aes128-wrap The BouncyCaste output is not decryptable on Windows. Is there a way generate a CMS with ECC compatible with Windows? Meik > On 15. Nov 2019, at 12:18, Meik Kreyenkoetter wrote: > > Hello, > > when generating a CMS with OpenSSL 1.1.1d or OpenSSL 1.0.2g using only ECC Keys, Windows 10 is unable to decrypt the CMS. > All Passwords for keys is "test". > > Encrypting: > > openssl cms -encrypt -outform PEM -recip bob.pem -in Test.eml -out opensslencrypted.cms -aes256 -aes128-wrap > > Decryption on Windows 10 (with installed Keys in Store): > > Unprotect-CmsMessage -Path .\opensslencrypted.cms > > Unprotect-CmsMessage : Die Daten sind unzul?ssig. > In Zeile:1 Zeichen:1 > + Unprotect-CmsMessage -Path .\opensslencrypted.cms > + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + CategoryInfo : NotSpecified: (:) [Unprotect-CmsMessage], CryptographicException > + FullyQualifiedErrorId : System.Security.Cryptography.CryptographicException,Microsoft.PowerShell.Commands.Unprot > ectCmsMessageCommand > > > The file outlookencrypted.cms contains a CMS with ECC keys generated on Windows 10. It's decryptable by Windows and OpenSSL. > > Inspecting the Windows and Openssl generated CMS, they both look ok. The only difference if have seen in CMS -print output is parameter absent in openssl generated and NULL in Windows generated: > > OpenSSL, openssl cms -in opensslencrypted.cms -cmsout -print -inform PEM: > > recipientInfos: > d.kari: > version: 3 > d.originatorKey: > algorithm: > algorithm: id-ecPublicKey (1.2.840.10045.2.1) > parameter: > publicKey: (0 unused bits) > > Windows generated, openssl cms -in outlookencrypted.cms -cmsout -print -inform PEM: > > recipientInfos: > d.kari: > version: 3 > d.originatorKey: > algorithm: > algorithm: id-ecPublicKey (1.2.840.10045.2.1) > parameter: NULL > publicKey: (0 unused bits) > > I have changed the OpenSSL sources to include "parameter: NULL" in CMS generation, but that makes no difference. The CMS with changed sources is decryptable by OpenSSL, but not on Windows: > > openssl cms -decrypt -in opensslencrypted_changed_sources.cms -inform PEM -recip bob.pem > > I have attached all keys and output. > > Anything i am missing here? > > > Meik > > > > > > From pneumiller at directstream.com Fri Nov 15 22:03:38 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Fri, 15 Nov 2019 15:03:38 -0700 (MST) Subject: How do I turn off EC point formats from showing up in TLS 1.3 client hello? Message-ID: <1573855418193-0.post@n7.nabble.com> TLS 1.3 doesn't use EC point formats right? I don't know why they are in my TLS 1.3 client hello. Extension: ec_point_formats (len=4) Type: ec_point_formats (11) Length: 4 EC point formats Length: 3 Elliptic curves point formats (3) EC point format: uncompressed (0) EC point format: ansiX962_compressed_prime (1) EC point format: ansiX962_compressed_char2 (2) There is a flag OPENSSL_NO_EC do I need to set that somewhere in config or make? Is there an API call do disable these? There is a get function SSL_get0_ec_point_formats() but no set. Thanks, Phil ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From kwebb at teradactyl.com Fri Nov 15 22:10:55 2019 From: kwebb at teradactyl.com (Kristen Webb) Date: Fri, 15 Nov 2019 15:10:55 -0700 Subject: Can a linux service work as both TLS client and server? Message-ID: Is there a way for a single program to act as both a TLS client and a TLS server after a TCP/IP accept() call? Today, I simply have the TCP connecting process issue a 1 or 0 to indicate how it is acting. This is then used to determine who does SSL_accept and SSL_connect and everything works out. My original idea was that I could then configure any number of supporting services on the same port going forward. I'd like to remove this 1 time TCP write/read operation. For example, I cannot see how this will work with Apple's network framework going forward. I currently have 3 authentication use cases Server side cert (currently only works in one direction without my workaround) Server side cert with GSSAPI (currently only works in one direction without my workaround) Client/Server certs (so this one should work either way) Will PSK allow my service to say, always act as a TLS server without a server certificate? Could I then proceed with additional certificate functions (e.g. for GSSAPI)? Or should I go back and grovel for another port and use this information to explain why I need one? Many thanks in advance for any insights! Kris -- This message is NOT encrypted -------------------------------- Mr. Kristen J. Webb Chief Technology Officer Teradactyl LLC. 2450 Baylor Dr. S.E. Albuquerque, New Mexico 87106 Phone: 1-505-338-6000 Email: kwebb at teradactyl.com Web: http://www.teradactyl.com Providers of Scalable Backup Solutions for Unique Data Environments -------------------------------- NOTICE TO RECIPIENTS: Any information contained in or attached to this message is intended solely for the use of the intended recipient(s). If you are not the intended recipient of this transmittal, you are hereby notified that you received this transmittal in error, and we request that you please delete and destroy all copies and attachments in your possession, notify the sender that you have received this communication in error, and note that any review or dissemination of, or the taking of any action in reliance on, this communication is expressly prohibited. Regular internet e-mail transmission cannot be guaranteed to be secure or error-free. Therefore, we do not represent that this information is complete or accurate, and it should not be relied upon as such. If you prefer to communicate with Teradactyl LLC. using secure (i.e., encrypted and/or digitally signed) e-mail transmission, please notify the sender. Otherwise, you will be deemed to have consented to communicate with Teradactyl via regular internet e-mail transmission. Please note that Teradactyl reserves the right to intercept, monitor, and retain all e-mail messages (including secure e-mail messages) sent to or from its systems as permitted by applicable law -------------- next part -------------- An HTML attachment was scrubbed... URL: From pneumiller at directstream.com Fri Nov 15 22:36:31 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Fri, 15 Nov 2019 15:36:31 -0700 (MST) Subject: Can a linux service work as both TLS client and server? In-Reply-To: References: Message-ID: <1573857391051-0.post@n7.nabble.com> Sure, you just need additional threads. Note: accept is a blocking call so the thread that runs in (i.e. your server side will block until a packet is received). You can write a polling loop using select, that doesn't block. The cleanest thing to do is have a thread for client(s) and one for server. I have done this with C++17 with TLS1_3_Client and TLS1_3_Server classes with accept loop member functions started as std::thread. ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From kwebb at teradactyl.com Fri Nov 15 22:49:04 2019 From: kwebb at teradactyl.com (Kristen Webb) Date: Fri, 15 Nov 2019 15:49:04 -0700 Subject: Can a linux service work as both TLS client and server? In-Reply-To: <1573857391051-0.post@n7.nabble.com> References: <1573857391051-0.post@n7.nabble.com> Message-ID: Hi Phil, Thanks for such a fast response! I am doing the polling today. I believe I left something very important out of my original question. I only have 1 well known port to accept all of my connections. TLS_client_app -> service on portA (needs to be a TLS_server) TLS_server_app -> service on portA (needs to be a TLS_client) The problem is that when the service accept()'s the connection it does not know what type of app made the connection so it cannot decide if it should act as the TLS client or server (unless I send a 1/0 hint over TCP first). Kris On Fri, Nov 15, 2019 at 3:28 PM Phil Neumiller wrote: > Sure, you just need additional threads. Note: accept is a blocking call so > the thread that runs in (i.e. your server side will block until a packet is > received). You can write a polling loop using select, that doesn't block. > The cleanest thing to do is have a thread for client(s) and one for > server. > I have done this with C++17 with TLS1_3_Client and TLS1_3_Server classes > with accept loop member functions started as std::thread. > > > > ----- > Phillip Neumiller > Platform Engineering > Directstream, LLC > -- > Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html > -- This message is NOT encrypted -------------------------------- Mr. Kristen J. Webb Chief Technology Officer Teradactyl LLC. 2450 Baylor Dr. S.E. Albuquerque, New Mexico 87106 Phone: 1-505-338-6000 Email: kwebb at teradactyl.com Web: http://www.teradactyl.com Providers of Scalable Backup Solutions for Unique Data Environments -------------------------------- NOTICE TO RECIPIENTS: Any information contained in or attached to this message is intended solely for the use of the intended recipient(s). If you are not the intended recipient of this transmittal, you are hereby notified that you received this transmittal in error, and we request that you please delete and destroy all copies and attachments in your possession, notify the sender that you have received this communication in error, and note that any review or dissemination of, or the taking of any action in reliance on, this communication is expressly prohibited. Regular internet e-mail transmission cannot be guaranteed to be secure or error-free. Therefore, we do not represent that this information is complete or accurate, and it should not be relied upon as such. If you prefer to communicate with Teradactyl LLC. using secure (i.e., encrypted and/or digitally signed) e-mail transmission, please notify the sender. Otherwise, you will be deemed to have consented to communicate with Teradactyl via regular internet e-mail transmission. Please note that Teradactyl reserves the right to intercept, monitor, and retain all e-mail messages (including secure e-mail messages) sent to or from its systems as permitted by applicable law -------------- next part -------------- An HTML attachment was scrubbed... URL: From pneumiller at directstream.com Fri Nov 15 23:08:37 2019 From: pneumiller at directstream.com (Phil Neumiller) Date: Fri, 15 Nov 2019 16:08:37 -0700 (MST) Subject: Can a linux service work as both TLS client and server? In-Reply-To: References: <1573857391051-0.post@n7.nabble.com> Message-ID: <1573859317865-0.post@n7.nabble.com> Yes, so you accept thread needs to either fork() or spawn another thread to process the packet and go back into the accept loop for another connection. ----- Phillip Neumiller Platform Engineering Directstream, LLC -- Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html From kwebb at teradactyl.com Fri Nov 15 23:27:20 2019 From: kwebb at teradactyl.com (Kristen Webb) Date: Fri, 15 Nov 2019 16:27:20 -0700 Subject: Can a linux service work as both TLS client and server? In-Reply-To: <1573859317865-0.post@n7.nabble.com> References: <1573857391051-0.post@n7.nabble.com> <1573859317865-0.post@n7.nabble.com> Message-ID: In the future, I will not have an initial TCP 1/0 packet (clue) to process. So I have no way to decide if my forked/spawned process should SSL_connect or SSL_accept. For example, I cannot see how this can be done with Apple's network framework (at least not yet). It appears to be so high level as to not allow me to process a TCP packet within a TLS style connection. I realize that this is not an openssl issue. And I do have things working today with Apples security framework and openssl (with that extra TCP packet clue in place). I am more familiar with openssl and I'm trying to code everything there first. Also my entire application runs on linux so I am able to test all the combinations easily from there. And I'll need it to work with Apple's networking in the future as their security APIs go away. Thank you for bearing with me so far! On Fri, Nov 15, 2019 at 4:01 PM Phil Neumiller wrote: > Yes, so you accept thread needs to either fork() or spawn another thread to > process the packet and go back into the accept loop for another connection. > > > > ----- > Phillip Neumiller > Platform Engineering > Directstream, LLC > -- > Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html > -- This message is NOT encrypted -------------------------------- Mr. Kristen J. Webb Chief Technology Officer Teradactyl LLC. 2450 Baylor Dr. S.E. Albuquerque, New Mexico 87106 Phone: 1-505-338-6000 Email: kwebb at teradactyl.com Web: http://www.teradactyl.com Providers of Scalable Backup Solutions for Unique Data Environments -------------------------------- NOTICE TO RECIPIENTS: Any information contained in or attached to this message is intended solely for the use of the intended recipient(s). If you are not the intended recipient of this transmittal, you are hereby notified that you received this transmittal in error, and we request that you please delete and destroy all copies and attachments in your possession, notify the sender that you have received this communication in error, and note that any review or dissemination of, or the taking of any action in reliance on, this communication is expressly prohibited. Regular internet e-mail transmission cannot be guaranteed to be secure or error-free. Therefore, we do not represent that this information is complete or accurate, and it should not be relied upon as such. If you prefer to communicate with Teradactyl LLC. using secure (i.e., encrypted and/or digitally signed) e-mail transmission, please notify the sender. Otherwise, you will be deemed to have consented to communicate with Teradactyl via regular internet e-mail transmission. Please note that Teradactyl reserves the right to intercept, monitor, and retain all e-mail messages (including secure e-mail messages) sent to or from its systems as permitted by applicable law -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Fri Nov 15 23:54:22 2019 From: matt at openssl.org (Matt Caswell) Date: Fri, 15 Nov 2019 23:54:22 +0000 Subject: How do I turn off EC point formats from showing up in TLS 1.3 client hello? In-Reply-To: <1573855418193-0.post@n7.nabble.com> References: <1573855418193-0.post@n7.nabble.com> Message-ID: On 15/11/2019 22:03, Phil Neumiller wrote: > > TLS 1.3 doesn't use EC point formats right? I don't know why they are in my > TLS 1.3 client hello. No, its not used in TLSv1.3 but is used in TLSv1.2 or below. A ClientHello is sent before version negotiation takes place so you don't know what version will eventually be selected. Therefore, if EC is enabled, this extension is always added regardless. A possible improvement would be for OpenSSL to detect whether TLSv1.3 is the only enabled protocol version on the client and disable it in those circumstances. But it doesn't currently check this. Currently the only way to disable this extension is to disable EC. However that means (assuming you are using 1.1.1 instead of master) that TLSv1.3 will not work since EC is required for the 1.1.1 TLSv1.3 support. In master that isn't the case so I guess it might be possible there but I've not tried it. Matt From vieuxtech at gmail.com Sat Nov 16 01:02:33 2019 From: vieuxtech at gmail.com (Sam Roberts) Date: Fri, 15 Nov 2019 17:02:33 -0800 Subject: Can a linux service work as both TLS client and server? In-Reply-To: References: <1573857391051-0.post@n7.nabble.com> <1573859317865-0.post@n7.nabble.com> Message-ID: I'm curious, its pretty unusual to not know which side of a TCP connection is the client or server, not just TLS, HTTP, SMTP, .... etc. Its almost always the side that makes the accept() call that's the server, but that doesn't have to be. Why is it that you do not in this context? Without it, you are fairly far off the beaten path. A normal TLS client would not be able to connect to your "accept()" side if it tried to be a client, and a normal TLS server would never initiate a connection to your "accept()" in the hopes that the acceptor would turn around and be a client. The 1/0 isn't so terrible (well, maybe the protocol is terrible :-). To do a custom negotiation, then "step up" to TLS is done by other protocols. You could also sniff the TCP after accept, and wait a while to see if a client hello arrives to know if the other side is a client (or wait for any data, I don't think a server starts sending data until it gets something, but its been a while since I looked). Hope thats helpful, and even if not, it'd be interesting to know what apple is doing that is pushing you down this path. Sam -------------- next part -------------- An HTML attachment was scrubbed... URL: From Michael.Wojcik at microfocus.com Sat Nov 16 02:04:24 2019 From: Michael.Wojcik at microfocus.com (Michael Wojcik) Date: Sat, 16 Nov 2019 02:04:24 +0000 Subject: Can a linux service work as both TLS client and server? In-Reply-To: References: <1573857391051-0.post@n7.nabble.com> <1573859317865-0.post@n7.nabble.com> Message-ID: > From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of Kristen Webb > Sent: Friday, November 15, 2019 18:27 > > So I have no way to decide if my forked/spawned process should SSL_connect > or SSL_accept. > > For example, I cannot see how this can be done with Apple's network framework > (at least not yet). It appears to be so high level as to not allow me > to process a TCP packet within a TLS style connection. It's not entirely clear to me what you're trying to do. Is this a fair description? 1. You have a process (which happens to be running on Linux and is a "service", though that's not a specific thing on Linux) which is accepting TCP connections. In technical terms, it's doing TCP passive opens: it has a stream socket bound to a port, in LISTENING state, and it calls accept on that socket. (Whether you multiplex that, or use non-blocking accept, or just block in accept until it's ready is irrelevant.) 2. When you get a conversation (i.e. ESTABLISHED state) socket from accept, you want to perform one of two actions: 2.1. If the peer intends to act as a TLS client, you want to call SSL_accept to create the TLS tunnel. That is, you'll do a passive TLS open, which is what normal TLS applications would do following a passive TCP open. 2.2. If the peer intends to act as a TLS server, you want to call SSL_connect, performing the active TLS open even though you did a passive TCP open. This is weird, frankly; but there's nothing it TCP or TLS to prevent it. 2.3. However, you don't know what the peer intends to do yet. There's no deterministic way of solving this. You only have one piece of information at this point: the conversation's 5-tuple address (address family, local IP address, local port, remote IP address, remote port). Conventionally, a TCP server (i.e. a process that does passive opens) which handles different client behaviors will establish the client's intent from either the local port or the initial transmission from the client. The former is out of the question, since the whole point of this exercise is to multiplex different behaviors on the same local port. So, your program needs information from the client to figure out what to do. Offhand I see only one way to resolve this: you need to interpose logic between OpenSSL and the actual communications endpoint. OpenSSL has a standard mechanism for this: the BIO pair. You'll want to do something along these lines. 1. Determine how long you'll wait for an initial transmission from the client. You have to have some timeout here, because if the client's doing a passive TLS open it will wait for data from you. But then the FPL theorem says in general a distributed system has to employ timeouts anyway if it wants to guarantee completion. We're just making that requirement a bit more concrete. Let's say you'll wait for a configurable number of seconds, and default to, oh, 5s. 3. Rather than creating a socket BIO around your conversation socket as a normal OpenSSL application would, you create a BIO pair. One half of the pair will be the external BIO; you'll transfer data between that BIO and the socket. The other half of the pair will be the internal BIO, and that will be the one associated with OpenSSL operations. Create your SSL object and associate it with the internal BIO. 4. Wait for your conversation socket to become readable, for your timeout period. 4.1. If the socket becomes readable, do a conventional recv (or read, or the system call of your choice) on it. Handle errors and conversation termination as necessary. Mark the conversation as passive (you're the server, doing the TLS accept) in your own conversation information. Write the data to the external BIO. 4.2. If the socket does not become readable, mark the conversation as active and perform the SSL_connect on the internal BIO. 5. From this point on you want OpenSSL to operate on the internal BIO, while copying data between the socket and the extternal BIO. So start a thread that does the following: 5.1. Check to see if the socket is readable. If so, receive data (do error and termination handling) and write it to the external BIO. (If the external BIO's buffer is full, you may have to buffer the data somewhere else, or just yield your timeslice and hope the conversation-processing thread gets its act together.) 5.2. Check to see if the external BIO is readable. (See the OpenSSL man page for BIO_make_bio_pair for more information.) If so, read data from it and write to the socket. You can check the socket for writability first if you want to avoid blocking. (For a maximally-robust solution this all could get somewhat complicated; using a decent set of abstractions would be a goodd plan.) 5.3. If neither is readable, go to sleep. You can use poll() or similar here to sleep with a timeout on the socket becoming readable. Unfortunately you can't trivially multiplex on a BIO, as its internal descriptor (if it even has one) isn't exposed by the public API. You *could* set a callback on the external BIO that writes a trigger byte to a control pipe that you add to the poll set, if you want to get fancy. (As I said, this can get complicated.) 6. Meanwhile, your conversation thread goes off and does its conversation stuff, using the SSL object you created before. Actually, having written all that, it occurs to me there's an easier approach. When you get the conversation socket, wait for the timeout period for it to become readable. If it does, mark the conversation as passive; if it doesn't, mark it as active. Start your conversation logic. The first thing that should do is look at that active/passive flag and do an SSL_connect or an SSL_accept. You don't need the BIO pair because you don't care about what the data before OpenSSL reads it or before its sent to the peer; you just care whether there *is* data from the peer, within the timeout window at the beginning of the conversation. The BIO-pair arrangement is useful if you need to intervene more extensively at the layer between OpenSSL and the network endpoint for some reason, e.g. to do sophisticated buffering or to communicate using some non-BIO-compatible interface. -- Michael Wojcik Distinguished Engineer, Micro Focus From karl at denninger.net Sat Nov 16 02:40:41 2019 From: karl at denninger.net (Karl Denninger) Date: Fri, 15 Nov 2019 20:40:41 -0600 Subject: Can a linux service work as both TLS client and server? In-Reply-To: References: <1573857391051-0.post@n7.nabble.com> <1573859317865-0.post@n7.nabble.com> Message-ID: <26f11347-95de-eaf6-b2bd-4fa0d4d75654@denninger.net> On 11/15/2019 17:27, Kristen Webb wrote: > In the future, I will not have an initial?TCP 1/0 packet (clue) to > process. > So I have no way to decide if my forked/spawned process should SSL_connect > or SSL_accept. > > For example, I cannot see how this can be done with Apple's network > framework > (at least not yet).? It appears to be so high level as to not allow me > to process a TCP packet within a TLS style connection.? I realize that > this is not > an openssl issue.? And I do have things working today with Apples security > framework and openssl (with that extra TCP packet clue in place).? I > am more familiar > with openssl and I'm trying?to code everything there first.? Also my > entire application > runs on linux so I am able to test all the combinations easily from > there.? And I'll > need it to work with Apple's networking in the future as their > security APIs go away. > > Thank you for bearing with me so far! > I don't quite understand what you're attempting to do, or why. I assume (since you're sending the initial packet) that the "thing" connecting to the OpenSSL end is under your control (it's your code.)? If so, why do you care which "way" the listening end comes up? By convention if you are doing a listen() on an a socket then you're a server.? You don't have to be from an SSL perspective, but from a socket perspective you absolutely are. So why do you want to select "which way" you do this on the TLS/SSL end?? Is it a function of which end has a certificate (or whether both do) and which one(s) you care to verify (or not)?? If so that can be dealt with through options and who checks what, rather than what you're doing now. I'm trying to understand the workflow you are attempting to implement, and why, because I suspect you may be going about this the hard way. -- Karl Denninger karl at denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4897 bytes Desc: S/MIME Cryptographic Signature URL: From openssl-users at dukhovni.org Sat Nov 16 03:10:32 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 15 Nov 2019 22:10:32 -0500 Subject: Can a linux service work as both TLS client and server? In-Reply-To: References: Message-ID: <20191116031032.GN34850@straasha.imrryr.org> On Fri, Nov 15, 2019 at 03:10:55PM -0700, Kristen Webb wrote: > Is there a way for a single program to act as both a TLS client and a TLS > server after a TCP/IP accept() call? Yes, but as you're aware and others have mentioned it has to decide which somehow. > Today, I simply have the TCP connecting process issue a 1 or 0 to indicate > how it is acting. This is then used to determine who does SSL_accept and > SSL_connect and everything works out. That's one valid way to do that. Whoever is the server will need some sort of server certificte or have a PSK in common with the client. The client can also use ia certificate, or authenticate via GSSAPI after the TLS connection is established, it could also perform GSSAPI channel binding to the server certificate. > Will PSK allow my service to say, always act as a TLS server without a > server certificate? Yes. And you can even negotiate the PSK via an initial GSSAPI session establishment. Then just use a nominal PSK id (say a single '\0' byte) that signals the just negotiated PSK. Here you might have the TCP client also always be the GSSAPI initiator, but tell the (TCP/GSSAPI) server who will be the TLS server. > Could I then proceed with additional certificate functions (e.g. for > GSSAPI)? With that you would not need after-the-fact GSSAPI, because GSSAPI authentication is implied via the PSK. -- Viktor. From kwebb at teradactyl.com Sat Nov 16 15:21:30 2019 From: kwebb at teradactyl.com (Kristen Webb) Date: Sat, 16 Nov 2019 08:21:30 -0700 Subject: Can a linux service work as both TLS client and server? In-Reply-To: <20191116031032.GN34850@straasha.imrryr.org> References: <20191116031032.GN34850@straasha.imrryr.org> Message-ID: First, thanks to everyone for all the useful feedback. I really appreciate it! Let me try to explain my situation again as to why I need a service to do both. 1. I have a single service listening on a well known port. it is a backup service that runs on the backup server which will always be the TLS_server (will always have a certificate). The same service runs on multiple backup clients where it acts as the TLS_client, listening for connections from the backup server. 2. In a common mode, a separate backup server process (TLS_server) will connect to it and the backup service needs to act as a TLS_client. 3. In another mode a backup client (TLS_client) will connect the backup service on the backup server and that service needs to act as a TLS_server It sounds like peeking at the port may be the simplest way to determine how it is being connected to. As I write this I realize that the service running on all of the other backup clients really only needs 2. In this case, the backup client is a TLS_client, but it will be doing the TCP accept. No need to decide which mode to use in this case.. It is the backup server's backup service that needs to do both and will always be a linux server, so setting up peek should work well. Apple has a network framework (supports TLS 1.3) that I believe will eventually replace the current security framework (supports up to TLS 1.2) that I am using today to get the job done. There is example code here: https://developer.apple.com/documentation/network/implementing_netcat_with_network_framework I have been experimenting with this code. I could not get it to work on Mojave, but I could get it to work on Catalina. To summarize how I think it works: 1. You set up all of the connection parameters up front (TCP, TLS, TLS-PSK, UDP, bonjour, etc, etc). 2. Next, create an endpoint using nw_endpoint_create_host 3. In the case of a service (listener) call nw_listener_create with the parameters. You now have an opaque object and in TLS mode, I do not see a way to access the underlying TCP socket. I've just realized that I need to do more testing against my common, but odd case, where my client's backup service needs to accept, but process w/o a cert as a TLS_client. Joy! Doing more digging today, I find in https://forums.developer.apple.com/thread/116221 __BEGIN_COMMENTS__ I've not seen an equivalent way of message peeking Such a mechanism does not exist. or even getting the ? socket. Nor does that, at least in general. Network framework was designed to work in conjunction with our user space networking stack, and that stack does not have an underlying socket because sockets are a kernel-only concept. Now, on macOS, Network framework does actually talk to the kernel, but that?s just a compatibility measure: We can?t enable the user space networking stack on the Mac while continuing to support Network Kernel Extensions (NKEs). NKEs have been informally deprecated for a while now, so the expectation is that they?ll be formally deprecated and then removed in future OS releases, at which point macOS will work like all our other OSes in this regard. You can learn more about the background to this in: WWDC 2017 Session 707 Advances in Networking, Part 1 WWDC 2018 Session 715 Introducing Network Framework All of this is to say that, if you can?t do what you want with Network framework, the time to an enhancement request for the facilities that you need is now. __END_COMMENTS It appears that if you do not keep up with the apple way of doing things on OSX at some point you will be locked out? Kris On Fri, Nov 15, 2019 at 8:10 PM Viktor Dukhovni wrote: > On Fri, Nov 15, 2019 at 03:10:55PM -0700, Kristen Webb wrote: > > > Is there a way for a single program to act as both a TLS client and a TLS > > server after a TCP/IP accept() call? > > Yes, but as you're aware and others have mentioned it has to decide > which somehow. > > > Today, I simply have the TCP connecting process issue a 1 or 0 to > indicate > > how it is acting. This is then used to determine who does SSL_accept and > > SSL_connect and everything works out. > > That's one valid way to do that. Whoever is the server will need > some sort of server certificte or have a PSK in common with the > client. The client can also use ia certificate, or authenticate > via GSSAPI after the TLS connection is established, it could also > perform GSSAPI channel binding to the server certificate. > > > Will PSK allow my service to say, always act as a TLS server without a > > server certificate? > > Yes. And you can even negotiate the PSK via an initial GSSAPI > session establishment. Then just use a nominal PSK id (say a single > '\0' byte) that signals the just negotiated PSK. Here you might > have the TCP client also always be the GSSAPI initiator, but tell > the (TCP/GSSAPI) server who will be the TLS server. > > > Could I then proceed with additional certificate functions (e.g. for > > GSSAPI)? > > With that you would not need after-the-fact GSSAPI, because GSSAPI > authentication is implied via the PSK. > > -- > Viktor. > -- This message is NOT encrypted -------------------------------- Mr. Kristen J. Webb Chief Technology Officer Teradactyl LLC. 2450 Baylor Dr. S.E. Albuquerque, New Mexico 87106 Phone: 1-505-338-6000 Email: kwebb at teradactyl.com Web: http://www.teradactyl.com Providers of Scalable Backup Solutions for Unique Data Environments -------------------------------- NOTICE TO RECIPIENTS: Any information contained in or attached to this message is intended solely for the use of the intended recipient(s). If you are not the intended recipient of this transmittal, you are hereby notified that you received this transmittal in error, and we request that you please delete and destroy all copies and attachments in your possession, notify the sender that you have received this communication in error, and note that any review or dissemination of, or the taking of any action in reliance on, this communication is expressly prohibited. Regular internet e-mail transmission cannot be guaranteed to be secure or error-free. Therefore, we do not represent that this information is complete or accurate, and it should not be relied upon as such. If you prefer to communicate with Teradactyl LLC. using secure (i.e., encrypted and/or digitally signed) e-mail transmission, please notify the sender. Otherwise, you will be deemed to have consented to communicate with Teradactyl via regular internet e-mail transmission. Please note that Teradactyl reserves the right to intercept, monitor, and retain all e-mail messages (including secure e-mail messages) sent to or from its systems as permitted by applicable law -------------- next part -------------- An HTML attachment was scrubbed... URL: From eureka6676 at gmail.com Sun Nov 17 01:43:15 2019 From: eureka6676 at gmail.com (Rafael Ferrer) Date: Sun, 17 Nov 2019 09:43:15 +0800 Subject: Is ED25519 on DTLS supported? Message-ID: It's DTLS-OK according to IANA. https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-16 I tested ED25519 certificates on TLS 1.2 and it worked fine. openssl s_server -port 4321 -cert server-cert.pem -key server-key.pem -CAfile client-cert.pem -tls1_2 -sigalgs ed25519 openssl s_client -bind localhost:1234 -connect localhost:4321 -cert client-cert.pem -key client-key.pem -CAfile server-cert.pem -tls1_2 -sigalgs ed25519 But I get a "no shared cipher" error (on the server) if I just replace -tls1_2 with -dtls1_2 on those two commands. The certs and keys are self-signed for both the server and client and where generated by this command. openssl req -x509 -newkey ed25519 -subj "/CN=localhost" -nodes -addext keyUsage=digitalSignature -keyout key.pem -out cert.pem -------------- next part -------------- An HTML attachment was scrubbed... URL: From Michael.Wojcik at microfocus.com Mon Nov 18 12:28:55 2019 From: Michael.Wojcik at microfocus.com (Michael Wojcik) Date: Mon, 18 Nov 2019 12:28:55 +0000 Subject: Static linking libssl.a and libcrypto.a on Linux x64 fails In-Reply-To: References: Message-ID: > From: Aijaz Baig [mailto:aijazbaig1 at gmail.com] > Sent: Wednesday, November 13, 2019 19:58 > However my current concern here is meeting libSSL and libCrypto's dependencies on > host libraries on Linux platform. For instance, when I talked about 'linking' errors > with respect to symbols like 'dlopen', so as I mentioned, I had to specify '-ldl' > and '-lz' to the gcc linker that suggests a dynamic dependency on these platform > libraries. The -l flag can specify static as well as dynamic libraries, so its use doesn't in itself imply dynamic linking. However, if you need libdl, then something in the application or one or more of the libraries contains references to runtime dynamic linking functions, which suggests the application may be doing explicit dynamic linking. And if memory serves, libdl is only available as a shared object, not as a static library. > I was trying to understand the components that would be needed to package the whole > shebang into an archive which I can later 'just run' on a similar Linux system that > has been completely stripped down for purposes of size and speed Ultimately, you can only statically link objects that you have available for static linking. If you only have the C runtime (glibc) or libz or whatever as a dynamic library, then dynamic linking is your only option for them. Also, static linking generally increases the size requirements, since you duplicate object code into applications rather than having it in a single shared object. -- Michael Wojcik Distinguished Engineer, Micro Focus From Michael.Wojcik at microfocus.com Mon Nov 18 12:28:56 2019 From: Michael.Wojcik at microfocus.com (Michael Wojcik) Date: Mon, 18 Nov 2019 12:28:56 +0000 Subject: Can a linux service work as both TLS client and server? In-Reply-To: References: <20191116031032.GN34850@straasha.imrryr.org> Message-ID: > From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of Kristen Webb > Sent: Saturday, November 16, 2019 10:22 > It sounds like peeking at the port may be the simplest way to determine how it is > being connected to. Using different ports for different types of services - which is what you have here - is the traditional approach, yes. I'm not sure why you refer to this as "peeking", since it's really a matter of having two endpoints; when you accept a conversation, you already know what endpoint it has arrived on. > Apple has a network framework (supports TLS 1.3) that I believe will eventually replace the > current security framework (supports up to TLS 1.2) that I am using today to get the job > done. It sounds like your problem is with Apple's API, not with OpenSSL, so I don't know that we can offer any further assistance. -- Michael Wojcik Distinguished Engineer, Micro Focus From thomas_floodeenjr at mentor.com Mon Nov 18 13:44:13 2019 From: thomas_floodeenjr at mentor.com (Floodeenjr, Thomas) Date: Mon, 18 Nov 2019 13:44:13 +0000 Subject: Static linking libssl.a and libcrypto.a on Linux x64 fails In-Reply-To: References: Message-ID: If you want to link statically, when dynamic libraries are also available, you need to tell the linker that you want to use static libraries, otherwise it will always assume dynamic LINK_LIBS = -Wl,-Bstatic -lstaticlibs -lcrypto -lssl -lz -Wl,-Bdynamic -ldynamiclibs -----Original Message----- From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of Michael Wojcik Sent: Monday, November 18, 2019 5:29 AM To: openssl-users at openssl.org Subject: RE: Static linking libssl.a and libcrypto.a on Linux x64 fails > From: Aijaz Baig [mailto:aijazbaig1 at gmail.com] > Sent: Wednesday, November 13, 2019 19:58 > However my current concern here is meeting libSSL and libCrypto's > dependencies on host libraries on Linux platform. For instance, when I > talked about 'linking' errors with respect to symbols like 'dlopen', so as I mentioned, I had to specify '-ldl' > and '-lz' to the gcc linker that suggests a dynamic dependency on > these platform libraries. The -l flag can specify static as well as dynamic libraries, so its use doesn't in itself imply dynamic linking. However, if you need libdl, then something in the application or one or more of the libraries contains references to runtime dynamic linking functions, which suggests the application may be doing explicit dynamic linking. And if memory serves, libdl is only available as a shared object, not as a static library. > I was trying to understand the components that would be needed to > package the whole shebang into an archive which I can later 'just run' > on a similar Linux system that has been completely stripped down for > purposes of size and speed Ultimately, you can only statically link objects that you have available for static linking. If you only have the C runtime (glibc) or libz or whatever as a dynamic library, then dynamic linking is your only option for them. Also, static linking generally increases the size requirements, since you duplicate object code into applications rather than having it in a single shared object. -- Michael Wojcik Distinguished Engineer, Micro Focus From matt at openssl.org Mon Nov 18 16:42:56 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 18 Nov 2019 16:42:56 +0000 Subject: Is ED25519 on DTLS supported? In-Reply-To: References: Message-ID: <820c8aeb-c83f-f367-bc5e-fff83b594e7f@openssl.org> On 17/11/2019 01:43, Rafael Ferrer wrote: > It's DTLS-OK according to IANA. > https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-16 > > > I tested ED25519 certificates on TLS 1.2 and it worked fine. > > openssl s_server -port 4321 -cert server-cert.pem -key server-key.pem > -CAfile client-cert.pem -tls1_2 -sigalgs ed25519 > openssl s_client -bind localhost:1234 -connect localhost:4321 -cert > client-cert.pem -key client-key.pem -CAfile server-cert.pem -tls1_2 > -sigalgs ed25519 > > But I get a "no shared cipher" error (on the server) if I just replace > -tls1_2 with -dtls1_2 on those two commands. > > > The certs and keys are self-signed for both the server and client and > where generated by this command. > > openssl req -x509 -newkey ed25519 -subj "/CN=localhost" -nodes -addext > keyUsage=digitalSignature -keyout key.pem -out cert.pem > This is a really good question. Currently Ed25519 certificates are not supported in DTLS. The function ssl_set_masks() in ssl_lib.c has this code: /* Allow Ed25519 for TLS 1.2 if peer supports it */ if (!(mask_a & SSL_aECDSA) && ssl_has_cert(s, SSL_PKEY_ED25519) && pvalid[SSL_PKEY_ED25519] & CERT_PKEY_EXPLICIT_SIGN && TLS1_get_version(s) == TLS1_2_VERSION) mask_a |= SSL_aECDSA; Note, this explicitly checks for TLSv1.2 and only allows ED25519 if it is true. There is no equivalent code for DTLSv1.2. Technically getting it to support DTLSv1.2 is easy. We just amend the above line to additionally check for DTLSv1.2 and it should work. The question is, is that correct? EdDSA support for TLSv1.2 was specified in RFC8422. That RFC only has one mention of DTLS here: "IANA has assigned one value from the "TLS HashAlgorithm" registry for Intrinsic (8) with DTLS-OK set to true (Y) and this document as reference. This keeps compatibility with TLS 1.3." That's in reference to IANA TLS HashAlgorithm registry. But for the TLS SignatureAlgorithm registry it says this: "IANA has assigned two values in the "TLS SignatureAlgorithm" registry for ed25519 (7) and ed448 (8) with this document as reference. This keeps compatibility with TLS 1.3." This is in the paragraph before the other one, and there is no reference to ed25519/ed448 being "ok" for DTLS, and in fact there is no mention of DTLS anywhere else in this RFC. So, I'm slightly perplexed as to why the IANA registry says something different to this (i.e. DTLS is "ok" for ed25519/ed448). Is this an error in the IANA registry? Or is this an error in the RFC? Or is there some other RFC somewhere that specifies ed25519/ed448 usage in DTLS? I looked to see if there were any errata for RFC8422, but nothing looked relevant. Matt From matt at openssl.org Mon Nov 18 16:50:27 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 18 Nov 2019 16:50:27 +0000 Subject: Is ED25519 on DTLS supported? In-Reply-To: <820c8aeb-c83f-f367-bc5e-fff83b594e7f@openssl.org> References: <820c8aeb-c83f-f367-bc5e-fff83b594e7f@openssl.org> Message-ID: <96d77c56-30f6-5903-c4ca-1624213daa4d@openssl.org> On 18/11/2019 16:42, Matt Caswell wrote: > > > On 17/11/2019 01:43, Rafael Ferrer wrote: >> It's DTLS-OK according to IANA. >> https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-16 >> >> >> I tested ED25519 certificates on TLS 1.2 and it worked fine. >> >> openssl s_server -port 4321 -cert server-cert.pem -key server-key.pem >> -CAfile client-cert.pem -tls1_2 -sigalgs ed25519 >> openssl s_client -bind localhost:1234 -connect localhost:4321 -cert >> client-cert.pem -key client-key.pem -CAfile server-cert.pem -tls1_2 >> -sigalgs ed25519 >> >> But I get a "no shared cipher" error (on the server) if I just replace >> -tls1_2 with -dtls1_2 on those two commands. >> >> >> The certs and keys are self-signed for both the server and client and >> where generated by this command. >> >> openssl req -x509 -newkey ed25519 -subj "/CN=localhost" -nodes -addext >> keyUsage=digitalSignature -keyout key.pem -out cert.pem >> > > > This is a really good question. Currently Ed25519 certificates are not > supported in DTLS. The function ssl_set_masks() in ssl_lib.c has this code: > > /* Allow Ed25519 for TLS 1.2 if peer supports it */ > if (!(mask_a & SSL_aECDSA) && ssl_has_cert(s, SSL_PKEY_ED25519) > && pvalid[SSL_PKEY_ED25519] & CERT_PKEY_EXPLICIT_SIGN > && TLS1_get_version(s) == TLS1_2_VERSION) > mask_a |= SSL_aECDSA; > > Note, this explicitly checks for TLSv1.2 and only allows ED25519 if it > is true. There is no equivalent code for DTLSv1.2. > > Technically getting it to support DTLSv1.2 is easy. We just amend the > above line to additionally check for DTLSv1.2 and it should work. The > question is, is that correct? EdDSA support for TLSv1.2 was specified in > RFC8422. That RFC only has one mention of DTLS here: > > "IANA has assigned one value from the "TLS HashAlgorithm" registry for > Intrinsic (8) with DTLS-OK set to true (Y) and this document as > reference. This keeps compatibility with TLS 1.3." > > That's in reference to IANA TLS HashAlgorithm registry. But for the TLS > SignatureAlgorithm registry it says this: > > "IANA has assigned two values in the "TLS SignatureAlgorithm" registry > for ed25519 (7) and ed448 (8) with this document as reference. This > keeps compatibility with TLS 1.3." > > This is in the paragraph before the other one, and there is no reference > to ed25519/ed448 being "ok" for DTLS, and in fact there is no mention of > DTLS anywhere else in this RFC. > > So, I'm slightly perplexed as to why the IANA registry says something > different to this (i.e. DTLS is "ok" for ed25519/ed448). Is this an > error in the IANA registry? Or is this an error in the RFC? Or is there > some other RFC somewhere that specifies ed25519/ed448 usage in DTLS? > > I looked to see if there were any errata for RFC8422, but nothing looked > relevant. Note, I just asked about this on the TLS WG list. Matt From fergtm at hyperion.io Mon Nov 18 18:44:54 2019 From: fergtm at hyperion.io (Fernando Gutierrez Mendez) Date: Mon, 18 Nov 2019 12:44:54 -0600 Subject: ssl3_get_record:decryption failed on some machines Message-ID: <20191118183950.GA4546@quickbeam.nil.mx> Hi, I wrote an application that uses OpenSSL (1.1.1) and for the past couple of weeks I have been unable to solve a very strange issue. I use non-blocking IO with a SSL BIO so a call to BIO_read eventually returns -1, when this happens I call BIO_should_retry to test if this is due an error or because of the underlying non-blocking transport. This code works correctly but after transferring between 1Mb to 5Mb (it varies every time) BIO_should_rety returns false and SSL_get_error returns SSL_ERROR_SSL. The error is "139964546914112:error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:677" The very strange thing is that this code has been working correctly and transferring several Gb without any issues on a couple of machines. I started getting the error in a virtual machine from a popular VPS provider that uses AMD CPUs and in one physical machine using an older Intel CPU. Works correctly on: Intel Celeron CPU J1800 Virtual Machine on Intel Core i7-5820K Virtual Machine on Intel Xeon E5-2697 Fails every time on: Intel Pentium G2020T Virtual Machine on AMD EPYC 7601 All machines are using "OpenSSL 1.1.1 11 Sep 2018" on "Ubuntu 18.04.3 LTS" Things I tried: - Playing with OPENSSL_ia32cap to force disable PCLMULQDQ/AES-NI, this makes no difference - Running my app under valgrind. It does not report any error but the problem does not reproduce - Instead of using the distro provided build I downloaded and compiled from https://github.com/openssl/openssl/archive/OpenSSL_1_1_1d.tar.gz, it also made no difference I understand this could be a bug in my code but I cant figure out why it only happens on some machines. Any help is appreciated. Thanks From openssl-users at dukhovni.org Mon Nov 18 19:19:30 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Mon, 18 Nov 2019 14:19:30 -0500 Subject: ssl3_get_record:decryption failed on some machines In-Reply-To: <20191118183950.GA4546@quickbeam.nil.mx> References: <20191118183950.GA4546@quickbeam.nil.mx> Message-ID: > On Nov 18, 2019, at 1:44 PM, Fernando Gutierrez Mendez wrote: > > I use non-blocking IO with a SSL BIO so a call to BIO_read eventually returns -1, when this happens I call BIO_should_retry to test if this is due an error or because of the underlying non-blocking transport. Is the writer side also non-blocking? Is it your own code? > This code works correctly but after transferring between 1Mb to 5Mb (it varies every time) BIO_should_rety returns false and SSL_get_error returns SSL_ERROR_SSL. The error is "139964546914112:error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:677" One way to get decryption integrity failure is for a non-blocking writer to not handle partial writes correctly, if on an incomplete write the writer resends the whole buffer, rather than only what it failed to send last time, the TCP stream ends up stuttering ciphertext, and the reader sees data integrity errors. This can be seen by looking for unexpected runs of repeated ciphertext in a PCAP capture of the data. Whether the data sent to a particular reader ever ends up blocked at the TCP layer for a given writer can depend on various network-layer issues making some machines more prone to problems than others. -- Viktor. From fergtm at hyperion.io Mon Nov 18 20:33:47 2019 From: fergtm at hyperion.io (Fernando Gutierrez Mendez) Date: Mon, 18 Nov 2019 14:33:47 -0600 Subject: ssl3_get_record:decryption failed on some machines In-Reply-To: References: <20191118183950.GA4546@quickbeam.nil.mx> Message-ID: <20191118203347.GA5194@quickbeam.nil.mx> The writer is my own code but I can also reproduce the problem when server is nginx and client is my app. In my code I do not use OpenSSL socket BIOs instead I do read/writes through a BIO pair: pairBase = BIO_new(BIO_s_bio()); pairInt = BIO_new(BIO_s_bio()); [...] BIO_make_bio_pair(pairBase, pairInt); [...] sslBIO = BIO_new_ssl(ssl_ctx, 1 /* Client */); [...] BIO_push(sslBIO, pairInt); After each BIO_read/BIO_write to sslBIO I read/write any available data from the network to pairBase. I think I'm handling partial writes correctly: SSL_CTX_set_mode(ssl_ctx, SSL_MODE_AUTO_RETRY | SSL_MODE_ENABLE_PARTIAL_WRITE | SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER); [..] ret = BIO_write(sslBIO, buf, (int)length); if (ret <= 0 && !BIO_should_retry(sslBIO)) { /* Handle error */ return; } if (ret > 0) { buf = ((uint8_t *)buf) + (size_t)ret; length -= (size_t)ret; } but again the problem reproduces even if the writer is nginx. Thanks On Mon, Nov 18, 2019 at 02:19:30PM -0500, Viktor Dukhovni wrote: > > On Nov 18, 2019, at 1:44 PM, Fernando Gutierrez Mendez wrote: > > > > I use non-blocking IO with a SSL BIO so a call to BIO_read eventually returns -1, when this happens I call BIO_should_retry to test if this is due an error or because of the underlying non-blocking transport. > > Is the writer side also non-blocking? Is it your own code? > > > This code works correctly but after transferring between 1Mb to 5Mb (it varies every time) BIO_should_rety returns false and SSL_get_error returns SSL_ERROR_SSL. The error is "139964546914112:error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:677" > > One way to get decryption integrity failure is for a non-blocking > writer to not handle partial writes correctly, if on an incomplete > write the writer resends the whole buffer, rather than only what > it failed to send last time, the TCP stream ends up stuttering > ciphertext, and the reader sees data integrity errors. > > This can be seen by looking for unexpected runs of repeated > ciphertext in a PCAP capture of the data. > > Whether the data sent to a particular reader ever ends up > blocked at the TCP layer for a given writer can depend on > various network-layer issues making some machines more > prone to problems than others. > > -- > Viktor. > From pankajnsarode at gmail.com Tue Nov 19 06:39:40 2019 From: pankajnsarode at gmail.com (Pankaj Sarode) Date: Tue, 19 Nov 2019 12:09:40 +0530 Subject: Problem building Linux shared library with static FIPS capable OpenSSL Message-ID: Hi, I am using following openssl versions openssl-1.0.2r.tar.gz openssl-fips-2.0.5.tar.gz Following are my files, *foo.c* : [root at data-domain-dev-vm poc]# cat foo.c #include #include #include int foo_func() { int rc, mode; mode = FIPS_mode(); if(mode == 0) { rc = FIPS_mode_set(1); if(rc == 0) { printf("Failed to enable FIPS mode, "); printf("%s\n",ERR_error_string(ERR_get_error(),NULL)); } else { printf("Enabled FIPS mode"); } } else { printf("Already in FIPS mode"); } return 0; } *poc.c* : [root at data-domain-dev-vm poc]# cat poc.c #include #include "foo.h" int main(int argc, char* argv[]) { foo_func(); return 0; } *Makefile* : [root at data-domain-dev-vm poc]# cat Makefile OPENSSLDIR = /usr/local/ssl/fips?2.0 OPENSSL_INCLUDEDIR = /root/poc/openssl-1.0.2r/include/ FIPSMODULE = $(OPENSSLDIR)/lib/fipscanister.o CC = /usr/bin/gcc FIPSLD = /usr/local/ssl/fips-2.0/bin/fipsld OBJS = poc.o LIBCRYPTO = /root/poc/openssl-1.0.2r/libcrypto.a LIBSSL = /root/poc/openssl-1.0.2r/libssl.a libfoo.so: env FIPSLD_CC=/usr/bin/gcc $(FIPSLD) -fPIC -shared -o libfoo.so foo.c $(LIBCRYPTO) $(LIBSSL) # Working poc.o: /usr/bin/gcc -I$(OPENSSL_INCLUDEDIR) -Wall -c poc.c POC: libfoo.so $(OBJS) env FIPSLD_CC=/usr/bin/gcc $(FIPSLD) $(OBJS) $(LIBCRYPTO) $(LIBSSL) -L/root/poc -lfoo -ldl -o POC #working #/usr/bin/gcc $(OBJS) -L/root/poc -lfoo -ldl -o POC #notworking #env FIPSLD_CC=/usr/bin/gcc $(FIPSLD) $(OBJS) -L/root/poc -lfoo -ldl -o POC #notworking What I am not able to understand is why any of the red colored lines are not working. when I try to do the FIPS_mode_set() when compiled using red highlighted tex, I get an error Failed to enable FIPS mode, error:2D06B06F:lib(45):func(107):reason(111) I actually want to compile the POC application without $(LIBCRYPTO) and $(LIBSSL) what can be the done to have $(LIBCRYPTO) and $(LIBSSL) linked to only foo.so and POC application can only use foo.so for successful operation. Some more details: [root at data-domain-dev-vm poc]# OPENSSL_FIPS=1 openssl md5 /dev/null Error setting digest md5 140539482445728:error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips:digest.c:256: [root at data-domain-dev-vm poc]# OPENSSL_FIPS=1 openssl sha1 /dev/null SHA1(/dev/null)= da39a3ee5e6b4b0d3255bfef95601890afd80709 Any help is appreciated, Please let me know where in code i am going wrong. Thanks a lot, Pankaj -------------- next part -------------- An HTML attachment was scrubbed... URL: From eureka6676 at gmail.com Tue Nov 19 22:48:12 2019 From: eureka6676 at gmail.com (Rafael Ferrer) Date: Wed, 20 Nov 2019 06:48:12 +0800 Subject: Is ED25519 on DTLS supported? In-Reply-To: <96d77c56-30f6-5903-c4ca-1624213daa4d@openssl.org> References: <96d77c56-30f6-5903-c4ca-1624213daa4d@openssl.org> Message-ID: <7f6261b3-a252-032e-ee11-aa3749160062@gmail.com> Thanks, I'll read up on your discussion there. My original use for this is to share the same certificate and key on a process that has both a TLS and DTLS connection. I went with just making the DTLS derive a PSK from the keying material of the TLS. From rsalz at akamai.com Thu Nov 21 02:05:48 2019 From: rsalz at akamai.com (Salz, Rich) Date: Thu, 21 Nov 2019 02:05:48 +0000 Subject: Reviewer statistics Message-ID: The attached script summarizes git log data and generates CSV lines for each commit. For example, ; git log -2 | ~/reviews.pl 8984, 2019-05-23, levitte at openssl.org, O, paul.dale at oracle.com, O 9624, 2019-08-18, bernd.edlinger at hotmail.de, C, levitte at openssl.org, O It ignores log entries that don?t follow the common format of having ?Reviewed-by? and ?Merged from? lines. This can be useful to get a handle on PR?s and reviewer activity. I don?t yet know what questions to ask :) but it will be nice to have numbers that prove that the project needs to improve its reviewer flow :( There?s rumors that the project will be announcing some restructuring or policy changes soon, and maybe this will be helpful input to that process. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: reviews.pl Type: text/x-perl-script Size: 2341 bytes Desc: reviews.pl URL: From navin.shivanna at gmail.com Thu Nov 21 02:47:49 2019 From: navin.shivanna at gmail.com (Naveen Shivanna) Date: Thu, 21 Nov 2019 08:17:49 +0530 Subject: Does 'no-hw' compile option disables the effect of opening the ASM assembler instruction ? Message-ID: Hi, If there is no 'no-asm' compile option while building the OpenSSL, then it enables the assembler instruction support for OpenSSL. Whether having 'no-hw' compile option disables the assembler instruction support in above case ? Rgds, Navi -------------- next part -------------- An HTML attachment was scrubbed... URL: From ca+ssl-users at esmtp.org Thu Nov 21 19:37:03 2019 From: ca+ssl-users at esmtp.org (Claus Assmann) Date: Thu, 21 Nov 2019 20:37:03 +0100 Subject: SSL_set_client_CA_list(ssl, NULL) problem? Message-ID: <20191121193703.GA99944@kiel.esmtp.org> I'm trying to find what's wrong when I use SSL_set_client_CA_list(ssl, NULL); in a server: openssl s_client still shows "Acceptable client CA names" (those which are previously set using SSL_CTX_load_verify_locations()) instead of the expected "No client certificate CA names sent" which happens if I use SSL_CTX_set_client_CA_list(ctx, NULL) Now sure what's wrong: - my program - openssl s_client - SSL_set_client_CA_list(ssl, NULL) - something else PS: openssl version is 1.0.2t From Tobias.Wolf at t-systems.com Fri Nov 22 09:26:16 2019 From: Tobias.Wolf at t-systems.com (Tobias.Wolf at t-systems.com) Date: Fri, 22 Nov 2019 09:26:16 +0000 Subject: Engine with custom evp method callbacks Message-ID: Hi everbody, I`m looking for a working example on how to implements a custom engine based on EVP methods callbacks. First I was implementing my custom engine based on RSA callbacks, but we found out that we cannot use this mechanism, therefore I need to change to EVP, details are written here https://github.com/openssl/openssl/issues/7968. RSA_METHOD* rsa_method = RSA_meth_new("OpenSSL Custom RSA method", 0); const RSA_METHOD* ossl_rsa_meth = RSA_PKCS1_OpenSSL(); rc = RSA_meth_set_priv_enc(rsa_method, gk_openssl_rsa_priv_enc); rc = ENGINE_set_RSA(e, rsa_method); if (rc != TRUE) { return 0; } if (flags & ENGINE_METHOD_RSA) { rc = ENGINE_register_RSA(e); if (rc != TRUE) { return 0; } } Now I try with EVP the following source code but it's not working: EVP_PKEY_METHOD* engine_pkey_methods = EVP_PKEY_meth_new(EVP_PKEY_RSA_PSS, 0); const EVP_PKEY_METHOD* ossl_pkey_methods = EVP_PKEY_meth_find(EVP_PKEY_RSA_PSS); EVP_PKEY_meth_copy(engine_pkey_methods, ossl_pkey_methods); // This shall be an equivalent to = RSA_PKCS1_OpenSSL(); const EVP_PKEY_METHOD* ossl_pkey_methods = EVP_PKEY_meth_find(EVP_PKEY_RSA_PSS); But how to set the evp method the engine like RSA(e, rsa_method);? This expects another callback, but I just want to set the method?! int ENGINE_set_pkey_meths(ENGINE *e, ENGINE_PKEY_METHS_PTR f); regards Tobi -------------- next part -------------- An HTML attachment was scrubbed... URL: From ca+ssl-users at esmtp.org Fri Nov 22 16:20:53 2019 From: ca+ssl-users at esmtp.org (Claus Assmann) Date: Fri, 22 Nov 2019 17:20:53 +0100 Subject: 1.1.1d build failure with no-shared Message-ID: <20191122162053.GA7705@kiel.esmtp.org> Just FYI: trying to build openssl 1.1.1d with no-shared fails (on OpenBSD 6.5) see below. I'm not sure why test/cipher_overhead_test is needed for the build. rm -f test/cipher_overhead_test ${LDCMD:-cc} -Wa,--noexecstack -Qunused-arguments -Wall -O3 -L. -o test/cipher_overhead_test test/cipher_overhead_test.o -lssl test/libtestutil.a -lcrypto ld: error: undefined symbol: ssl3_num_ciphers >>> referenced by cipher_overhead_test.c >>> test/cipher_overhead_test.o:(cipher_overhead) ld: error: undefined symbol: ssl3_get_cipher >>> referenced by cipher_overhead_test.c >>> test/cipher_overhead_test.o:(cipher_overhead) ld: error: undefined symbol: ssl_cipher_get_overhead >>> referenced by cipher_overhead_test.c >>> test/cipher_overhead_test.o:(cipher_overhead) cc: error: linker command failed with exit code 1 (use -v to see invocation) *** Error 1 in . (Makefile:8181 'test/cipher_overhead_test') *** Error 1 in [[path removed]]/openssl-1.1.1d (Makefile:174 'all') From Michal.Trojnara at stunnel.org Fri Nov 22 18:03:39 2019 From: Michal.Trojnara at stunnel.org (=?UTF-8?Q?Micha=c5=82_Trojnara?=) Date: Fri, 22 Nov 2019 19:03:39 +0100 Subject: stunnel 5.56 released Message-ID: Dear Users, I have released version 5.56 of stunnel. ### Version 5.56, 2019.11.22, urgency: HIGH * New features ? - Various text files converted to Markdown format. * Bugfixes ? - Support for realpath(3) implementations incompatible ??? with POSIX.1-2008, such as 4.4BSD or Solaris. ? - Support for engines without PRNG seeding methods (thx to ??? Petr Mikhalitsyn). ? - Retry unsuccessful port binding on configuration ??? file reload. ? - Thread safety fixes in SSL_SESSION object handling. ? - Terminate clients on exit in the FORK threading model. Home page: https://www.stunnel.org/ Download: https://www.stunnel.org/downloads.html SHA-256 hashes: 7384bfb356b9a89ddfee70b5ca494d187605bb516b4fff597e167f97e2236b22? stunnel-5.56.tar.gz e9d7dea3976219f0fc89cfb4f645f47b1291ebec8ce55cff46dbbfbb2e9b4084? stunnel-5.56-win64-installer.exe d8a5e359c7102b3c9619fca6b4ffbb39c16a9779dcecb426f204a7857cb33f67? stunnel-5.56-android.zip Best regards, ??? Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From beldmit at gmail.com Sat Nov 23 13:55:09 2019 From: beldmit at gmail.com (Dmitry Belyavsky) Date: Sat, 23 Nov 2019 16:55:09 +0300 Subject: Engine with custom evp method callbacks In-Reply-To: References: Message-ID: Dear Tobi, On Fri, Nov 22, 2019 at 12:27 PM wrote: > Hi everbody, > > > > I`m looking for a working example on how to implements a custom engine > based on EVP methods callbacks. First I was implementing my custom engine > based on RSA callbacks, but we found out that we cannot use this mechanism, > > therefore I need to change to EVP, details are written here > https://github.com/openssl/openssl/issues/7968. > > > > RSA_METHOD* rsa_method = RSA_meth_new("OpenSSL Custom RSA > method", 0); > > const RSA_METHOD* ossl_rsa_meth = RSA_PKCS1_OpenSSL(); > > > > rc = RSA_meth_set_priv_enc(rsa_method, > gk_openssl_rsa_priv_enc); > > > > rc = ENGINE_set_RSA(e, rsa_method); > > if (rc != TRUE) { > > return 0; > > } > > > > if (flags & ENGINE_METHOD_RSA) { > > rc = ENGINE_register_RSA(e); > > if (rc != TRUE) { > > > return 0; > > } > > } > > > > > > Now I try with EVP the following source code but it?s not working: > > > > EVP_PKEY_METHOD* engine_pkey_methods = EVP_PKEY_meth_new(EVP_PKEY_RSA_PSS, > 0); > > const EVP_PKEY_METHOD* ossl_pkey_methods = EVP_PKEY_meth_find( > EVP_PKEY_RSA_PSS); > > EVP_PKEY_meth_copy(engine_pkey_methods, ossl_pkey_methods); > > > > // This shall be an equivalent to = RSA_PKCS1_OpenSSL(); > > const EVP_PKEY_METHOD* ossl_pkey_methods = EVP_PKEY_meth_find( > EVP_PKEY_RSA_PSS); > > > > But how to set the evp method the engine like RSA(e, rsa_method);? > > This expects another callback, but I just want to set the method?! > > > > int ENGINE_set_pkey_meths(ENGINE *e, ENGINE_PKEY_METHS_PTR f); > > > I strongly suppose that you can't mix EVP_PKEY_METHOD and RSA_METHOD, but you should wrap the RSA_METHOD callbacks in the EVP_PKEY callbacks. I suggest you look at the https://github.com/gost-engine/engine as an example of providing the EVP operations via the engine. I also have an example of providing custom RSA_METHOD somewhere but it was designed to work with 1.0 and may be incompatible with the 1.1.* because of using the internal structures. -- SY, Dmitry Belyavsky -------------- next part -------------- An HTML attachment was scrubbed... URL: From levitte at openssl.org Sat Nov 23 20:49:50 2019 From: levitte at openssl.org (Richard Levitte) Date: Sat, 23 Nov 2019 21:49:50 +0100 Subject: 1.1.1d build failure with no-shared In-Reply-To: <20191122162053.GA7705@kiel.esmtp.org> References: <20191122162053.GA7705@kiel.esmtp.org> Message-ID: <87y2w6xqq9.wl-levitte@openssl.org> You don't happen to have a libcrypto.so lying around in your build directory? Just now, I noticed that test/build.info is a bit sloppy, this patch should help: diff --git a/test/build.info b/test/build.info index e4fff15883..e5e1eef153 100644 --- a/test/build.info +++ b/test/build.info @@ -449,7 +449,7 @@ IF[{- !$disabled{tests} -}] PROGRAMS{noinst}=cipher_overhead_test SOURCE[cipher_overhead_test]=cipher_overhead_test.c INCLUDE[cipher_overhead_test]=.. ../include ../apps/include - DEPEND[cipher_overhead_test]=../libcrypto ../libssl libtestutil.a + DEPEND[cipher_overhead_test]=../libcrypto.a ../libssl.a libtestutil.a ENDIF SOURCE[uitest]=uitest.c ../apps/lib/apps_ui.c Cheers, Richard ( I'll check if there are others with the same problem ) On Fri, 22 Nov 2019 17:20:53 +0100, Claus Assmann wrote: > > Just FYI: trying to build openssl 1.1.1d with no-shared fails (on > OpenBSD 6.5) see below. I'm not sure why test/cipher_overhead_test > is needed for the build. > > rm -f test/cipher_overhead_test > ${LDCMD:-cc} -Wa,--noexecstack -Qunused-arguments -Wall -O3 -L. -o test/cipher_overhead_test test/cipher_overhead_test.o -lssl test/libtestutil.a -lcrypto > ld: error: undefined symbol: ssl3_num_ciphers > >>> referenced by cipher_overhead_test.c > >>> test/cipher_overhead_test.o:(cipher_overhead) > > ld: error: undefined symbol: ssl3_get_cipher > >>> referenced by cipher_overhead_test.c > >>> test/cipher_overhead_test.o:(cipher_overhead) > > ld: error: undefined symbol: ssl_cipher_get_overhead > >>> referenced by cipher_overhead_test.c > >>> test/cipher_overhead_test.o:(cipher_overhead) > cc: error: linker command failed with exit code 1 (use -v to see invocation) > *** Error 1 in . (Makefile:8181 'test/cipher_overhead_test') > *** Error 1 in [[path removed]]/openssl-1.1.1d (Makefile:174 'all') > -- Richard Levitte levitte at openssl.org OpenSSL Project http://www.openssl.org/~levitte/ From hmurray at megapathdsl.net Sun Nov 24 00:42:50 2019 From: hmurray at megapathdsl.net (Hal Murray) Date: Sat, 23 Nov 2019 16:42:50 -0800 Subject: Clutter in log files, bogus connections Message-ID: <20191124004250.A08DF406063@ip-64-139-1-69.sjc.megapath.net> I see a lot of clutter in log files from things like error:1408F10B:SSL routines:ssl3_get_record:wrong version number I assume they are from bad guys probing for openings. Is the error code returned by ERR_get_error() constant across releases? Can I compile magic constants like 1408F10B into my code? If not, is there a suggested approach? Is there a list of well known attacks and their error codes? -- These are my opinions. I hate spam. From ca+ssl-users at esmtp.org Sun Nov 24 11:05:34 2019 From: ca+ssl-users at esmtp.org (Claus Assmann) Date: Sun, 24 Nov 2019 12:05:34 +0100 Subject: SSL_set_client_CA_list(ssl, NULL) problem? In-Reply-To: <20191121193703.GA99944@kiel.esmtp.org> References: <20191121193703.GA99944@kiel.esmtp.org> Message-ID: <20191124110534.GA57849@kiel.esmtp.org> Seems it is impossible to override the list with NULL for SSL, as the code will then use the list from CTX (if my limited understanding of the code is correct): STACK_OF(X509_NAME) *SSL_get_client_CA_list(const SSL *s) { ... if (s->client_CA != NULL) return (s->client_CA); else return (s->ctx->client_CA); Is this intentional? The man pages says: SSL_set_client_CA_list() sets the list of CAs sent to the client when requesting a client certificate for the chosen ssl, overriding the setting valid for ssl's SSL_CTX object. IMHO there should be some indication (flag) that the value from SSL should be used (to distinguish between the ways NULL is used: "this is NULL because of the initialization" and "this is explicitly set to NULL"). From kurt at roeckx.be Sun Nov 24 23:14:36 2019 From: kurt at roeckx.be (Kurt Roeckx) Date: Mon, 25 Nov 2019 00:14:36 +0100 Subject: Clutter in log files, bogus connections In-Reply-To: <20191124004250.A08DF406063@ip-64-139-1-69.sjc.megapath.net> References: <20191124004250.A08DF406063@ip-64-139-1-69.sjc.megapath.net> Message-ID: <20191124231436.GA23553@roeckx.be> On Sat, Nov 23, 2019 at 04:42:50PM -0800, Hal Murray wrote: > > I see a lot of clutter in log files from things like > error:1408F10B:SSL routines:ssl3_get_record:wrong version number > I assume they are from bad guys probing for openings. > > Is the error code returned by ERR_get_error() constant across releases? Can I > compile magic constants like 1408F10B into my code? If not, is there a > suggested approach? Use ERR_GET_LIB(error) == ERR_LIB_SSL && ERR_GET_REASON(error) == SSL_R_WRONG_VERSION_NUMBER. Kurt From ca+ssl-users at esmtp.org Mon Nov 25 05:26:08 2019 From: ca+ssl-users at esmtp.org (Claus Assmann) Date: Mon, 25 Nov 2019 06:26:08 +0100 Subject: 1.1.1d build failure with no-shared In-Reply-To: <87y2w6xqq9.wl-levitte@openssl.org> References: <20191122162053.GA7705@kiel.esmtp.org> <87y2w6xqq9.wl-levitte@openssl.org> Message-ID: <20191125052608.GA75155@kiel.esmtp.org> On Sat, Nov 23, 2019, Richard Levitte wrote: > You don't happen to have a libcrypto.so lying around in your build > directory? That was probably be the (my) mistake, sorry for the false alarm (I cannot reproduce the problem in a setup where I just untar'ed the source code). > Just now, I noticed that test/build.info is a bit sloppy, this patch > should help: At least something good came out of this :-) Thanks. From fergtm at hyperion.io Mon Nov 25 08:45:04 2019 From: fergtm at hyperion.io (fergtm at hyperion.io) Date: Mon, 25 Nov 2019 02:45:04 -0600 Subject: ssl3_get_record:decryption failed on some machines In-Reply-To: <20191118203347.GA5194@quickbeam.nil.mx> References: <20191118183950.GA4546@quickbeam.nil.mx> <20191118203347.GA5194@quickbeam.nil.mx> Message-ID: <001101d5a36c$a37d21e0$ea7765a0$@hyperion.io> Sorry to bring this up again but I really don't know how to fix. I already re-wrote my code to use SSL_read/SSL_write instead of a SSL filter BIO but I still get the same error. I can reproduce when the sender is nginx, socat openssl-listen or openssl s_server. Both the server and client are running in the same machine. The SSL object is not using a socket BIO instead I use a BIO pair. I may be using the BIO pair incorrectly but I haven't found any complete examples on how to use them. It works perfectly if I use a debug build of OpenSSL Thanks -----Original Message----- From: openssl-users On Behalf Of Fernando Gutierrez Mendez Sent: Monday, November 18, 2019 2:34 PM To: openssl-users at openssl.org Subject: Re: ssl3_get_record:decryption failed on some machines The writer is my own code but I can also reproduce the problem when server is nginx and client is my app. In my code I do not use OpenSSL socket BIOs instead I do read/writes through a BIO pair: pairBase = BIO_new(BIO_s_bio()); pairInt = BIO_new(BIO_s_bio()); [...] BIO_make_bio_pair(pairBase, pairInt); [...] sslBIO = BIO_new_ssl(ssl_ctx, 1 /* Client */); [...] BIO_push(sslBIO, pairInt); After each BIO_read/BIO_write to sslBIO I read/write any available data from the network to pairBase. I think I'm handling partial writes correctly: SSL_CTX_set_mode(ssl_ctx, SSL_MODE_AUTO_RETRY | SSL_MODE_ENABLE_PARTIAL_WRITE | SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER); [..] ret = BIO_write(sslBIO, buf, (int)length); if (ret <= 0 && !BIO_should_retry(sslBIO)) { /* Handle error */ return; } if (ret > 0) { buf = ((uint8_t *)buf) + (size_t)ret; length -= (size_t)ret; } but again the problem reproduces even if the writer is nginx. Thanks On Mon, Nov 18, 2019 at 02:19:30PM -0500, Viktor Dukhovni wrote: > > On Nov 18, 2019, at 1:44 PM, Fernando Gutierrez Mendez wrote: > > > > I use non-blocking IO with a SSL BIO so a call to BIO_read eventually returns -1, when this happens I call BIO_should_retry to test if this is due an error or because of the underlying non-blocking transport. > > Is the writer side also non-blocking? Is it your own code? > > > This code works correctly but after transferring between 1Mb to 5Mb (it varies every time) BIO_should_rety returns false and SSL_get_error returns SSL_ERROR_SSL. The error is "139964546914112:error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:677" > > One way to get decryption integrity failure is for a non-blocking > writer to not handle partial writes correctly, if on an incomplete > write the writer resends the whole buffer, rather than only what it > failed to send last time, the TCP stream ends up stuttering > ciphertext, and the reader sees data integrity errors. > > This can be seen by looking for unexpected runs of repeated ciphertext > in a PCAP capture of the data. > > Whether the data sent to a particular reader ever ends up blocked at > the TCP layer for a given writer can depend on various network-layer > issues making some machines more prone to problems than others. > > -- > Viktor. > From matt at openssl.org Mon Nov 25 10:35:01 2019 From: matt at openssl.org (Matt Caswell) Date: Mon, 25 Nov 2019 10:35:01 +0000 Subject: ssl3_get_record:decryption failed on some machines In-Reply-To: <001101d5a36c$a37d21e0$ea7765a0$@hyperion.io> References: <20191118183950.GA4546@quickbeam.nil.mx> <20191118203347.GA5194@quickbeam.nil.mx> <001101d5a36c$a37d21e0$ea7765a0$@hyperion.io> Message-ID: <10846377-1696-401b-d78d-f081d7180574@openssl.org> On 25/11/2019 08:45, fergtm at hyperion.io wrote: > Sorry to bring this up again but I really don't know how to fix. I already > re-wrote my code to use SSL_read/SSL_write instead of a SSL filter BIO but I > still get the same error. > > I can reproduce when the sender is nginx, socat openssl-listen or openssl > s_server. Both the server and client are running in the same machine. > > The SSL object is not using a socket BIO instead I use a BIO pair. I may be > using the BIO pair incorrectly but I haven't found any complete examples on > how to use them. > > It works perfectly if I use a debug build of OpenSSL This suggests it *could* be a compiler bug. You might want to experiment with different optimization levels to see if that makes a difference. Matt > > Thanks > > -----Original Message----- > From: openssl-users On Behalf Of > Fernando Gutierrez Mendez > Sent: Monday, November 18, 2019 2:34 PM > To: openssl-users at openssl.org > Subject: Re: ssl3_get_record:decryption failed on some machines > > The writer is my own code but I can also reproduce the problem when server > is nginx and client is my app. > > In my code I do not use OpenSSL socket BIOs instead I do read/writes through > a BIO pair: > > pairBase = BIO_new(BIO_s_bio()); > pairInt = BIO_new(BIO_s_bio()); > > [...] > > BIO_make_bio_pair(pairBase, pairInt); > > [...] > > sslBIO = BIO_new_ssl(ssl_ctx, 1 /* Client */); > > [...] > > BIO_push(sslBIO, pairInt); > > After each BIO_read/BIO_write to sslBIO I read/write any available data from > the network to pairBase. > > I think I'm handling partial writes correctly: > > SSL_CTX_set_mode(ssl_ctx, SSL_MODE_AUTO_RETRY | > SSL_MODE_ENABLE_PARTIAL_WRITE | SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER); > > [..] > > ret = BIO_write(sslBIO, buf, (int)length); > > if (ret <= 0 && !BIO_should_retry(sslBIO)) > { > /* Handle error */ > return; > } > > if (ret > 0) > { > buf = ((uint8_t *)buf) + (size_t)ret; > length -= (size_t)ret; > } > > but again the problem reproduces even if the writer is nginx. > > Thanks > > On Mon, Nov 18, 2019 at 02:19:30PM -0500, Viktor Dukhovni wrote: >>> On Nov 18, 2019, at 1:44 PM, Fernando Gutierrez Mendez > wrote: >>> >>> I use non-blocking IO with a SSL BIO so a call to BIO_read eventually > returns -1, when this happens I call BIO_should_retry to test if this is due > an error or because of the underlying non-blocking transport. >> >> Is the writer side also non-blocking? Is it your own code? >> >>> This code works correctly but after transferring between 1Mb to 5Mb (it > varies every time) BIO_should_rety returns false and SSL_get_error returns > SSL_ERROR_SSL. The error is "139964546914112:error:1408F119:SSL > routines:ssl3_get_record:decryption failed or bad record > mac:../ssl/record/ssl3_record.c:677" >> >> One way to get decryption integrity failure is for a non-blocking >> writer to not handle partial writes correctly, if on an incomplete >> write the writer resends the whole buffer, rather than only what it >> failed to send last time, the TCP stream ends up stuttering >> ciphertext, and the reader sees data integrity errors. >> >> This can be seen by looking for unexpected runs of repeated ciphertext >> in a PCAP capture of the data. >> >> Whether the data sent to a particular reader ever ends up blocked at >> the TCP layer for a given writer can depend on various network-layer >> issues making some machines more prone to problems than others. >> >> -- >> Viktor. >> > From deepak.redmi2 at gmail.com Mon Nov 25 18:10:18 2019 From: deepak.redmi2 at gmail.com (Dipak B) Date: Mon, 25 Nov 2019 23:40:18 +0530 Subject: Fingerprint mismatch only for 32-bit FIPS binary Message-ID: Hi, Appreciate any help on the following. 1) Built static binaries of FIPS capable OSSL which statically link to the windows runtime. 2) Consumed these binaries (libeaycompat32.lib, libeayfips32.lib and ssleay32.lib) into myapp.dll using msincore.pl. Result 1) FIPS mode gets set and working with 64-bit myapp.dll 2) But for 32-bit myapp.dll with same configuration, FIPS_mode_set() fails with reason 111 (Fingerprint mismatch) Tried following Since above 32-bit myapp.dll did not work, some additional configuration changes were made. 1) ReBuilt FIPS capable OSSL with additional LFLAGS of "/DynamicBase:No /Fixed". 2) ReBuilt 32-bit myapp.dll with above additional LFLAGS But 32-bit myapp DLL does fail with fingerprint mismatch. Questions 1) How do I get 32-bit myapp.dll working in FIPS mode? FIPS_mode_set() returns (100:error:2D06B06F:lib(45):func(107): reason (111):/FIPS/FIPS.c:232) 2) Why does "nmake -f nt.mak install" not copy "libeaycompat32.lib"? Is this simply dropped because of EOL of FOM? 3) Are additional LFLAGS required ? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rohit.kaushal at gmail.com Tue Nov 26 08:11:15 2019 From: rohit.kaushal at gmail.com (Rohit Kaushal) Date: Tue, 26 Nov 2019 00:11:15 -0800 Subject: SSL_CTX_set1_curves to specify curves in Client Hello Supported Group Extension, not working Message-ID: Hi, I would like to ask if anyone here has actually been able to trim the EC Supported Groups in the Client Hello with a TLS-ECDHE cipher using the APIs described in the OpenSSL v1.0.2 manpage for SSL_CTX_set1_curves() My shared objects are built using OpenSSL v1.0.2t with FIPS Object Module v2.0.5 . The TLS handshake pcap has always shown myCipher ( "ECDHE-RSA-AES128-GCM-SHA256") in the Client Hello correctly, accompanied with 13 curves in the Supported Group Extension. However, enhancing SSL_CTX myCtx to restrict the curves to just 3 (P-521:P384:P256) still show the same 13 curves as in the pcap. The man page doesn't suggest nor do i see any reason why FIPS should prevent this from working. //existing working code if((myCtx = SSL_CTX_new(SSL_METHOD *)TLSv1_2_method()) == NULL) return; if((SSL_CTX_set_cipher_list(myCtx, myCipher) != 1)) return; //enhancement, not working if(SSL_CTX_set1_curves(myCtx, "P-521:P-384:P-256", 3)) != 1)) return; The API returns 0 (suggesting no error), but pcap shows no change, i.e. still shows the std. 13 curves. Trying a code snippet involving SSL_CONF_cmd(myConfCtx, "-named-curve", "P-256") suggested in this link , available prior to SSL_CTX_set1_curves() introduction in v1.0.2, didn?t help either. Thank you for any guidance you can provide. Rohit -------------- next part -------------- An HTML attachment was scrubbed... URL: From rashok.svks at gmail.com Tue Nov 26 08:34:59 2019 From: rashok.svks at gmail.com (Raja Ashok) Date: Tue, 26 Nov 2019 14:04:59 +0530 Subject: SSL_CTX_set1_curves to specify curves in Client Hello Supported Group Extension, not working In-Reply-To: References: Message-ID: Hi, Need to use *SSL_CTX_set1_curves_list()*, for ECC curves configuration using string. SSL_CTX_set1_curves_list(ctx, "P-521:P-384:P-256") Regards, R Ashok On Tue, Nov 26, 2019 at 1:42 PM Rohit Kaushal wrote: > Hi, > > I would like to ask if anyone here has actually been able to trim the EC > Supported Groups in the Client Hello with a TLS-ECDHE cipher using the APIs > described in the OpenSSL v1.0.2 manpage for SSL_CTX_set1_curves() > > > > > My shared objects are built using OpenSSL v1.0.2t with FIPS Object Module > v2.0.5 . The TLS handshake pcap has always shown myCipher ( > "ECDHE-RSA-AES128-GCM-SHA256") in the Client Hello correctly, accompanied > with 13 curves in the Supported Group Extension. However, enhancing SSL_CTX > myCtx to restrict the curves to just 3 (P-521:P384:P256) still show the > same 13 curves as in the pcap. The man page doesn't suggest nor do i see > any reason why FIPS should prevent this from working. > > > //existing working code > > if((myCtx = SSL_CTX_new(SSL_METHOD *)TLSv1_2_method()) == NULL) > > return; > > if((SSL_CTX_set_cipher_list(myCtx, myCipher) != 1)) > > return; > > > > //enhancement, not working > > if(SSL_CTX_set1_curves(myCtx, "P-521:P-384:P-256", 3)) != 1)) > > return; > > > > The API returns 0 (suggesting no error), but pcap shows no change, i.e. > still shows the std. 13 curves. > > > > Trying a code snippet involving SSL_CONF_cmd(myConfCtx, "-named-curve", > "P-256") suggested in this link > , > available prior to SSL_CTX_set1_curves() introduction in v1.0.2, didn?t > help either. > > > Thank you for any guidance you can provide. > > Rohit > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nospam001-lists at jan-kohnert.de Tue Nov 26 08:47:15 2019 From: nospam001-lists at jan-kohnert.de (Jan Kohnert) Date: Tue, 26 Nov 2019 09:47:15 +0100 Subject: SSL_CTX_set1_curves to specify curves in Client Hello Supported Group Extension, not working In-Reply-To: References: Message-ID: <4908121.XxfC8Ic6UX@kohnert-n4> Hi, Am Dienstag, 26. November 2019, 09:11:15 CET schrieb Rohit Kaushal: > //enhancement, not working > > if(SSL_CTX_set1_curves(myCtx, "P-521:P-384:P-256", 3)) != 1)) > > return; > > > > The API returns 0 (suggesting no error), but pcap shows no change, i.e. > still shows the std. 13 curves. According to [1] SSL_CTX_set1_curves() returns 1 for success, 0 for failure. [1] https://www.openssl.org/docs/man1.1.0/man3/SSL_CTX_set1_curves.html -- Kind regards Jan From raja.ashok at huawei.com Tue Nov 26 11:18:27 2019 From: raja.ashok at huawei.com (Raja ashok) Date: Tue, 26 Nov 2019 11:18:27 +0000 Subject: Usage of Secure C (memcpy_s, strcpy_s etc) functions on OpenSSL Message-ID: Hi All, We are using OpenSSL in our projects and we found some of the C standard functions (like memcpy, strcpy) used in OpenSSL may induce security vulnerablities like buffer overflow. Currently we have not found any instances which causes such issues. But we feel better to change these calls to C11 standard's secure functions like memcpy_s, strcpy_s etc. By defining a secure calls method (list of func pointers) and allowing application to register the method. I understand that this affects performance because of return value check added for xxxx_s calls, but this will make sure it removes buffer overflow kind of issues completely from code. And also currently using secure c calls is a general industry practice. Please share your opinion on it, and if any discussion happened in OpenSSL community to do this change in future. Thanks in advance. Raja Ashok -------------- next part -------------- An HTML attachment was scrubbed... URL: From raja.ashok at huawei.com Tue Nov 26 11:07:25 2019 From: raja.ashok at huawei.com (Raja ashok) Date: Tue, 26 Nov 2019 11:07:25 +0000 Subject: Usage of Secure C (memcpy_s, strcpy_s etc) functions on OpenSSL Message-ID: Hi All, We are using OpenSSL in our projects and we found some of the C standard functions (like memcpy, strcpy) used in OpenSSL may induce security vulnerablities like buffer overflow. Currently we have not found any instances which causes such issues. But we feel better to change these calls to C11 standard's secure functions like memcpy_s, strcpy_s etc. By defining a secure calls method (list of func pointers) and allowing application to register the method. I understand that this affects performance because of return value check added for xxxx_s calls, but this will make sure it removes buffer overflow kind of issues completely from code. And also currently using secure c calls is a general industry practice. Please share your opinion on it, and if any discussion happened in OpenSSL coummunity to do this change in future. Thanks in advance. Raja Ashok -------------- next part -------------- An HTML attachment was scrubbed... URL: From shivakumar2696 at gmail.com Tue Nov 26 11:38:34 2019 From: shivakumar2696 at gmail.com (shiva kumar) Date: Tue, 26 Nov 2019 17:08:34 +0530 Subject: OpenSSL 1.0.2 EOL Message-ID: Hi, As we know that OpenSSL 1.0.2 support will end in 31st December 2019. and the latest version is 1.0.2t, is there will be any release by EOL? can we expect a release before EOL? Regards Shivakumar -------------- next part -------------- An HTML attachment was scrubbed... URL: From ossl at mts.cz Tue Nov 26 12:51:47 2019 From: ossl at mts.cz (Libor Chocholaty) Date: Tue, 26 Nov 2019 13:51:47 +0100 Subject: Usage of Secure C (memcpy_s, strcpy_s etc) functions on OpenSSL In-Reply-To: References: Message-ID: <51aca1cd06e1991f4e3c95476627e27b@mts.cz> Hello, these _s functions are not so much safer than the standard ones. Using them makes just MSVC happy. Regards, Libor On 2019-11-26 12:07, Raja ashok wrote: > Hi All, > > We are using OpenSSL in our projects and we found some of the C standard functions (like memcpy, strcpy) used in OpenSSL may induce security vulnerablities like buffer overflow. Currently we have not found any instances which causes such issues. > > But we feel better to change these calls to C11 standard's secure functions like memcpy_s, strcpy_s etc. By defining a secure calls method (list of func pointers) and allowing application to register the method. I understand that this affects performance because of return value check added for xxxx_s calls, but this will make sure it removes buffer overflow kind of issues completely from code. And also currently using secure c calls is a general industry practice. > > Please share your opinion on it, and if any discussion happened in OpenSSL coummunity to do this change in future. > > Thanks in advance. > > Raja Ashok -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Tue Nov 26 13:01:06 2019 From: matt at openssl.org (Matt Caswell) Date: Tue, 26 Nov 2019 13:01:06 +0000 Subject: OpenSSL 1.0.2 EOL In-Reply-To: References: Message-ID: <315a1f70-4cb5-4a7d-c890-391a581285f7@openssl.org> On 26/11/2019 11:38, shiva kumar wrote: > Hi, > As we know that OpenSSL 1.0.2 support will end in 31st? December 2019. > and the latest version is 1.0.2t, is there will be any release by EOL? > can we expect a release before EOL? This is as yet undecided. We issue releases on an as-needed basis and so far we have not needed another release. That might change before the end of the year. Matt From Matthias.St.Pierre at ncp-e.com Tue Nov 26 13:38:56 2019 From: Matthias.St.Pierre at ncp-e.com (Matthias St. Pierre) Date: Tue, 26 Nov 2019 14:38:56 +0100 Subject: Usage of Secure C (memcpy_s, strcpy_s etc) functions on OpenSSL In-Reply-To: References: Message-ID: <83a7a6ea-1343-94f3-000a-35c0c592ce4c@ncp-e.com> Hello Raja, according to our policy, the OpenSSL C source code needs to conform to ISO C90, which makes it impossible to adopt C11 features. Chapter 20: Portability To maximise portability the version of C defined in ISO/IEC 9899:1990 should be used. This is more commonly referred to as C90. ISO/IEC 9899:1999 (also known as C99) is not supported on some platforms that OpenSSL is used on and therefore should be avoided. (see https://www.openssl.org/policies/codingstyle.html) Matthias On 26.11.19 12:07, Raja ashok wrote: > > Hi All, > > We are using OpenSSL in our projects and we found some of the C standard functions (like memcpy, strcpy) used in OpenSSL may induce security vulnerablities like buffer overflow. Currently we have not found any instances which causes such issues. > > But we feel better to change these calls to C11 standard's secure functions like memcpy_s, strcpy_s etc. By defining a secure calls method (list of func pointers) and allowing application to register the method. I understand that this affects performance because of return value check added for xxxx_s calls, but this will make sure it removes buffer overflow kind of issues completely from code. And also currently using secure c calls is a general industry practice. > > Please share your opinion on it, and if any discussion happened in OpenSSL coummunity to do this change in future. > > Thanks in advance. > > Raja Ashok > From Michael.Wojcik at microfocus.com Tue Nov 26 15:41:18 2019 From: Michael.Wojcik at microfocus.com (Michael Wojcik) Date: Tue, 26 Nov 2019 15:41:18 +0000 Subject: Usage of Secure C (memcpy_s, strcpy_s etc) functions on OpenSSL In-Reply-To: References: Message-ID: The Appendix K functions (memcpy_s, etc) do NOT "remove buffer overflow kind of issues completely", and anyone who thinks they do is making a serious error. The Appendix K functions impose an additional check. That's all they do. It is possible, and in some use cases quite easy, for the developer to pass the wrong value for the destsz parameter and invalidate that check. Some C experts have argued that the length-checking versions of the library functions, either the C90 ones such as strncat or the Appendix K ones, are essentially pointless anyway; that the caller needs to handle truncation and so ought to know whether truncation (or overflow) would occur before attempting the operation. On some platforms there are issues with using the Appendix K functions, either because the major C implementations for that platform do not implement them (they predate C99, or didn't implement Appendix K which was optional in C99), or because they have limitations. For example, with at least some versions of the Solaris C runtime they can't be safely used in multithreaded applications because the Runtime Constraint Handler is not thread-safe. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunchunjie_0_0 at 163.com Tue Nov 26 17:20:40 2019 From: sunchunjie_0_0 at 163.com (=?GBK?B?wq/W7g==?=) Date: Wed, 27 Nov 2019 01:20:40 +0800 (CST) Subject: need your help about fipsld in Example OpenSSL Based Application Message-ID: <7d89ace6.334.16ea8babdf3.Coremail.sunchunjie_0_0@163.com> Dear all, As a newbie, I am following the details in document UserGuide-2.0.pdf,Appendix C Example OpenSSL Based Application,C1, which creates one Makefile and one c source code file. when run make command, it always use ld to do the link, not the fipsld, but when I run command like : make CC=/path/to/fipsld it seems ran into loop, so, I need your help about what to fix to make the example works. -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl at jordan.maileater.net Tue Nov 26 23:47:20 2019 From: openssl at jordan.maileater.net (Jordan Brown) Date: Tue, 26 Nov 2019 23:47:20 +0000 Subject: Usage of Secure C (memcpy_s, strcpy_s etc) functions on OpenSSL In-Reply-To: References: Message-ID: <0101016eaa1cbc95-b85269d9-a66e-4f61-8acb-504a62067aa1-000000@us-west-2.amazonses.com> On 11/26/2019 7:41 AM, Michael Wojcik wrote: > The Appendix K functions (memcpy_s, etc) do NOT "remove buffer > overflow kind of issues completely", and anyone who thinks they do is > making a serious error. The Appendix K functions impose an additional > check. That's all they do. It is possible, and in some use cases quite > easy, for the developer to pass the wrong value for the destsz > parameter and invalidate that check. > > Some C experts have argued that the length-checking versions of the > library functions, either the C90 ones such as strncat or the Appendix > K ones, are essentially pointless anyway; that the caller needs to > handle truncation and so ought to know whether truncation (or > overflow) would occur before attempting the operation. I was initially a fan of them when I first heard of them, but have since soured on them, as have others.? They are very nearly useless for libraries, because their behavior is controlled on a process-global basis.? The library cannot assume that the "bad" cases will result in aborts, because the application might have chosen to have them return errors instead.? That means that the library has to check for and handle all of those "should be impossible" error cases. Here's a paper on the subject:? http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm -- Jordan Brown, Oracle ZFS Storage Appliance, Oracle Solaris -------------- next part -------------- An HTML attachment was scrubbed... URL: From pankajnsarode at gmail.com Wed Nov 27 06:16:51 2019 From: pankajnsarode at gmail.com (Pankaj Sarode) Date: Wed, 27 Nov 2019 11:46:51 +0530 Subject: need your help about fipsld in Example OpenSSL Based Application In-Reply-To: <7d89ace6.334.16ea8babdf3.Coremail.sunchunjie_0_0@163.com> References: <7d89ace6.334.16ea8babdf3.Coremail.sunchunjie_0_0@163.com> Message-ID: Hi, You will need to pass your native compiler i.e gcc to fipsld script and make the fipsld script refer to your native compiler. Example: Make CC= FIPSLD_CC=/USR/BIN/GCC refer section 5.3.1 of fips-openssl-userguide-2.0 Pankaj On Tue, Nov 26, 2019, 11:06 PM ?? wrote: > Dear all, > As a newbie, I am following the details in document > UserGuide-2.0.pdf,Appendix C Example OpenSSL Based Application,C1, which > creates one Makefile and one c source code file. when run make command, it > always use ld to do the link, not the fipsld, but when I run command like : > make CC=/path/to/fipsld > it seems ran into loop, so, I need your help about what to fix to make > the example works. > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shivakumar2696 at gmail.com Wed Nov 27 11:07:31 2019 From: shivakumar2696 at gmail.com (shiva kumar) Date: Wed, 27 Nov 2019 16:37:31 +0530 Subject: OpenSSL 1.0.2 EOL In-Reply-To: <315a1f70-4cb5-4a7d-c890-391a581285f7@openssl.org> References: <315a1f70-4cb5-4a7d-c890-391a581285f7@openssl.org> Message-ID: but still the update is going on in the GitHub repository for 102 branch, is that mean there will be a release by end of this year? On Tue, Nov 26, 2019 at 6:31 PM Matt Caswell wrote: > > > On 26/11/2019 11:38, shiva kumar wrote: > > Hi, > > As we know that OpenSSL 1.0.2 support will end in 31st December 2019. > > and the latest version is 1.0.2t, is there will be any release by EOL? > > can we expect a release before EOL? > > This is as yet undecided. We issue releases on an as-needed basis and so > far we have not needed another release. That might change before the end > of the year. > > Matt > > -- *With Best Regards* *Shivakumar S* -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at openssl.org Wed Nov 27 11:33:20 2019 From: matt at openssl.org (Matt Caswell) Date: Wed, 27 Nov 2019 11:33:20 +0000 Subject: OpenSSL 1.0.2 EOL In-Reply-To: References: <315a1f70-4cb5-4a7d-c890-391a581285f7@openssl.org> Message-ID: <130cc715-7157-164f-7f8b-2ef3aa6c57da@openssl.org> On 27/11/2019 11:07, shiva kumar wrote: > but still the update is going on in the GitHub repository for 102 > branch, is that mean there will be a release by end of this year? There are no commits against the 1.0.2 branch that would qualify for a CVE - they are all relatively minor commits. Like I said, it is currently undecided whether we will do another release or not. Historically we have tended to do a "mop up" release shortly before the end of support, and so we might speculate that we will do another one in this case - but it would be speculation only. Matt > > On Tue, Nov 26, 2019 at 6:31 PM Matt Caswell > wrote: > > > > On 26/11/2019 11:38, shiva kumar wrote: > > Hi, > > As we know that OpenSSL 1.0.2 support will end in 31st? December 2019. > > and the latest version is 1.0.2t, is there will be any release by EOL? > > can we expect a release before EOL? > > This is as yet undecided. We issue releases on an as-needed basis and so > far we have not needed another release. That might change before the end > of the year. > > Matt > > > > -- > *With Best Regards* > *Shivakumar S* From phill at thesusis.net Wed Nov 27 14:59:35 2019 From: phill at thesusis.net (Phillip Susi) Date: Wed, 27 Nov 2019 09:59:35 -0500 Subject: Usage of Secure C (memcpy_s, strcpy_s etc) functions on OpenSSL In-Reply-To: References: Message-ID: <87d0ddgyaw.fsf@vps.thesusis.net> Michael Wojcik writes: > Some C experts have argued that the length-checking versions of the library functions, either the C90 ones such as strncat or the Appendix K ones, are essentially pointless anyway; that the caller needs to handle truncation and so ought to know whether truncation (or overflow) would occur before attempting the operation. Isn't this normally/easilly handled simply by passing sizeof( buffer ) - 1? Then the last byte is always \0 whether or not the copy was truncated. From paul at mad-scientist.net Wed Nov 27 15:06:44 2019 From: paul at mad-scientist.net (Paul Smith) Date: Wed, 27 Nov 2019 10:06:44 -0500 Subject: Usage of Secure C (memcpy_s, strcpy_s etc) functions on OpenSSL In-Reply-To: <0101016eaa1cbc95-b85269d9-a66e-4f61-8acb-504a62067aa1-000000@us-west-2.amazonses.com> References: <0101016eaa1cbc95-b85269d9-a66e-4f61-8acb-504a62067aa1-000000@us-west-2.amazonses.com> Message-ID: On Tue, 2019-11-26 at 23:47 +0000, Jordan Brown wrote: > Here's a paper on the subject: > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm I love the fact that the "correct and safe" example they give in "Unnecessary Uses" is neither correct nor safe (it has a potential DOS due to memory leak). However I definitely do agree that the Appendix K functions are of marginal use _at best_... I don't use them or recommend them myself. From Michael.Wojcik at microfocus.com Wed Nov 27 15:57:04 2019 From: Michael.Wojcik at microfocus.com (Michael Wojcik) Date: Wed, 27 Nov 2019 15:57:04 +0000 Subject: Usage of Secure C (memcpy_s, strcpy_s etc) functions on OpenSSL In-Reply-To: <87d0ddgyaw.fsf@vps.thesusis.net> References: , <87d0ddgyaw.fsf@vps.thesusis.net> Message-ID: > From: Phillip Susi > > Michael Wojcik writes: > > Some C experts have argued that the length-checking versions of the library functions, either the C90 > > ones such as strncat or the Appendix K ones, are essentially pointless anyway; that the caller needs to > > handle truncation and so ought to know whether truncation (or overflow) would occur before > > attempting the operation. > Isn't this normally/easilly handled simply by passing sizeof( buffer ) - > 1?? Then the last byte is always \0 whether or not the copy was truncated. No, for a number of reasons: - The buffer may be dynamically allocated, in which case sizeof buffer is very much the wrong thing. - A trailing nul byte is not always desirable (not everything is a string), and not all of the Appendix K functions will append one. Neither will strncpy, if the limit is reached; strncpy has broken semantics. - The buffer size (less 1) is the wrong value to pass when using strncat with a destination that does not start with a nul character. Developers also frequently miss the fact that strncat can append length+1 bytes to the buffer. (strncat's semantics are dangerously non-obvious.) - Non-trivial programs *will still have to handle truncation properly*. It's rarely the case for a non-trivial program that "silently truncate the data" is a good way to handle it. A well-behaved program will treat truncation as an error or a special case that needs further handling, or at least needs to be flagged to the user or some other diagnostic sink. The C90 length-checked functions don't communicate truncation to the caller; the Annex K ones only return "a non-zero value" if a constraint is violated, so there's no standardization as to how a particular constraint violation is communicated. Also, sizeof is an operator, not a function. There's no reason to parenthesize its argument unless the argument is a type name, in which case it's parenthesized as a special syntactic case. Incidentally, I misremembered; it's Annex K, not Appendix K, and it was introduced in C11, not C99. So the Annex K functions weren't even standardized until C11, and they're still optional there (though Annex K is normative). From andrew.tucker at salesforce.com Wed Nov 27 18:03:35 2019 From: andrew.tucker at salesforce.com (Andrew Tucker) Date: Wed, 27 Nov 2019 10:03:35 -0800 Subject: Usage of Secure C (memcpy_s, strcpy_s etc) functions on OpenSSL In-Reply-To: <87d0ddgyaw.fsf@vps.thesusis.net> References: <87d0ddgyaw.fsf@vps.thesusis.net> Message-ID: Unless buffer is a char* instead of a char[] in which case its completely wrong. A very common case among buggy C code. On Wed, Nov 27, 2019 at 7:09 AM Phillip Susi wrote: > > Michael Wojcik writes: > > > Some C experts have argued that the length-checking versions of the > library functions, either the C90 ones such as strncat or the Appendix K > ones, are essentially pointless anyway; that the caller needs to handle > truncation and so ought to know whether truncation (or overflow) would > occur before attempting the operation. > > Isn't this normally/easilly handled simply by passing sizeof( buffer ) - > 1? Then the last byte is always \0 whether or not the copy was truncated. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunchunjie_0_0 at 163.com Thu Nov 28 04:53:38 2019 From: sunchunjie_0_0 at 163.com (=?GBK?B?wq/W7g==?=) Date: Thu, 28 Nov 2019 12:53:38 +0800 (CST) Subject: need your help about fipsld in Example OpenSSL Based Application In-Reply-To: References: <7d89ace6.334.16ea8babdf3.Coremail.sunchunjie_0_0@163.com> Message-ID: <347853e9.9206.16eb05b84d0.Coremail.sunchunjie_0_0@163.com> Thank you for the suggestion. When I do like make CC=/path/to/fipsld FIPSLD_CC=/point/to/gcc It gave out /bin/sh: warning: shell level (1000) too high, resetting to 1 then putty exit. At 2019-11-27 13:16:51, "Pankaj Sarode" wrote: Hi, You will need to pass your native compiler i.e gcc to fipsld script and make the fipsld script refer to your native compiler. Example: Make CC= FIPSLD_CC=/USR/BIN/GCC refer section 5.3.1 of fips-openssl-userguide-2.0 Pankaj On Tue, Nov 26, 2019, 11:06 PM ?? wrote: Dear all, As a newbie, I am following the details in document UserGuide-2.0.pdf,Appendix C Example OpenSSL Based Application,C1, which creates one Makefile and one c source code file. when run make command, it always use ld to do the link, not the fipsld, but when I run command like : make CC=/path/to/fipsld it seems ran into loop, so, I need your help about what to fix to make the example works. -------------- next part -------------- An HTML attachment was scrubbed... URL: From d3ck0r at gmail.com Fri Nov 29 00:31:38 2019 From: d3ck0r at gmail.com (J Decker) Date: Thu, 28 Nov 2019 16:31:38 -0800 Subject: Differently named symbols between OpenSSL and RFC Message-ID: I made this issue on LibreSSL's github... https://github.com/libressl-portable/portable/issues/537 It's about ... TLSEXT_TYPE_psk_kex_modes: from openssl/tls1.h 1.1.1b # define TLSEXT_TYPE_psk_kex_modes 45 from libressl/2.9.2 tls1.h #define TLSEXT_TYPE_psk_key_exchange_modes 45 They argue that the names they defined follow the RFC https://tools.ietf.org/html/rfc8446#section-4.2 pre_shared_key(41), psk_key_exchange_modes(45), are what the names should be based on, and that OpenSSL's definition is also inaccurate TLSEXT_TYPE_psk_kex_modes TLSEXT_TYPE_psk https://github.com/openssl/openssl/blob/master/include/openssl/tls1.h#L143-L147 instead of 'pre_shared_key' and 'key_exchange_modes' (sorry, this looks kind of ragged, get what I'm sayin?) -------------- next part -------------- An HTML attachment was scrubbed... URL: From openssl-users at dukhovni.org Fri Nov 29 18:16:32 2019 From: openssl-users at dukhovni.org (Viktor Dukhovni) Date: Fri, 29 Nov 2019 13:16:32 -0500 Subject: Differently named symbols between OpenSSL and RFC In-Reply-To: References: Message-ID: <20191129181632.GM34850@straasha.imrryr.org> On Thu, Nov 28, 2019 at 04:31:38PM -0800, J Decker wrote: > from openssl/tls1.h 1.1.1b > > # define TLSEXT_TYPE_psk_kex_modes 45 This was added in 1.1.1-dev. > pre_shared_key(41), psk_key_exchange_modes(45), are what the names should be > based on, and that OpenSSL's definition is also inaccurate > > TLSEXT_TYPE_psk_kex_modes > TLSEXT_TYPE_psk > > instead of 'pre_shared_key' and 'key_exchange_modes' The TLS RFCs do not define an API, they only define a wire protocol. Implementations are free to use whatever names they see fit for the various RFC code-points. OpenSSL chose to abbreviate "key_exchange" to "kex" and "pre_shared_key" to "psk". The LibreSSL maintainers are free to make their own choices. -- Viktor. From d3ck0r at gmail.com Sat Nov 30 20:12:00 2019 From: d3ck0r at gmail.com (J Decker) Date: Sat, 30 Nov 2019 12:12:00 -0800 Subject: Differently named symbols between OpenSSL and RFC In-Reply-To: <20191129181632.GM34850@straasha.imrryr.org> References: <20191129181632.GM34850@straasha.imrryr.org> Message-ID: On Fri, Nov 29, 2019 at 10:16 AM Viktor Dukhovni wrote: > On Thu, Nov 28, 2019 at 04:31:38PM -0800, J Decker wrote: > > > from openssl/tls1.h 1.1.1b > > > > # define TLSEXT_TYPE_psk_kex_modes 45 > > This was added in 1.1.1-dev. > > > pre_shared_key(41), psk_key_exchange_modes(45), are what the names > should be > > based on, and that OpenSSL's definition is also inaccurate > > > > TLSEXT_TYPE_psk_kex_modes > > TLSEXT_TYPE_psk > > > > instead of 'pre_shared_key' and 'key_exchange_modes' > > The TLS RFCs do not define an API, they only define a wire protocol. > > Implementations are free to use whatever names they see fit for the > various RFC > code-points. OpenSSL chose to abbreviate "key_exchange" to "kex" and > "pre_shared_key" to "psk". The LibreSSL maintainers are free to make > their own > choices. > Well; since they are the clone of the original, they're not exactly 'free' to make choices; there are constraints issued by consumers of their libraries (me) that they should be compatible. So; Their argument for name choice is 'RFC' , I thought 'understandable easily' but making it the same as the RFC allows one to search for information on what that extension might be. Looking for 'psk' and 'kex' doesn't lead to related information very well, which is an argument that OpenSSL should also provide alternate names which align better with other information about such names... I looked a while to find those symbols when I initially ran into this issue; and psk just kept coming back to openssl headers/docs which themselves don't give very good information about the block. > > -- > Viktor. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From uri at ll.mit.edu Sat Nov 30 23:58:00 2019 From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL) Date: Sat, 30 Nov 2019 23:58:00 +0000 Subject: Writing an ENGINE for OpenSSL-1.1.1 and 3.0 Message-ID: In preparation for writing a new engine that supports message digest and asymmetric crypto (sign and decrypt), I am trying to port the existing simple/demo engines from the Engine Corner examples (thanks, Richard!). The fork of https://github.com/engine-corner/Lesson-2-A-digest.git that compiles and runs correctly (apparently) on OpenSSL-1.1.1 is https://github.com/mouse07410/Lesson-2-A-digest.git . But no matter what, I was unable to make either of these two repos to run successfully with OpenSSL-3.0 (master), even though the latter repo at least seems to compile correctly, and answers ?Available?. Here?s what I?m getting (for both versions I built the sample ?emd5?engine and copied to the appropriate subdir of the lib/; ?openssl3? is an alias that points OPENSSL_CONF at the correct file, and invokes the correct OpenSSL-3.0 ?openssl? binary, as it?s not on the main path): $ penssl version OpenSSL 1.1.1d 10 Sep 2019 $ openssl engine -t -c emd5 (emd5) A simple md5 engine for demonstration purposes [MD5] [ available ] $ echo "shoot" | openssl dgst -md5 -engine emd5 engine "emd5" set. (stdin)= 61a08703a6a4c774cad650afaedd9c10 $ echo "shoot" | openssl dgst -md5 (stdin)= 61a08703a6a4c774cad650afaedd9c10 $ $ openssl3 version OpenSSL 3.0.0-dev xx XXX xxxx (Library: OpenSSL 3.0.0-dev xx XXX xxxx) $ openssl3 engine -t -vv -c emd5 (emd5) A simple md5 engine for demonstration purposes [MD5] [ available ] $ echo "shoot" | openssl3 dgst -md5 MD5(stdin)= 61a08703a6a4c774cad650afaedd9c10 $ echo "shoot" | openssl3 dgst -md5 -engine emd5 engine "emd5" set. Error setting digest C0:05:98:0C:01:00:00:00:error:digital envelope routines:EVP_DigestInit_ex:initialization error:crypto/evp/digest.c:224: $ Something must be missing from the configuration/setup ? but what?? Help would be greatly appreciated! Thanks! ? Regards, Uri -------------- next part -------------- An HTML attachment was scrubbed... URL: