Application accessing 'ex_kusage'

Thu Nov 19 16:24:38 UTC 2020

Hi Matt,
                Thanks for the response, the application code has been around a very long time and no one knows the rationale behind it
As I understand from your reply that the application need not have to do these operations internally, so we will go ahead and stub it out for now

Regards,
Sunil

From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of openssl-users-request at openssl.org
Sent: 17 November 2020 08:44
To: openssl-users at openssl.org
Subject: openssl-users Digest, Vol 72, Issue 14

________________________________
NOTICE: This email was received from an EXTERNAL sender
________________________________

Send openssl-users mailing list submissions to
openssl-users at openssl.org<mailto:openssl-users at openssl.org>

To subscribe or unsubscribe via the World Wide Web, visit
https://mta.openssl.org/mailman/listinfo/openssl-users<https://mta.openssl.org/mailman/listinfo/openssl-users>
or, via email, send a message with subject or body 'help' to
openssl-users-request at openssl.org<mailto:openssl-users-request at openssl.org>

You can reach the person managing the list at
openssl-users-owner at openssl.org<mailto:openssl-users-owner at openssl.org>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of openssl-users digest..."

Today's Topics:

1. Re: ## Application accessing 'ex_kusage' ## (Matt Caswell)
2. Re: RAND_bytes() thread safety (Matt Caswell)
3. Re: test cases failed after enabling ktls (Matt Caswell)
4. Handling BIO errors (Jo?o Santos)
5. Re: test cases failed after enabling ktls (rui zang)
6. Re: Server application hangs on SS_read, even when client
disconnects (Jakob Bohm)

----------------------------------------------------------------------

Message: 1
Date: Mon, 16 Nov 2020 10:16:24 +0000
From: Matt Caswell <matt at openssl.org<mailto:matt at openssl.org>>
To: openssl-users at openssl.org<mailto:openssl-users at openssl.org>
Subject: Re: ## Application accessing 'ex_kusage' ##
Message-ID: <b854f116-568a-d795-9186-098a6d8f4281 at openssl.org<mailto:b854f116-568a-d795-9186-098a6d8f4281 at openssl.org>>
Content-Type: text/plain; charset=utf-8

On 13/11/2020 19:10, Narayana, Sunil Kumar wrote:
> Hi ,
>
> ??????????????? We are porting our Application from ?openssl 1.0.1 to
> openssl 3.0. in related to this activity we require to access the
> variable ?*ex_kusage*? pointed by *X509*
>
> But there are no set utilities available to access this variable. Only
> ?X509_get_key_usage Is available.
>
> ?
>
> Our code for 1.0.1 is as below. Please suggest the right way to achieve
> this.

I'd like to ask why you feel you need to do this at all. It seems to me
like you are replicating libcrypto internal code in your own
application. This is code in libcrypto:

/* Handle (basic) key usage */
if ((usage = X509_get_ext_d2i(x, NID_key_usage, &i, NULL)) != NULL) {
x->ex_kusage = 0;
if (usage->length > 0) {
x->ex_kusage = usage->data[0];
if (usage->length > 1)
x->ex_kusage |= usage->data[1] << 8;
}
x->ex_flags |= EXFLAG_KUSAGE;
ASN1_BIT_STRING_free(usage);
/* Check for empty key usage according to RFC 5280 section
4.2.1.3<http://4.2.1.3> */
if (x->ex_kusage == 0) {
ERR_raise(ERR_LIB_X509, X509V3_R_EMPTY_KEY_USAGE);
x->ex_flags |= EXFLAG_INVALID;
}
} else if (i != -1) {
x->ex_flags |= EXFLAG_INVALID;
}

So it seems very similar to what you are trying to do, and I guess some
earlier version of this code was the original source of what is in your
application now.

The purpose of this code is to decode the key usage extension and cache
it in the internal `ex_flags` value. This code gets called in numerous
code paths whenever we need to query extension data - including if you
were to call X509_get_key_usage().

Your application seems to want to manage for itself when libcrypto does
this caching. It should not need to do so - it's entirely internal. My
guess is that, perhaps, in some older version of OpenSSL the caching
didn't happen when it was supposed to and you implemented this
workaround?? Or possibly the workaround is still needed due to a bug in
OpenSSL that still doesn't do the caching when needed? If so I'd like to
understand the circumstances behind that.

Matt

------------------------------

Message: 2
Date: Mon, 16 Nov 2020 10:35:06 +0000
From: Matt Caswell <matt at openssl.org<mailto:matt at openssl.org>>
To: openssl-users at openssl.org<mailto:openssl-users at openssl.org>
Subject: Re: RAND_bytes() thread safety
Message-ID: <f84a0efa-0a0b-d118-8430-18ea43957251 at openssl.org<mailto:f84a0efa-0a0b-d118-8430-18ea43957251 at openssl.org>>
Content-Type: text/plain; charset=utf-8

On 14/11/2020 11:00, Rahul Godbole wrote:
> Is OpenSSL function RAND_bytes () thread safe?

Short answer: Yes

Longer answer: Yes assuming that:
- you are using >= OpenSSL 1.1.0
or
- you are using OpenSSL 1.0.2 or below and you have set up the locking
callbacks

AND

- You have not compiled OpenSSL with "no-threads"

Matt

------------------------------

Message: 3
Date: Mon, 16 Nov 2020 11:45:20 +0000
From: Matt Caswell <matt at openssl.org<mailto:matt at openssl.org>>
To: openssl-users at openssl.org<mailto:openssl-users at openssl.org>
Subject: Re: test cases failed after enabling ktls
Message-ID: <749b7676-2c84-1b63-9b75-575f903e14f7 at openssl.org<mailto:749b7676-2c84-1b63-9b75-575f903e14f7 at openssl.org>>
Content-Type: text/plain; charset=utf-8

On 16/11/2020 07:56, rui zang wrote:
> Resend in plain text.
> ======================================
>
> Greetings,
>
> I am trying openssl+ktls on ubuntu 20.04.
> I have tried openssl-3.0.0-alpha8 from https://www.openssl.org/source/openssl-3.0.0-alpha8.tar.gz<https://www.openssl.org/source/openssl-3.0.0-alpha8.tar.gz>
> and also the current master branch from the github repo.
> The kernels I have tried are v5.9 and v5.9.8.
> On every combination, same group of test case could not pass.

Please can you open this as a github issue?

Thanks

Matt

>
> Test Summary Report
> -------------------
> 70-test_key_share.t (Wstat: 1536 Tests: 22 Failed: 6)
> Failed tests: 1, 4, 6-7, 13-14
> Non-zero exit status: 6
> 70-test_sslextension.t (Wstat: 256 Tests: 8 Failed: 1)
> Failed test: 8
> Non-zero exit status: 1
> 70-test_sslrecords.t (Wstat: 768 Tests: 20 Failed: 3)
> Failed tests: 18-20
> Non-zero exit status: 3
> 70-test_sslsigalgs.t (Wstat: 1536 Tests: 26 Failed: 6)
> Failed tests: 1, 6, 22-23, 25-26
> Non-zero exit status: 6
> 70-test_sslsignature.t (Wstat: 256 Tests: 4 Failed: 1)
> Failed test: 1
> Non-zero exit status: 1
> 70-test_sslversions.t (Wstat: 512 Tests: 8 Failed: 2)
> Failed tests: 5, 7
> Non-zero exit status: 2
> 70-test_tls13cookie.t (Wstat: 512 Tests: 2 Failed: 2)
> Failed tests: 1-2
> Non-zero exit status: 2
> 70-test_tls13downgrade.t (Wstat: 256 Tests: 6 Failed: 1)
> Failed test: 6
> Non-zero exit status: 1
> 70-test_tls13kexmodes.t (Wstat: 7424 Tests: 1 Failed: 1)
> Failed test: 1
> Non-zero exit status: 29
> Parse errors: Bad plan. You planned 11 tests but ran 1.
> 70-test_tls13messages.t (Wstat: 7424 Tests: 1 Failed: 0)
> Non-zero exit status: 29
> Parse errors: Bad plan. You planned 17 tests but ran 1.
> 70-test_tls13psk.t (Wstat: 7424 Tests: 1 Failed: 1)
> Failed test: 1
> Non-zero exit status: 29
> Parse errors: Bad plan. You planned 5 tests but ran 1.
> 70-test_tlsextms.t (Wstat: 256 Tests: 10 Failed: 1)
> Failed test: 10
> Non-zero exit status: 1
> Files=223, Tests=3571, 615 wallclock secs (11.00 usr 0.93 sys + 343.65 cusr 84.69 csys = 440.27 CPU)
> Result: FAIL
> make[1]: *** [Makefile:3197: _tests] Error 1
> make[1]: Leaving directory '/home/ubuntu/openssl'
> make: *** [Makefile:3194: tests] Error 2
>
> Complete `make test` output (kernel v5.9.8 + openssl master) is copied here https://cl1p.net/openssl_ktls_make_test_failure<https://cl1p.net/openssl_ktls_make_test_failure> (due to the 100K limit of this mailing list)
> I am sure that the kernel tls module is loaded correctly since /proc/net/tls_stat is exposed correctly and I can see the counters increasing while doing `make test`.
> So is this supposed to happen? What should I do to make ktls work?
>
> Thanks,
> Rui Zang
>

------------------------------

Message: 4
Date: Tue, 17 Nov 2020 02:33:30 +0000
From: Jo?o Santos <jps at xce.pt<mailto:jps at xce.pt>>
To: openssl-users at openssl.org<mailto:openssl-users at openssl.org>
Subject: Handling BIO errors
Message-ID: <009450BD-7D9B-47D7-A1D2-8A4476EE52BE at xce.pt<mailto:009450BD-7D9B-47D7-A1D2-8A4476EE52BE at xce.pt>>
Content-Type: text/plain; charset=us-ascii

I'm writing a daemon that talks to a server using HTTP/2 over TLS 1.2+ and leveraging OpenSSL 1.1.1h to provide the TLS support.

At the moment I think that I have the whole TLS part figured, and I could probably have the project running by now if I used SSL_set_fd to assign a connected socket to the underlying BIO of an SSL object, but I want to simplify the code as much as possible by using the highest level interfaces at my disposal, which in the case of OpenSSL means using BIO objects.

Unfortunately I'm having a problem which is that I can't figure out how to convert error codes returned by ERR_get_error and split by ERR_GET_LIB, ERR_GET_FUNC, and ERR_GET_REASON into constants that I can use in a switch statement to react to BIO errors. This is not a problem for SSL filter BIOs since those have their own error reporting functions, but is a problem for Internet socket source BIOs since BIO_do_connect in particular can fail due to a system call error, a DNS error,, or even an error generated by lower level OpenSSL functions and other BIOs in the chain, and I cannot find any manual pages documenting these error constants, if they even exist.

Here's a small working example that illustrates the problem that I'm having:

#include <stdio.h>
#include <openssl/bio.h>
#include <openssl/err.h>

int main(void) {
ERR_load_ERR_strings();
BIO *bio = BIO_new_connect("http://wwx.google.com:80<http://wwx.google.com:80>");
printf("Connected: %ld\n", BIO_do_connect(bio));
ERR_print_errors_fp(stderr);
return 0;
}

Running this code, which has a misspelled hostname on purpose so that it can fail, results in the following printed out to the console:

Connected: -1
4667342272:error:2008F002:BIO routines:BIO_lookup_ex:system lib:crypto/bio/b_addr.c:726:nodename nor servname provided, or not known

What could I do in that code to use a switch statement on the kind of information printed by ERR_print_errors_fp? I know that, in this example, the error is from getaddrinfo, since I recognize the error message, but assuming that I want to handle that specific error, what can I match the library, function, and reason error codes against?

Thanks in advance!

------------------------------

Message: 5
Date: Tue, 17 Nov 2020 11:08:27 +0800
From: rui zang <rui.zang at yandex.com<mailto:rui.zang at yandex.com>>
To: Matt Caswell <matt at openssl.org<mailto:matt at openssl.org>>, "openssl-users at openssl.org<mailto:openssl-users at openssl.org>"
<openssl-users at openssl.org<mailto:openssl-users at openssl.org>>
Subject: Re: test cases failed after enabling ktls
Message-ID: <1089471605582408 at mail.yandex.com<mailto:1089471605582408 at mail.yandex.com>>
Content-Type: text/plain; charset=utf-8

Thanks, please check out https://github.com/openssl/openssl/issues/13424<https://github.com/openssl/openssl/issues/13424>

Regards,
Rui Zang

16.11.2020, 19:45, "Matt Caswell" <matt at openssl.org<mailto:matt at openssl.org>>:
> On 16/11/2020 07:56, rui zang wrote:
>> ?Resend in plain text.
>> ?======================================
>>
>> ?Greetings,
>>
>> ?I am trying openssl+ktls on ubuntu 20.04.
>> ?I have tried openssl-3.0.0-alpha8 from https://www.openssl.org/source/openssl-3.0.0-alpha8.tar.gz<https://www.openssl.org/source/openssl-3.0.0-alpha8.tar.gz>
>> ?and also the current master branch from the github repo.
>> ?The kernels I have tried are v5.9 and v5.9.8.
>> ?On every combination, same group of test case could not pass.
>
> Please can you open this as a github issue?
>
> Thanks
>
> Matt
>
>> ?Test Summary Report
>> ?-------------------
>> ?70-test_key_share.t (Wstat: 1536 Tests: 22 Failed: 6)
>> ???Failed tests: 1, 4, 6-7, 13-14
>> ???Non-zero exit status: 6
>> ?70-test_sslextension.t (Wstat: 256 Tests: 8 Failed: 1)
>> ???Failed test: 8
>> ???Non-zero exit status: 1
>> ?70-test_sslrecords.t (Wstat: 768 Tests: 20 Failed: 3)
>> ???Failed tests: 18-20
>> ???Non-zero exit status: 3
>> ?70-test_sslsigalgs.t (Wstat: 1536 Tests: 26 Failed: 6)
>> ???Failed tests: 1, 6, 22-23, 25-26
>> ???Non-zero exit status: 6
>> ?70-test_sslsignature.t (Wstat: 256 Tests: 4 Failed: 1)
>> ???Failed test: 1
>> ???Non-zero exit status: 1
>> ?70-test_sslversions.t (Wstat: 512 Tests: 8 Failed: 2)
>> ???Failed tests: 5, 7
>> ???Non-zero exit status: 2
>> ?70-test_tls13cookie.t (Wstat: 512 Tests: 2 Failed: 2)
>> ???Failed tests: 1-2
>> ???Non-zero exit status: 2
>> ?70-test_tls13downgrade.t (Wstat: 256 Tests: 6 Failed: 1)
>> ???Failed test: 6
>> ???Non-zero exit status: 1
>> ?70-test_tls13kexmodes.t (Wstat: 7424 Tests: 1 Failed: 1)
>> ???Failed test: 1
>> ???Non-zero exit status: 29
>> ???Parse errors: Bad plan. You planned 11 tests but ran 1.
>> ?70-test_tls13messages.t (Wstat: 7424 Tests: 1 Failed: 0)
>> ???Non-zero exit status: 29
>> ???Parse errors: Bad plan. You planned 17 tests but ran 1.
>> ?70-test_tls13psk.t (Wstat: 7424 Tests: 1 Failed: 1)
>> ???Failed test: 1
>> ???Non-zero exit status: 29
>> ???Parse errors: Bad plan. You planned 5 tests but ran 1.
>> ?70-test_tlsextms.t (Wstat: 256 Tests: 10 Failed: 1)
>> ???Failed test: 10
>> ???Non-zero exit status: 1
>> ?Files=223, Tests=3571, 615 wallclock secs (11.00 usr 0.93 sys + 343.65 cusr 84.69 csys = 440.27 CPU)
>> ?Result: FAIL
>> ?make[1]: *** [Makefile:3197: _tests] Error 1
>> ?make[1]: Leaving directory '/home/ubuntu/openssl'
>> ?make: *** [Makefile:3194: tests] Error 2
>>
>> ?Complete `make test` output (kernel v5.9.8 + openssl master) is copied here https://cl1p.net/openssl_ktls_make_test_failure<https://cl1p.net/openssl_ktls_make_test_failure> (due to the 100K limit of this mailing list)
>> ?I am sure that the kernel tls module is loaded correctly since /proc/net/tls_stat is exposed correctly and I can see the counters increasing while doing `make test`.
>> ?So is this supposed to happen? What should I do to make ktls work?
>>
>> ?Thanks,
>> ?Rui Zang

------------------------------

Message: 6
Date: Tue, 17 Nov 2020 04:13:54 +0100
From: Jakob Bohm <jb-openssl at wisemo.com<mailto:jb-openssl at wisemo.com>>
To: openssl-users at openssl.org<mailto:openssl-users at openssl.org>
Subject: Re: Server application hangs on SS_read, even when client
disconnects
Message-ID: <66c3b7e7-871f-b0c0-3e4c-1968c8d6b91c at wisemo.com<mailto:66c3b7e7-871f-b0c0-3e4c-1968c8d6b91c at wisemo.com>>
Content-Type: text/plain; charset=utf-8; format=flowed

(Top posting to match what Mr. Andr? does):

TCP without keepalive will time out the connection a few minutes after
sending any data that doesn't get a response.

TCP without keepalive with no outstanding send (so only a blocking
recv) and nothing outstanding at the other end will probably hang
almost forever as there is nothing indicating that there is actual
data lost in transit.

On 2020-11-13 17:13, Brice Andr? wrote:
> Hello,
>
> And many thanks for the answer.
>
> "Does the server parent process close its copy of the conversation
> socket?" : I checked in my code, but it seems that no. Is it needed? ?
> May it explain my problem ?
>
> " Do you have keepalives enabled?" To be honest, I did not know it was
> possible to not enable them. I checked with command "netstat -tnope"
> and it tells me that it is not enabled.
>
> I suppose that, if for some reason, the communication with the client
> is lost (crash of client, loss of network, etc.) and keepalive is not
> enabled, this may fully explain my problem ?
>
> If yes, do you have an idea of why keepalive is not enabled ? I
> thought that by default on linux it was ?
>
> Many thanks,
> Brice
>
>
> Le?ven. 13 nov. 2020 ??15:43, Michael Wojcik
> <Michael.Wojcik at microfocus.com <mailto:Michael.Wojcik at microfocus.com<mailto:Michael.Wojcik at microfocus.com%20%3cmailto:Michael.Wojcik at microfocus.com>>>
> a ?crit?:
>
> > From: openssl-users <openssl-users-bounces at openssl.org
<mailto:openssl-users-bounces at openssl.org%0b>> <mailto:openssl-users-bounces at openssl.org>> On Behalf Of Brice Andr?
> > Sent: Friday, 13 November, 2020 05:06
>
> > ... it seems that in some rare execution cases, the server
> performs a SSL_read,
> > the client disconnects in the meantime, and the server never
> detects the
> > disconnection and remains stuck in the SSL_read operation.
>
> ...
>
> > #0? 0x00007f836575d210 in __read_nocancel () from
> /lib/x86_64-linux-gnu/libpthread.so.0
> > #1? 0x00007f8365c8ccec in ?? () from
> /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
> > #2? 0x00007f8365c8772b in BIO_read () from
> /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
>
> So OpenSSL is in a blocking read of the socket descriptor.
>
> > tcp? ? ? ? 0? ? ? 0 http://5.196.111.132:5413<http://5.196.111.132:5413>
> <http://5.196.111.132:5413<http://5.196.111.132:5413>> http://85.27.92.8:25856<http://85.27.92.8:25856>
> <http://85.27.92.8:25856<http://85.27.92.8:25856>> ? ? ? ESTABLISHED 19218/./MabeeServer
> > tcp? ? ? ? 0? ? ? 0 http://5.196.111.132:5412<http://5.196.111.132:5412>
> <http://5.196.111.132:5412<http://5.196.111.132:5412>> http://85.27.92.8:26305<http://85.27.92.8:26305>
> <http://85.27.92.8:26305<http://85.27.92.8:26305>> ? ? ? ESTABLISHED 19218/./MabeeServer
>
> > From this log, I can see that I have two established connections
> with remote
> > client machine on IP 109.133.193.70<http://109.133.193.70>. Note that it's normal to
> have two connexions
> > because my client-server protocol relies on two distinct TCP
> connexions.
>
> So the client has not, in fact, disconnected.
>
> When a system closes one end of a TCP connection, the stack will
> send a TCP packet
> with either the FIN or the RST flag set. (Which one you get
> depends on whether the
> stack on the closing side was holding data for the conversation
> which the application
> hadn't read.)
>
> The sockets are still in ESTABLISHED state; therefore, no FIN or
> RST has been
> received by the local stack.
>
> There are various possibilities:
>
> - The client system has not in fact closed its end of the
> conversation. Sometimes
> this happens for reasons that aren't immediately apparent; for
> example, if the
> client forked and allowed the descriptor for the conversation
> socket to be inherited
> by the child, and the child still has it open.
>
> - The client system shut down suddenly (crashed) and so couldn't
> send the FIN/RST.
>
> - There was a failure in network connectivity between the two
> systems, and consequently
> the FIN/RST couldn't be received by the local system.
>
> - The connection is in a state where the peer can't send the
> FIN/RST, for example
> because the local side's receive window is zero. That shouldn't be
> the case, since
> OpenSSL is (apparently) blocked in a receive on the connection.
> but as I don't have
> the complete picture I can't rule it out.
>
> > This let me think that the connexion on which the SSL_read is
> listening is
> > definitively dead (no more TCP keepalive)
>
> "definitely dead" doesn't have any meaning in TCP. That's not one
> of the TCP states,
> or part of the other TCP or IP metadata associated with the local
> port (which is
> what matters).
>
> Do you have keepalives enabled?
>
> > and that, for a reason I do not understand, the SSL_read keeps
> blocked into it.
>
> The reason is simple: The connection is still established, but
> there's no data to
> receive. The question isn't why SSL_read is blocking; it's why you
> think the
> connection is gone, but the stack thinks otherwise.
>
> > Note that the normal behavior of my application is : client
> connects, server
> > daemon forks a new instance,
>
> Does the server parent process close its copy of the conversation
> socket?
>
>

Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com<https://www.wisemo.com>
Transformervej 29, 2860 S?borg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

------------------------------

Subject: Digest Footer

_______________________________________________
openssl-users mailing list
openssl-users at openssl.org<mailto:openssl-users at openssl.org>
https://mta.openssl.org/mailman/listinfo/openssl-users<https://mta.openssl.org/mailman/listinfo/openssl-users>

------------------------------

End of openssl-users Digest, Vol 72, Issue 14
*********************************************

-----------------------------------------------------------------------------------------------------------------------
Notice: This e-mail together with any attachments may contain information of Ribbon Communications Inc. that
is confidential and/or proprietary for the sole use of the intended recipient.  Any review, disclosure, reliance or
distribution by others or forwarding without express permission is strictly prohibited.  If you are not the intended
recipient, please notify the sender immediately and then delete all copies, including any attachments.
-----------------------------------------------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20201119/fe612648/attachment-0001.html>