Issues with ASYNC_pause_job() wake up

Tue Feb 11 15:39:45 UTC 2020

Thanks Matt, this was very helpful. I'll dig into libcurl to understand better. I appreciated your help.

Kind regards,
Valerio 

--
Valerio Di Gregorio
ENGINEER.SOFTWARE ENGINEERING
vadigreg at cisco.com | +41 21 694 3840

.:|:.:|:. Cisco Systems International Sàrl

EPFL, Quartier de l'Innovation
Batiment E, 2nd Floor, Desk A4-7
1015 Ecublens, Vaud
Switzerland

-----Original Message-----
From: Matt Caswell <matt at openssl.org> 
Sent: Tuesday, February 11, 2020 4:14 PM
To: Valerio Di Gregorio (vadigreg) <vadigreg at cisco.com>; openssl-users at openssl.org
Subject: Re: Issues with ASYNC_pause_job() wake up

On 11/02/2020 14:49, Valerio Di Gregorio (vadigreg) wrote:
> My understanding is that libcurl's multi interface is async.

I know nothing about curl, but a quick search of the codebase does turn up some evidence that it seems to be ASYNC aware. At least it is prepared to accept SSL_ERROR_WANT_ASYNC as a return value from
SSL_get_error():

https://github.com/curl/curl/blob/355a5a310019659d9bf6818d2fd66fbb214dfed7/lib/vtls/openssl.c#L2940-L2955

> That's
> the interface I'm using. I configured SSL_CTX_set_mode(ssl_ctx,
> SSL_MODE_ASYNC) as you pointed out. I did in the same libcurl callback 
> I used for SSL_CTX_set_cert_verify_callback(). I can see
> ASYNC_pause_job() actually pausing and returning with no error, so I 
> must assume I'm into an async job.
> 
> Should I call ASYNC_start_job() right after the write() to wake-up and 
> read()?

When your application knows that the callback is ready to continue it must ensure that whatever OpenSSL I/O operation was in progress prior to the pause is then invoked again. So if SSL_connect() was being called at the time of the pause, then SSL_connect() needs to be called again.

I don't know what that means you need to do in libcurl terms.

Matt

> 
> Val
> 
> -----Original Message----- From: Matt Caswell <matt at openssl.org>
> Sent: Tuesday, February 11, 2020 3:36 PM To: Valerio Di Gregorio
> (vadigreg) <vadigreg at cisco.com>; openssl-users at openssl.org Subject:
> Re: Issues with ASYNC_pause_job() wake up
> 
> 
> 
> On 11/02/2020 14:26, Valerio Di Gregorio (vadigreg) wrote:
>> Hi Matt, Thanks for your reply. HTTPS request is initiated using 
>> libcurl. I've installed a callback in libcurl as 
>> CURLOP_T_SSL_CTX_FUNCTION which in turn installs a callback using 
>> SSL_CTX_set_cert_verify_callback(). So my application never calls 
>> ASYNC_start_job().
>> 
>> I expected my application to write() on the write file descriptor of 
>> the pipe and that was enough to un-pause, without any need of calling 
>> ASYNC_start_job() explicitily.
>> 
>> What's the right usage of this async APIs in my case?
> 
> Hmm. Well, unfortunately if libcurl is managing the interaction with 
> libssl then this won't work unless libcurl is async aware (I don't 
> know if it is or not).
> 
> ASYNC_pause_job() does nothing at all if you are not running within an 
> async job. An async job is *not* started automatically by libssl.
> In order for that to happen you have to put the SSL object into ASYNC 
> mode using a call to SSL_set_mode() or SSL_CTX_set_mode(), and passing 
> the `SSL_MODE_ASYNC` value. See:
> 
> https://www.openssl.org/docs/man1.1.1/man3/SSL_set_mode.html
> 
> Once set into async mode libssl will call ASYNC_start_job() 
> automatically for all IO functions (e.g. SSL_read()/SSL_write() etc)
> 
> Further, when the application calls SSL_get_error(), it must be 
> prepared to handle SSL_ERROR_WANT_ASYNC as a return value.
> 
> So, probably, you need to investigate whether libcurl can do this.
> 
> Matt
> 
>> 
>> Many thanks for helping!
>> 
>> Val
>> 
>> 
>> -----Original Message----- From: openssl-users 
>> <openssl-users-bounces at openssl.org> On Behalf Of Matt Caswell
>> Sent: Tuesday, February 11, 2020 2:52 PM To:
>> openssl-users at openssl.org Subject: Re: Issues with
>> ASYNC_pause_job() wake up
>> 
>> 
>> 
>> On 11/02/2020 13:05, Valerio Di Gregorio (vadigreg) via openssl-users 
>> wrote:
>>> 1. ASYNC_pause_job() can wake up before write(). It will then block 
>>> on the read(), which is too bad in my single-threaded code.
>>> 2. I fixed case 1 by making read() non-blocking, I then run
>>> ASYNC_pause_job() again and again until write() is actually 
>>> performed. So now I hit another issue. Time between
>>> printf("pause") and write() is ~100/200 milliseconds. However time 
>>> between write() and printf("resume") is usually ~4.5 seconds and 
>>> this way too much.
>> 
>> 
>> It's not clear from your description how your application starts the 
>> ASYNC job in the first place. Are you using libssl to do this, or 
>> does your application call ASYNC_start_job() directly?
>> 
>> When your certificate validation code calls ASYNC_pause_job() control 
>> will return to your application. Control will not return until 
>> ASYNC_start_job() is called again. Depending on how you started the 
>> job in the first place this will either be due to your application 
>> explicitly calling ASYNC_start_job() directly, or (if it used libssl 
>> to start the job), then by recalling the I/O function that hit the 
>> pause originally.
>> 
>> Once ASYNC_start_job() is invoked then context swap should happen 
>> immediately (so ~4.5 seconds would be totally unexpected).
>> 
>> So please explain a little more about how your application starts the 
>> job, and how it monitors its progress in order to know when to 
>> resume.
>> 
>> Matt
>> 
>> 
>>> 
>>> 
>>> 
>>> Is there a reliable way (only when data is really ready) and a 
>>> prompt way (in order of micro/milli-seconds) to wake up from 
>>> ASYNC_pause_job()?
>>> 
>>> 
>>> 
>>> Many thanks, any help would be much appreciated.
>>> 
>>> 
>>> 
>>> Kind Regards,
>>> 
>>> Valerio Di Gregorio
>>> 
>> 