SSL_read() returning SSL_ERROR_SYSCALL with errno 11 EAGAIN

[Viktor Dukhovni]

On Thu, May 02, 2019 at 04:10:31PM +0000, John Unsworth wrote:

> > Do you wait for the non-blocking connect to complete at this point?
> We connect in blocking mode then switch to non-blocking.

Thanks that rules connection setup out of the picture.

> > Are multiple threads writing to the same SSL connection?  How do you
> > ensure orderly use of the SSL connection?  Sharing connections across
> > threads without application level synchronization is not supported in
> > OpenSSL.
> 
> We use mutexes to synchronize of course.

OK.

> > How are further requests locked out when you're performing reads?
> > What is the granularity of the relevant locks?
> 
> The mutex only allows one SSL call at a time.

So a serialized mix of reads and writes with possibly multiple
outstanding writes interleaved with the reader?  Right?

> > At this point you'd be calling SSL_get_error(), is there a lock that
> > prevents writes between SSL_read() and SSL_read() and SSL_get_error()?
> 
> The mutex does not protect SSL_get_error() calls.

I think that's an application bug.  The SSL_get_error() is using
the same SSL handle as the SSL_read(), which can be materially
altered by concurrent writes.  (Matt, if you're still reading this
thread, do you agree?)

I would not release the mutex until after the call to SSL_get_error().

> > I gather the protocol is full-duplex and multiple outstanding requests
> > can be written before the corresponding replies are read?  Or is it strict
> > half-duplex request-response?
> 
> It is full duplex and there can be multiple operations in progress.

Please retest with both the SSL_read() and SSL_get_error() running
under a single lock.  And similarly on the write side.

Do keep in mind that a server may close the socket for further
writes after replying to a number of requests (ideally sending an
SSL close notify, i.e. SSL_shutdown() as it does so), at which point
the SSL connection is half-closed.  With a full-duplex protocol,
you may also need to handle reading outstanding replies on a
connection that no longer supports writes.

-- 
	Viktor.