[openssl-users] Performance problems with OpenSSL and threading

[Bryan Call]

It is in non-blocking mode.

After removing the call to SSL_get_error for the SSL_read case (0 return value) I discovered we call SSL_get_error in a couple more places.  Here is a simple request and the return codes on SSL_accept, SSL_read, and SSL_write and the return codes from SSL_get_error.  Every time we get a return code that is <= 0 we call SSL_get_error.  For a simple request we call it 5 times!

[May  8 09:38:01.075] Server {0xe95c000} DEBUG: <SSLUtils.cc:2111 (SSLAccept)> (bcall) SSL_accept - return value: -1 SSL_get_error: 2 errno: 0 shutdown: 0
[May  8 09:38:01.081] Server {0xe95c000} DEBUG: <SSLUtils.cc:2099 (SSLAccept)> (bcall) SSL_accept - return value: 1 SSL_get_error: 0 errno: 0 shutdown: 0
[May  8 09:38:01.081] Server {0xe95c000} DEBUG: <SSLUtils.cc:2087 (SSLReadBuffer)> (bcall) SSL_read - bytes: 0 return value: -1 SSL_get_error: 2 errno: 35 shutdown: 0
[May  8 09:38:01.103] Server {0xe95c000} DEBUG: <SSLUtils.cc:2066 (SSLReadBuffer)> (bcall) SSL_read - bytes: 0 return value: 73 SSL_get_error: 0 errno: 0 shutdown: 0
[May  8 09:38:01.103] Server {0xe95c000} DEBUG: <SSLUtils.cc:2087 (SSLReadBuffer)> (bcall) SSL_read - bytes: 0 return value: -1 SSL_get_error: 2 errno: 35 shutdown: 0
[May  8 09:38:01.113] Server {0xe95c000} DEBUG: <SSLUtils.cc:2033 (SSLWriteBuffer)> (bcall) SSL_write - bytes: 0 return value: 364 SSL_get_error: 0 errno: 0 shutdown: 0
[May  8 09:38:01.113] Server {0xe95c000} DEBUG: <SSLUtils.cc:2033 (SSLWriteBuffer)> (bcall) SSL_write - bytes: 0 return value: 249 SSL_get_error: 0 errno: 0 shutdown: 0
[May  8 09:38:01.113] Server {0xe95c000} DEBUG: <SSLUtils.cc:2087 (SSLReadBuffer)> (bcall) SSL_read - bytes: 0 return value: -1 SSL_get_error: 2 errno: 35 shutdown: 0
[May  8 09:38:01.113] Server {0xe95c000} DEBUG: <SSLUtils.cc:2087 (SSLReadBuffer)> (bcall) SSL_read - bytes: 0 return value: 0 SSL_get_error: 6 errno: 0 shutdown: 2

errno value from above (on OSX):
sys/errno.h:#define	EAGAIN		35		/* Resource temporarily unavailable */

If someone knows of a good way to reduce the number of calls to SSL_get_error that would be really helpful.  I am trying to push 20Gbits/second on this server.  I am hoping that I can get the same results as I would have from SSL_get_error, by looking at other variables in the SSL structure or errno or should I wait for the lock handling to be cleaned up in the error-stack?

-Bryan




> On May 7, 2015, at 10:59 AM, John Foley <foleyj at cisco.com> wrote:
> 
> Not sure.  
> 
> Are you using blocking or non-blocking IO?  
> Have you tried SSL_MODE_AUTO_RETRY?
> Do you notice a different return value from SSL_read() after a zero byte read compared to other errors?
>