[openssl-users] Intermittent SSL_shutdown error

[Sahib Jakhar]

Hi,

I am trying to create a SSL connection (call it connection 2) inside
another SSL connection (call it connection 1). Connection 2 is used to
connect to a machine which is hidden behind the machine to which I
connect using connection 1. Once Connection 2 is established (i.e. SSL
handshake finishes) I close connection 1.

The problem I am facing is SSL_shutdown is returning error (return
code < 0). Most of the time it works, and I am successfully able to
communicate over connection 2 after connection 1 has been closed. But
suddenly once out of the blue it returns error. Subsequent SSL_read on
connection 2 fail with error.

**SSL_get_error returns: 2**

**ERR_error_string returns: error:00000002:lib(0):func(0):system lib**


Note connection 1 is with a java server and connection 2 is with a
openssl based server. Can somebody please help me identify what could
be going wrong over here?

Here is some code for your reference to help you understand what I am doing:


    SSL *ssl; // connection 1
    SSL *ssl2; //connection 2

    // ssl is already established at this point
    // i.e. connection 1 already exists.

    BIO *rbio = BIO_new (BIO_s_mem());
    BIO *wbio = BIO_new (BIO_s_mem());

    SSL_set_bio (ssl, rbio, wbio);
    SSL_set_connect_state (ssl);

    while (!SSL_is_init_finished(ssl))
    {
        ret = SSL_do_handshake (ssl);
        if (ret == 1) /* Handshake was successful */
        {
            break;
        }

        ssl_error = SSL_get_error (ssl, ret);

        if (ssl_error != SSL_ERROR_WANT_READ
        && ssl_error != SSL_ERROR_WANT_WRITE)
        {
            // report failed
            return;
        }

        size = BIO_ctrl_pending (wbio);
        if (size > 0)
        {
            size = BIO_read (wbio, buff, size);
            if (size <= 0)
            {
                // report failed this shouldn't happen
                continue;
            }

            // Write using SSL_WRITE to connection 1
        }

        /* Read only if SSL_do_handshake expects */
        if (ssl_error == SSL_ERROR_WANT_READ)
        {
            // read into buff from SSL_READ from connection 1
            // continue if noting available

            BIO_write (rbio, buff, size);
            free (buff);
        }
    }

    // Write to SSL connection 1 handshake is successful
    // to let Javaserver know that it should close ssl part of connection 1

    SSL_set_quiet_shutdown (ssl, 0);
    ret = SSL_shutdown (ssl);
    if (ret == 0)
        ret = SSL_shutdown (ssl);
    if (ret < 0)
    {
        /* FAILS HERE: This is where it fails sometimes
        (may be once or twice out of 10)*/
    }

    // Some clean up

    /* We are done with cleanup of old SSL connection,
     * and establishing new SSL connection to the new
     * one. Now let's start communicating using connection 2.
     * But before that we need to do a few things. */

    sock_bio = BIO_new_socket (INTERNAL (agentbi)->sock_fd, BIO_NOCLOSE);
    SSL_set_bio (ssl2, sock_bio, sock_bio);

    // From this point onwards we use connection 2 to communicate as
    // connection 1 has been reduced to tcp only.



On the java side I can see close_notify being sent and received:
I call close on SSLSocket once client tells that its SSL handshake for
connection 2 is complete.