[openssl-users] OpenSSL sending close_notify right after responding to a heartbeat request

[R Kaja Mohideen]

I have used backtrace function (execinfo.h) as documented here
(http://www.gnu.org/software/libc/manual/html_node/Backtraces.html) in
couple of OpenSSL source files - ssl_lib.c (ssl_shutdown) and s3_pkt.c
(ssl3_send_alert). I have actually used the exact same example from
that page for getting the stack trace printed from those two
functions.

When I reproduce the issue - following is the stacktrace I got

Obtained 3 stack frames.
/OpenSSL/libssl.so.1.0.0(printStackTrace+0x2d) [0x7f13927f482d]
/OpenSSL/libssl.so.1.0.0(SSL_shutdown+0x9) [0x7f13927f74a9]
[0x7f14a901f9e4]
Obtained 4 stack frames.
/OpenSSL/libssl.so.1.0.0(printStackTrace1+0x2d) [0x7f13927da4dd]
/OpenSSL/libssl.so.1.0.0(ssl3_send_alert+0x11) [0x7f13927dbe11]
/OpenSSL/libssl.so.1.0.0(ssl3_shutdown+0xa2) [0x7f13927d8662]
[0x7f14a901f9e4]

I'm surprised to see that I'm not able to get the caller details using
backtrace. Is it due to architecture of OpenSSL or something which
makes OpenSSL to use a new thread for invoking ssl_shutdown?

Any OpenSSL developers?

// Kaja

On Fri, Mar 24, 2017 at 7:10 PM, R Kaja Mohideen <reachme at kajasweb.com> wrote:
> Hi,
>
> We have a TLS Server (Written in C) and Client (Written in Java using
> Netty + OpenSSL).
>
> I see that when Server sends a TLS extension Heartbeat request to
> client - OpenSSL responds to it and sends a close_notify alert right
> after it - causing the server to close the session with client.
>
> I don't have any callback registered in client (HB request recipient
> side - Java/Netty doesn't really have that support) and so I'm sure
> that it is OpenSSL by itself is responding to that heartbeat request.
> But, who or what is making OpenSSL to send an alert & close the
> session upon responding to heartbeat remains a mystery.
>
> Any help / suggestions to investigate this issue is highly appreciated.
>
> Thanks & regards,
> R Kaja Mohideen