[openssl-dev] Memory leak in application when we use ECDH
Mody, Darshan (Darshan)
darshanmody at avaya.com
Mon Mar 27 08:40:30 UTC 2017
Matt, Richard,
I have not created any patch. Instead I am using s_server app provided from Openssl. Below is the server side command run
openssl s_server -accept <port> -key <custom-key> -cert <custom-certificate signed by custom CA> -CAfile <custom self signed CA> -verify <custom self signed CA>
On the client side I am running script below
#/bin/sh
while [ 1 ]
do
openssl s_client -connect <server-ip>:<server-port> -cert <custom client certificate not trusted by server> -key <custom client certificate key> &>/dev/null
done
Running the client once and printing the results showed the cipher selected was ECDHE cipher. I believe there is inherent leak in the ECDHE side of openssl.
I find that the server started leaking was leaking. Check the Resident memory and the CPU. Resident memory increases after every 3 second. When I had stopped the client traffic the resident memory did not increase however it did not returned back to what it was earlier. Hope this will help to reproduce the issue.
top - 10:16:37 up 46 days, 1:29, 2 users, load average: 0.03, 0.06, 0.08
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.6%us, 0.1%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3924288k total, 3757716k used, 166572k free, 78200k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219736k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 39908 3528 2740 S 0.0 0.1 0:00.00 openssl
top - 10:16:40 up 46 days, 1:29, 2 users, load average: 0.03, 0.06, 0.08
Tasks: 1 total, 1 running, 0 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.6%us, 0.9%sy, 0.0%ni, 97.4%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 3924288k total, 3757856k used, 166432k free, 78208k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219744k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 39908 3532 2740 R 4.3 0.1 0:00.13 openssl
top - 10:16:43 up 46 days, 1:29, 2 users, load average: 0.03, 0.06, 0.08
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 2.6%us, 0.8%sy, 0.1%ni, 96.2%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st
Mem: 3924288k total, 3757980k used, 166308k free, 78232k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219744k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 39908 3544 2740 S 7.0 0.1 0:00.34 openssl
top - 10:16:46 up 46 days, 1:30, 2 users, load average: 0.03, 0.06, 0.08
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 2.1%us, 1.1%sy, 0.0%ni, 96.5%id, 0.0%wa, 0.0%hi, 0.2%si, 0.0%st
Mem: 3924288k total, 3757964k used, 166324k free, 78248k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219732k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 39908 3560 2740 S 6.7 0.1 0:00.54 openssl
top - 10:16:49 up 46 days, 1:30, 2 users, load average: 0.02, 0.06, 0.08
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 2.7%us, 1.1%sy, 0.0%ni, 96.0%id, 0.0%wa, 0.0%hi, 0.2%si, 0.0%st
Mem: 3924288k total, 3758212k used, 166076k free, 78264k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219744k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 39908 3572 2740 S 8.0 0.1 0:00.78 openssl
top - 10:16:52 up 46 days, 1:30, 2 users, load average: 0.02, 0.06, 0.08
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.2%us, 1.2%sy, 0.0%ni, 94.8%id, 0.1%wa, 0.0%hi, 0.6%si, 0.0%st
Mem: 3924288k total, 3758212k used, 166076k free, 78288k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219732k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 39908 3592 2740 S 11.0 0.1 0:01.11 openssl
top - 10:16:55 up 46 days, 1:30, 2 users, load average: 0.02, 0.06, 0.08
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.3%us, 1.3%sy, 0.1%ni, 94.7%id, 0.0%wa, 0.0%hi, 0.6%si, 0.0%st
Mem: 3924288k total, 3758584k used, 165704k free, 78288k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219744k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 39908 3608 2740 S 11.3 0.1 0:01.45 openssl
top - 10:16:58 up 46 days, 1:30, 2 users, load average: 0.02, 0.06, 0.08
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.3%us, 1.6%sy, 0.0%ni, 94.7%id, 0.0%wa, 0.0%hi, 0.4%si, 0.0%st
Mem: 3924288k total, 3758864k used, 165424k free, 78296k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219744k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 39908 3628 2740 S 11.3 0.1 0:01.79 openssl
top - 10:17:01 up 46 days, 1:30, 2 users, load average: 0.02, 0.06, 0.08
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.4%us, 1.5%sy, 0.0%ni, 94.7%id, 0.0%wa, 0.0%hi, 0.4%si, 0.0%st
Mem: 3924288k total, 3758956k used, 165332k free, 78312k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219740k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 40016 3648 2740 S 10.7 0.1 0:02.11 openssl
top - 10:17:04 up 46 days, 1:30, 2 users, load average: 0.02, 0.06, 0.07
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.9%us, 1.9%sy, 0.0%ni, 91.9%id, 1.6%wa, 0.0%hi, 0.7%si, 0.0%st
Mem: 3924288k total, 3759708k used, 164580k free, 78328k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219916k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 40016 3668 2740 S 11.0 0.1 0:02.44 openssl
top - 10:17:07 up 46 days, 1:30, 2 users, load average: 0.02, 0.06, 0.07
Tasks: 1 total, 1 running, 0 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.4%us, 1.2%sy, 0.1%ni, 93.8%id, 0.0%wa, 0.0%hi, 1.4%si, 0.0%st
Mem: 3924288k total, 3759336k used, 164952k free, 78360k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219900k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 40016 3688 2740 R 10.7 0.1 0:02.76 openssl
top - 10:17:10 up 46 days, 1:30, 2 users, load average: 0.02, 0.06, 0.07
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.2%us, 1.6%sy, 0.0%ni, 94.2%id, 0.0%wa, 0.0%hi, 1.0%si, 0.0%st
Mem: 3924288k total, 3758584k used, 165704k free, 78360k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219916k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 40016 3708 2740 S 11.3 0.1 0:03.10 openssl
top - 10:17:13 up 46 days, 1:30, 2 users, load average: 0.02, 0.06, 0.07
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.6%us, 1.1%sy, 0.0%ni, 94.5%id, 0.0%wa, 0.0%hi, 0.8%si, 0.0%st
Mem: 3924288k total, 3758476k used, 165812k free, 78376k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219920k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 40016 3728 2740 S 10.7 0.1 0:03.42 openssl
top - 10:17:16 up 46 days, 1:30, 2 users, load average: 0.02, 0.06, 0.07
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.8%us, 1.3%sy, 0.0%ni, 93.3%id, 0.0%wa, 0.0%hi, 1.7%si, 0.0%st
Mem: 3924288k total, 3759824k used, 164464k free, 78384k buffers
Swap: 8388604k total, 999556k used, 7389048k free, 219920k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 40124 3748 2740 S 11.3 0.1 0:03.76 openssl
---------------
top - 10:23:10 up 46 days, 1:36, 2 users, load average: 0.03, 0.05, 0.06
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.6%us, 1.3%sy, 0.1%ni, 94.4%id, 0.0%wa, 0.0%hi, 0.6%si, 0.0%st
Mem: 3924288k total, 3756048k used, 168240k free, 75888k buffers
Swap: 8388604k total, 998972k used, 7389632k free, 216576k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 42392 6048 2740 S 10.7 0.2 0:41.95 openssl
top - 10:23:13 up 46 days, 1:36, 2 users, load average: 0.02, 0.04, 0.06
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.2%us, 1.3%sy, 0.0%ni, 94.8%id, 0.0%wa, 0.0%hi, 0.6%si, 0.0%st
Mem: 3924288k total, 3756064k used, 168224k free, 75904k buffers
Swap: 8388604k total, 998972k used, 7389632k free, 216572k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 42392 6068 2740 S 10.7 0.2 0:42.27 openssl
top - 10:23:16 up 46 days, 1:36, 2 users, load average: 0.02, 0.04, 0.06
Tasks: 1 total, 1 running, 0 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.2%us, 1.3%sy, 0.1%ni, 94.9%id, 0.0%wa, 0.0%hi, 0.4%si, 0.0%st
Mem: 3924288k total, 3756188k used, 168100k free, 75912k buffers
Swap: 8388604k total, 998972k used, 7389632k free, 216572k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 42392 6088 2740 R 11.0 0.2 0:42.60 openssl
top - 10:23:19 up 46 days, 1:36, 2 users, load average: 0.02, 0.04, 0.06
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.1%us, 1.2%sy, 1.6%ni, 93.5%id, 0.0%wa, 0.0%hi, 0.6%si, 0.0%st
Mem: 3924288k total, 3756528k used, 167760k free, 75920k buffers
Swap: 8388604k total, 998972k used, 7389632k free, 216584k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 42392 6108 2740 S 10.7 0.2 0:42.92 openssl
top - 10:23:22 up 46 days, 1:36, 2 users, load average: 0.02, 0.04, 0.06
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 11.1%us, 4.6%sy, 20.2%ni, 63.6%id, 0.0%wa, 0.0%hi, 0.6%si, 0.0%st
Mem: 3924288k total, 3756924k used, 167364k free, 75952k buffers
Swap: 8388604k total, 998972k used, 7389632k free, 216600k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27303 root 20 0 42500 6124 2740 S 10.3 0.2 0:43.23 openssl
Thanks
Darshan
-----Original Message-----
From: openssl-dev [mailto:openssl-dev-bounces at openssl.org] On Behalf Of Mody, Darshan (Darshan)
Sent: Thursday, March 23, 2017 7:40 PM
To: openssl-dev at openssl.org
Cc: Bahr, William G (Bill)
Subject: Re: [openssl-dev] Memory leak in application when we use ECDH
Thanks Richard and Matt,
I will patch it and send the patch. It will take me couple of days.
Regards
Darshan
-----Original Message-----
From: openssl-dev [mailto:openssl-dev-bounces at openssl.org] On Behalf Of Richard Levitte
Sent: Thursday, March 23, 2017 7:31 PM
To: openssl-dev at openssl.org
Subject: Re: [openssl-dev] Memory leak in application when we use ECDH
I think that Matt is asking for example code that exhibits this leak.
You could patch apps/s_server.c with your callback, or ssl/ssltest.c, and give us that patch.
The reason is that we can't know what assumptions you're going with in your callback or application, so if we code an example together, it will be with Our conditions, not yours, and therefore a pretty bad method to figure this out.
Cheers,
Richard
In message <25D2EC755404B4409F263AC6D050FEBB2A10781E at AZ-FFEXMB03.global.avaya.com> on Thu, 23 Mar 2017 13:47:10 +0000, "Mody, Darshan (Darshan)" <darshanmody at avaya.com> said:
darshanmody> Matt,
darshanmody>
darshanmody> Below is the scenario.
darshanmody>
darshanmody> 1. Have server open a listen socket which always validates the client certificate and chain.
darshanmody> 2. On server support ECDHE using callback. Ensure the EC_KEY passed to openssl from app is cleaned up by the app.
darshanmody> 3. Connect client with certificates that server does not trust.
darshanmody> 4. The connections from client to server fails
darshanmody>
darshanmody> In course of time the app running the server has been leaking. Even after accounting for the EC_KEY passed by the server app to openssl we find there seems to be leak. Further investigation on the core dumps generated from the server app shows that it has the certificates from the client saved.
darshanmody>
darshanmody> Hope this helps
darshanmody>
darshanmody> Thanks
darshanmody> Darshan
darshanmody>
darshanmody> -----Original Message-----
darshanmody> From: openssl-dev [mailto:openssl-dev-bounces at openssl.org]
darshanmody> On Behalf Of Matt Caswell
darshanmody> Sent: Thursday, March 23, 2017 6:55 PM
darshanmody> To: openssl-dev at openssl.org
darshanmody> Subject: Re: [openssl-dev] Memory leak in application when
darshanmody> we use ECDH
darshanmody>
darshanmody>
darshanmody>
darshanmody> On 23/03/17 13:19, Mody, Darshan (Darshan) wrote:
darshanmody> > Can you further elaborate?
darshanmody> >
darshanmody> > What we did is to create a TLS connection and with
darshanmody> > invalid certificates from the client and server on
darshanmody> > verification would reject the certificate. The cipher
darshanmody> > negotiated was ECDHE cipher between client and server.
darshanmody> >
darshanmody> > This was done with load (multiple while 1 script trying
darshanmody> > to connect to server using invalid certificates and in
darshanmody> > course of time the memory was increasing).
darshanmody>
darshanmody> Without being able to recreate the problem its going to be very difficult/impossible for us to fix it (assuming the problem is in OpenSSl itself). We would need some simple reproducer code that demonstrates the problem occurring.
darshanmody>
darshanmody> Matt
darshanmody>
darshanmody>
darshanmody> >
darshanmody> > Thanks Darshan
darshanmody> >
darshanmody> > -----Original Message----- From: openssl-dev
darshanmody> > [mailto:openssl-dev-bounces at openssl.org] On Behalf Of
darshanmody> > Matt Caswell
darshanmody> > Sent: Thursday, March 23, 2017 4:09 PM To:
darshanmody> > openssl-dev at openssl.org
darshanmody> > Subject: Re: [openssl-dev] Memory leak in application
darshanmody> > when we use ECDH
darshanmody> >
darshanmody> >
darshanmody> >
darshanmody> > On 23/03/17 10:13, Mody, Darshan (Darshan) wrote:
darshanmody> >> Matt,
darshanmody> >>
darshanmody> >> Even after accounting for the EC_KEY we still observe some leak.
darshanmody> >> The leak started after we started using supporting EC
darshanmody> >> with callback SSL_set_tmp_ecdh_callback().
darshanmody> >>
darshanmody> >> The core dump shows the string data of the far-end certificates.
darshanmody> >> I cannot pin point the code in openssl with this regard.
darshanmody> >
darshanmody> > Are you able to create a simple reproducer demonstrating
darshanmody> > the problem with the callback?
darshanmody> >
darshanmody> > Matt
darshanmody> >
darshanmody> --
darshanmody> openssl-dev mailing list
darshanmody> To unsubscribe:
darshanmody> https://urldefense.proofpoint.com/v2/url?u=https-3A__mta.op
darshanmody> enssl.org_mailman_listinfo_openssl-2Ddev&d=DwICAg&c=BFpWQw8
darshanmody> bsuKpl1SgiZH64Q&r=bsEULbVnjelD7InzgsegHBEbtXzaIDagy9EuEhJrK
darshanmody> fQ&m=VbrRgO8PZIVkFM4PjeK7TEgKDHnbLu_QfbyqRhmvx8I&s=u0cR7sQf
darshanmody> _Zz8FoCnrzgLc3drBSR8Ou1qDUyxV8z1xYQ&e=
darshanmody> --
darshanmody> openssl-dev mailing list
darshanmody> To unsubscribe:
darshanmody> https://urldefense.proofpoint.com/v2/url?u=https-3A__mta.op
darshanmody> enssl.org_mailman_listinfo_openssl-2Ddev&d=DwICAg&c=BFpWQw8
darshanmody> bsuKpl1SgiZH64Q&r=bsEULbVnjelD7InzgsegHBEbtXzaIDagy9EuEhJrK
darshanmody> fQ&m=OtZlUFiavvOVqXL900IST85y3pZLikUdEgekBIIyZoI&s=3T5xlm8q
darshanmody> 92-eP1ItbDzGOU972l4wFrkJUgLrBNR4Qx8&e=
darshanmody>
--
openssl-dev mailing list
To unsubscribe: https://urldefense.proofpoint.com/v2/url?u=https-3A__mta.openssl.org_mailman_listinfo_openssl-2Ddev&d=DwICAg&c=BFpWQw8bsuKpl1SgiZH64Q&r=bsEULbVnjelD7InzgsegHBEbtXzaIDagy9EuEhJrKfQ&m=OtZlUFiavvOVqXL900IST85y3pZLikUdEgekBIIyZoI&s=3T5xlm8q92-eP1ItbDzGOU972l4wFrkJUgLrBNR4Qx8&e=
--
openssl-dev mailing list
To unsubscribe: https://urldefense.proofpoint.com/v2/url?u=https-3A__mta.openssl.org_mailman_listinfo_openssl-2Ddev&d=DwICAg&c=BFpWQw8bsuKpl1SgiZH64Q&r=bsEULbVnjelD7InzgsegHBEbtXzaIDagy9EuEhJrKfQ&m=aM8Nc6bSJzLs-HdERDikYMeQK6uUqvjYvWnfqtvREM4&s=5DECvTKmFHf9urfCwMDaDnk14k9fM_XDkpl9X5r5aJA&e=
More information about the openssl-dev
mailing list