[openssl-dev] Failure of MD_Init not checked in crypto/rand/md_rand.c

[Pascal Cuoq]

I tried to report this issue one week ago through RT, but failed somehow.
I didn't insist because I then noticed that the issue had already been reported as:
https://rt.openssl.org/Ticket/Display.html?user=guest&pass=guest&id=3159

However, the patch provided in that report makes some unrelated changes.

A better candidate for a patch should be the one I attach.

In the first chunk, the function being patched, ssleay_rand_add, returns
void. Therefore a choice should be made between returning when
failure is detected or calling abort(). Note that the current behavior
when MD_init fails is to dereference a null pointer a few instructions
later, so that calling abort() would not be a loss of functionality (but
it would improve the safety of the code. Dereferencing a null pointer
is undefined behavior and is not guaranteed to crash the program).

The remaining chunks patch the function ssleay_rand_bytes, which
has a convention to signaling errors through a return code, documented at:
https://www.openssl.org/docs/crypto/RAND_bytes.html

The patch makes the function return 0 in case of allocation failure,
which seems to best match the existing documented cases:

“RAND_bytes() returns 1 on success, 0 otherwise.
The error code can be obtained by ERR_get_error.
RAND_pseudo_bytes() returns 1 if the bytes generated are cryptographically strong,
0 otherwise.
Both functions return -1 if they are not supported by the current RAND method.”

Pascal Cuoq
TrustInSoft Chief Scientist

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20150407/5e6313f8/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: check_MD_Init.patch
Type: application/octet-stream
Size: 1736 bytes
Desc: check_MD_Init.patch
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20150407/5e6313f8/attachment.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20150407/5e6313f8/attachment.htm>