<div dir="ltr"><div dir="ltr"><div dir="ltr">Hi,<div>This is how I have initialized my variables:-</div><div><br></div><div>EVP_MD_CTX *mdctx;</div><div>const EVP_MD *md;</div><div>int i;</div><div>HASH hash_data;</div><div>unsigned char message_data[BUFFER_SIZE];</div><div><br></div><div>BUFFER_SIZE has been defined as 131072</div><div>and HASH is my hash structure defined to hold the message digest, message digest length and message digest type</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, 23 Feb 2019 at 00:17, Jordan Brown <<a href="mailto:openssl@jordan.maileater.net">openssl@jordan.maileater.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div class="gmail-m_-3919589885526903920moz-cite-prefix">The most obvious question is "how are
you allocating your message_data buffer?". You don't show that.</div>
<div class="gmail-m_-3919589885526903920moz-cite-prefix"><br>
</div>
<div class="gmail-m_-3919589885526903920moz-cite-prefix">On 2/22/2019 2:27 AM, prithiraj das
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr"><br>
<div>
<div>Hi All,
<div><br>
</div>
<div>Using OpenSSL 1.0.2g, I have written a code to
generate the hash of a file in an embeddded device
having linux OS and low memory capacity and the files
are generally of size 44 MB or more. The first time or
even the second time on some occasions, the hash of
any file is successfully generated. On the 3rd or 4th
time (possibly due to lack of memory/memory leak), the
system reboots before the hash can be generated.
After restart, the same thing happens when the
previous steps are repeated.</div>
<div>The stats below shows the memory usage before and
after computing the hash. </div>
</div>
</div>
<div><br>
</div>
<div>
<div><b>root@at91sam9m10g45ek:~# free</b></div>
<div><b> total used
free shared buff/cache available</b></div>
<div><b>Mem: 252180 13272 223048
280 15860 230924</b></div>
<div><b>Swap: 0 0 0</b></div>
</div>
<div><b><br>
</b></div>
<div><b>After computing hash :-</b></div>
<div>
<div><b>root@at91sam9m10g45ek:~# free</b></div>
<div><b> total used
free shared buff/cache available</b></div>
<div><b>Mem: 252180 13308 179308
280 59564 230868</b></div>
<div><b>Swap: 0 0
0</b></div>
<div style="font-weight:bold"><br>
</div>
</div>
<div style="font-weight:bold">
<div style="font-weight:400">Buff/cache increases by
almost 44MB (same as file size) everytime I generate the
hash and free decreases. I believe the file is being
loaded into buffer and not being freed. <br>
</div>
<div style="font-weight:400"><br>
</div>
<div style="font-weight:400">I am using the below code to
compute the message digest. This code is part of a
function ComputeHash and the file pointer here is fph.</div>
<div style="font-weight:400"><br>
</div>
<div style="font-weight:400"> <b> EVP_add_digest(EVP_sha256());</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> md
= EVP_get_digestbyname("sha256");</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> </b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> if(!md)
{</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span>
printf("Unknown message digest \n");</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span>
exit(1);</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> }</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("Message
digest algorithm successfully loaded\n");</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> mdctx
= EVP_MD_CTX_create();</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> EVP_DigestInit_ex(mdctx,
md, NULL);</b></div>
<div style="font-weight:400"><b><br>
</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> //
Reading data to array of unsigned chars<span style="white-space:pre-wrap"> </span></b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> long
long int bytes_read = 0;</b></div>
<div style="font-weight:400"><b><br>
</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("FILE
size of the file to be hashed is %ld",filesize);<span style="white-space:pre-wrap"> </span></b></div>
<div style="font-weight:400"><b><br>
</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> //reading
image file in chunks below and fph is the file pointer
to the 44MB file</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> while
((bytes_read = fread (message_data, 1, BUFFER_SIZE,
fph)) != 0)</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> <span style="white-space:pre-wrap"> </span>EVP_DigestUpdate(mdctx,
message_data, bytes_read);</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> EVP_DigestFinal_ex(mdctx,
hash_data.md_value, &hash_data.md_len);</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("\n%d\n",EVP_MD_CTX_size(mdctx));</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("\n%d\n",EVP_MD_CTX_type(mdctx));</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> hash_data.md_type=EVP_MD_CTX_type(mdctx);</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> EVP_MD_CTX_destroy(mdctx);</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> //fclose(fp);</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("Generated
Digest is:\n ");</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> for(i
= 0; i < hash_data.md_len; i++)</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span>
printf("%02x", hash_data.md_value[i]);</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("\n");</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> EVP_cleanup();</b></div>
<div style="font-weight:400"><b> return hash_data;</b></div>
<div style="font-weight:400"><b><br>
</b></div>
<div style="font-weight:400">
<div>In the the code below, I have done fclose(fp)</div>
<div><b>verify_hash=ComputeHash(fp,size1);</b></div>
<div><b>fclose(fp);</b></div>
</div>
<div style="font-weight:400"><b><br>
</b></div>
<div style="font-weight:400">I believe that instead of
loading the entire file all at once I am reading the
44MB file in chunks and computing the hash using the
piece of code below: (fph is the file pointer)</div>
<div style="font-weight:400"><b>while ((bytes_read = fread
(message_data, 1, BUFFER_SIZE, fph)) != 0)</b></div>
<div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> <span style="white-space:pre-wrap"> </span>EVP_DigestUpdate(mdctx,
message_data, bytes_read);</b></div>
<div style="font-weight:400"><b><br>
</b></div>
<div style="font-weight:400">Where I am going wrong? How
can I free the buff/cache after computation of message
digest? Please suggest ways to tackle this.</div>
<div style="font-weight:400"><br>
</div>
<div style="font-weight:400"><br>
</div>
<div style="font-weight:400">Thanks and Regards,</div>
<div style="font-weight:400">Prithiraj</div>
</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
<p><br>
</p>
<pre class="gmail-m_-3919589885526903920moz-signature" cols="72">--
Jordan Brown, Oracle ZFS Storage Appliance, Oracle Solaris</pre>
</div>
</blockquote></div></div>