<div dir="ltr"><div dir="ltr"><div dir="ltr">Hi,<div>This is how I have initialized my variables:-</div><div><br></div><div>EVP_MD_CTX *mdctx;</div><div>const EVP_MD *md;</div><div>int i;</div><div>HASH hash_data;</div><div>unsigned char message_data[BUFFER_SIZE];</div><div><br></div><div>BUFFER_SIZE has been defined as 131072</div><div>and HASH is my hash structure defined to hold the message digest, message digest length and message digest type</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, 23 Feb 2019 at 00:17, Jordan Brown <<a href="mailto:openssl@jordan.maileater.net">openssl@jordan.maileater.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF">
    <div class="gmail-m_-3919589885526903920moz-cite-prefix">The most obvious question is "how are
      you allocating your message_data buffer?".  You don't show that.</div>
    <div class="gmail-m_-3919589885526903920moz-cite-prefix"><br>
    </div>
    <div class="gmail-m_-3919589885526903920moz-cite-prefix">On 2/22/2019 2:27 AM, prithiraj das
      wrote:<br>
    </div>
    <blockquote type="cite">
      
      <div dir="ltr">
        <div dir="ltr">
          <div dir="ltr"><br>
            <div>
              <div>Hi All,
                <div><br>
                </div>
                <div>Using OpenSSL 1.0.2g, I have written a code to
                  generate the hash of a file in an embeddded device
                  having linux OS and low memory capacity and the files
                  are generally of size 44 MB or more. The first time or
                  even the second time on some occasions, the hash of
                  any file is successfully generated. On the 3rd or 4th
                  time (possibly due to lack of memory/memory leak), the
                  system reboots before the hash can be generated. 
                  After restart, the same thing happens when the
                  previous steps are repeated.</div>
                <div>The stats below shows the memory usage before and
                  after computing the hash. </div>
              </div>
            </div>
            <div><br>
            </div>
            <div>
              <div><b>root@at91sam9m10g45ek:~# free</b></div>
              <div><b>                      total        used         
                  free         shared    buff/cache   available</b></div>
              <div><b>Mem:         252180       13272      223048       
                   280          15860          230924</b></div>
              <div><b>Swap:                0           0               0</b></div>
            </div>
            <div><b><br>
              </b></div>
            <div><b>After computing hash :-</b></div>
            <div>
              <div><b>root@at91sam9m10g45ek:~# free</b></div>
              <div><b>                      total        used         
                  free       shared    buff/cache   available</b></div>
              <div><b>Mem:         252180       13308      179308       
                  280       59564           230868</b></div>
              <div><b>Swap:             0                0             
                  0</b></div>
              <div style="font-weight:bold"><br>
              </div>
            </div>
            <div style="font-weight:bold">
              <div style="font-weight:400">Buff/cache increases by
                almost 44MB (same as file size) everytime I generate the
                hash and free decreases. I believe the file is being
                loaded into buffer and not being freed. <br>
              </div>
              <div style="font-weight:400"><br>
              </div>
              <div style="font-weight:400">I am using the below code to
                compute the message digest. This code is part of a
                function ComputeHash and the file pointer here is fph.</div>
              <div style="font-weight:400"><br>
              </div>
              <div style="font-weight:400">  <b> EVP_add_digest(EVP_sha256());</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> md
                  = EVP_get_digestbyname("sha256");</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> </b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> if(!md)
                  {</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> 
                        printf("Unknown message digest \n");</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> 
                        exit(1);</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap">         </span> }</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("Message
                  digest algorithm successfully loaded\n");</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> mdctx
                  = EVP_MD_CTX_create();</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> EVP_DigestInit_ex(mdctx,
                  md, NULL);</b></div>
              <div style="font-weight:400"><b><br>
                </b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> //
                  Reading data to array of unsigned chars<span style="white-space:pre-wrap">    </span></b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> long
                  long int bytes_read = 0;</b></div>
              <div style="font-weight:400"><b><br>
                </b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("FILE
                  size of the file to be hashed is %ld",filesize);<span style="white-space:pre-wrap">      </span></b></div>
              <div style="font-weight:400"><b><br>
                </b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> //reading
                  image file in chunks below and fph is the file pointer
                  to the 44MB file</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> while
                  ((bytes_read = fread (message_data, 1, BUFFER_SIZE,
                  fph)) != 0)</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> <span style="white-space:pre-wrap">      </span>EVP_DigestUpdate(mdctx,
                  message_data, bytes_read);</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> EVP_DigestFinal_ex(mdctx,
                  hash_data.md_value, &hash_data.md_len);</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("\n%d\n",EVP_MD_CTX_size(mdctx));</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("\n%d\n",EVP_MD_CTX_type(mdctx));</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> hash_data.md_type=EVP_MD_CTX_type(mdctx);</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> EVP_MD_CTX_destroy(mdctx);</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> //fclose(fp);</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("Generated
                  Digest is:\n ");</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> for(i
                  = 0; i < hash_data.md_len; i++)</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> 
                        printf("%02x", hash_data.md_value[i]);</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> printf("\n");</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> EVP_cleanup();</b></div>
              <div style="font-weight:400"><b>         return hash_data;</b></div>
              <div style="font-weight:400"><b><br>
                </b></div>
              <div style="font-weight:400">
                <div>In the the code below, I have done fclose(fp)</div>
                <div><b>verify_hash=ComputeHash(fp,size1);</b></div>
                <div><b>fclose(fp);</b></div>
              </div>
              <div style="font-weight:400"><b><br>
                </b></div>
              <div style="font-weight:400">I believe that instead of
                loading the entire file all at once I am reading the
                44MB file in chunks and computing the hash using the
                piece of code below: (fph is the file pointer)</div>
              <div style="font-weight:400"><b>while ((bytes_read = fread
                  (message_data, 1, BUFFER_SIZE, fph)) != 0)</b></div>
              <div style="font-weight:400"><b><span style="white-space:pre-wrap"> </span> <span style="white-space:pre-wrap">      </span>EVP_DigestUpdate(mdctx,
                  message_data, bytes_read);</b></div>
              <div style="font-weight:400"><b><br>
                </b></div>
              <div style="font-weight:400">Where I am going wrong? How
                can I free the buff/cache after computation of message
                digest?  Please suggest ways to tackle this.</div>
              <div style="font-weight:400"><br>
              </div>
              <div style="font-weight:400"><br>
              </div>
              <div style="font-weight:400">Thanks and Regards,</div>
              <div style="font-weight:400">Prithiraj</div>
            </div>
            <div><br>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
    <p><br>
    </p>
    <pre class="gmail-m_-3919589885526903920moz-signature" cols="72">-- 
Jordan Brown, Oracle ZFS Storage Appliance, Oracle Solaris</pre>
  </div>

</blockquote></div></div>