[openssl-users] openssl cms -decrypt failing due to malloc(3) failure

Michael Wojcik Michael.Wojcik at microfocus.com
Wed Aug 1 12:49:34 UTC 2018

> From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf
> Of Christian Böhme
> Sent: Tuesday, July 31, 2018 10:16
> On 30.07.2018 20:12, Michael Wojcik wrote:
> > FWIW, SUS Issue 5 defines RLIMIT_AS as applying to both malloc and mmap, but RLIMIT_DATA as
> > applying only to malloc. (That is, mmap'd pages do not count against the data limit.)
> mmap() , by defninition, populates the process' virtual address space, and if  that
> is limited in size, artificially or not, then the call is going to fail, eventually.

That's irrelevant to the statement you quoted, which was about the SUS process-limit mechanism (setrusage et al.), not the process address space.

> > Agreed. And I'm not endorsing the mmap approach for this problem anyway - I'd use a streaming
> > approach, so I'm not limited by address space.
> This structure, if held in a regular file, can be processed quite non-linearly,
> and without  mmap()'ing  its entire contents.

Indeed. I still don't see any compelling reason to mmap it at all.

> The pure streaming approach may be appropriate for file descriptors that are not
> seekable like sockets, pipes, tty ends etcpp., whereas with egular files, random
> access schemes using either  pread(v)(2)  or  lseek(2)  in combination with
> read(v)(2)  can be employed.

Or regular files could also be processed sequentially. What's the advantage of making seekable sources a special case?

In any case, the OpenSSL apps are a convenience and a set of samples. You can always write your own version of the cms app.

Michael Wojcik
Distinguished Engineer, Micro Focus

More information about the openssl-users mailing list