[openssl-users] openssl cms -decrypt failing due to malloc(3) failure

[Michael Wojcik]

> From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of Jordan Brown
> Sent: Monday, July 30, 2018 10:46

> I have never heard of a malloc implementation that has artificial limits;

Er... setrlimit(RLIMIT_DATA). For OSes that claim Single UNIX Specification compliance.

>if it's failing it's because it can't find that much contiguous virtual address space, and mmap won't be able to
> find it either.

FWIW, SUS Issue 5 defines RLIMIT_AS as applying to both malloc and mmap, but RLIMIT_DATA as applying only to malloc. (That is, mmap'd pages do not count against the data limit.)

> If you're a 32-bit process, then malloc'ing or mmap'ing a 2GB object will be difficult at best.

Agreed. And I'm not endorsing the mmap approach for this problem anyway - I'd use a streaming approach, so I'm not limited by address space.

--
Michael Wojcik
Distinguished Engineer, Micro Focus