[openssl-users] openssl cms -decrypt failing due to malloc(3) failure

[Viktor Dukhovni]

> On Jul 30, 2018, at 12:46 PM, Jordan Brown <openssl at jordan.maileater.net> wrote:
> 
> If you can't malloc the space, you probably can't mmap it either.  I have never heard of a malloc implementation that has artificial limits; if it's failing it's because it can't find that much contiguous virtual address space, and mmap won't be able to find it either.
> 
> If you're a 32-bit process, then malloc'ing or mmap'ing a 2GB object will be difficult at best.

Getting out of the weeds, the core issue is that CMS message input processing
doesn't stream.  The entire CMS message has to fit into memory.  A different
data format is required for streaming large payloads.  The data would need
to be chunked with integrity protection and protection applied to each
chunk (packet) and appropriate sequence number integrity in place to
prevent reordering, insertion or deletion of chunks.

CMS works fine for small messages, and could even be used to construct
the integrity-protected chunks in a higher-level protocol.  CMS is
not appropriate for multi-gigabyte or terabyte, ... datasets.

-- 
	Viktor.