[openssl-users] Max size on ASN1_item_d2i_bio()?

[Dave Thompson]

> From: openssl-users On Behalf Of Dr. Stephen Henson
> Sent: Friday, February 20, 2015 17:24

> On Fri, Feb 20, 2015, Nathaniel McCallum wrote:
> 
> > I'd like to use ASN1_item_d2i_bio() (or something similar) to parse an
> > incoming message. However, given that types like ASN1_OCTET_STRING
> > have (essentially) unbounded length, how do I prevent an attacker from
> > DOS'ing via OOM?
> >
> > Is there some way to set a max packet size?
> >
> 
> No there isn't but if the input is in DER form you can peek the first few
> bytes and get the tag+length fields to determine the size of the
structure. If
> the input uses indefinite length encoding that isn't possible however.
> 
Some other possibilities:

If the bio is memBIO or fileBIO its input size is known before you start,
at least if it contains only one root item. More generally you could layer 
a simple filter BIO that limits total reads to a chosen amount like 1M, 
probably measured from a CTRL operation  -- or a more complex one 
that looks dynamically at your memory-used and/or memory-available 
and chooses whether/when to force EOF, but that would be dependent 
on your particular platform and not portable.

Alternatively or in addition, OpenSSL allows you to provide your own 
malloc/realloc/free implementations used instead of the standard ones. 
But these are used for *all* OpenSSL heap allocations, so you might need 
some care to count the space used "for" or at least during a d2i 
as opposed to other purposes and times.