[openssl-dev] [openssl.org #4094] Nonsensical pointer comparison in PACKET_buf_init

[Kaduk, Ben via RT]

On 10/16/2015 03:32 AM, Matt Caswell via RT wrote:
>
> On 15/10/15 20:53, Alexander Cherepanov via RT wrote:
>> What was not entirely clear from the original bug report is that, while 
>> the check is not compiled away, it's compiled into something completely 
>> different from what is written in the source. Specifically, the check 
>> "buf + len < buf" is optimized into "len >> 63" on 64-bit platform, i.e. 
>> "(ssize_t)len < 0" or "len > SIZE_MAX / 2". This is not a check for 
>> overflow at all, it doesn't even depend on the value of "buf".
>>
>> If this is what was intended then it's better to write it explicitly. If 
>> this is not what was intended then some other approach is required.
> I'd say that is an instance of the compiler knowing better than us how
> big |len| would have to be in order to trigger an overflow. Those rules
> are going to be platform specific so we should not attempt to second
> guess them, but instead let the optimiser do its job.
>

I hope I am not dragging this thread on too long, but with all due
respect, we are not asking the compiler/optimizer to detect overflow --
we are asking the compiler to instantiate undefined behavior in a way
that is convenient for us.  This will only happen by chance, as a side
effect of some other decisions made by the compiler authors, in the
present state of compiler development.

-Ben

P.S. If you haven't encountered it yet,
http://blog.regehr.org/archives/213 et. seq. make for fun reading.