[openssl-dev] [openssl.org #4552] Bug report: hex string is too long, problem in set_hex()?

[Richard Levitte via RT]

That hex key string looks off. It seems to include an ending \n (0a), which I
suspect is because at an earlier time, someone forgot to peal off the ending
linefeed. Take away the endine 0a and I'm sure things will be fine.

The 'set_hex' check is exactly the same in the 1.0.1, 1.0.2 and upcoming 1.1.0
series, so I'm pretty sure *that* part is a red herring.

Why 1.0.1 accepts a too long hex string is a good question, but it's too late
in its life cycle to bother.

Cheers,
Richard

On Tue May 31 20:37:19 2016, david-e.young at db.com wrote:
> Classification: Public
> OS: SUSE Linux Enterprise Server 11 SP2 (x86_64)
> OpenSSL: versions 1.0.1m, 1.1.0-pre5
>
> Using this command sequence:
>
> echo WuNhPwuWAOiG86RfO4A5jITR9WZ+kF1L+iBgGPQJ4dEJk8Sxiqb014bJsEGDbCfk
> | $ssl/bin/openssl enc -aes128 -d -a -iv
> 57fd56a7e47b9482096ab4707ca9d383 -K 617364696f69636a61736f3932336b3b0a
>
> we get different behavior between version 1.0.1m and later versions.
> With version 1.0.1m the above sequence works fine. Using later
> versions, including 1.1.0-pre5, we get:
>
> hex string is too long
> invalid hex key value
>
> The trouble is in set_hex():
>
> static int set_hex(char *in, unsigned char *out, int size)
> {
> ...
> if (n > (size * 2)) {
> // hex string is too long
> ...
> }
> }
>
> If the size multiplier is changed to, say, 4, then the problem goes
> away with no apparent ill effects. Reading the code for set_hex() and
> its caller, it does not appear that the size multiplier is related to
> a buffer size or some other limitation.
>
> I've read that the "hex string is too long" issue is a bug in OpenSSL
> that was supposed to be fixed in the 1.0.2 branch, but that appears
> not to be the case. Please let us know if we've missed something or if
> this issue is in fact an OpenSSL bug.
>
> Kind Regards,
>
> David
>
>
>
> ---
> This communication may contain confidential and/or privileged
> information. If you are not the intended recipient (or have received
> this communication in error) please notify the sender immediately and
> destroy this communication. Any unauthorized copying, disclosure or
> distribution of the material in this communication is strictly
> forbidden.
>
> Deutsche Bank does not render legal or tax advice, and the information
> contained in this communication should not be regarded as such.


--
Richard Levitte
levitte at openssl.org

-- 
Ticket here: http://rt.openssl.org/Ticket/Display.html?id=4552
Please log in as guest with password guest if prompted