[openssl-dev] [PATCH] Support broken PKCS#12 key generation.

[Andy Polyakov]

First of all. *Everything* that is said below (and what modifications in
question are about) applies to non-ASCII passwords. ASCII-only passwords
are not affected by this and keep working as they were.

>> commit 647ac8d3d7143e3721d55e1f57730b6f26e72fc9
>>
>> OpenSSL versions before 1.1.0 didn't convert non-ASCII
>> UTF8 PKCS#12 passwords to Unicode correctly.
>>
>> To correctly decrypt older files, if MAC verification fails
>> with the supplied password attempt to use the broken format
>> which is compatible with earlier versions of OpenSSL.
>>
>> Reviewed-by: Richard Levitte <levitte at openssl.org>
> 
> Hm, this sounds like something that other crypto libraries also ought
> to try to work around. 
> 
> Please could you elaborate on the specific problem, and/or show a test
> case?

Note that there is 80-test_pkcs12.t that refers to shibboleth.pfx.

> I'm not quite sure how to interpret the patch itself. You pass the
> password through OPENSSL_asc2uni() and then OPENSSL_uni2utf8() — which
> is essentially converting ISO8859-1 to UTF-8.

You make it sound like these two are called one after another. But
that's not what happens. It *tries* to access data with passwords
converted with OPENSSL_asc2uni *or* OPENSSL_utf82uni, effectively
independently of each other, in order to see which one is right. So that
you can *transparently* access old broken data, as well as
specification-compliant one.

> So, if my password is "naïve". In UTF-8 that's 6e 61 c3 af 76 65, which
> is the correct sequence of bytes to use for the password?

00 6e 00 61 00 ef 00 76 00 65, big-endian UTF-16.

> And you now convert that sequence of bytes to 6e 61 c3 83 c2 af 76 65
> by assuming it's ISO8859-1 (which would be 'naïve') and converting to UTF-8?

I don't follow. Why would it have to be converted to this sequence?

> So... what was the bug that was actually being worked around?

6e 61 c3 af 76 65 was expanded to 00 6e 00 61 00 c3 00 af 00 76 00 65,
in violation of specification.

> That
> older versions were converting from the local charset to UTF-8 twice in
> a row? So you've implemented a "convert ISO8859-1 to UTF-8" fallback
> which will cope with that in *some* locales but not all...?

Yeah, something like that. Note that if you have created PKCS#12 file
with a non-UTF8 locale, it's not given that you can read it with another
locale, UTF-8 or not. This was *always* the case. And that's *not* what
we try to address here. We assume that you don't change locale when you
upgrade OpenSSL version. Ultimate goal is to make it compliant with
specification and therefore interoperable with other compliant
implementations. But we can't just switch, because then it stops
interoperate with older OpenSSL versions. This is the reason for why it
looks messy. It's about having it both ways... Though there is one
ambiguity in this. Said interoperability assumes UTF-8 locale (on *ix,
or OPENSSL_WIN32_UTF8 opt-in environment variable on Windows). I.e. it's
not given that users that use non-UTF8 locale can actually interoperate
with other implementations. It's assumed that such users are not
actually interested to, and if they express interest, they will be
advised to switch to UTF-8 locale and convert their data to
interoperable format.

Is this helpful?