[openssl-dev] [PATCH] Support broken PKCS#12 key generation.

[Andy Polyakov]

> Hm, words fail me.
> 
> Well, that's not entirely true. But *polite* words fail me...

:-)

> Let me try to understand this... you have always ignored, and still
> ignore, the actual LC_CTYPE which tells you the character set in which
> the password was provided from the user.
> 
> You *used* to convert it through your horribly misnamed
> OPENSSL_asc2uni() function, which actually converts ISO8859-1 to
> UTF16BE by simply expanding it and inserting 0x00 between each byte of
> the original. So effectively you were "assuming" the input was
> ISO8859-1.

Unfortunately yes.

> Now you assume it's UTF-8, and convert it with OPENSSL_utf8touni().
> (And now what happens when the locale input *isn't* valid UTF-8,
> because it's a legacy 8-bit charset? That conversion should fail,
> right?)

Right. Though more correct formulation probably "is likely to fail"
instead of "should fail".

> Your latest workaround (from which I started this thread) is addressing
> the first problem *purely* for the case of the UTF-8 locale. It checks
> for the "we treated UTF-8 as if it were ISO8859-1" case, which is what
> leads to the 006e 0061 00c3 00af 0076 0065 example you gave above.

Yes.

> But you *haven't* actually implemented any workaround for the other
> whole set of "we treated locale XXXXX as if it were ISO8859-1" bugs
> that your original code had. Or the whole set of "we treated local
> XXXXX as if it were UTF-8" bugs that the new code has.

Yes.

> So for other applications to try to read OpenSSL's PKCs#12 files, what
> we need to do is first convert the sane Unicode rendition of the
> password into the native locale charset (e.g. Windows-1252), then take
> that bytestream and *pretend* it's ISO8859-1, and convert to UTF-16BE
> to check the MAC. Then if that fails, take the same Windows-1252
> bytestream and *pretend* it's UTF-8, and convert *that* to UTF-16BE to
> see if it works.

Are you sure you want to know? I mean you already said "I wish I hadn't
ask", and now you are bringing Windows into conversation :-) :-) :-)
Yes, it's as bad as you can possibly imagine, and trouble is that there
is no "right thing" to do *if* you formulate "keep old data accessible"
as goal. I mean the right thing to do is to perform all due conversions
to obtain locale-neutral big-endian UTF-16 (though it's not obvious if
it's actually desired to bring in dependency on something like iconv
into libcrypto), but trouble is that it will more than likely render old
data inaccessible. That's why somewhat opportunistic approach is taken
in this version, unfortunate as it is. Main goal is that given otherwise
unchanged circumstances new version *has to* be able to read old data
generated by previous version on the *same* system, irregardless how
broken it is. At the same time a way to access and generate
specification-compliant data is provided (on *ix it takes *an* UTF8
locale, which is default in absolute majority of cases bu now, and on
Windows it's an *opt-in* environment variable for the time being).