[openssl-commits] [openssl] master update
Richard Levitte
levitte at openssl.org
Mon May 14 15:15:41 UTC 2018
The branch master has been updated
via 8481434439e421d513350a81a01ba2e2a8e61b6e (commit)
via 491c35324c9f4392ec3d59bb260bf9b7e588f881 (commit)
from de03cc92d1f3758ad1a525893c93e16b15e6cf45 (commit)
- Log -----------------------------------------------------------------
commit 8481434439e421d513350a81a01ba2e2a8e61b6e
Author: Richard Levitte <levitte at openssl.org>
Date: Sun May 13 11:35:14 2018 +0200
Link in passphrase-encoding(7) in relevant documentation
Reviewed-by: Andy Polyakov <appro at openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6179)
commit 491c35324c9f4392ec3d59bb260bf9b7e588f881
Author: Richard Levitte <levitte at openssl.org>
Date: Sat May 5 00:06:07 2018 +0200
Docs: add general document on how pass phrases are handled
Reviewed-by: Andy Polyakov <appro at openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6179)
-----------------------------------------------------------------------
Summary of changes:
doc/man1/openssl.pod | 3 +
doc/man3/OSSL_STORE_open.pod | 11 ++-
doc/man3/PEM_bytes_read_bio.pod | 7 +-
doc/man3/PEM_read.pod | 7 +-
doc/man3/PEM_read_CMS.pod | 9 +-
doc/man3/PEM_read_bio_PrivateKey.pod | 7 +-
doc/man3/PKCS12_create.pod | 7 +-
doc/man3/PKCS12_newpass.pod | 3 +-
doc/man3/PKCS12_parse.pod | 3 +-
doc/man3/PKCS5_PBKDF2_HMAC.pod | 6 +-
doc/man3/d2i_PKCS8PrivateKey_bio.pod | 7 +-
doc/man7/passphrase-encoding.pod | 182 +++++++++++++++++++++++++++++++++++
12 files changed, 242 insertions(+), 10 deletions(-)
create mode 100644 doc/man7/passphrase-encoding.pod
diff --git a/doc/man1/openssl.pod b/doc/man1/openssl.pod
index 03c07c0..a713269 100644
--- a/doc/man1/openssl.pod
+++ b/doc/man1/openssl.pod
@@ -439,6 +439,9 @@ password argument is given and a password is required then the user is
prompted to enter one: this will typically be read from the current
terminal with echoing turned off.
+Note that character encoding may be relevant, please see
+L<passphrase-encoding(7)>.
+
=over 4
=item B<pass:password>
diff --git a/doc/man3/OSSL_STORE_open.pod b/doc/man3/OSSL_STORE_open.pod
index 13f3722..b1467f4 100644
--- a/doc/man3/OSSL_STORE_open.pod
+++ b/doc/man3/OSSL_STORE_open.pod
@@ -112,6 +112,14 @@ URI, or if it's a different error (such as memory allocation
failures); if the URI was parsable but the scheme unregistered, the
top error will have the reason C<OSSL_STORE_R_UNREGISTERED_SCHEME>.
+These functions make no direct assumption regarding the pass phrase received
+from the password callback.
+The loaders may make assumptions, however.
+For example, the B<file:> scheme loader inherits the assumptions made by
+OpenSSL functionality that handles the different file types; this is mostly
+relevant for PKCS#12 objects.
+See L<passphrase-encoding(7)> for further information.
+
=head1 RETURN VALUES
OSSL_STORE_open() returns a pointer to a B<OSSL_STORE_CTX> on success, or
@@ -132,7 +140,8 @@ OSSL_STORE_ctrl() and OSSL_STORE_close() returns 1 on success, or 0 on failure.
=head1 SEE ALSO
-L<ossl_store(7)>, L<OSSL_STORE_INFO(3)>, L<OSSL_STORE_register_loader(3)>
+L<ossl_store(7)>, L<OSSL_STORE_INFO(3)>, L<OSSL_STORE_register_loader(3)>,
+L<passphrase-encoding(7)>
=head1 HISTORY
diff --git a/doc/man3/PEM_bytes_read_bio.pod b/doc/man3/PEM_bytes_read_bio.pod
index d16ccd8..cd05582 100644
--- a/doc/man3/PEM_bytes_read_bio.pod
+++ b/doc/man3/PEM_bytes_read_bio.pod
@@ -55,6 +55,10 @@ use of BIO_s_file() indicates the use of the operating system stdio
functionality, which includes buffering as a feature; BIO_s_fd() is likely
to be more appropriate in such cases.
+These functions make no assumption regarding the pass phrase received from the
+password callback.
+It will simply be treated as a byte sequence.
+
=head1 RETURN VALUES
PEM_bytes_read_bio() and PEM_bytes_read_bio_secmem() return 1 for success or
@@ -63,7 +67,8 @@ PEM_bytes_read_bio() and PEM_bytes_read_bio_secmem() return 1 for success or
=head1 SEE ALSO
L<PEM(3)>,
-L<PEM_read_bio_ex(3)>
+L<PEM_read_bio_ex(3)>,
+L<passphrase-encoding(7)>
=head1 HISTORY
diff --git a/doc/man3/PEM_read.pod b/doc/man3/PEM_read.pod
index 66cbc7d..2a017c6 100644
--- a/doc/man3/PEM_read.pod
+++ b/doc/man3/PEM_read.pod
@@ -110,10 +110,15 @@ Instead, private keys should be stored in PKCS#8 form, with a strong PKCS#5
v2.0 PBE.
See L<PEM_write_PrivateKey(3)> and L<d2i_PKCS8PrivateKey_bio(3)>.
+PEM_do_header() makes no assumption regarding the pass phrase received from the
+password callback.
+It will simply be treated as a byte sequence.
+
=head1 SEE ALSO
L<ERR_peek_last_error(3)>, L<ERR_GET_LIB(3)>,
-L<d2i_PKCS8PrivateKey_bio(3)>.
+L<d2i_PKCS8PrivateKey_bio(3)>,
+L<passphrase-encoding(7)>
=head1 COPYRIGHT
diff --git a/doc/man3/PEM_read_CMS.pod b/doc/man3/PEM_read_CMS.pod
index 65a114d..0c22618 100644
--- a/doc/man3/PEM_read_CMS.pod
+++ b/doc/man3/PEM_read_CMS.pod
@@ -73,6 +73,12 @@ PEM_write_TYPE() writes the PEM encoding of the object B<a> to the file B<fp>.
PEM_write_bio_TYPE() similarly writes to the BIO B<bp>.
+=head1 NOTES
+
+These functions make no assumption regarding the pass phrase received from the
+password callback.
+It will simply be treated as a byte sequence.
+
=head1 RETURN VALUES
PEM_read_TYPE() and PEM_read_bio_TYPE() return a pointer to an allocated
@@ -83,7 +89,8 @@ or zero on error.
=head1 SEE ALSO
-L<PEM_read(3)>
+L<PEM_read(3)>,
+L<passphrase-encoding(7)>
=head1 COPYRIGHT
diff --git a/doc/man3/PEM_read_bio_PrivateKey.pod b/doc/man3/PEM_read_bio_PrivateKey.pod
index 4ff5b2d..9f62140 100644
--- a/doc/man3/PEM_read_bio_PrivateKey.pod
+++ b/doc/man3/PEM_read_bio_PrivateKey.pod
@@ -386,6 +386,10 @@ this:
this is a bug because an attempt will be made to reuse the data at B<x>
which is an uninitialised pointer.
+These functions make no assumption regarding the pass phrase received from the
+password callback.
+It will simply be treated as a byte sequence.
+
=head1 PEM ENCRYPTION FORMAT
These old B<PrivateKey> routines use a non standard technique for encryption.
@@ -465,7 +469,8 @@ as they will be formally deprecated in a future releases.
=head1 SEE ALSO
-L<EVP_EncryptInit(3)>, L<EVP_BytesToKey(3)>
+L<EVP_EncryptInit(3)>, L<EVP_BytesToKey(3)>,
+L<passphrase-encoding(7)>
=head1 COPYRIGHT
diff --git a/doc/man3/PKCS12_create.pod b/doc/man3/PKCS12_create.pod
index 37b53be..0dcd2ab 100644
--- a/doc/man3/PKCS12_create.pod
+++ b/doc/man3/PKCS12_create.pod
@@ -60,13 +60,18 @@ should be used.
B<mac_iter> can be set to -1 and the MAC will then be omitted entirely.
+PKCS12_create() makes assumptions regarding the encoding of the given pass
+phrase.
+See L<passphrase-encoding(7)> for more information.
+
=head1 RETURN VALUES
PKCS12_create() returns a valid B<PKCS12> structure or NULL if an error occurred.
=head1 SEE ALSO
-L<d2i_PKCS12(3)>
+L<d2i_PKCS12(3)>,
+L<passphrase-encoding(7)>
=head1 COPYRIGHT
diff --git a/doc/man3/PKCS12_newpass.pod b/doc/man3/PKCS12_newpass.pod
index 58207f5..c27c652 100644
--- a/doc/man3/PKCS12_newpass.pod
+++ b/doc/man3/PKCS12_newpass.pod
@@ -102,7 +102,8 @@ this function.
=head1 SEE ALSO
-L<PKCS12_create(3)>, L<ERR_get_error(3)>
+L<PKCS12_create(3)>, L<ERR_get_error(3)>,
+L<passphrase-encoding(7)>
=head1 COPYRIGHT
diff --git a/doc/man3/PKCS12_parse.pod b/doc/man3/PKCS12_parse.pod
index c03c371..d61b172 100644
--- a/doc/man3/PKCS12_parse.pod
+++ b/doc/man3/PKCS12_parse.pod
@@ -57,7 +57,8 @@ Attributes currently cannot be stored in the private key B<EVP_PKEY> structure.
=head1 SEE ALSO
-L<d2i_PKCS12(3)>
+L<d2i_PKCS12(3)>,
+L<passphrase-encoding(7)>
=head1 COPYRIGHT
diff --git a/doc/man3/PKCS5_PBKDF2_HMAC.pod b/doc/man3/PKCS5_PBKDF2_HMAC.pod
index e5d1689..174831d 100644
--- a/doc/man3/PKCS5_PBKDF2_HMAC.pod
+++ b/doc/man3/PKCS5_PBKDF2_HMAC.pod
@@ -52,6 +52,9 @@ Increasing the B<iter> parameter slows down the algorithm which makes it
harder for an attacker to perform a brute force attack using a large number
of candidate passwords.
+These functions make no assumption regarding the given password.
+It will simply be treated as a byte sequence.
+
=head1 RETURN VALUES
PKCS5_PBKDF2_HMAC() and PBKCS5_PBKDF2_HMAC_SHA1() return 1 on success or 0 on error.
@@ -59,7 +62,8 @@ PKCS5_PBKDF2_HMAC() and PBKCS5_PBKDF2_HMAC_SHA1() return 1 on success or 0 on er
=head1 SEE ALSO
L<evp(7)>, L<RAND_bytes(3)>,
-L<EVP_BytesToKey(3)>
+L<EVP_BytesToKey(3)>,
+L<passphrase-encoding(7)>
=head1 COPYRIGHT
diff --git a/doc/man3/d2i_PKCS8PrivateKey_bio.pod b/doc/man3/d2i_PKCS8PrivateKey_bio.pod
index cc588df..43a218a 100644
--- a/doc/man3/d2i_PKCS8PrivateKey_bio.pod
+++ b/doc/man3/d2i_PKCS8PrivateKey_bio.pod
@@ -45,6 +45,10 @@ Currently all the functions use BIOs or FILE pointers, there are no functions wh
work directly on memory: this can be readily worked around by converting the buffers
to memory BIOs, see L<BIO_s_mem(3)> for details.
+These functions make no assumption regarding the pass phrase received from the
+password callback.
+It will simply be treated as a byte sequence.
+
=head1 RETURN VALUES
d2i_PKCS8PrivateKey_bio() and d2i_PKCS8PrivateKey_fp() return a valid B<EVP_PKEY>
@@ -55,7 +59,8 @@ and i2d_PKCS8PrivateKey_nid_fp() return 1 on success or 0 on error.
=head1 SEE ALSO
-L<PEM_read_PrivateKey(3)>
+L<PEM_read_PrivateKey(3)>,
+L<passphrase-encoding(7)>
=head1 COPYRIGHT
diff --git a/doc/man7/passphrase-encoding.pod b/doc/man7/passphrase-encoding.pod
new file mode 100644
index 0000000..bc777c9
--- /dev/null
+++ b/doc/man7/passphrase-encoding.pod
@@ -0,0 +1,182 @@
+=pod
+
+=encoding utf8
+
+=head1 NAME
+
+password encoding
+- How diverse parts of OpenSSL treat pass phrases character encoding
+
+=head1 DESCRIPTION
+
+In a modern world with all sorts of character encodings, the treatment of pass
+phrases has become increasingly complex.
+This manual page attempts to give an overview over how this problem is
+currently addressed in different parts of the OpenSSL library.
+
+=head2 The general case
+
+The OpenSSL library doesn't treat pass phrases in any special way as a general
+rule, and trusts the application or user to choose a suitable character set
+and stick to that throughout the lifetime of affected objects.
+This means that for an object that was encrypted using a pass phrase encoded in
+ISO-8859-1, that object needs to be decrypted using a pass phrase encoded in
+ISO-8859-1.
+Using the wrong encoding is expected to cause a decryption failure.
+
+=head2 PKCS#12
+
+PKCS#12 is a bit different regarding pass phrase encoding.
+The standard stipulates that the pass phrase shall be encoded as an ASN.1
+BMPString, which consists of the code points of the basic multilingual plane,
+encoded in big endian (UCS-2 BE).
+
+OpenSSL tries to adapt to this requirements in one of the following manners:
+
+=over 4
+
+=item 1.
+
+Treats the received pass phrase as UTF-8 encoded and tries to re-encode it to
+UTF-16 (which is the same as UCS-2 for characters U+0000 to U+D7FF and U+E000
+to U+FFFF, but becomes an expansion for any other character), or failing that,
+proceeds with step 2.
+
+=item 2.
+
+Assumes that the pass phrase is encoded in ASCII or ISO-8859-1 and
+opportunistically prepends each byte with a zero byte to obtain the UCS-2
+encoding of the characters, which it stores as a BMPString.
+
+Note that since there is no check of your locale, this may produce UCS-2 /
+UTF-16 characters that do not correspond to the original pass phrase characters
+for other character sets, such as any ISO-8859-X encoding other than
+ISO-8859-1 (or for Windows, CP 1252 with exception for the extra "graphical"
+characters in the 0x80-0x9F range).
+
+=back
+
+OpenSSL versions older than 1.1.0 do variant 2 only, and that is the reason why
+OpenSSL still does this, to be able to read files produced with older versions.
+
+It should be noted that this approach isn't entirely fault free.
+
+A passphrase encoded in ISO-8859-2 could very well have a sequence such as
+0xC3 0xAF (which is the two characters "LATIN CAPITAL LETTER A WITH BREVE"
+and "LATIN CAPITAL LETTER Z WITH DOT ABOVE" in ISO-8859-2 encoding), but would
+be misinterpreted as the perfectly valid UTF-8 encoded code point U+00EF (LATIN
+SMALL LETTER I WITH DIARESIS) I<if the passphrase doesn't contain anything that
+would be invalid UTF-8>.
+A pass phrase that contains this kind of byte sequence will give a different
+outcome in OpenSSL 1.1.0 and newer than in OpenSSL older than 1.1.0.
+
+ 0x00 0xC3 0x00 0xAF # OpenSSL older than 1.1.0
+ 0x00 0xEF # OpenSSL 1.1.0 and newer
+
+On the same accord, anything encoded in UTF-8 that was given to OpenSSL older
+than 1.1.0 was misinterpreted as ISO-8859-1 sequences.
+
+=head2 OSSL_STORE
+
+L<ossl_store(7)> acts as a general interface to access all kinds of objects,
+potentially protected with a pass phrase, a PIN or something else.
+This API currently doesn't stipulate any specific encoding of pass phrases, but
+uses the underlying routines with their behaviours.
+This means that when using the built-in C<file:> scheme loader, the pass phrase
+to unlock a PKCS#12 file will be treated as described for PKCS#12 above, and
+the pass phrase for a PEM files will be treated as the general case described
+above, since that loader uses the same underlying routines.
+I<Note that other loaders will have their own behaviours>.
+
+=head1 RECOMMENDATIONS
+
+This section assumes that you know what pass phrase was used for encryption,
+but that it may have been encoded in a different character encoding than the
+one used by your current input method.
+For example, the pass phrase may have been used at a time when your default
+encoding was ISO-8859-1 (i.e. "naïve" resulting in the byte sequence 0x6E 0x61
+0xEF 0x76 0x65), and you're now in an environment where your default encoding
+is UTF-8 (i.e. "naïve" resulting in the byte sequence 0x6E 0x61 0xC3 0xAF 0x76
+0x65).
+Whenever it's mentioned that you should use a certain character encoding, it
+should be understood that you either change the input method to use the
+mentioned encoding when you type in your pass phrase, or use some suitable tool
+to convert your pass phrase from your default encoding to the target encoding.
+
+Also note that the sub-sections below discuss human readable pass phrases.
+This is particularly relevant for PKCS#12 objects, where human readable pass
+phrases are assumed.
+For other objects, it's as legitimate to use any byte sequence (such as a
+sequence of bytes from `/dev/urandom` that's been saved away), which makes any
+character encoding discussion irrelevant; in such cases, simply use the same
+byte sequence as it is.
+
+=head2 Creating new objects
+
+For creating new pass phrase protected objects, make sure the pass phrase is
+encoded using UTF-8.
+This is default on most modern Unixes, but may involve an effort on other
+platforms.
+Specifically for Windows, setting the environment variable
+C<OPENSSL_WIN32_UTF8> will have anything entered on [Windows] console prompt
+converted to UTF-8 (command line and separately prompted pass phrases alike).
+
+=head2 Opening existing objects
+
+For opening pass phrase protected objects where you know what character
+encoding was used for the encryption pass phrase, make sure to use the same
+encoding again.
+
+For opening pass phrase protected objects where the character encoding that was
+used is unknown, or where the producing application is unknown, try one of the
+following:
+
+=over 4
+
+=item 1.
+
+Try the password that you have as it is in the character encoding of your
+environment.
+It's possible that its byte sequence is exactly right.
+
+=item 2.
+
+Convert the pass phrase to UTF-8 and try with the result.
+Specifically with PKCS#12, this should open up any object that was created
+according to the specification.
+
+=item 3.
+
+Do a naïve (i.e. purely mathematical) ISO-8859-1 to UTF-8 conversion and try
+with the result.
+This differs from the previous attempt because ISO-8859-1 maps directly to
+U+0000 to U+00FF, which other non-UTF-8 character sets do not.
+
+This also takes care of the case when a UTF-8 encoded string was used with
+OpenSSL older than 1.1.0.
+(for example, C<ï>, which is 0xC3 0xAF when encoded in UTF-8, would become 0xC3
+0x83 0xC2 0xAF when re-encoded in the naïve manner.
+The conversion to BMPString would then yield 0x00 0xC3 0x00 0xA4 0x00 0x00, the
+erroneous/non-compliant encoding used by OpenSSL older than 1.1.0)
+
+=back
+
+=head1 SEE ALSO
+
+L<evp(7)>,
+L<ossl_store(7)>,
+L<EVP_BytesToKey(3)>, L<EVP_DecryptInit(3)>,
+L<PEM_do_header(3)>,
+L<PKCS12_parse(3)>, L<PKCS12_newpass(3)>,
+L<d2i_PKCS8PrivateKey_bio(3)>
+
+=head1 COPYRIGHT
+
+Copyright 2018 The OpenSSL Project Authors. All Rights Reserved.
+
+Licensed under the OpenSSL license (the "License"). You may not use
+this file except in compliance with the License. You can obtain a copy
+in the file LICENSE in the source distribution or at
+L<https://www.openssl.org/source/license.html>.
+
+=cut
More information about the openssl-commits
mailing list