error: ASN1_mbstring_ncopy:illegal characters

[Viktor Dukhovni]

On Wed, Apr 12, 2023 at 03:20:51PM +1000, raf via openssl-users wrote:

> Thanks. I thought that might be the case, but I didn't
> know what kind of encoding was appropriate for openssl
> usage. There are different encodings for different
> purposes. My interest in Unicode domain names relates
> to DNS usage where IDNA2008/UTC#46 is useful. But this
> makes sense since it's an email address.
> 
> It would be great if openssl performed the necessary
> encoding, especially when it has been instructed (with
> the -utf8 option) to interperet input as UTF-8 (but the
> locale should probably be enough of an indication), and
> to also perform the corresponding decoding on output. I
> think that requiring users to perform the correct
> encoding is asking too much. But maybe expecting
> openssl to include code for encoding and decoding email
> addresses is asking too much.
> 
> I have a shell script that will need to decode
> international email addresses in S/MIME certificates,
> and then encode the domain as IDNA2008/UTC#46.

[ Doing X.509 validation in shell scripts sounds particularly unwise,
  Python, or a programming language less prone to code injection
  attacks or other serialisation/deserialisation issues would be
  a better choice, or even some robust C++ code on top of the OpenSSL
  API. ]

This isn't a question of *encoding*, rather addresses with UTF-8
localparts, or a U-label domain, are stored in a different subtype of
the SAN discriminated union.

You need to specify a SAN "otherName" of type smtpUtf8Name, rather than
an rfc822Name.  With OpenSSL 3.0, you can use "id-on-SmtpUTF8Mailbox"
instead of the numeric OID:

    [extensions]
    subjectAltName = @sans

    [sans]
    otherName.1 = 1.3.6.1.5.5.7.8.9;FORMAT:UTF8,UTF8String:потребитель@домен.example

Full support for this in certificate verification requires OpenSSL 3.0.

-- 
    Viktor.