[openssl-dev] [openssl.org #3887] PATCH: rsautl and intelligent retry for Public Key parse after Traditional/Subject Public Key Info parse fails

[Richard Levitte via RT]

Nice idea, I'm however thinking that much of the trying different formats could
be moved to load_key / load_pubkey, all that would be needed is a keyformat
denoting "try anything". -1, perhaps?

On Sun May 31 09:46:28 2015, noloader at gmail.com wrote:
> apps.c has a couple of parsing routines called load_pubkey and
> load_key. rsautl uses those routines.
>
> However, there's no option in rsautil to use anything other than a
> ASN.1/DER or PEM encoded traditional key (or subject public key info).
> The code paths are present, we just can't seem to get to them.
>
> Folks in the field have problem with it on occasion. See, for example,
> "Can't load programmatically generated public key,"
> http://stackoverflow.com/q/30547646.
>
> This patch adds an intelligent fallback:
>
> /* rsautl does not offer a way to specify just a public key. It
> requires a */
> /* subjectPublicKeyInfo, and there's no argument or option to
> switch to */
> /* the other type with either ASN.1/DER or PEM. This attempts to
> make an */
> /* intelligent retry. */
> if(keyformat == FORMAT_PEM) {
> next_format = FORMAT_PEMRSA;
> } if(keyformat == FORMAT_ASN1) {
> next_format = FORMAT_ASN1RSA;
> } else {
> next_format = -1;
> }
>
> switch (key_type) {
> case KEY_PRIVKEY:
> pkey = load_key(keyfile, keyformat, 0, passin, e, "Private
> Key", 0 /*last_try*/);
> if(!pkey && next_format != -1) { pkey = load_key(keyfile,
> next_format, 0, passin, e, "Private Key", 1 /*last_try*/); }
> break;
>
> case KEY_PUBKEY:
> pkey = load_pubkey(keyfile, keyformat, 0, NULL, e, "Public
> Key", 0 /*last_try*/);
> if(!pkey && next_format != -1) { pkey = load_pubkey(keyfile,
> next_format, 0, NULL, e, "Public Key", 1 /*last_try*/); }
> break;
>
> Then, functions load_key and load_pubkey suppress the error messages
> if last_try is 0:
>
> if (pkey == NULL && last_try) {
> BIO_printf(bio_err, "unable to load %s\n", key_descrip);
> }
>
> All in all, existing behavior and error messages are maintained. The
> subcommand is just a little more robust.
>
> Related, we might be able to make similar changes to rsa.c. But
> there's some extra special sauce present, so it was not a clear
> cut-in. I think the special sauce kid of attempts to do the same thing
> as above. (Maybe this patch should act more like rsa.c?)


--
Richard Levitte
levitte at openssl.org