[openssl-users] openssl 1.0.1p PEM_write_bio_RSAPrivateKey fail. error: ASN1_get_object:too long

[Viktor Dukhovni]

> On Mar 18, 2016, at 2:14 AM, Tekale, Sharad <sharad.tekale at zebra.com> wrote:
> 
> Thanks a lot for your reply.
>  
> I've actually used password of 64 characters in my program, for simplicity I've showcased as 6 byte password in below example.
>  
> Looks like there is some other issue or some stringent check that is added in 1.0.1p as the same code works fine in 0.9.8zg version.
>  
> Can you please give us pointers to debug this issue.

There's not much to debug.  The code fragment you posted works fine with 1.0.1.
You've not posted a complete program, nor how what steps you take to compile it,
or any compiler warnings, ..., so it is difficult to help you.

For comparison, this is what I get: 

$ OSSL=/.../OpenSSL_1_0_1

$ ${OSSL}/bin/openssl version -a
OpenSSL 1.0.1s-dev  xx XXX xxxx
built on: Fri Feb 12 23:23:01 2016
platform: darwin64-x86_64-cc
options:  bn(64,64) rc4(ptr,char) des(idx,cisc,16,int) idea(int) blowfish(idx) 
compiler: cc -I. -I.. -I../include  -fPIC -fno-common -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch x86_64 -O3 -DL_ENDIAN -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
OPENSSLDIR: "/.../OpenSSL_1_0_1/ssl"

$ cc -I${OSSL}/include -L${OSSL}/lib -lssl -lcrypto -o foo foo.c

$ ./foo
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-256-CBC,C87AA39820A10CA005471EA1E17E3CFD

yaoT0FSlhKY77KC1GNlWYN/0Gfk3L9hCl9AKiTqJ1NibvpwuemXYld7OmybHW9ZO
m7lSnApS91GDqsQGXOlhcKD0WgPf+YFbza7/RATxq5A+rs+CohEa7cg8C6eEgdqC
v2f8+AaSl7hdjtNaRFMcxp4ySq6oZPAsmXbbB/4l/AYK/7q//snqOgPbmGp4nLI9
J7gmJvf8igT0LjmC6IIiUgzb2wqHt/U/pFXj+9mm13tNlZtBVKInQMR3wfHqly4p
p2x3UD1xG982rfPvbefU80Zi8l5UnPAgd3jcYVxuZdCmABWqr30Wc/DuiRXzRhJe
LGKEIl+atfjfjH2cnytShkLvSDo/Ml2xKpnP3PQD7OrmCQpPWcFxphPCjHCsIRPz
c3kbx5LE2NArHT7FuOX8e4h422OIuqbxJc2KM8dLKvuoSB6ZWK8kcd1sesMcXu6p
/Q24EBSPCKd/MaRSqo2m41wVTrtnHHJ1R3MxzjtFLLXTWasT0qDbjj+45jyLzJHd
Xt/fhZJaK2mjmDCoLQFXNeaBqj7vN8ts8AI383hTcACeINzvsIirbKzWkH+els4e
ifrrge5mFvn7vvlAycIx73tlkTqI3o0CImXfIXp+necgEYkAAIwrTQxmYBz6jxhh
UcWN3sr1klrGLNakznnu4FWMFzPnE/Hjz1nRdMcXdS1j4k3gM48Zy3zXtBeQIAex
Fb+vpALzV7sk7H6wKpIjMTcNr66Z1WM+Vu3b4pQr/RnBtOjpHUt6uupr0gR6B61F
W7S0pztQXGI0tT4wL6KHa94XvPTNOJwvsPQyRId/I2J9iTMuPxm/xwx5nwL6nVwo
UPcbKbP4S4PWQpkCEL+CG6hLOWV1rkbpC7Rbk8RuB53CBmXJrMBVLpQe5T08OtEr
DXFuWs0Q00Z4y0U6nfjNzpPzevkTJN/ywjxufsri1J9U50zOLseOCauNzkSsEcoK
2kS1FiaxrGKWpCtd1H0klGoIbV2DtFsSMeyqMWfouMbT6dtw2fRZQcypB9nqgVD1
EUmhyPEk1QgRQKmFZ1VyU7gOygJ5RmEwNgy4gGIlXHT+Z7v/lKPyQfFrlqV3qskv
VDsjWp4GplzqIufwU7KHQj1yz3TM5k7BGDUf8DtP3UEioa9k0LdN5byc1Omk8Wgn
mX+sNXiOf5droA6QfAvZky/TCp/bptiEeabIRyyAL0gvQClf78vIrRUlBQ3wMYUF
wNS/XIZVU5szkIqPO7elJEhX149phcln7BvOLVHtUmSo0vzbFuQ+6gmfhAbU6Ozl
cIupo0flQ9Mz5TvE5HTi2Oe0Hg2DeRAYa8KzTYOkaGVnjXJKqVOvjW2fnen9kJOe
X5vjEvHnZIJ1L7z/YhGU4xKF4uSseV0AUgnyLItIa+Vd6iMu/l7Qxuqby+M1dLft
UQhrnJmo98Xr8zrE7Q9++CwH2O+v8/8vDkopLaZrmyktgr6yfbGuQcuT9/tByEVu
D2vbBCZWybwYmbiwS1i+tbl3sg/JvruCUXZ8HTu5JV2NLIPBGf+i4XmF5ZTuXSAU
ba0l8XIAHYGLADvdnDlhew99bvpmU3fUEUHC1PF0C09r9pSkDDHtedmRwxVZCOUp
-----END RSA PRIVATE KEY-----

$ echo $?
0

$ cat foo.c
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

#include <openssl/rsa.h>
#include <openssl/evp.h>
#include <openssl/pem.h>
#include <openssl/bio.h>

int main(int argc, char *argv[])
{   
    char *pass = "abcdef";
    size_t passlen;
    int len;
    int ret;
    RSA *rsa;
    BIO *bio = BIO_new_fp(stdout, BIO_NOCLOSE);
    BIO *priv_bio = BIO_new(BIO_s_mem());
    char buf[4096];

    if (argc > 1)
        pass = argv[1];
    passlen = strlen(pass);

    OpenSSL_add_all_algorithms();
    rsa = RSA_generate_key(2048, 65537, NULL, NULL);
    ret = PEM_write_bio_RSAPrivateKey(priv_bio, rsa, EVP_aes_256_cbc(),
                                      (unsigned char *)pass, (int)passlen,
                                      NULL, NULL);
    while((len = BIO_gets(priv_bio, buf, sizeof(buf))) > 0)
        BIO_write(bio, buf, len);
    BIO_free(priv_bio);
    BIO_free(bio);
    exit(!ret);
}

-- 
	Viktor.