[openssl-users] d2i_RSAPrivateKey not working on a private key

[Frank Migge]

Hi David,

I didn't spot the error in your code. But since d2i_RSAPrivateKey() 
fails on the DER data, how about a workaround? If your remaining code 
works fine, you could extract the RSA key from the EVP_PKEY object 
(which you are getting with d2i_PrivateKey), e.g.

    RSA *rsakey;
    rsakey = EVP_PKEY_get1_RSA(privkey);  // this reads EVP_PKEY, best 
to test before to ensure it is RSA

    if (RSA_check_key(rsakey)) { printf("RSA key is valid.\n"); }
    else { printf("Error validating RSA key.\n"); }

    RSA_print_fp(stdout, rsakey, 3);
    ...

Alternatively, building a test case around d2i_RSAPrivateKey() to see if 
the DER format is valid, e.g. converting the PEM key into DER using the 
OpenSSL commandline, and binary-compare with the programs DER data.

openssl rsa -inform PEM -in test-key.pem -outform DER -out key.bin

Best Wishes,
Frank
> David Lobron <mailto:dlobron at akamai.com>
> Monday, October 12, 2015 11:34 PM
>
> Thanks very much, Frank. My code reads the PEM file, base64-decodes 
> it, and passes the resulting DER value to d2i_RSAPrivateKey. I 
> verified that I can extract the private key with d2i_PrivateKey from 
> the DER formatted data, and I can call SSL_CTX_use_PrivateKey with it 
> on my SSL context without a problem. It's only when I call 
> d2i_RSAPrivateKey I encounter a problem. I have included the code 
> below, with annotations (it's in Objective-C).
>
> I've got custom classes for X509Certificate and X509PrivateKey. I use 
> them like this:
>
> X509Certificate *cert = [X509Certificate 
> certificateWithPemEncodedFile:certFile];
> X509PrivateKey *privKey = [X509PrivateKey 
> privateKeyWithPemEncodedFile:keyFile];
> [cert validateWithPrivateKey:privKey];
> [privKey validate];
>
> That last call to [privKey validate] is where things fail currently. 
> The validateWithPrivateKey method works fine, and it looks like this:
>
> - (void)validateWithPrivateKey:(X509PrivateKey *)key
> {
> SSL_CTX *sslContext;
>
> [self validate];
> sslContext = SSL_CTX_new(TLSv1_server_method());
> NS_DURING {
> NSData *d = [key der];
> const unsigned char *p = (const unsigned char *)[d bytes];
> EVP_PKEY *pkey = d2i_PrivateKey(EVP_PKEY_RSA, NULL, &p, [d length]);
> if (!sslContext)
> [NSException raise:X509CertificateExcInternalError 
> format:@"SSL_CTX_new failed: %@", sslErrorString()];
> if (SSL_CTX_use_certificate(sslContext, _x) != 1)
> [NSException raise:X509CertificateExcInvalidCertificate 
> format:@"SSL_CTX_use_certificate failed: %@", sslErrorString()];
> if (SSL_CTX_use_PrivateKey(sslContext, pkey) != 1)
> [NSException raise:X509CertificateExcInvalidPrivateKey 
> format:@"SSL_CTX_use_PrivateKey_ASN1 failed: %@", sslErrorString()];
> SSL_CTX_free(sslContext);
> } NS_HANDLER {
> if (sslContext)
> SSL_CTX_free(sslContext);
> [localException raise];
> } NS_ENDHANDLER
> }
>
> (That initial call to "self validate" simply validates the cert 
> object's SSL_CTX).
>
> I initialize my private key object as follows:
>
> - (id)initWithPemEncodedFile:(NSString *)path
> {
> NSData *d = nil;
> NS_DURING {
> NSString *s;
> NSArray *inputLines;
>
> // read the file
> s = [NSString stringWithContentsOfFile:path];
> if (s == nil || [s length] == 0)
> [NSException raise:X509CertificateExcParameterError format:@"File %@ 
> is empty or cannot be read", path];
> inputLines = [s componentsSeparatedByString:@"\n"];
> d = [X509Certificate decodePemFragmentFromLines:inputLines 
> withBoundaryPhrases:[NSArray arrayWithObjects:@"PRIVATE KEY", @"RSA 
> PRIVATE KEY", nil]];
> } NS_HANDLER {
> [self release];
> [localException raise];
> } NS_ENDHANDLER
> return [self initWithDer:d];
>
> }
>
> The decodePemFragmentFromLines method looks like this:
>
> // Extract part of a PEM-encoded message, base64-decode it, and return 
> an NSData object
> + (NSData *)decodePemFragmentFromLines:(NSArray *)inputLines 
> withBoundaryPhrases:(NSArray *)boundaryPhrases
> {
> NSEnumerator *e = [inputLines objectEnumerator];
> NSMutableString *b64 = [NSMutableString string];
> NSString *s;
> NSString *boundaryPhrase = nil;
> NSString *startBoundary = nil;
> NSString *endBoundary = nil;
>
> while ((s = [e nextObject]) != nil) {
> NSEnumerator *e = [boundaryPhrases objectEnumerator];
> while ((boundaryPhrase = [e nextObject]) != nil) {
> startBoundary = [NSString stringWithFormat:@"-----BEGIN %@-----", 
> boundaryPhrase];
> if ([s isEqualToString:startBoundary]) {
> endBoundary = [NSString stringWithFormat:@"-----END %@-----", 
> boundaryPhrase];
> break;
> }
> }
> if (endBoundary != nil)
> break;
> }
> if (s == nil)
> [NSException raise:X509CertificateExcParameterError format:@"Start 
> boundary \"%@\" not found", startBoundary];
> while ((s = [e nextObject]) != nil) {
> if ([s isEqualToString:endBoundary])
> break;
> [b64 appendString:s];
> }
> if (s == nil)
> [NSException raise:X509CertificateExcParameterError format:@"End 
> boundary \"%@\" not found", endBoundary];
> return base64Decode(b64);
> }
>
> The initWithDer method is simply:
>
> - (id)initWithDer:(NSData *)der
> {
> if ((self = [super init]) != nil) {
> _der = [der copy];
> }
> return self;
> }
>
> All of the above works as expected, but when I call d2i_RSAPrivateKey 
> on the _der object, it fails. Here is the code that throws the exception:
>
> // validate; throws exception if key invalid
> - (void)validate
> {
> const unsigned char *p = (unsigned char *)[_der bytes];
> RSA *r = d2i_RSAPrivateKey(0, &p, [_der length]);
> int n;
> if (r == 0)
> [NSException raise:X509CertificateExcInvalidPrivateKey format:@"cannot 
> decode RSA private key"];
> NS_DURING {
> switch (n = RSA_check_key(r)) {
> case 1: // ok
> break;
> default:
> [NSException raise:X509CertificateExcInvalidPrivateKey 
> format:@"RSA_check_key() returned %d", n];
> }
> } NS_HANDLER {
> RSA_free(r);
> [localException raise];
> } NS_ENDHANDLER
> RSA_free(r);
>
> }
>
> Thanks for any help you can give here!
>
> --David
>
>
> _______________________________________________
> openssl-users mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

-- 
Sent with Postbox <http://www.getpostbox.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20151014/3ed4d8c0/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: compose-unknown-contact.jpg
Type: image/jpeg
Size: 770 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20151014/3ed4d8c0/attachment-0001.jpg>