[openssl-users] openssl-users Digest, Vol 11, Issue 5

[David Lobron]

> Your attached sample certificate and private key (1024 bit RSA) works fine.
> I am reading it with PEM_read_PrivateKey( fp, &key, NULL, NULL), and also
> PEM_read_bio_PrivateKey(pkeybio, NULL, 0, NULL) works.
> 
> If you could post the code or code fragment that creates the problem?
> d2i_RSAPrivateKey() is not reading PEM, just making sure...

Thanks very much, Frank.  My code reads the PEM file, base64-decodes it, and passes the resulting DER value to d2i_RSAPrivateKey.  I verified that I can extract the private key with d2i_PrivateKey from the DER formatted data, and I can call SSL_CTX_use_PrivateKey with it on my SSL context without a problem.  It's only when I call d2i_RSAPrivateKey I encounter a problem.  I have included the code below, with annotations (it's in Objective-C).  

I've got custom classes for X509Certificate and X509PrivateKey.  I use them like this:

X509Certificate *cert = [X509Certificate certificateWithPemEncodedFile:certFile];
X509PrivateKey *privKey = [X509PrivateKey privateKeyWithPemEncodedFile:keyFile];
[cert validateWithPrivateKey:privKey];
[privKey validate];

That last call to [privKey validate] is where things fail currently.  The validateWithPrivateKey method works fine, and it looks like this:

- (void)validateWithPrivateKey:(X509PrivateKey *)key
{
    SSL_CTX *sslContext;
    
    [self validate];
    sslContext = SSL_CTX_new(TLSv1_server_method());
    NS_DURING {
        NSData *d = [key der];
        const unsigned char *p = (const unsigned char *)[d bytes];
        EVP_PKEY *pkey = d2i_PrivateKey(EVP_PKEY_RSA, NULL, &p, [d length]);
        if (!sslContext)
            [NSException raise:X509CertificateExcInternalError format:@"SSL_CTX_new failed: %@", sslErrorString()];
        if (SSL_CTX_use_certificate(sslContext, _x) != 1)
            [NSException raise:X509CertificateExcInvalidCertificate format:@"SSL_CTX_use_certificate failed: %@", sslErrorString()];
        if (SSL_CTX_use_PrivateKey(sslContext, pkey) != 1)
            [NSException raise:X509CertificateExcInvalidPrivateKey format:@"SSL_CTX_use_PrivateKey_ASN1 failed: %@", sslErrorString()];
        SSL_CTX_free(sslContext);
    } NS_HANDLER {
        if (sslContext)
            SSL_CTX_free(sslContext);
        [localException raise];
    } NS_ENDHANDLER
}

(That initial call to "self validate" simply validates the cert object's SSL_CTX). 

I initialize my private key object as follows:

- (id)initWithPemEncodedFile:(NSString *)path
{
    NSData *d = nil;
    NS_DURING {
        NSString *s;
        NSArray *inputLines;
        
        // read the file
        s = [NSString stringWithContentsOfFile:path];
        if (s == nil || [s length] == 0)
            [NSException raise:X509CertificateExcParameterError format:@"File %@ is empty or cannot be read", path];
        inputLines = [s componentsSeparatedByString:@"\n"];
        d = [X509Certificate decodePemFragmentFromLines:inputLines withBoundaryPhrases:[NSArray arrayWithObjects:@"PRIVATE KEY", @"RSA PRIVATE KEY", nil]];
    } NS_HANDLER {
        [self release];
        [localException raise];
    } NS_ENDHANDLER
    return [self initWithDer:d];

}

The decodePemFragmentFromLines method looks like this:

// Extract part of a PEM-encoded message, base64-decode it, and return an NSData object
+ (NSData *)decodePemFragmentFromLines:(NSArray *)inputLines withBoundaryPhrases:(NSArray *)boundaryPhrases
{
    NSEnumerator *e = [inputLines objectEnumerator];
    NSMutableString *b64 = [NSMutableString string];
    NSString *s;
    NSString *boundaryPhrase = nil;
    NSString *startBoundary = nil;
    NSString *endBoundary = nil;
    
    while ((s = [e nextObject]) != nil) {
        NSEnumerator *e = [boundaryPhrases objectEnumerator];
        while ((boundaryPhrase = [e nextObject]) != nil) {
            startBoundary = [NSString stringWithFormat:@"-----BEGIN %@-----", boundaryPhrase];
            if ([s isEqualToString:startBoundary]) {
                endBoundary = [NSString stringWithFormat:@"-----END %@-----", boundaryPhrase];
                break;
            }
        }
        if (endBoundary != nil)
            break;
    }
    if (s == nil)
	[NSException raise:X509CertificateExcParameterError format:@"Start boundary \"%@\" not found", startBoundary];
    while ((s = [e nextObject]) != nil) {
	if ([s isEqualToString:endBoundary])
	    break;
	[b64 appendString:s];
    }
    if (s == nil)
	[NSException raise:X509CertificateExcParameterError format:@"End boundary \"%@\" not found", endBoundary];
    return base64Decode(b64);
}

The initWithDer method is simply:

- (id)initWithDer:(NSData *)der
{
    if ((self = [super init]) != nil) {
        _der = [der copy];
    }
    return self;
}

All of the above works as expected, but when I call d2i_RSAPrivateKey on the _der object, it fails.  Here is the code that throws the exception:

// validate; throws exception if key invalid
- (void)validate
{
    const unsigned char *p = (unsigned char *)[_der bytes];
    RSA *r = d2i_RSAPrivateKey(0, &p, [_der length]);
    int n;
    if (r == 0)
	[NSException raise:X509CertificateExcInvalidPrivateKey format:@"cannot decode RSA private key"];
    NS_DURING {
	switch (n = RSA_check_key(r)) {
	    case 1:		// ok
		break;
	    default:
		[NSException raise:X509CertificateExcInvalidPrivateKey format:@"RSA_check_key() returned %d", n];
	}
    } NS_HANDLER {
	RSA_free(r);
	[localException raise];
    } NS_ENDHANDLER
    RSA_free(r);

}

Thanks for any help you can give here!

--David


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2863 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20151012/aa54b63e/attachment.bin>