[openssl-users] Lost in STACK_OF again (porting M2Crypto to OpenSSL 1.1.* API)

[Dr. Stephen Henson]

On Tue, Sep 12, 2017, Mat??j Cepl wrote:

> Hi,
> 
> I am working on porting M2Crypto to OpenSSL 1.1.* API (in branch
> https://gitlab.com/mcepl/m2crypto/commits/openssl-1.1.0 ) and I
> got lost in STACK_OF structures.
> 
> Simplified function I have troubles with is (the real stuff with
> all Python2/Python3 shims is https://is.gd/Nbq3Qp ; the similar problem
> is couple of lines below in the function get_der_encoding_stack).
> 
>     #include <openssl/asn1.h>
>     #include <openssl/x509.h>
>     #include <openssl/x509v3.h>
> 
>     #include <openssl/asn1t.h>
> 
>     typedef STACK_OF(X509) SEQ_CERT;
> 
>     ASN1_ITEM_TEMPLATE(SEQ_CERT) =
>         ASN1_EX_TEMPLATE_TYPE(ASN1_TFLG_SEQUENCE_OF, 0,
>                               SeqCert, X509)
>     ASN1_ITEM_TEMPLATE_END(SEQ_CERT)
> 
>     IMPLEMENT_ASN1_FUNCTIONS(SEQ_CERT)
> 
>     ...
> 
>     STACK_OF(X509) *
>       make_stack_from_der_sequence(PyObject * pyEncodedString){
>           STACK_OF(X509) *certs;
>           Py_ssize_t encoded_string_len;
>           char *encoded_string;
> 
>           encoded_string_len = PyString_Size(pyEncodedString);
> 
>           if (encoded_string_len > INT_MAX) {
>               PyErr_SetString(PyExc_ValueError,
>                               "object too large");
>               return NULL;
>           }
> 
>           encoded_string = PyString_AsString(pyEncodedString);
> 
>           if (!encoded_string) {
>               return NULL;
>           }
> 
>           certs = ASN1_seq_unpack(
>                   (unsigned char *)encoded_string,
>                   encoded_string_len,
>                   d2i_X509, X509_free );
>           if (!certs) {
>               PyErr_SetString(_x509_err,
>                               ERR_reason_error_string(
>                                   ERR_get_error()));
>               return NULL;
>           }
> 
>           return certs;
>       }
> 
> Obviously this fails to compile with these errors:
> 
> SWIG/_m2crypto_wrap.c: In function
> ???make_stack_from_der_sequence???:
> SWIG/_m2crypto_wrap.c:8718:13: warning: implicit declaration of
> function ???ASN1_seq_unpack???; did you mean ???ASN1_item_unpack???? [-
> Wimplicit-function-declaration]
>      certs = ASN1_seq_unpack((unsigned char *)encoded_string,
> encoded_string_len, d2i_X509, X509_free );
>              ^~~~~~~~~~~~~~~
>              ASN1_item_unpack
> SWIG/_m2crypto_wrap.c:8718:11: warning: assignment makes pointer
> from integer without a cast [-Wint-conversion]
>      certs = ASN1_seq_unpack((unsigned char *)encoded_string,
> encoded_string_len, d2i_X509, X509_free );
>            ^
> Obviously I have missed something from STACK_OF API, but I cannot
> for the love of the world find what. Did truly *_seq_unpack
> functions got lost on the way to 1.1 API? If I have to do the
> unpacking "manually", how to do it?
> 
> How can I get STACK_OF(X509) from the string with DER
> certificate?
> 
> I was looking also to the discussion by Jim Carroll on
> https://goo.gl/ZUxQH8 but I have probably misunderstood
> something. I believe I do everything I am supposed to, but still
> there is something apparently missing.
> 

Yes *_seq_unpack() is no longer in 1.1. What happens is that code above it
generates a function d2i_SEQ_CERT() which does the same as ASN1_seq_unpack()
for a certificate.

So something like this should work:

const unsigned char *tmp = (unsigned char *)encoded_string;

...

certs = d21_SEQ_CERT(NULL, &tmp, encoded_string_len);

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org