[openssl-users] Lost in STACK_OF again (porting M2Crypto to OpenSSL 1.1.* API)

[Matěj Cepl]

Hi,

I am working on porting M2Crypto to OpenSSL 1.1.* API (in branch
https://gitlab.com/mcepl/m2crypto/commits/openssl-1.1.0 ) and I
got lost in STACK_OF structures.

Simplified function I have troubles with is (the real stuff with
all Python2/Python3 shims is https://is.gd/Nbq3Qp ; the similar problem
is couple of lines below in the function get_der_encoding_stack).

    #include <openssl/asn1.h>
    #include <openssl/x509.h>
    #include <openssl/x509v3.h>

    #include <openssl/asn1t.h>

    typedef STACK_OF(X509) SEQ_CERT;

    ASN1_ITEM_TEMPLATE(SEQ_CERT) =
        ASN1_EX_TEMPLATE_TYPE(ASN1_TFLG_SEQUENCE_OF, 0,
                              SeqCert, X509)
    ASN1_ITEM_TEMPLATE_END(SEQ_CERT)

    IMPLEMENT_ASN1_FUNCTIONS(SEQ_CERT)

    ...

    STACK_OF(X509) *
      make_stack_from_der_sequence(PyObject * pyEncodedString){
          STACK_OF(X509) *certs;
          Py_ssize_t encoded_string_len;
          char *encoded_string;

          encoded_string_len = PyString_Size(pyEncodedString);

          if (encoded_string_len > INT_MAX) {
              PyErr_SetString(PyExc_ValueError,
                              "object too large");
              return NULL;
          }

          encoded_string = PyString_AsString(pyEncodedString);

          if (!encoded_string) {
              return NULL;
          }

          certs = ASN1_seq_unpack(
                  (unsigned char *)encoded_string,
                  encoded_string_len,
                  d2i_X509, X509_free );
          if (!certs) {
              PyErr_SetString(_x509_err,
                              ERR_reason_error_string(
                                  ERR_get_error()));
              return NULL;
          }

          return certs;
      }

Obviously this fails to compile with these errors:

SWIG/_m2crypto_wrap.c: In function
‘make_stack_from_der_sequence’:
SWIG/_m2crypto_wrap.c:8718:13: warning: implicit declaration of
function ‘ASN1_seq_unpack’; did you mean ‘ASN1_item_unpack’? [-
Wimplicit-function-declaration]
     certs = ASN1_seq_unpack((unsigned char *)encoded_string,
encoded_string_len, d2i_X509, X509_free );
             ^~~~~~~~~~~~~~~
             ASN1_item_unpack
SWIG/_m2crypto_wrap.c:8718:11: warning: assignment makes pointer
from integer without a cast [-Wint-conversion]
     certs = ASN1_seq_unpack((unsigned char *)encoded_string,
encoded_string_len, d2i_X509, X509_free );
           ^
Obviously I have missed something from STACK_OF API, but I cannot
for the love of the world find what. Did truly *_seq_unpack
functions got lost on the way to 1.1 API? If I have to do the
unpacking "manually", how to do it?

How can I get STACK_OF(X509) from the string with DER
certificate?

I was looking also to the discussion by Jim Carroll on
https://goo.gl/ZUxQH8 but I have probably misunderstood
something. I believe I do everything I am supposed to, but still
there is something apparently missing.

Thank you for any help,

Matěj

-- 
http://matej.ceplovi.cz/blog/, Jabber: mcepl<at>ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5  BC1D 7920 5802 880B C9D8
 A philosopher like Plato, according to Luther's colorful imagery,
remains like a cow who looks at a new door, refusing to enter?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20170912/13359147/attachment.sig>