[openssl-users] Adding EVP cipher into SSL library

[Schmicker, Robert]

Hello,

Can anyone give some insight on how to implement a new EVP symmetric
cipher into the SSL library? I have the cipher integrated into the EVP
and tested as working.

I know it's old but I followed AES's integration from this commit:
https://github.com/openssl/openssl/commit/deb2c1a1c58fb738b3216b663212572170de8183

Does anyone know of a more updated commit for a symmetric cipher I could
follow?

When debugging a client/server test program I receive the following
error client side:
    SSL routines:ssl_cipher_list_to_bytes:no ciphers
available:ssl/statem/statem_clnt.c:3564:

This leads me to believe I'm missing a crucial step somewhere for the
SSL library to find my EVP instance?

Best,
Rob Schmicker

P.S. I have done the following so far:

Added defines in include/openssl/tls1.h:
    # define TLS1_CK_ECDHE_ECDSA_WITH_MYCIPHER_SHA384        0x03001306
    # define TLS1_TXT_ECDHE_ECDSA_WITH_MYCIPHER_SHA384       
"ECDHE-ECDSA-MYCHIPHER-SHA384"

Added a define in include/openssl/ssl.h:
    # define SSL_TXT_MYCIPHER       "MYCIPHER"

Integrated into ssl/s3_lib.c:
    static SSL_CIPHER ssl3_ciphers[] = {
   
    {
     1,
     TLS1_TXT_ECDHE_ECDSA_WITH_MYCIPHER_SHA384,
     TLS1_CK_ECDHE_ECDSA_WITH_MYCIPHER_SHA384,
     SSL_kECDHE,
     SSL_aECDSA,
     SSL_MYCIPHER,
     SSL_AEAD,
     TLS1_2_VERSION, TLS1_2_VERSION,
     DTLS1_2_VERSION, DTLS1_2_VERSION,
     SSL_HIGH | SSL_FIPS,
     SSL_HANDSHAKE_MAC_SHA384 | TLS1_PRF_SHA384,
     64,
     64,
    },


Added the binary representation in ssl/ssl_locl.h:
    # define SSL_MYCIPHER           0x00100000U

Integrated into ssl/ssl_ciph.c:
    #define SSL_ENC_CHACHA_IDX      19
    #define SSL_ENC_MYCIPHER           20
    #define SSL_ENC_NUM_IDX             21
   
    /* Table of NIDs for each cipher */
    static const ssl_cipher_table
ssl_cipher_table_cipher[SSL_ENC_NUM_IDX] = {
        {SSL_MYCIPHER, NID_MYCIPHER},

    static const SSL_CIPHER cipher_aliases[] = {
        {0, SSL_TXT_MYCIPHER, 0, 0, 0, SSL_MYCIPHER},

Added the loading of the cipher into ssl/ssl_init.c:
    DEFINE_RUN_ONCE_STATIC(ossl_init_ssl_base)
    {
    #ifdef OPENSSL_INIT_DEBUG
        fprintf(stderr, "OPENSSL_INIT: ossl_init_ssl_base: "
                "Adding SSL ciphers and digests\n");
    #endif

        EVP_add_cipher(EVP_mycipher());

    #ifndef OPENSSL_NO_DES
        EVP_add_cipher(EVP_des_cbc());
        EVP_add_cipher(EVP_des_ede3_cbc());
    #endif