AES and EVP_CIPHER question

[Tomas Mraz]

The EVP_CIPHER_CTX_set_padding(ctx, 0) must be called after the
EVP_CipherInit() to have an effect.

Also what is the AST_CRYPTO_AES_BLOCKSIZE value? Is it in bits (i.e,
128)?

Also res should be initialized to -1 so you do not return uninitialized
value on error.

Tomas Mraz

On Fri, 2022-05-13 at 09:49 -0600, Philip Prindeville wrote:
> Hi,
> 
> I'm trying to rewrite some legacy AES_* code to use EVP_CIPHER_* so
> it's forward compatible into 3.x.
> 
> My code, in a nutshell, looks like:
> 
> static int evp_cipher_aes_decrypt(const unsigned char *in, unsigned
> char *out, unsigned inlen, const ast_aes_decrypt_key *key)
> {
>         EVP_CIPHER_CTX *ctx;
>         int res, outlen, finallen;
>         unsigned char final[AST_CRYPTO_AES_BLOCKSIZE / 8];
> 
>         if ((ctx = EVP_CIPHER_CTX_new()) == NULL) {
>                 return -1;
>         }
> 
>         EVP_CIPHER_CTX_set_padding(ctx, 0);
> 
>         do {
>                 if ((res = EVP_CipherInit(ctx, EVP_aes_128_ecb(),
> key->raw, NULL, 0)) <= 0) {
>                         break;
>                 }
>                 if ((res = EVP_CipherUpdate(ctx, out, &outlen, in,
> inlen)) <= 0) {
>                         break;
>                 }
>                 /* for ECB, this is a no-op */
>                 if ((res = EVP_CipherFinal(ctx, final, &finallen)) <=
> 0) {
>                         break;
>                 }
> 
>                 res = outlen;
>         } while (0);
> 
>         EVP_CIPHER_CTX_free(ctx);
> 
>         return res;
> }
> 
> It's ECB, so there's no IV.  Or padding.  The block size and key size
> are both 128 bits.
> 
> One thing I noticed right away is that EVP_CipherUpdate() returns 1,
> and sees "outlen" to zero.
> 
> And then EVP_CipherFinal() returns 0, and sets "finallen" to zero.
> 
> What's wrong with this code?
> 
> I'm trying to write "naive" code that counts on the primitives to
> indicate how much resultant output is generated for the input I've
> given (yes, I know that it's 1:1 in the case of ECB, but I shouldn't
> have to hard-code that in case I want to use the same code with
> multiple block modes).
> 
> The function is supposed to return <= 0 on error, otherwise the
> number of bytes decrypted into "out" on success.
> 
> Thanks,
> 
> -Philip
> 

-- 
Tomáš Mráz, OpenSSL