[openssl-users] How to define EVP_EncryptUpdate and EVP_EncryptFinal functions for an AES engine? (and a separate question re: padding).

[Matt Caswell]

On 26/06/17 15:21, Brett R. Nicholas wrote:
> Hi there,
> 
> 
> I'm building a dynamic engine to support a custom AES hardware module
> that I've implemented in FPGA logic****, but after reading all available
> documentation, and pouring over the source code, I'm still very confused
> about the following two things.....
> 
> 
>  1. *How and where I should define the
>     EVP_[En/De]cryptInit_ex(..),    EVP_[En/De]cryptUpdate(..), and
>     EVP_[En/De]cryptFinal_ex(..)
>     functions in my Engine code? *
> 
> 
> Prior to this, I successfully built an engine for my sha256 accelerator,
> and now I'm trying to follow the same steps for AES. For sha256, the
> EVP_MD structure allowed me to declare pointers to my init, update, and
> final functions. This all worked flawlessly.
> 
> 
> Now, when I'm building the AES engine, I see that the EVP_CIPHER
> structure does not have these pointers (init, update, final), but
> instead has a pointer to init_key and do_cipher functions. However, the
> EVP encryption interface still has these functions defined.
> 
> 
> AFAIK (and please correct me if this is wrong)  my init_key function is
> invoked by the EVP interface when I call the EVP_[En/De]cryptInit_ex
> function, and the do_cipher function is called upon
> EVP_[En/De]cryptUpdate. But how should I handle
> the EVP_[En/De]cryptFinal functions? Should I not be implementing them
> in my engine? Or am I missing something here....

The behaviour varies a little depending on whether you have set the
EVP_CIPH_FLAG_CUSTOM_CIPHER flag for your cipher.

If you have not set the EVP_CIPH_FLAG_CUSTOM_CIPHER flag:

do_cipher() will only ever be invoked where the length is a multiple of
a full block length. EVP_[En/De]cryptUpdate and EVP_[En/De]cryptFinal
will invoke do_cipher() as required. There may not be a one-to-one
correspondence between calls to those functions and calls to
do_cipher(). The EVP functions handle dealing with partial blocks and
padding the final block as required.


If you have set the EVP_CIPH_FLAG_CUSTOM_CIPHER flag:

EVP_[En/De]cryptUpdate just call do_cipher() directly. The cipher
implementation itself has to handle partial blocks etc. Similarly
EVP_[En/De]cryptFinal just calls do_cipher() directly, but will pass
NULL for the input buffer, and 0 for the input length. Your cipher
implementation will have to handle any padding etc itself.


Matt