[openssl-users] [FIPS compliance] ssl reneg when counter overflows(AES_GCM)

[Marcus Meissner]

On Fri, Nov 04, 2016 at 10:03:21AM +0530, Akshar Kanak wrote:
> Dear team
>     as per the documnet http://csrc.nist.gov/groups/
> STM/cmvp/documents/fips140-2/FIPS1402IG.pdf
>     page 150 , Its mentioned
>     The implementation of the nonce_explicit management logic inside the
> module shall ensure that
>     when the nonce_explicit part of the IV exhausts the maximum number of
> possible values for a given
>     session key (e.g., a 64-bit counter starting from 0 and increasing,
> when it reaches the maximum value
>     of 2 64 -1),
> *either party (the client or the server) that encounters this condition
> triggers a handshake to    establish a new encryption key – see Sections
> 7.4.1.1 and 7.4.1.2 in RFC 5246*.
> 
>     is this being handled by openssl ? in the source code of openssl i am
> not able find out the
>     exact location where this renegotiation is initiated when the counter
> over flows ?

(my understanding might be limited)


I think we currently do an error if the calling frontend does not initiate renegotiation.

Code is here:
crypto/modes/gcm128.c

CRYPTO_gcm128_encrypt_ctr32

These kind of checks avoid the 32bit counter overflow:

    if (mlen > ((U64(1) << 36) - 32) || (sizeof(len) == 8 && mlen < len))
            return -1;

The calling instance needs to re-iv with CRYPTO_gcm128_setiv.

For TLS, if I understand correctly, the stack does intiailize the GCM cipher
with a new IV on every TLS record and these cannot be that large.

Ciao, Marcus