[openssl-users] [FIPS compliance] ssl reneg when counter overflows(AES_GCM)

[Jakob Bohm]

On 04/11/2016 09:26, Marcus Meissner wrote:
> On Fri, Nov 04, 2016 at 10:03:21AM +0530, Akshar Kanak wrote:
>> Dear team
>>      as per the documnet http://csrc.nist.gov/groups/
>> STM/cmvp/documents/fips140-2/FIPS1402IG.pdf
>>      page 150 , Its mentioned
>>      The implementation of the nonce_explicit management logic inside the
>> module shall ensure that
>>      when the nonce_explicit part of the IV exhausts the maximum number of
>> possible values for a given
>>      session key (e.g., a 64-bit counter starting from 0 and increasing,
>> when it reaches the maximum value
>>      of 2 64 -1),
>> *either party (the client or the server) that encounters this condition
>> triggers a handshake to    establish a new encryption key – see Sections
>> 7.4.1.1 and 7.4.1.2 in RFC 5246*.
>>
>>      is this being handled by openssl ? in the source code of openssl i am
>> not able find out the
>>      exact location where this renegotiation is initiated when the counter
>> over flows ?
> (my understanding might be limited)
>
>
> I think we currently do an error if the calling frontend does not initiate renegotiation.
>
> Code is here:
> crypto/modes/gcm128.c
>
> CRYPTO_gcm128_encrypt_ctr32
>
> These kind of checks avoid the 32bit counter overflow:
>
>      if (mlen > ((U64(1) << 36) - 32) || (sizeof(len) == 8 && mlen < len))
>              return -1;
>
> The calling instance needs to re-iv with CRYPTO_gcm128_setiv.
>
> For TLS, if I understand correctly, the stack does intiailize the GCM cipher
> with a new IV on every TLS record and these cannot be that large.
>
> Ciao, Marcus
As I understand the GCM mode, the limitation is that the same
IV must not be used twice, and only a limited number of all
the possible IVs may be used before changing the *key*.
Setting a "new" IV for each TLS record is always needed,
regardless if it comes from a simple record counter or is
transmitted for each record (the TLS RFCs presumable have a
fixed choice for this).

But to get a new *key* before too much data has been encrypted
with it (such limits exist for *all* algorithms and modes), I
believe that an actual TLS renegotiation is needed, involving
at least a partial handshake without closing the connection.

Enjoy

Jakob
-- 
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded