[openssl-dev] sizeof (HMAC_CTX) changes with update, breaks binary compatibility

[mancha]

On Thu, Jun 11, 2015 at 09:07:18PM -0400, Dan McDonald wrote:
> I noticed that a new field was added to HMAC_CTX in the 1.0.2a->b or
> 1.0.1m->n update:
> 
> typedef struct hmac_ctx_st { const EVP_MD *md; EVP_MD_CTX md_ctx;
> EVP_MD_CTX i_ctx; EVP_MD_CTX o_ctx; unsigned int key_length; unsigned
> char key[HMAC_MAX_MD_CBLOCK]; + int key_init; } HMAC_CTX;
> 
> This breaks binary compatibility.  I found this out the hard way
> during an attempt to update OmniOS's OpenSSL to 1.0.2b ('014, bloody)
> or 1.0.1n (006, 012).  Observe our use of HMAC_CTX in illumos (which
> OmniOS is a distribution of):
> 
> struct Mac { char            *name; int             enabled; u_int
> mac_len; u_char          *key; u_int           key_len; int
> type; const EVP_MD    *evp_md; HMAC_CTX        evp_ctx; }; struct Comp
> { int     type; int     enabled; char    *name; }; struct Newkeys {
> Enc     enc; Mac     mac; Comp    comp; /* XXX KEBE SAYS THIS GETS
> CLOBBERED!!! */ };
> 
> You can see the code here:
> 
> 	http://src.illumos.org/source/xref/illumos-gate/usr/src/cmd/ssh/include/kex.h#100
> 
> What is supposed to happen in this situation?  I was under the
> impression that letter releases don't break binary compatibility.  The
> SSH in illumos breaks because of this, but it appears OpenSSH has
> worked around such a situation.
> 
> Clues are welcome.
> 
> Thanks, Dan McDonald -- OmniOS Engineering

Hi Dan. Many thanks for your report. I've checked and the issue you've
identified potentially affects OpenSSH 4.7 through 6.5, inclusive.

OpenSSH 6.6 replaces the OpenSSL HMAC with its own implementation making
the ABI change a NOP for OpenSSH 6.6 onwards.  

Cheers.

--mancha
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20150612/4226bf3e/attachment.sig>