[openssl-dev] [openssl.org #4046] Fix xmm6 register clobbering in crypto/bn/asm/x86_64-mont5.pl:bn_power5() under Win64

[Uwe Granzow via RT]

Hi,

i had some problems on Win64 using BIO_do_handshake/BIO_should_retry in a loop. The compiler optimizer placed a local variable value in the xmm6 register.
The content of this register was destroyed after calling BIO_do_handshake. I debugged this and found that the xmm6/xmm7 registers were not restored.
I fixed this with following diff for openssl-1.0.2d / x86_64-mont5.pl (bn_power5 and bn_from_mont8x):

1013a1014,1020
> ___
> $code.=<<___ if ($win64);
> movaps	-88(%rsi),%xmm6
> movaps	-72(%rsi),%xmm7
> ___
> $code.=<<___;
> 
2005a2013,2019
> ___
> $code.=<<___ if ($win64);
> movaps	-88(%rsi),%xmm6
> movaps	-72(%rsi),%xmm7
> ___
> $code.=<<___;
> 

The fix in bn_from_mont8x was not necessary for me but i think the lines are also missing there.

Best regards
Uwe Granzow
________________________________________
Celemony Software GmbH
Uwe Granzow
ug at celemony.com
________________________________________




-------------- next part --------------
_______________________________________________
openssl-bugs-mod mailing list
openssl-bugs-mod at openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-bugs-mod