[openssl-dev] [openssl.org #3694] WinCE openSSL 1.0.1L with FIPS 2.0.8 - fingerprint does not match

[Andy Polyakov via RT]

> One thing to try would be to try both ways of the define for __thumb. This can explain the fingerprint failure. 
> 
> In fips_canister.c around line 188
> 
> # if defined(__thumb__) || defined(__thumb)
>     return (void *)((size_t)instruction_pointer&~1);
> # else
>     return (void *)instruction_pointer;
> # endif

Well, procedure is build fipscanister.obj, build fips_algvs.exe, use
fipscanister.obj *binary* with application. So that that whatever thumb
definition was it worked, in fips_algvs.exe context. Besides, that mask
is more of an optimization thing. I mean it should work in either case,
just that without masking the bit all memory references will be
unaligned. In other words, problem ought to be elsewhere...

> -----Original Message-----
> From: openssl-dev [mailto:openssl-dev-bounces at openssl.org] On Behalf Of Lior Gotian via RT
> Sent: Tuesday, February 10, 2015 11:16
> Cc: openssl-dev at openssl.org
> Subject: [openssl-dev] [openssl.org #3694] WinCE openSSL 1.0.1L with FIPS 2.0.8 - fingerprint does not match
> 
> I was successful at compiling the FIPS 2.0.8 module for Windows CE exactly as provided without any modifications.
> Additionally, I built fips_algvs.exe to successfully validate the canister on the target system.
> 
> After tweaking some #ifdef directives in the openSSL 1.0.1L, I was able to get it to successfully build for WinCE.
> 
> The build appears to complete successfully.  However, at run-time, entering FIPS mode fails with an error messages:
> FIPS_check_incore_fingerprint:fingerprint does not match
> 
> I have reviewed the build instructions carefully and believe all the build instructions have been adhered to.  What needs to be changed for the signature to be properly embedded?

Well, this is not exactly fair question. I mean how can one tell
something without knowing something more specific than "some tweaking"
and "appears successful" :-) Well, tweaking probably is as important as
how you link your application, or rather how is the procedure different
from fips_algvs.exe. Can you confirm that msincore was actually
executed? What happens if you run it manually? Does application have
relocations? See with dumpbin /relocations. What's preferred load
address? See with dumpbin /headers. Is application loaded on preferred
address? This you should be able to see with debugger by comparing e.g.
main's address in debugger with one collected with dumpbin /symbols...