[openssl-users] fingerprint mismatch issue with fips build on Win32

[Zeke Evans]

Hi,

Our win32 applications will sometimes fail to start due to a
fingerprint mismatch in the fips module.  It appears this is caused by
the fixed baseaddr used to verify the checksum.  We are building with
the /FIXED and /DYNAMICBASE:NO options.

The User Guide states:
"The standard OpenSSL build with the fips option will use a base
address for libeay32.dll of 0xFB00000 by default. This value was
chosen because it is unlikely to conflict with other dynamically
loaded libraries. In the event of a clash with another dynamically
loaded library which will trigger runtime relocation of libeay32.dll,
the integrity check will fail with the error
FIPS_R_FINGERPRINT_DOES_NOT_MATCH_NONPIC_RELOCATED

A base address conflict can be resolved by shuffling the other DLLs or
re-compiling OpenSSL with an alternative base address specified with
the --with-baseaddr= option."

How is 0xFB00000 unlikely to conflict with other DLLs?  How would an
application select a base address that is less likely to have a
conflict?  Or, how can an application realistically shuffle the other
DLLs?  FWIW, the applications load many other DLLs and some are .NET
applications.  One train of thought is to load the OpenSSL DLL early
on, but in some cases that is not practical such as a .NET
application.

As a side note, the issue appears more frequently when the application
is loaded through Visual Studio (not a real world scenario).

Thanks,
Zeke