[openssl-users] shouldn't fipslink.pl include the fipscanister.lib in the link line?

[Jakob Bohm]

On 21/07/2017 00:56, Sam Roberts wrote:
> On Wed, Jul 19, 2017 at 6:27 PM, Dr. Stephen Henson <steve at openssl.org> wrote:
>> Try linking with fipscanister.lib too: that's where those symbols are located.
> OK, I'd tried that before with no luck, but I tried harder. I found
> that if my lib line has the fips_premain.obj, then the
> fipscanister.lib, then the rest of the program's obj files and libs in
> exactly that order, the symbols are resolved.
>
> That's progress!
>
> Now I'm hitting the dreaded MSVCRT conflicts with use of other libs problem.
>
> Most of the application is compiled with /MT, but openssl-fips-2.0.16
> is using /MD, could this be an issue? Can I/should I convince
> ms\do_fips to build against the multi-threaded runtime?
>
> It may also be other obj files causing the issue, the MSVC message is
> not so helpful, I'm continuing to look.
>
> I used /nodedefaultlib:msvcrt (even though its supposed to not be
> recommended) and I got a link of openssl-cli, though with lots of
> "LNK4049 locally defined symbol _exit imported", which sounds like its
> another symbol of /MT and /MD mismatch.
That's the warning that something (hopefully only the fips blob and not
the fips-enabled OpenSSL) was compiled with /MD.  This should be harmless
as the linker generates the needed glue (dummy pointer variables pointing
to the symbol, so the /MD compiled code can dereference those as if they
were import table entries).

> I also almost got a link of node, but it died with
> fipscanister.lib(cryptolib.obj): error LNK2001: unresolved symbol
> __imp_wcsstr, and __imp___stdio_common_vsscanf, both of which sound
> suspiciously like a problem with the runtime compilation flags to me.
This is strange, as those two should also have been handled by the
LNK4049 mechanism,unless there is one of the following:

A) Mismatch between compiler versions or C runtime header versions
   between the FIPS canister compile and your final link.

B) Some oddity in how those two symbols are defined in your visual
   studio runtime version.

Either should be possible to work around with some tenacity.  Of cause
doing such work is not "supported" by Microsoft, but what is these days.

If the issue is item B, then linking in a stub file providing the two
missing functions (as wrappers around functions actually in the static
C runtime) might do the trick.

Of cause, you will need to do thorough testing of the resulting
program/dll to make sure there are no other C runtime differences
causing trouble.

P.S.

I kind of wonder what in the fips canister uses wcsstr(), but since
that cannot be changed while retaining the FIPS validation status,
that's just curiousness.

Enjoy

Jakob
-- 
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded