Error building OpenSSL-1.1.1g

Fri Jun 26 20:20:07 UTC 2020

Two follow-up issues - someone please tell me if I should have started a new thread - these are not really related to the initial point of this one.

While my build and test worked after re-installing MSYS2, the install failed.  Unfortunately I inadvertently over-wrote the logfile I was keeping about this thread.

1) OPENSSLDIR was not correctly set.  In essence, while the documentation leads one to assume that the configure option --openssldir is derived from --prefix, this was NOT the case.  All of the installs went into the PREFIX I had passed to Configure until something barfed when it tried to write to c:\Program Files\SSL.  I then re-configured/re-built with both --prefix=PREFIX and --openssldir=PReFIX/SSL specified.  This seems to have worked correctly, with the relevant files then being put where they should be.

Perhaps this should be reported as a possible bug in the build process?

2) The documentation was not successfully installed.  There appears to be a complete file tree for it, but all of the html files are empty.  The failure is:

/c/usr/local/OPENSSL/openssl-1.1.1g/x86_64-8.1.0-posix-seh-rt_v6-rev0/share/doc/openssl/html/man7/X448.html -> /c/usr/local/OPENSSL/openssl-1.1.1g/x86_64-8.1.0-posix-seh-rt_v6-rev0/share/doc/openssl/html/man7/X25519.html
sh: pod2html: command not found

pod2html is a perl script in C:\msys64\usr\bin\core_perl.  I added that to my PATH.

Because I never saw any suggestion to do so, this might also be more appropriately submitted as a potential bug in the build process.

-----Original Message-----
From: mhkelley2017 at gmail.com <mhkelley2017 at gmail.com> 
Sent: Friday, June 26, 2020 12:09 PM
To: openssl-users at openssl.org
Cc: 'Michael Wojcik' <Michael.Wojcik at microfocus.com>
Subject: RE: Error building OpenSSL-1.1.1g

Thank you very much.  While my problem has already been resolved,
you raise a couple of issues that are worth noting for
completeness.

> > Can someone point me to sources of information about how to
> > resolve this issue?
> 
> Probably not, because it's not clear how you got into this state
> in the first place. You have a mix of 32- and 64-bit tools in
> your toolchain for this build. I don't know what you did to get
> there; I've never seen that happen with any of my OpenSSL builds.
> 
*** snip ***
> 
> My suggestion: Follow the recommended process by installing nasm
> (it's free) and configuring for that instead of MASM.

Did that as soon I decided to give up on mingw64 and try Visual
Studio.  Granted, my Y-chromosome does give me some challenges
when it comes to reading/following instructions, but that
requirement was clearly enough stated that I noticed and followed
up on it :)

> As Matt
> suggested, start from a "Visual Studio Command Prompt" window for
> the bitness you want to build, and then make sure you configure
> for that bitness. Get rid of any existing build artifacts first -
> probably the easist way is to move your working directory aside
> and extract a fresh one from the tarball.

Actualy repeated that step several times throughout to ensure I
was always starting clean.

> As Matt noted, you can confirm whether you're using the 32- or
> 64-bit version of the Visual Studio toolchain just by running
> cl.exe with no parameters; the banner says "for x86" or "for
> x64".

What cl.exe responded was what I expected, so didn't report it:

C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.26.28801\bin\Hostx86\x86\cl.exe

It is still a mystery why/how anything related to 32-bit ever arose in the build process.  But I no longer care because it was apparently self-inflicted and not likely to bother anyone else.

*** snip ***

> Regarding MSYS2 and Perl:
> 
> - Did you read the updated NOTES.WIN that Matt posted a link to?

Yes, found them at the beginning of my efforts, but (in case you
know the old joke about magicians), "the rabbit was already in
the hat" by that point.

*** snip ***

> - You haven't told us what Perl implementation you're
>   using.

My initial post:
   Perl: ActivePerl 5.24.3 Build_2404 (64-bit)

>   Knowing the path to it doesn't help us. (Though, to be
>   honest, knowing which implementation it is probably isn't
>   terribly useful either.) It would be nice if Sergio told us
>   which one *he's* using, since it's known to work with the MSYS2
>   configurations.

I think he did tell is - implicitly.  He used whatever he has
installed with his MSYS2 package.  If I'd had the same one
installed with my package, I'd have missed all this fun.
> 
> - I don't know whether there are Perl implementations for Windows
>   specifically intended to be used with MinGW or MSYS2, and if so
>   whether they work for the OpenSSL build. Oh, wait: I just did a
>   quick search, and apparently there is an MSYS2 Perl package
>   available from repo.msys2.org. Does it work with the OpenSSL
>   build? I don't know, but it seems like it's worth a try.

This is actually an important question.  In response to Matt's
suggestion, I went to MSYS2 to see what packages they have.
Their list of packages *did not* include a choice like "the one
we include with our initial installation in case you have somehow
lost it". Instead, they list the two give in one of my more
recent posts.  One listed is explicitly designed for use with
mingw-64.  It fails for OpenSSL the same way as ActivePerl.

Bottom line:  MSYS2 *does* include perl in its basic installation
(at least the most recent version) and it works.  Don't mess that
one up or your in for some fun.

Thanks to all.

-----Original Message-----
From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of Michael Wojcik
Sent: Friday, June 26, 2020 11:25 AM
To: openssl-users at openssl.org
Subject: RE: Error building OpenSSL-1.1.1g

[Removed the top-posted mess of history from this reply.]

In response to various questions posed in this thread:

> Can someone point me to sources of information about how to resolve this issue?

Probably not, because it's not clear how you got into this state in the first place. You have a mix of 32- and 64-bit tools in your toolchain for this build. I don't know what you did to get there; I've never seen that happen with any of my OpenSSL builds.

> I simply don't believe I'm the only one who wants to build OpenSSL for use in a
> Windows 10 environment - someone must have been successful and be able to point
> me to additional information.

That's good, because that would be a silly thing to believe. We have multiple teams that routinely build OpenSSL for Windows (and other platforms). I've built it myself any number of times. We build both 32- and 64-bit variants, without problems.

My suggestion: Follow the recommended process by installing nasm (it's free) and configuring for that instead of MASM. As Matt suggested, start from a "Visual Studio Command Prompt" window for the bitness you want to build, and then make sure you configure for that bitness. Get rid of any existing build artifacts first - probably the easist way is to move your working directory aside and extract a fresh one from the tarball.

As Matt noted, you can confirm whether you're using the 32- or 64-bit version of the Visual Studio toolchain just by running cl.exe with no parameters; the banner says "for x86" or "for x64". Note that it's important to let Visual Studio set up the environment (using one of those "Visual Studio Command Prompt" options) rather than trying to set it up manually - the latter is possible (I do it for my Cygwin bash sessions), but it's a pain to get it right.

> I messed around quite a bit trying to figure out how I might modify the build
> process to work with my usual set of tools - mostly UNIX-tools ported to Windows
> environment.  My preference would be to find pointers to information of how to
> accomplish that.

Unless your usual set of tools matches what OpenSSL wants for the MinGW+MSYS2 configuration option, I'd strongly recommend not trying to do this unless and until you understand the OpenSSL build process very well.

Regarding MSYS2 and Perl:

- Did you read the updated NOTES.WIN that Matt posted a link to?

- Perl implementations for Windows vary widely in their idiosyncracies, and OpenSSL is finicky about them. For years we used Cygwin Perl, but had to run it under a wrapper program which altered the command-line arguments. These days, we use Strawberry Perl. ActiveState Perl has also worked, but has licensing issues when used to build commercial software; that probably doesn't apply in your case. Note Strawberry and ActiveState have Windows-native behavior, and thus can be expected to work for the Visual Studio configurations but probably not for the MSYS2 configurations.
   In my experience, getting Perl to cooperate is the biggest hurdle in building OpenSSL on Windows, and historically for us has been the most fragile aspect. As the OpenSSL build process has made heavier use of Perl, it's become more picky about getting a Perl it likes. (This is probably my least favorite aspect of OpenSSL. People complain about the OpenSSL source, but I'd much rather debug that than the Perl build scripts.)

- You haven't told us what Perl implementation you're using. Knowing the path to it doesn't help us. (Though, to be honest, knowing which implementation it is probably isn't terribly useful either.) It would be nice if Sergio told us which one *he's* using, since it's known to work with the MSYS2 configurations.

- I don't know whether there are Perl implementations for Windows specifically intended to be used with MinGW or MSYS2, and if so whether they work for the OpenSSL build. Oh, wait: I just did a quick search, and apparently there is an MSYS2 Perl package available from repo.msys2.org. Does it work with the OpenSSL build? I don't know, but it seems like it's worth a try.

--
Michael Wojcik
Distinguished Engineer, Micro Focus