OpenSSL 1.1.1 Windows dependencies

[David Harris]

On 20 Oct 2022 at 20:04, Michael Wojcik wrote:

> OpenSSL 1.1.1 uses Windows cryptographic routines in two areas I'm
> aware of: rand_win.c and the CAPI engine. I don't offhand see a way
> that a problem with the calls in rand_win.c would cause the particular
> symptom you described. My guess is that you're not using the CAPI
> engine, but you might check your OpenSSL configuration on the failing
> system.

For a variety of reasons to do with redistributables, I build OpenSSL as 
no-shared, and because of the compiler I prefer to use (an older build of Visual 
C), I have to compile with no-capi as well, so CAPI clearly isn't an issue in this 
case. But to be sure, I tried rebuilding OpenSSL with Visual C 2022 (using 
Visual C 2019 as the compile unit) and according to the customer, the result 
was the same.

> I think more plausible causes of this failure are things like OpenSSL
> configuration and interference from other software such as an endpoint
> firewall. Getting SYSCALL from SSL_accept *really* looks like
> network-stack-level interference, from a firewall or similar
> mechanism.

That was my initial thought too, except that if it were firewall-related, the initial 
port 587 connection would be blocked, and it isn't - the failure doesn't happen 
until after STARTTLS has been issued. Furthermore, the OpenSSL 
configuration is identical between the systems/combinations of OpenSSL that 
work and those that don't.

> Personally, if I ran into this, I'd just build OpenSSL for debug and
> debug into it. But I know that's not everyone's cup of tea.

Unfortunately, I don't have that level of access to the customer's systems. 

I was really just wondering if the combination of factors rang any bells with 
anyone before I started digging much deeper; it's altogether possible that I might 
just have to write this one off to experience and tell the user to use a 1.1.1g build 
of OpenSSL (which I build exactly the same way, and which works correctly in 
the same setup).

Thanks for the help - appreciated.

Cheers!

-- David --