[openssl-dev] [openssl.org #4156] Wrong expectation of value of _IOB_ENTRIES with Visual Studio 2015

[Andy Polyakov via RT]

Hi,

> when using OpenSSL (1.0.1p) in our applications, we get the error 'No
> OPENSSL_Applink' when calling PEM_read_X509_CRL. We build both OpenSSL
> and the application using the same C runtime configuration (/MDd), using
> Visual Studio 2015 on Windows 10.

Original intention with OPENSSL_Applink was to facilitate certain things
for old code. This kind of implies that new code should be written in
such manner that would *avoid* Applink. For this reason I would
recommend to modify application and switch to PEM_read_bio_X509_CRL. But
the keyword is that you should let OpenSSL open the file for you by
calling BIO_new_file. As opposite to opening yourself and calling
BIO_new_fp that is.

> After some hours of debugging and comparing with the same scenario using
> Visual Studio 2010, I discovered that _IOB_ENTRIES is defined
> differently in the two versions of Visual Studio:
> 
> VS2015: c:\Program Files (x86)\Windows
> Kits\10\Include\10.0.10150.0\ucrt\stdio.h: #define _IOB_ENTRIES 3
> VS2010: c:\Program Files (x86)\Microsoft Visual Studio
> 10.0\VC\include\stdio.h: #define _IOB_ENTRIES 20
> 
> Now there's code in openssl-1.0.1p\crypto\bio\bss_file.c that seems to
> expect _IOB_ENTRIES to have the value 20:
> 
> #   if defined(__MINGW32__) && defined(__MSVCRT__) && !defined(_IOB_ENTRIES)
> #    define _IOB_ENTRIES 20
> #   endif
> #   if defined(_IOB_ENTRIES)
>         /* Safety net to catch purely internal BIO_set_fp calls */
>         if ((size_t)ptr >= (size_t)stdin &&
>             (size_t)ptr < (size_t)(stdin + _IOB_ENTRIES))
>             BIO_clear_flags(b, BIO_FLAGS_UPLINK);
> #   endif
> #  endif
> #  ifdef UP_fsetmod
>         if (b->flags & BIO_FLAGS_UPLINK)
>             UP_fsetmod(b->ptr, (char)((num & BIO_FP_TEXT) ? 't' : 'b'));
>         else
> #  endif
> 
> The 'safety net' does not work here, as _IOB_ENTRIES is too small in
> VS2015 and the flags don't get cleared (as for VS2010), so UP_fsetmod is
> being called which leads to the 'No OPENSSL_Applink' error.

Well, commentary is probably misleading in the context, because "purely
internal BIO_set_fp" refers to BIO_set_fp called with FILE * created
internally, rather than just any BIO_set_fp called internally. So that
if application passes FILE * it created itself, it should expect UPLINK
calls. The fact that it worked for you without applink.c in VS2010 is
circumstantial.

> We kindly ask you to provide a solution for our problem, as we currently
> are not able to use OpenSSL in combination with VS2015. Linking the
> applink.c into our applications is no option for us, as we would have to
> deploy multiple C runtimes (one for OpenSSL and one for the application).

Last sentence is false in sense that linking applink.c into application
does *not* force you into using multiple C runtimes. It allows to use
multiple runtimes, but does not prohibit/prevent using single one. In
other words if this is the only reason for not linking the applink.c,
then it should be disregarded. But, once again, the right thing to do is
to adhere to PEM_read_bio_X509_CRL. This applies to all interfaces that
accept FILE *.