"no OPENSSL_Applink " in OpenSSL3.0.5

[Imazu Setsuo]

On 2022/08/10 18:44, Matt Caswell wrote:
> 
> 
> On 10/08/2022 00:11, Imazu Setsuo wrote:
>> Hello, my name is Imazu.
>> I am using OpenSSL3.0.5 to develop a windows program.
>> When I call PEM_read_PrivateKey() I get the following error:
>> OPENSSL_Uplink(00007FF8011DD3E0,08): no OPENSSL_Applink
>>
>> The source that calls PEM_read_PrivateKey() includes the following sources.
>>
>> #include <openssl/applink.c>
>>
>> In the test program
>> test.exe --> my_appl.dll --> openssl.dll
>> , the above include is the source in my_appl.dll.
>> The depends tool (Dependency Walker) can check the entry of
>> OPENSSL_Applink.
>>
>> By the way, if you describe "#include <openssl/applink.c>" in the source
>> inside test.exe, it will work.
>> However, I want to complete the processing within my_appl.dll.
>> Please tell me a good way.
>>
>> Best regards.
>>
> 
> The problem comes when your code opens a FILE* and then passes it for
> use to OpenSSL. If your code and OpenSSL are using a different C runtime
> then that's going to fail.
> 
> A possible workaround is not call PEM_read_PrivateKey() at all and
> instead use PEM_read_bio_PrivateKey(). You can then (for example) load
> the file into memory completely within your code and store the data in a
> mem BIO (e.g. using BIO_new_mem_buf()). This removes the need to pass
> FILE * pointers from one dll to another and avoids the problem.
> 
> Matt

hello.
After fixing it with your advice, it worked without any errors.  Thank you.

I've been thinking about this for a while, what do you think of the following?

The "HMODULE hModule" that can be referenced by DllMain is retained in the variable (BASE_HANDLE) when the "DLL_PROCESS_ATTACH" condition occurs.

Use the retained value instead of the return value of GetModuleHandle(NULL) in uplink.c.
------
          if ((h = apphandle) == NULL) {
       　　h=BASE_HANDLE;
          
              if ( (NULL == h) && (h = GetModuleHandle(NULL)) == NULL) {
                  apphandle = (HMODULE) - 1;
                  _tcscpy(msg + len, _T("no host application"));
                  break;
              }
              apphandle = h;
          }
------
I don't know the detailed behavior of the DLL, so I'm not sure if it will work.

Imazu