server key exchange signature behavior

[Jan Just Keijser]

On 25/06/20 20:02, Bruce Cloutier wrote:
> I agree that I am not being explicit regarding my terminology. I don't
> mean to confuse. I just cannot get anywhere on this in a vacuum. So, I
> need to reach out.
>
> Specifically, the Signature covering the EC Diffe-Hellman Server Params
> in the server_key_exchange message that I eventually receive in making
> an outgoing client connection from my TLS implementation, does not
> decrypt into a valid PKCS1 ASN.1 structure (or anything recognizable).
> That is happening with this one server installation using Apache in a
> Bitnami stack (Windows machine). My TLS code is running as part of the
> operating system on our separate single board computer controller (JNIOR).
>
> I use the public key extracted from the supplied certificate to decrypt
> and then compare the calculated hash. This procedure has been successful
> (and signatures verify/validate/match) in every other connection attempt
> to other servers (including google.com). It also works with this Apache
> server before we move away from the default key and certificate files.
> Basically, our server is using some other key for this. Maybe still the
> default. After we point configuration to a new certificate and key.
>
> Yeah, I had at first thought that there was an intrusion. But in testing
> with a clean sandboxed server we see the same results. Yes, it is most
> likely an Apache configuration issue but, we've dug through all of that
> and nothing jumps out.
>
> So either we haven't configured every possible corner of this Apache
> server correctly, or there is some odd extension to TLS that I've
> missed. But as you all know, debugging in this area is difficult.
>
> jnior.com is obviously public and we need to know if we have it
> improperly configured. But, I need to know why in this one instance my
> TLS implementation fails. In either case, it is an symptom that
> shouldn't be ignored as you would agree.
>
> The good news is that if there were an OpenSSL bug somewhat associated
> with this, you all would have already mentioned it (I assume).
>

I'd suggest to run something like wireshark and capture the TLS 
handshake - if I do that against jnior.com I see that the jnior.com 
certificate has a X509 cert extension unknown to my rather old Wireshark 
version, but that should not matter much. Wireshark will give you 
detailed info about the EC Diffie-Hellman Server Params that were sent 
over the wire, including the signature. Wireshark does not indicate any 
problems there, although I am not sure if it *would* , if there was a 
problem.

However, it will give you an easy method to see what wireshark thinks 
the signature is compared to what your code thinks the signature is.

As a side-remark: I did notice your server cert is using 3072 bit RSA 
whereas the CAs are based on 2048bit RSA - nothing wrong with that, but 
do make sure your code can handle that (under normal circumstances it 
would not be an issue).

HTH,

JJK

> On 6/25/20 1:32 PM, Michael Wojcik wrote:
>>> From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of
>>> Bruce Cloutier
>>> Sent: Thursday, June 25, 2020 12:10
>>>
>>> By "If OpenSSL fails to validate this particular digital signature that
>>> would be the case." I meant to question whether or not OpenSSL is in
>>> fact doing the validation? In the case that the signature is being
>>> ignored then clients wouldn't complain. They wouldn't notice.
>> s_client should be verifying the signature.[1] That is, it should be verifying every signature that's part of the actual TLS protocol. I admit it's not entirely clear to me which signature isn't being verified successfully by your client.
>>
>>
>> [1] I'm not sure "validate" is the proper term here, technically speaking. In my experience, the literature usually uses "verify" for confirming a signature. "Validate" is generally used for more complex protocols, such as certificate validation, which involves a large number of steps with various types of checks.
>>
>> --
>> Michael Wojcik
>> Distinguished Engineer, Micro Focus
>>
>>
>>