<div dir="ltr"><div class="gmail_default" style="font-family:verdana,sans-serif">I checked this further and the issue was indeed with my code.. I did some recent async io completion handler refactor because of which the bio's socket write completion was triggering the observer's read completion callback.. the records were actually client side write bio buffers which got mixed with the ssl_read data on the read path. Thanks for the pointer, we are doing good. Nothing further is needed at this point.</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">Neelabh</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Nov 24, 2022 at 4:51 PM Neelabh Mam <<a href="mailto:neelabh.mam@gmail.com">neelabh.mam@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div class="gmail_default"><font face="monospace">I hook an observer for </font><span style="font-family:monospace">decrypted data</span><font face="monospace"> immediately after the handshake is successful (SSL_do_handshake rc 1) and it is this observer which gets the ccs+list data on the vert next ssl_read cycle. Now, it could be that my code is at fault here.. But I do see the decrypted dummy ccs and one more record as part of the list data. Annotated hex dump of the buffer reproduced below. Btw this is against the latest FZ windows FTPS server which I think recently moved to GNUtls. </font><font face="monospace">As suggested earlier I'll check my code aga</font><span style="font-family:monospace">in and open a github ticket. </span></div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default"><font face="monospace">Command : PASV<br>sock-cc 2570925053120 OnWrite, n 29<br>sock-cc 2570925053120 OnRead, n 273<br><b style="background-color:rgb(182,215,168)">New session callback 1</b><br>sock-cc 2570925053120 OnRead, n 68<br>Response : 227 Entering Passive Mode (127,0,0,1,249,75)<br>Command : LIST /<br>sock-dc 2570925624800 OnConnect, n 0<br>sock-dc OnConnect()<br>sock-cc 2570925053120 OnWrite, n 30<br>sock-cc 2570925053120 OnRead, n 51<br>Response : 150 Starting data transfer.<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 32<br>sock-dc 2570925624800 OnWrite, n 10<br>sock-dc 2570925624800 OnRead, n 133<br>sock-dc Handshake failed : -1 2<br>sock-dc 2570925624800 OnRead, n 108<br><b style="background-color:rgb(182,215,168)">sock-dc Handshake done. TLS session reused : 1</b><br>sock-dc 2570925624800 OnWrite, n 32<br>observer OnDataChannelIoCompletion write<br>sock-dc 2570925624800 OnWrite, n 32<br>observer OnDataChannelIoCompletion write<br>sock-dc 2570925624800 OnWrite, n 16<br>observer OnDataChannelIoCompletion write<br>sock-dc 2570925624800 OnRead, n 86<br>observer OnDataChannelIoCompletion read<br>sock-cc 2570925053120 OnRead, n 48<br>Response : 226 Operation successful<br><br>sock-dc 2570925624800 OnRead, n 816<br>sock-dc SSL_RECEIVED_SHUTDOWN, flags : 2<br>sock-dc : ssl_shutdown() rc : 1<br>sock-dc SSL_RECEIVED_SHUTDOWN, flags : 3<br>sock-dc : shutdown(sd_send)<br>observer OnDataChannelIoCompletion read<br>sock-dc 2570925624800 OnRead, n 0<br>sock-dc shutdown mode : 3<br><span style="background-color:rgb(234,153,153)">¢â╣ª5d²┼l⌐\╒═£t</span><span style="background-color:rgb(255,229,153)">drwxrwxrwx 1 ftp ftp 0 Nov 01 08:55 $RECYCLE.BIN<br>-rw-rw-rw- 1 ftp ftp 1468320 Oct 15 17:37 accesschk.exe<br>-rw-rw-rw- 1 ftp ftp 810416 Oct 15 17:37 accesschk64.exe<br>-rw-rw-rw- 1 ftp ftp 264088 Oct 15 17:37 AccessEnum.exe<br>-rw-rw-rw- 1 ftp ftp 2502032 Oct 15 17:37 Autoruns.exe<br>-rw-rw-rw- 1 ftp ftp 2928520 Oct 15 17:37 Autoruns64.exe<br>-rw-rw-rw- 1 ftp ftp 712080 Oct 15 17:37 autorunsc.exe<br>-rw-rw-rw- 1 ftp ftp 788400 Oct 15 17:37 autorunsc64.exe<br>drwxrwxrwx 1 ftp ftp 0 Nov 06 16:52 System Volume Information<br>-rw-rw-rw- 1 ftp ftp 5 Nov 24 02:26 x.txt</span><br></font><pre style="color:rgb(0,0,0)">14 TLS Change Cipher Spec protocol
03 03 SSL 3.3 (TLS 1.2)
00 01 length of change cipher spec message
01 payload (irrelevant)</pre><pre style="color:rgb(0,0,0)"><pre>17 TLS application data
03 03 version 1.2
00 45 length of payload (69 bytes)</pre></pre><font face="monospace"><span style="background-color:rgb(159,197,232)"> 14 03 03 00 01 01</span> <span style="background-color:rgb(217,210,233)"><u>17 03 03 00 45</u> 4b 60 8c 72 be<br> 99 60 ed da 3a a0 09 9c 35 24 a5 3d 48 b5 78 bc<br> 2e 2c c3 e7 19 b4 74 e9 7b 70 90 d4 17 d4 2a 46<br> 8c e7 1a 94 6b c8 ea 54 b2 72 8b 03 8e cb 0d 9b<br> 83 b9 a6 35 64 fd c5 55 08 6c a9 5c d5 cd 9c 74</span><br><span style="background-color:rgb(255,229,153)"> 64 72 77 78 72 77 78 72 77 78 20 31 20 66 74 70<br> 20 66 74 70 20 20 20 20 20 20 20 20 20 20 20 20<br> 20 20 20 30 20 4e 6f 76 20 30 31 20 30 38 3a 35<br> 35 20 24 52 45 43 59 43 4c 45 2e 42 49 4e 0d 0a<br> 2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74 70<br> 20 66 74 70 20 20 20 20 20 20 20 20 20 31 34 36<br> 38 33 32 30 20 4f 63 74 20 31 35 20 31 37 3a 33<br> 37 20 61 63 63 65 73 73 63 68 6b 2e 65 78 65 0d<br> 0a 2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74<br> 70 20 66 74 70 20 20 20 20 20 20 20 20 20 20 38<br> 31 30 34 31 36 20 4f 63 74 20 31 35 20 31 37 3a<br> 33 37 20 61 63 63 65 73 73 63 68 6b 36 34 2e 65</span></font><br></div></div><div class="gmail_default" style="font-family:verdana,sans-serif"> :</div><div class="gmail_default" style="font-family:verdana,sans-serif"> : goes on full list</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Nov 24, 2022 at 3:15 PM Matt Caswell <<a href="mailto:matt@openssl.org" target="_blank">matt@openssl.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
<br>
On 24/11/2022 07:57, Neelabh Mam wrote:<br>
> Hi,<br>
> <br>
> With my openssl based FTPS client (non-blocking bio) targeting TLS1.3, I <br>
> see that immediately after a successful data channel handshake (with <br>
> session reuse), a dummy change_cipher_spec record and a non-application <br>
> data record are sent as part of the directory listing data. Same holds <br>
> true for file downloads as well..(again with session reuse). This seems <br>
> to be in line with the last paragraph in Appendix D.4 of RFC8446 which <br>
> mandates a change_cipher_spec record to be sent from the server in case <br>
> of session reuse. I could manually filter out the non-application data <br>
> records from the actual data.. However, I was wondering if there's some <br>
> api/setting in openssl that would do that at an API level.. as this <br>
> looks like it should ideally be part of openssl workflow. But then what <br>
> I am also not sure about is why the receipt of new session ticket data <br>
> on the control channel does not bubble up as application data ? Openssl, <br>
> from what I can tell, seems to be "consuming" it (new session data) <br>
> internally. The earlier 2 messages however, do get exposed out of <br>
> ssl_read and eventually become part of application data.<br>
<br>
The only data you should be getting out of an `SSL_read` call is <br>
application data. If you're getting handshake or CCS messages as part of <br>
app data then something has gone very surprisingly wrong. I cannot <br>
imagine what would cause that.<br>
<br>
You might want to open a github issue about that to dive into it in more <br>
detail.<br>
<br>
Matt<br>
<br>
<br>
> Is it like : <br>
> the message type warrants one to be exposed (change_cipher_spec) and the <br>
> other to be handled internally (new session data) ? Could we please <br>
> advise on openssl's standard operating workflow here ? Also, would I <br>
> have to add logic to manually separate these 2 non-application records <br>
> from the application data ? Is that how it is supposed to be ? Thank you.<br>
> <br>
> Neelabh<br>
</blockquote></div></div>
</blockquote></div>