<div dir="ltr"><div dir="ltr"><div dir="auto">Hi Matt,<div dir="auto"> </div><div dir="auto"> Thanks for your update. Will try to implement one scheme using the same. </div><div dir="auto"><br></div><div dir="auto"> I have one doubt regarding the points which we have discussed earlier. </div><div dir="auto"><br></div><div dir="auto"> I am now facing an issue of getting an invalid waitctx address while trying to get it using <span style="font-family:sans-serif">ASYNC_get_wait_ctx(). I have checked using gdb, in which got memory inaccessible print when I tried to dump the waitctx structure. If I put some delay in the engine, things are working fine, ie. no corruption of waitctx structure address.</span></div><div dir="auto"><span style="font-family:sans-serif"><br></span></div><div dir="auto"> But I am just copying the address of job structure before the ASYNC_pause(). After the HW operation is completed I will use this address to get waitctx and fd using ASYNC_get_wait_ctx() and ASYNC_WAIT_CTX_get_fd() from the thread which I have created from the engine. I haven't access the job address directly from the thread. <br></div><div dir="auto"><br></div><div dir="auto"> <span style="font-family:sans-serif">I know you have already told me that I should be careful while handling ASYNC_JOB structure from outside threads. But I have only a little experience on handling these ASYNC_JOB structures. If you can provide a solution for this, it would be helpful.</span></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 14 Jan 2019 5:33 pm, "Matt Caswell" <<a href="mailto:matt@openssl.org" target="_blank">matt@openssl.org</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
<br>
On 13/01/2019 04:36, Ananthu Unnikrishnan wrote:<br>
> Hi Matt,<br>
> <br>
> Thanks a lot for the response. The alternative method to resume job<br>
> operation is a good one. <br>
> <br>
> I need some more clarifications regarding the async job operation.<br>
> Please help.<br>
> In ssl/ss_lib.c, ssl_start_async_job() function found. In which<br>
> ASYNC_start_job function is calling but no mechanism found for polling the fd's.<br>
> Is it written elsewhere?. <br>
<br>
You are supposed to write the fd polling code yourself, e.g. by using<br>
SSL_get_all_async_fds() or SSL_get_changed_async_fds() to get hold of the fds,<br>
and then followed by "select" or similar.<br>
<br>
See:<br>
<br>
<a href="https://www.openssl.org/docs/man1.1.1/man3/SSL_get_all_async_fds.html" rel="noreferrer" target="_blank">https://www.openssl.org/docs/man1.1.1/man3/SSL_get_all_async_fds.html</a><br>
<br>
mATT<br>
<br>
<br>
<br>
> <br>
> <br>
> On Fri, Jan 11, 2019 at 3:12 PM Matt Caswell <<a href="mailto:matt@openssl.org" target="_blank">matt@openssl.org</a><br>
> <mailto:<a href="mailto:matt@openssl.org" target="_blank">matt@openssl.org</a>>> wrote:<br>
> <br>
> <br>
> <br>
> On 10/01/2019 18:09, Ananthu Unnikrishnan wrote:<br>
> > Hi Matt,<br>
> ><br>
> > Thanks a lot for the reply.<br>
> ><br>
> > After calling ASYNC_pause_job() from the engine, control will<br>
> transfer to<br>
> > the place where we start the ASYNC_start_job right? So how can we write<br>
> the code<br>
> > to put a trigger on fd in the same thread?<br>
> <br>
> I'm not saying you can't use threads to do this - only that you need to be<br>
> careful about using them and must obey the rules.<br>
> <br>
> <br>
> > If I am wrong please correct me.<br>
> > Also if u can suggest where resume operation should be done, it would be<br>
> helpful.<br>
> <br>
> Typically the calling application will wait on the fd (e.g. using "select" or<br>
> similar) until it is readable. Once it sees the fd as readable then it calls<br>
> ASYNC_start_job() again.<br>
> <br>
> <br>
> As an aside you may also be interested in this PR that is currently being<br>
> reviewed:<br>
> <br>
> <a href="https://github.com/openssl/openssl/pull/7573" rel="noreferrer" target="_blank">https://github.com/openssl/openssl/pull/7573</a><br>
> <br>
> This adds an alternative mechanism for signalling other than using fds (i.e. to<br>
> use a callback instead) and will be available in OpenSSL 3.0 when it gets<br>
> released.<br>
> <br>
> Matt<br>
> <br>
> <br>
> <br>
> ><br>
> > On Thu, Jan 10, 2019 at 10:11 PM Matt Caswell <<a href="mailto:matt@openssl.org" target="_blank">matt@openssl.org</a><br>
> <mailto:<a href="mailto:matt@openssl.org" target="_blank">matt@openssl.org</a>><br>
> > <mailto:<a href="mailto:matt@openssl.org" target="_blank">matt@openssl.org</a> <mailto:<a href="mailto:matt@openssl.org" target="_blank">matt@openssl.org</a>>>> wrote:<br>
> ><br>
> ><br>
> ><br>
> > On 10/01/2019 09:39, Ananthu Unnikrishnan wrote:<br>
> > > Hi all,<br>
> > ><br>
> > > We are not able to access the waitctx address from the job<br>
> address<br>
> > using<br>
> > > ASYNC_get_wait_ctx(job) from a thread which starts in the bind<br>
> section of the<br>
> > > dynamic engine. The job address is the same as that we got<br>
> > > using ASYNC_get_current_job. Can anyone help on this?<br>
> ><br>
> > It's very unclear from the your various emails what you are trying to<br>
> achieve<br>
> > and the problem that you are experiencing.<br>
> ><br>
> > From the above it sounds like you are starting threads from inside<br>
> your dynamic<br>
> > engine? ASYNC_JOBs are always local to a thread. When a job is started<br>
> it is<br>
> > associated with the current thread. So ASYNC_get_current_job() returns<br>
> the job<br>
> > associated with the current thread. Similarly if you pause a job,<br>
> > ASYNC_pause_job() must be called from that same thread. Finally if you<br>
> restart a<br>
> > paused job it must also be restarted on the same thread.<br>
> ><br>
> > If you are starting new threads from within your engine then extreme<br>
> care must<br>
> > be taken to obey the above rules - otherwise you are likely to get strange<br>
> > results.<br>
> ><br>
> > Matt<br>
> ><br>
> ><br>
> > ><br>
> > > <br>
> > ><br>
> > > On Tue, Jan 8, 2019 at 11:26 AM Ananthu Unnikrishnan<br>
> > <<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>><br>
> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>>><br>
> > > <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>><br>
> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>>>>> wrote:<br>
> > ><br>
> > > Can anyone please help on this? If u need any additional information<br>
> > please<br>
> > > let me know.<br>
> > ><br>
> > > On Mon, Jan 7, 2019 at 6:25 PM Ananthu Unnikrishnan<br>
> > <<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>><br>
> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>>><br>
> > > <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>><br>
> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>>>>> wrote:<br>
> > ><br>
> > > Hi all,<br>
> > ><br>
> > > Adding more details to the previous mail. We have edited the<br>
> OpenSSL<br>
> > > code for implementing the polling for changed fd's as in<br>
> OpenSSL speed<br>
> > > command. Attached the code snippet of the same along with<br>
> this mail.<br>
> > > Mentioned below some observations which found doubtful:<br>
> > ><br>
> > > 1) We have got prints in ASYNC_FINISH case, before getting<br>
> print in<br>
> > > ASYNC_PAUSE case. Attaching the log also, so you can understand<br>
> > easily. <br>
> > > 2) Shown below the code snippet which we have written to<br>
> provide a<br>
> > write<br>
> > > on fd.<br>
> > ><br>
> > > if ((waitctx = ASYNC_get_wait_ctx((ASYNC_JOB *)job)) == NULL) {<br>
> > > printf("In dynamic engine | waitctx == NULL : %d\n",<br>
> > __LINE__);<br>
> > > return ret;<br>
> > > }<br>
> > ><br>
> > > printf("\n----- In dynamic engine | After pausing | job<br>
> is %lx | <br>
> > > waitctx in resume job %lx |-----\n", job, waitctx);<br>
> > ><br>
> > > if ((ret = ASYNC_WAIT_CTX_get_fd(waitctx, engine_id,<br>
> &efd,<br>
> > > &custom)) > 0) {<br>
> > > if (write(efd, &buf, sizeof(uint64_t)) == -1) {<br>
> > > printf("\nFailed to write\n");<br>
> > > }<br>
> > > }<br>
> > ><br>
> > > Here waitctx is getting NULL when we tried to fetch waitctx<br>
> using<br>
> > > ASYNC_get_wait_ctx(). We have printed data of sizeof(ASYNC_JOB)<br>
> > ie.1176<br>
> > > bytes contained in the job address from the engine, we got<br>
> > correct data<br>
> > > before calling ASYNC_pause() and but got half data zero data<br>
> when we<br>
> > > tried to access the same address after ASYNC_pause(). <br>
> > ><br>
> > > Attaching both job structure contents before and after calling<br>
> > > ASYNC_pause(). Also, we got the correct data in the same address<br>
> > when we<br>
> > > have printed from ssl_start_async_job() before polling starts,<br>
> > where we<br>
> > > have started the async_job<br>
> > > . <br>
> > > The prints starting with "In dynamic engine" are prints<br>
> inside the<br>
> > > engine, rest prints are from ss/ssl_lib.c. Kindly check this and<br>
> > please<br>
> > > point out if anything is wrong somewhere. Thanks in advance.<br>
> > ><br>
> > ><br>
> > > On Sun, Jan 6, 2019 at 10:30 AM Ananthu Unnikrishnan<br>
> > > <<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>><br>
> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>>><br>
> > <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>><br>
> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a> <mailto:<a href="mailto:ananthuu741@gmail.com" target="_blank">ananthuu741@gmail.com</a>>>>> wrote:<br>
> > ><br>
> > > Hi all,<br>
> > ><br>
> > > We have implemented a dynamic engine and tested<br>
> in the<br>
> > async<br>
> > > mode using OpenSSL speed command. But in a real network<br>
> > scenario, we<br>
> > > have seen only starting the async_job(in file ssl/ssl_lib.c,<br>
> > > function: ssl_start_async_job) in the OpenSSL. We<br>
> haven't seen<br>
> > > polling async_fd's for resuming the job as in<br>
> apps/speed.c(in<br>
> > speed<br>
> > > command case).<br>
> > > So can anyone please suggest any solution for<br>
> resuming the<br>
> > > async_jobs in real network scenario? Thanks in advance.<br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > > --<br>
> > > With best Regards, <br>
> > ><br>
> > > Ananthu<br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > > --<br>
> > > With best Regards, <br>
> > ><br>
> > > Ananthu<br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > > --<br>
> > > With best Regards, <br>
> > ><br>
> > > Ananthu<br>
> > ><br>
> > ><br>
> > ><br>
> > --<br>
> > openssl-users mailing list<br>
> > To unsubscribe: <a href="https://mta.openssl.org/mailman/listinfo/openssl-users" rel="noreferrer" target="_blank">https://mta.openssl.org/mailman/listinfo/openssl-users</a><br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > With best Regards, <br>
> ><br>
> > Ananthu<br>
> ><br>
> ><br>
> ><br>
> -- <br>
> openssl-users mailing list<br>
> To unsubscribe: <a href="https://mta.openssl.org/mailman/listinfo/openssl-users" rel="noreferrer" target="_blank">https://mta.openssl.org/mailman/listinfo/openssl-users</a><br>
> <br>
> <br>
> <br>
> -- <br>
> With best Regards, <br>
> <br>
> Ananthu<br>
> <br>
> <br>
> <br>
-- <br>
openssl-users mailing list<br>
To unsubscribe: <a href="https://mta.openssl.org/mailman/listinfo/openssl-users" rel="noreferrer" target="_blank">https://mta.openssl.org/mailman/listinfo/openssl-users</a><br>
</blockquote></div></div>