<div dir="ltr">Improved testing to me is something that is a good thing - and a value judgement.<div>It doesn't change libcrypto or libssl - and that to me is the way I think of it.</div><div>Fixing tests and apps and Makefiles to me are different from adding features to libcrypto or libssl.</div><div><br></div><div>On this one - the fuzz testing has been sufficiently slow to reduce its usefulness - and this is a step in the right direction.</div><div><br></div><div>It is however also a bit outside of our current policy on such things - so perhaps we need to update that.</div><div><br></div><div>Tim.</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 29, 2018 at 11:45 PM, Richard Levitte <span dir="ltr"><<a href="mailto:levitte@openssl.org" target="_blank">levitte@openssl.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">In message <<a href="mailto:e32db73b-b91d-d3c2-274b-6e13e969af34@openssl.org">e32db73b-b91d-d3c2-274b-<wbr>6e13e969af34@openssl.org</a>> on Thu, 29 Mar 2018 14:03:06 +0100, Matt Caswell <<a href="mailto:matt@openssl.org">matt@openssl.org</a>> said:<br>
<br>
matt><br>
matt><br>
matt> On 29/03/18 14:00, Salz, Rich wrote:<br>
matt> > Please see <a href="https://github.com/openssl/openssl/pull/5788" rel="noreferrer" target="_blank">https://github.com/openssl/<wbr>openssl/pull/5788</a><br>
matt> ><br>
matt> > I don’t think it is, but I’d like to know what others think.<br>
matt><br>
matt> I do think this should be applied. The tests in question are not just<br>
matt> slow but *really* slow to the point that I often exit them before they<br>
matt> have completed. This removes the benefits of having the tests in the<br>
matt> first place. From that perspective I view this as a bug fix.<br>
<br>
Something to remember is that no user will ever complain about this,<br>
because we don't deliver the contents of fuzz/corpora in our tarballs.<br>
<br>
In other words, this is a developer only change of our current tests,<br>
and you will only hear from developers who do engage in fuzz testing,<br>
i.e. those who do these tests as part of a release, just to pick a<br>
very recent example.<br>
<br>
Also, you may note that this test re-engages fuzz testing as part of<br>
our normal tests that are run for every PR, which means that we will<br>
catch errors that the fuzzers can detect much earlier. Because the<br>
fuzz testing took so long time, we had them only engaged with<br>
[extended tests], something that's almost never used.<br>
<br>
So I would argue that faster fuzz testing means more fuzz testing, and<br>
hopefully better testing of stuff that's harder to catch otherwise.<br>
<br>
Cheers,<br>
Richard ( plus, from a very personal point of view, it's *my* time,<br>
and Matt's, and whoever else's who tests for releases, that<br>
gets substantially less wasted! )<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Richard Levitte <a href="mailto:levitte@openssl.org">levitte@openssl.org</a><br>
OpenSSL Project <a href="http://www.openssl.org/~levitte/" rel="noreferrer" target="_blank">http://www.openssl.org/~<wbr>levitte/</a><br>
</font></span><div class="HOEnZb"><div class="h5">______________________________<wbr>_________________<br>
openssl-project mailing list<br>
<a href="mailto:openssl-project@openssl.org">openssl-project@openssl.org</a><br>
<a href="https://mta.openssl.org/mailman/listinfo/openssl-project" rel="noreferrer" target="_blank">https://mta.openssl.org/<wbr>mailman/listinfo/openssl-<wbr>project</a></div></div></blockquote></div><br></div>