[openssl-dev] who wants to fix travis builds?

[Alessandro Ghedini]

On Fri, Oct 16, 2015 at 10:56:43am +0200, Andy Polyakov wrote:
> >>>> I've opened the following PR to add support for GCC v5 and
> >>>> address sanitizer (not sure if we want valgrind as well...): 
> >>>> https://github.com/openssl/openssl/pull/429
> > 
> > I've commented there on other -fsanitize options as well as about
> > option to execute them without --debug and/or --strict-warnings. But I
> > failed to recall all the details from last time the question was risen
> > in context of RT#3422 through 3324. -fsanitize=undefined is expected
> > to work only with --strict-warnings (or more specifically with
> > -DPEDANTIC) and I wouldn't be surprised if -fsanitize=memory works
> > only with -DPURIFY.
> 
> The main weakness of the endeavour is that testing is limited to x86
> hardware. I don't mean specifically -fsanitize, but in general. But
> let's concentrate on -fsanitize for a moment. Or more specifically why
> are there limitations for it, ones mentioned above. In the nutshell,
> remaining -fsanitize warnings were effectively about the fact that we
> sometimes rely on platform- or implementation-specific behaviour. But it
> is *conscious* choice and there always is platform-neutral fallback. So
> that above mentioned limitations are actually about nothing else but
> driving compiler toward these platform-neutral fallbacks. One can even
> take next step and say that in order to maximize -fsanitize utility, one
> should expose as much C code to -fsanitize as possible, or in other
> words adhere even to no-asm with -fsanitize. Incidentally this would
> also at least partially mitigate the main weakness, as -fsanitize should
> expose spots where platform-neutral fallback is not as platform-neutral
> as it should be, wouldn't it?

FWIW both gcc and clang support some sort of blacklist to exclude particular
functions from sanitizers. This would allow us to mark special cases manually
in order to avoid missing accidental errors in non-debug builds... but that's
work for some other time.

For now, I added ubsan debug-only builds for Travis at [0], and fixed a couple
of very minor issues (a test called memcmp() with a NULL pointer, and there was
a missing, but probably irrelevant, cast). I'm not sure if the fixes are
correct or not though.

Sort of related: the usefulness of sanitizers is directly related to the
coverage of the test suite, which, according to gcov, is currently around 50%
or so of OpenSSL's code. Would it be ok to steal^Wget inspiration from other
projects' test suites (e.g. BoringSSL and LibreSSL added some new tests) to try
to increse that?

One nice thing from BoringSSL's test suite is the Go-based test runner thingy.
It'd be interesting to see how good or bad OpenSSL does against it.

Cheers

[0] https://github.com/openssl/openssl/pull/445
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20151020/c7790947/attachment.sig>