From matt at openssl.org  Mon May  1 22:33:43 2017
From: matt at openssl.org (Matt Caswell)
Date: Mon, 1 May 2017 23:33:43 +0100
Subject: [openssl-dev] Fwd: Code Health Tuesday - old issues
In-Reply-To: <c6358119-022e-7564-a359-e7ae96077aeb@openssl.org>
References: <c6358119-022e-7564-a359-e7ae96077aeb@openssl.org>
Message-ID: <e7a95c7a-328a-4283-7fd0-a67fcfebe501@openssl.org>

Just a reminder about our code health Tuesday event this week. Please
see the details below.

Matt


-------- Forwarded Message --------
Subject: Code Health Tuesday - old issues
Date: Thu, 27 Apr 2017 08:43:18 +0100
From: Matt Caswell <matt at openssl.org>
To: openssl-dev at openssl.org <openssl-dev at openssl.org>



Hi all

Our next "Code Health Tuesday" event will be on Tuesday 2nd May.

This time we would like to focus on closing off Github issues (with a
particular focus on old ones - including all those we brought over from RT).

Thanks!

Matt

FAQ:

Q: How do I participate?
A: Review the old issues in Github. If an issue can be closed then post
a comment in the issue suggesting its closure along with your reasons.
If an issue still needs to be fixed then raise a PR to fix it and
include "Fixes #XXX" in the commit message (where XXX is the issue
number) Include the words "code health" in the PR title.

Q: Which branches should I target?
A: Any currently supported branch (i.e. master, 1.1.0 and 1.0.2) where
the issue applies

Q: What are good reasons for recommending the closure of an issue?
A: There are lots of possible reasons. Some are:
 - It has already been fixed
 - It is no longer relevant
 - It does not affect currently supported versions (master, 1.1.0 and 1.02)
 - We can no longer reproduce it
 - The issue is incorrect
 - It is a duplicate of some other issue

Q: How do I find the issues that were brought over from RT?
A: Use this link:
https://github.com/openssl/openssl/issues?q=is%3Aissue%20is%3Aopen%20rt

From matt at openssl.org  Wed May  3 16:33:15 2017
From: matt at openssl.org (Matt Caswell)
Date: Wed, 3 May 2017 17:33:15 +0100
Subject: [openssl-dev] OpenSSL master now at TLSv1.3 draft-20
Message-ID: <1beb730c-9a7f-8b65-15ea-0ce344e72535@openssl.org>

FYI, I have just made the necessary updates to bring the OpenSSL master
branch up to TLSv1.3 draft-20 compatibility. I don't know of any other
draft-20 implementations yet so treat with caution and please test!

As with all new drafts, draft-20 is not compatible with draft-19. If you
were using the previous draft-19 code (or the draft-18 code) then you
could encounter issues.

I will ask Richard to create a new tls1.3-draft-19 branch for those that
want to stick with the previous version for now.

Matt


From matt at openssl.org  Thu May  4 11:59:53 2017
From: matt at openssl.org (Matt Caswell)
Date: Thu, 4 May 2017 12:59:53 +0100
Subject: [openssl-dev] Code Health Tuesday - old issues
In-Reply-To: <c6358119-022e-7564-a359-e7ae96077aeb@openssl.org>
References: <c6358119-022e-7564-a359-e7ae96077aeb@openssl.org>
Message-ID: <1cbfdde8-77cd-b903-6bea-65dd0bd16f0b@openssl.org>

Just to let you all know that we managed to close 58 old tickets as a
result of this event. Fantastic!

Thanks

Matt


On 27/04/17 08:43, Matt Caswell wrote:
> 
> 
> Hi all
> 
> Our next "Code Health Tuesday" event will be on Tuesday 2nd May.
> 
> This time we would like to focus on closing off Github issues (with a
> particular focus on old ones - including all those we brought over from RT).
> 
> Thanks!
> 
> Matt
> 
> FAQ:
> 
> Q: How do I participate?
> A: Review the old issues in Github. If an issue can be closed then post
> a comment in the issue suggesting its closure along with your reasons.
> If an issue still needs to be fixed then raise a PR to fix it and
> include "Fixes #XXX" in the commit message (where XXX is the issue
> number) Include the words "code health" in the PR title.
> 
> Q: Which branches should I target?
> A: Any currently supported branch (i.e. master, 1.1.0 and 1.0.2) where
> the issue applies
> 
> Q: What are good reasons for recommending the closure of an issue?
> A: There are lots of possible reasons. Some are:
>  - It has already been fixed
>  - It is no longer relevant
>  - It does not affect currently supported versions (master, 1.1.0 and 1.02)
>  - We can no longer reproduce it
>  - The issue is incorrect
>  - It is a duplicate of some other issue
> 
> Q: How do I find the issues that were brought over from RT?
> A: Use this link:
> https://github.com/openssl/openssl/issues?q=is%3Aissue%20is%3Aopen%20rt
> 

From matt at openssl.org  Thu May  4 13:21:39 2017
From: matt at openssl.org (Matt Caswell)
Date: Thu, 4 May 2017 14:21:39 +0100
Subject: [openssl-dev] Using TLS1.3 with OpenSSL
Message-ID: <4cb4f07b-0200-8572-429c-bada345f93b4@openssl.org>

Hi all

OpenSSL 1.1.1, when it is released, will support TLSv1.3 and it will be
binary and source compatible with OpenSSL 1.1.0. If your application
already supports 1.1.0 then, in theory, all you need to do to support
TLSv1.3 is to drop in the new OpenSSL version. However there are various
issues that application developers and application deployers need to be
aware of. I have written a blog post to cover some of those things. You
can read it here:

https://www.openssl.org/blog/blog/2017/05/04/tlsv1.3/

Matt

From michel.sales at free.fr  Fri May  5 20:57:17 2017
From: michel.sales at free.fr (Michel)
Date: Fri, 5 May 2017 22:57:17 +0200
Subject: [openssl-dev] Using TLS1.3 with OpenSSL
In-Reply-To: <4cb4f07b-0200-8572-429c-bada345f93b4@openssl.org>
References: <4cb4f07b-0200-8572-429c-bada345f93b4@openssl.org>
Message-ID: <001b01d2c5e2$2f5e84d0$8e1b8e70$@sales@free.fr>

Found it very helpful and highly informative.
Thanks (again :-) Matt.

-----Message d'origine-----
De?: openssl-dev [mailto:openssl-dev-bounces at openssl.org] De la part de Matt
Caswell
Envoy??: jeudi 4 mai 2017 15:22
??: openssl-users at openssl.org; openssl-dev at openssl.org
Objet?: [openssl-dev] Using TLS1.3 with OpenSSL

Hi all

OpenSSL 1.1.1, when it is released, will support TLSv1.3 and it will be
binary and source compatible with OpenSSL 1.1.0. If your application already
supports 1.1.0 then, in theory, all you need to do to support
TLSv1.3 is to drop in the new OpenSSL version. However there are various
issues that application developers and application deployers need to be
aware of. I have written a blog post to cover some of those things. You can
read it here:

https://www.openssl.org/blog/blog/2017/05/04/tlsv1.3/

Matt
--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev


From zubinmeva at qbitlogic.com  Tue May  9 00:22:31 2017
From: zubinmeva at qbitlogic.com (Zubin Mevawalla)
Date: Mon, 8 May 2017 20:22:31 -0400
Subject: [openssl-dev] Null pointer dereferences?
Message-ID: <CANn+0XOZe-hk+x0Dcg2L2Kwp4S+rQEmssp1KmnuEDO30TDQbWg@mail.gmail.com>

I was curious if these were real null pointer dereference issues?

CodeAi, an automated repair tool being developed at Qbit logic,
suggested a couple of if-guards as fixes.

The first was in crypto/async/async_wait.c on line 157. `prev` is
assigned to NULL on 144, and it doesn't look like it is assigned to
anything in the while loop. Access to the field `next` from variable
`prev` thus results in a null pointer dereference.

diff --git a/crypto/async/async_wait.c b/crypto/async/async_wait.c
--- a/crypto/async/async_wait.c
+++ b/crypto/async/async_wait.c
@@ -154,7 +154,9 @@ int ASYNC_WAIT_CTX_clear_fd(ASYNC_WAIT_CTX *ctx,
const void *key)
                 if (ctx->fds == curr) {
                     ctx->fds = curr->next;
                 } else {
-                    prev->next = curr->next;
+                    if(prev) {
+                        prev->next = curr->next;
+                    }
                 }

                 /* It is responsibility of the caller to cleanup before calling


The second fix was in ssl/statem/statem_srvr.c on line 1442 having
seen a path through the control flow where access to the field
`pre_proc_exts` from variable `clienthello` results in a null pointer
dereference. On line 1243 if `clienthello` is NULL then control jumps
to line 1440, and `clienthello` is dereferenced on line 1442 without
having being assigned.

diff --git a/ssl/statem/statem_srvr.c b/ssl/statem/statem_srvr.c
--- a/ssl/statem/statem_srvr.c
+++ b/ssl/statem/statem_srvr.c
@@ -1439,7 +1439,9 @@ MSG_PROCESS_RETURN tls_process_client_hello(SSL
*s, PACKET *pkt)
  err:
     ossl_statem_set_error(s);

-    OPENSSL_free(clienthello->pre_proc_exts);
+    if(clienthello) {
+        OPENSSL_free(clienthello->pre_proc_exts);
+    }
     OPENSSL_free(clienthello);

     return MSG_PROCESS_ERROR;

Could I submit these as patches if they look alright?

Thanks so much,

Zubin

From rsalz at akamai.com  Tue May  9 01:55:59 2017
From: rsalz at akamai.com (Salz, Rich)
Date: Tue, 9 May 2017 01:55:59 +0000
Subject: [openssl-dev] Null pointer dereferences?
In-Reply-To: <CANn+0XOZe-hk+x0Dcg2L2Kwp4S+rQEmssp1KmnuEDO30TDQbWg@mail.gmail.com>
References: <CANn+0XOZe-hk+x0Dcg2L2Kwp4S+rQEmssp1KmnuEDO30TDQbWg@mail.gmail.com>
Message-ID: <d016f923595744a5b87b700ba8c92da7@usma1ex-dag1mb1.msg.corp.akamai.com>

> The first was in crypto/async/async_wait.c on line 157. `prev` is assigned to
> NULL on 144, and it doesn't look like it is assigned to anything in the while
> loop.

Line 177.
 
> -    OPENSSL_free(clienthello->pre_proc_exts);
> +    if(clienthello) {
> +        OPENSSL_free(clienthello->pre_proc_exts);
> +    }

Without the curly braces :)  yes, this seems like a bug.

From matt at openssl.org  Tue May  9 07:50:10 2017
From: matt at openssl.org (Matt Caswell)
Date: Tue, 9 May 2017 08:50:10 +0100
Subject: [openssl-dev] Null pointer dereferences?
In-Reply-To: <CANn+0XOZe-hk+x0Dcg2L2Kwp4S+rQEmssp1KmnuEDO30TDQbWg@mail.gmail.com>
References: <CANn+0XOZe-hk+x0Dcg2L2Kwp4S+rQEmssp1KmnuEDO30TDQbWg@mail.gmail.com>
Message-ID: <1363a8ec-28ed-824b-854d-fb24f0f7d14f@openssl.org>



On 09/05/17 01:22, Zubin Mevawalla wrote:
> I was curious if these were real null pointer dereference issues?
> 
> CodeAi, an automated repair tool being developed at Qbit logic,
> suggested a couple of if-guards as fixes.
> 
> The first was in crypto/async/async_wait.c on line 157. `prev` is
> assigned to NULL on 144, and it doesn't look like it is assigned to
> anything in the while loop. Access to the field `next` from variable
> `prev` thus results in a null pointer dereference.
> 
> diff --git a/crypto/async/async_wait.c b/crypto/async/async_wait.c
> --- a/crypto/async/async_wait.c
> +++ b/crypto/async/async_wait.c
> @@ -154,7 +154,9 @@ int ASYNC_WAIT_CTX_clear_fd(ASYNC_WAIT_CTX *ctx,
> const void *key)
>                  if (ctx->fds == curr) {
>                      ctx->fds = curr->next;
>                  } else {
> -                    prev->next = curr->next;
> +                    if(prev) {
> +                        prev->next = curr->next;
> +                    }
>                  }
> 
>                  /* It is responsibility of the caller to cleanup before calling
> 
> 
> The second fix was in ssl/statem/statem_srvr.c on line 1442 having
> seen a path through the control flow where access to the field
> `pre_proc_exts` from variable `clienthello` results in a null pointer
> dereference. On line 1243 if `clienthello` is NULL then control jumps
> to line 1440, and `clienthello` is dereferenced on line 1442 without
> having being assigned.
> 
> diff --git a/ssl/statem/statem_srvr.c b/ssl/statem/statem_srvr.c
> --- a/ssl/statem/statem_srvr.c
> +++ b/ssl/statem/statem_srvr.c
> @@ -1439,7 +1439,9 @@ MSG_PROCESS_RETURN tls_process_client_hello(SSL
> *s, PACKET *pkt)
>   err:
>      ossl_statem_set_error(s);
> 
> -    OPENSSL_free(clienthello->pre_proc_exts);
> +    if(clienthello) {
> +        OPENSSL_free(clienthello->pre_proc_exts);
> +    }
>      OPENSSL_free(clienthello);
> 
>      return MSG_PROCESS_ERROR;
> 
> Could I submit these as patches if they look alright?

The first one is not a bug. For the second one, please do via our github
site. No curly braces for just a single line, and make the "if"
expression "if (clienthello != NULL)".

Matt

From nreilly at blackberry.com  Tue May  9 13:43:45 2017
From: nreilly at blackberry.com (Nick Reilly)
Date: Tue, 9 May 2017 09:43:45 -0400
Subject: [openssl-dev] shlibloadtest failure on non-Linux
Message-ID: <063e1617-1513-4c5b-9e2c-05d17885e698@blackberry.com>

Hi,
I just ported OpenSSL 1.1.0e to QNX and ran in to an issue on 
shlibloadtest and it trying to unload the shared libraries in 
conjunction with the code in crypto/init.c

1) crypto/init.c at line 106 deliberately leaks a reference to prevent 
the library from unloading unless OPENSSL_USE_NODELETE is defined, yet 
shlibloadtest doesn't test for whether this is set. I think the 
shlibloadtest should only test the dlclose() return value on if 
OPENSSL_USE_NODELETE is set.

2) crypto/init.c at line 77 does "atexit(OPENSSL_cleanup)". If I try 
defining OPENSSL_USE_NODELETE then this atexit() handler is meant to 
cleanup on unload of the shared library, but this meaning of atexit() is 
Linux specific. It is not required in POSIX and the Linux manpage lists 
this usage under the "Linux notes" section. In QNX there must be no 
atexit() handlers registered if the library is being unloaded, otherwise 
there is a forced abort() called from the dynamic linker. I think that 
rather than using the atexit() mechanism some other mechanism e.g. 
function attribute of destructor on gcc should be used.

Regards,
Nick.

From rsalz at akamai.com  Tue May  9 14:03:00 2017
From: rsalz at akamai.com (Salz, Rich)
Date: Tue, 9 May 2017 14:03:00 +0000
Subject: [openssl-dev] shlibloadtest failure on non-Linux
In-Reply-To: <063e1617-1513-4c5b-9e2c-05d17885e698@blackberry.com>
References: <063e1617-1513-4c5b-9e2c-05d17885e698@blackberry.com>
Message-ID: <30f4717aaf5f41ac8633a04c79fd3822@usma1ex-dag1mb1.msg.corp.akamai.com>

> doesn't test for whether this is set. I think the shlibloadtest should only test
> the dlclose() return value on if OPENSSL_USE_NODELETE is set.

Please see https://github.com/openssl/openssl/pull/3399
 
> 2) crypto/init.c at line 77 does "atexit(OPENSSL_cleanup)". If I try defining
> OPENSSL_USE_NODELETE then this atexit() handler is meant to cleanup on
> unload of the shared library, but this meaning of atexit() is Linux specific. It is
> not required in POSIX and the Linux manpage lists this usage under the
> "Linux notes" section.

Does changing the guard to this work?
#if !defined(OPENSSL_SYS_UEFI) &&  !defined(OPENSSL_SYS_QNX)


From matt at openssl.org  Tue May  9 14:08:25 2017
From: matt at openssl.org (Matt Caswell)
Date: Tue, 9 May 2017 15:08:25 +0100
Subject: [openssl-dev] shlibloadtest failure on non-Linux
In-Reply-To: <063e1617-1513-4c5b-9e2c-05d17885e698@blackberry.com>
References: <063e1617-1513-4c5b-9e2c-05d17885e698@blackberry.com>
Message-ID: <c39c6414-67fb-cb7b-1fe9-5bd82594e2d4@openssl.org>



On 09/05/17 14:43, Nick Reilly wrote:
> Hi,
> I just ported OpenSSL 1.1.0e to QNX and ran in to an issue on
> shlibloadtest and it trying to unload the shared libraries in
> conjunction with the code in crypto/init.c
> 
> 1) crypto/init.c at line 106 deliberately leaks a reference to prevent
> the library from unloading unless OPENSSL_USE_NODELETE is defined, yet
> shlibloadtest doesn't test for whether this is set. I think the
> shlibloadtest should only test the dlclose() return value on if
> OPENSSL_USE_NODELETE is set.

I'm not sure why this makes a difference. The return value of dlclose()
tells you whether there has been an error or not. Not whether the
library has actually been removed from address space. Or am I missing
your point?

> 
> 2) crypto/init.c at line 77 does "atexit(OPENSSL_cleanup)". If I try
> defining OPENSSL_USE_NODELETE then this atexit() handler is meant to
> cleanup on unload of the shared library, but this meaning of atexit() is
> Linux specific.

No. The whole point of OPENSSL_USE_NODELETE is to indicate that the
library won't be unloaded until process exit because we've ensured that
through some other mechanism (e.g. because we built it using
"-znodelete" on Linux). We do not rely on the linux specific behaviour.
If your platform doesn't have some other mechanism then you need to use
the default "deliberate leak" approach.

Matt

From matt at openssl.org  Tue May  9 14:11:16 2017
From: matt at openssl.org (Matt Caswell)
Date: Tue, 9 May 2017 15:11:16 +0100
Subject: [openssl-dev] shlibloadtest failure on non-Linux
In-Reply-To: <30f4717aaf5f41ac8633a04c79fd3822@usma1ex-dag1mb1.msg.corp.akamai.com>
References: <063e1617-1513-4c5b-9e2c-05d17885e698@blackberry.com>
 <30f4717aaf5f41ac8633a04c79fd3822@usma1ex-dag1mb1.msg.corp.akamai.com>
Message-ID: <57fc60de-ad07-b42b-dc05-6682e5221dee@openssl.org>



On 09/05/17 15:03, Salz, Rich via openssl-dev wrote:
>> doesn't test for whether this is set. I think the shlibloadtest should only test
>> the dlclose() return value on if OPENSSL_USE_NODELETE is set.
> 
> Please see https://github.com/openssl/openssl/pull/3399
>  
>> 2) crypto/init.c at line 77 does "atexit(OPENSSL_cleanup)". If I try defining
>> OPENSSL_USE_NODELETE then this atexit() handler is meant to cleanup on
>> unload of the shared library, but this meaning of atexit() is Linux specific. It is
>> not required in POSIX and the Linux manpage lists this usage under the
>> "Linux notes" section.
> 
> Does changing the guard to this work?
> #if !defined(OPENSSL_SYS_UEFI) &&  !defined(OPENSSL_SYS_QNX)
> 

That presumably would mean that the library would not clean up at all on
QNX, which is probably undesirable.

Matt

From nreilly at blackberry.com  Tue May  9 17:12:23 2017
From: nreilly at blackberry.com (Nick Reilly)
Date: Tue, 9 May 2017 13:12:23 -0400
Subject: [openssl-dev] shlibloadtest failure on non-Linux
In-Reply-To: <c39c6414-67fb-cb7b-1fe9-5bd82594e2d4@openssl.org>
References: <063e1617-1513-4c5b-9e2c-05d17885e698@blackberry.com>
 <c39c6414-67fb-cb7b-1fe9-5bd82594e2d4@openssl.org>
Message-ID: <12969be7-b3f4-fafa-6876-ebfc94e7c341@blackberry.com>



On 2017-05-09 10:08 AM, Matt Caswell wrote:
>
> I'm not sure why this makes a difference. The return value of dlclose()
> tells you whether there has been an error or not. Not whether the
> library has actually been removed from address space. Or am I missing
> your point?

dlclose() is returning an error and dlerror() reports that the library 
cannot be closed as it is still in use (from the leaked reference).

> No. The whole point of OPENSSL_USE_NODELETE is to indicate that the
> library won't be unloaded until process exit because we've ensured that
> through some other mechanism (e.g. because we built it using
> "-znodelete" on Linux). We do not rely on the linux specific behaviour.
> If your platform doesn't have some other mechanism then you need to use
> the default "deliberate leak" approach.

Ok, misunderstood the intention of OPENSSL_USE_NODELETE, didn't realise 
it meant -znodelete.

Thanks,
Nick.

From nreilly at blackberry.com  Tue May  9 17:13:22 2017
From: nreilly at blackberry.com (Nick Reilly)
Date: Tue, 9 May 2017 13:13:22 -0400
Subject: [openssl-dev] shlibloadtest failure on non-Linux
In-Reply-To: <30f4717aaf5f41ac8633a04c79fd3822@usma1ex-dag1mb1.msg.corp.akamai.com>
References: <063e1617-1513-4c5b-9e2c-05d17885e698@blackberry.com>
 <30f4717aaf5f41ac8633a04c79fd3822@usma1ex-dag1mb1.msg.corp.akamai.com>
Message-ID: <d775e02c-a6c4-b9ab-f3a4-9b4a9ee90b0f@blackberry.com>



On 2017-05-09 10:03 AM, Salz, Rich via openssl-dev wrote:
>> doesn't test for whether this is set. I think the shlibloadtest should only test
>> the dlclose() return value on if OPENSSL_USE_NODELETE is set.
>
> Please see https://github.com/openssl/openssl/pull/3399
>

Sense of OPENSSL_USE_NODELETE has been reversed i.e. you want #ifdef and 
not #ifndef

Also I think its still fair game to try a dlclose() just that the 
dlclose() may return an error if OPENSSL_USE_NODELETE is not defined.

Regards,
Nick.

From rsalz at akamai.com  Tue May  9 17:16:43 2017
From: rsalz at akamai.com (Salz, Rich)
Date: Tue, 9 May 2017 17:16:43 +0000
Subject: [openssl-dev] shlibloadtest failure on non-Linux
In-Reply-To: <d775e02c-a6c4-b9ab-f3a4-9b4a9ee90b0f@blackberry.com>
References: <063e1617-1513-4c5b-9e2c-05d17885e698@blackberry.com>
 <30f4717aaf5f41ac8633a04c79fd3822@usma1ex-dag1mb1.msg.corp.akamai.com>
 <d775e02c-a6c4-b9ab-f3a4-9b4a9ee90b0f@blackberry.com>
Message-ID: <7f3830c047fe47f3b4ebfe0ce5082c2a@usma1ex-dag1mb1.msg.corp.akamai.com>

> Sense of OPENSSL_USE_NODELETE has been reversed i.e. you want #ifdef
> and not #ifndef

Argh, you're right.
 
> Also I think its still fair game to try a dlclose() just that the
> dlclose() may return an error if OPENSSL_USE_NODELETE is not defined.

I'll just leave it as-is.  Thanks for looking at this!

From matt at openssl.org  Tue May  9 17:45:29 2017
From: matt at openssl.org (Matt Caswell)
Date: Tue, 9 May 2017 18:45:29 +0100
Subject: [openssl-dev] shlibloadtest failure on non-Linux
In-Reply-To: <12969be7-b3f4-fafa-6876-ebfc94e7c341@blackberry.com>
References: <063e1617-1513-4c5b-9e2c-05d17885e698@blackberry.com>
 <c39c6414-67fb-cb7b-1fe9-5bd82594e2d4@openssl.org>
 <12969be7-b3f4-fafa-6876-ebfc94e7c341@blackberry.com>
Message-ID: <afa01908-1591-d638-e30f-455d9192abe8@openssl.org>



On 09/05/17 18:12, Nick Reilly wrote:
> 
> 
> On 2017-05-09 10:08 AM, Matt Caswell wrote:
>>
>> I'm not sure why this makes a difference. The return value of dlclose()
>> tells you whether there has been an error or not. Not whether the
>> library has actually been removed from address space. Or am I missing
>> your point?
> 
> dlclose() is returning an error and dlerror() reports that the library
> cannot be closed as it is still in use (from the leaked reference).

Hmmm. Arguably that is broken dlclose() behaviour. dlclose() is just
supposed to inform the system that a handle previously returned by
dlopen() is no longer required by the application. AFAIK there is no
requirement for all other handles returned from other dlopen() open
calls for the same shared library to be closed first (how would that
work anyway...unless there was a strict requirement for the handle
returned from the first call to dlopen() to always be the last one to be
used in a call to dlclose()?).

This is not a problem on other non-linux Unix based systems that we have
tried it on, so it looks to be something peculiar to QNX.

Does QNX have a "-znodelete" equivalent? That would be the most
preferable fix.

Another option might be something like this:

diff --git a/test/shlibloadtest.c b/test/shlibloadtest.c
index 6f220ba530..bc701b4333 100644
--- a/test/shlibloadtest.c
+++ b/test/shlibloadtest.c
@@ -65,8 +65,16 @@ static int shlib_sym(SHLIB lib, const char *symname,
SHLIB_SYM *sym)

 static int shlib_close(SHLIB lib)
 {
+#ifdefined OPENSSL_SYS_QNX
+    /*
+     * Ignore errors from dlclose() on QNX to avoid spurious complaints
about
+     * being unable to close it due to it still being in use.
+     */
+    dlclose(lib);
+#else
     if (dlclose(lib) != 0)
         return 0;
+#endif

     return 1;
 }

Matt


From uri at ll.mit.edu  Fri May 12 18:34:05 2017
From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL)
Date: Fri, 12 May 2017 18:34:05 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
Message-ID: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>

I?m sorry to report that in the current OpenSSL 1.1 master running ?make test? freezes up the machine. Mac OS X 10.11.6, Xcode-8.2, current Github master. Here?s the configuration:

 

./Configure darwin64-x86_64-cc enable-threads enable-shared enable-zlib enable-ec_nistp_64_gcc_128 enable-rfc3779 enable-rc5 enable-tls1_3 --prefix=/Users/uri/src/openssl-1.1 --openssldir=/Users/uri/src/openssl-1.1/etc

 

Then of course ?make depend && make clean && make all && make test?

 

../test/recipes/90-test_ige.t ................. ok   

../test/recipes/90-test_memleak.t ............. ok   

../test/recipes/90-test_overhead.t ............ skipped: Only supported in no-shared builds

../test/recipes/90-test_secmem.t .............. 

 

At this point the machine has to be power-cycled.

? 

Regards,

Uri

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/d8bd3eff/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5211 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/d8bd3eff/attachment-0001.bin>

From bkaduk at akamai.com  Fri May 12 18:49:39 2017
From: bkaduk at akamai.com (Benjamin Kaduk)
Date: Fri, 12 May 2017 13:49:39 -0500
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
Message-ID: <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>

On 05/12/2017 01:34 PM, Blumenthal, Uri - 0553 - MITLL wrote:
>
> I?m sorry to report that in the current OpenSSL 1.1 master running
> ?make test? freezes up the machine. Mac OS X 10.11.6, Xcode-8.2,
> current Github master. Here?s the configuration:
>

A commit hash would be more useful than "current github master"

>  
>
> ./Configure darwin64-x86_64-cc enable-threads enable-shared
> enable-zlib enable-ec_nistp_64_gcc_128 enable-rfc3779 enable-rc5
> enable-tls1_3 --prefix=/Users/uri/src/openssl-1.1
> --openssldir=/Users/uri/src/openssl-1.1/etc
>
>  
>
> Then of course ?make depend && make clean && make all && make test?
>
>  
>
> ../test/recipes/90-test_ige.t ................. ok   
>
> ../test/recipes/90-test_memleak.t ............. ok   
>
> ../test/recipes/90-test_overhead.t ............ skipped: Only
> supported in no-shared builds
>
> ../test/recipes/90-test_secmem.t .............. 
>
>  
>
> At this point the machine has to be power-cycled.
>
> ?
>


I can understand not wanting to have to power-cycle the machine again,
but the 'make TESTS=test_secmem V=1 test' output (or some
dtruss/similar) would be helpful in tracking things down.

How locked up is the machine?  Can you get memory usage stats or is it
completely unresponsive?

-Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/803d467f/attachment.html>

From rsalz at akamai.com  Fri May 12 18:51:42 2017
From: rsalz at akamai.com (Salz, Rich)
Date: Fri, 12 May 2017 18:51:42 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
Message-ID: <32bfda79569e48deb91d5bd4ee8ab0ad@usma1ex-dag1mb1.msg.corp.akamai.com>

Current master 1d0f116 fails on my machine.

../test/recipes/90-test_secmem.t ..
1..1
    # Subtest: ./secmemtest
    1..1
    # INFO:  @ test/secmemtest.c:65
    # Possible infinite loop: allocate more than available

    # INFO:  @ test/secmemtest.c:71
    # Possible infinite loop: small arena

    # INFO:  @ test/secmemtest.c:79
    # Possible infinite loop: 1<<31 limit

crypto/mem_sec.c:428: OpenSSL internal error: assertion failed: sh.map_result != MAP_FAILED
../util/shlib_wrap.sh ./secmemtest => 134
not ok 1 - running secmemtest


From uri at ll.mit.edu  Fri May 12 19:05:06 2017
From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL)
Date: Fri, 12 May 2017 19:05:06 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
Message-ID: <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>

On 5/12/17, 2:49 PM, "openssl-dev on behalf of Benjamin Kaduk via openssl-dev" <openssl-dev-bounces at openssl.org on behalf of openssl-dev at openssl.org> wrote:

? I?m sorry to report that in the current OpenSSL 1.1 master running ?make test?
?  freezes up the machine. Mac OS X 10.11.6, Xcode-8.2, current Github master. 
? Here?s the configuration:

      A commit hash would be more useful than "current github master"

I thought you know what?s in the current master right now. But here?s the last few hashes for your pleasure:

$ git log
commit 80a2fc4100daf6f1001eee33ef2f9b9eee05bedf (HEAD -> master, origin/master, origin/HEAD)
Author: Todd Short <tshort at akamai.com>
Date:   Wed May 10 11:44:55 2017 -0400

    Clean up SSL_OP_* a bit
    
    Reviewed-by: Matt Caswell <matt at openssl.org>
    Reviewed-by: Rich Salz <rsalz at openssl.org>
    (Merged from https://github.com/openssl/openssl/pull/3439)

commit 33242d9d79e7f06151e905b83dc8f995006fa7cd
Author: Rich Salz <rsalz at openssl.org>
Date:   Thu May 11 20:42:32 2017 -0400

    Use scalar, not length; fixes test_evp
    
    Reviewed-by: Stephen Henson <steve at openssl.org>
    Reviewed-by: Richard Levitte <levitte at openssl.org>
    (Merged from https://github.com/openssl/openssl/pull/3452)


      I can understand not wanting to have to power-cycle the machine again,
      but the 'make TESTS=test_secmem V=1 test' output (or some dtruss/similar)
      would be helpful in tracking things down.

Sorry.

      How locked up is the machine?? Can you get memory usage stats or is it completely unresponsive?

Completely unresponsive. Totally. No memory usage. The only thing that works at this point is the power button.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5211 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/a8301a74/attachment.bin>

From bkaduk at akamai.com  Fri May 12 19:15:57 2017
From: bkaduk at akamai.com (Benjamin Kaduk)
Date: Fri, 12 May 2017 14:15:57 -0500
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
Message-ID: <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>

On 05/12/2017 02:05 PM, Blumenthal, Uri - 0553 - MITLL wrote:
> On 5/12/17, 2:49 PM, "openssl-dev on behalf of Benjamin Kaduk via openssl-dev" <openssl-dev-bounces at openssl.org on behalf of openssl-dev at openssl.org> wrote:
>
> ? I?m sorry to report that in the current OpenSSL 1.1 master running ?make test?
> ?  freezes up the machine. Mac OS X 10.11.6, Xcode-8.2, current Github master. 
> ? Here?s the configuration:
>
>       A commit hash would be more useful than "current github master"
>
> I thought you know what?s in the current master right now. But here?s the last few hashes for your pleasure:

Well, you're right to think that I do, and I do trust you to be accurate
when you make that claim.  But there are many people for which I would
not fully trust the veracity of such a claim, the commit hash is
completely unambiguous, and it is a whole lot easier for those poor
folks reading this thread in the archive two years from now who are
trying to track down a similar-looking issue.  So, on the whole, I
recommend always using commit hashes, with an optional annotation of how
it relates to a branch.

> $ git log
> commit 80a2fc4100daf6f1001eee33ef2f9b9eee05bedf (HEAD -> master, origin/master, origin/HEAD)
> Author: Todd Short <tshort at akamai.com>
> Date:   Wed May 10 11:44:55 2017 -0400
>
>     Clean up SSL_OP_* a bit
>     
>     Reviewed-by: Matt Caswell <matt at openssl.org>
>     Reviewed-by: Rich Salz <rsalz at openssl.org>
>     (Merged from https://github.com/openssl/openssl/pull/3439)
>
> commit 33242d9d79e7f06151e905b83dc8f995006fa7cd
> Author: Rich Salz <rsalz at openssl.org>
> Date:   Thu May 11 20:42:32 2017 -0400
>
>     Use scalar, not length; fixes test_evp
>     
>     Reviewed-by: Stephen Henson <steve at openssl.org>
>     Reviewed-by: Richard Levitte <levitte at openssl.org>
>     (Merged from https://github.com/openssl/openssl/pull/3452)
>
>
>       I can understand not wanting to have to power-cycle the machine again,
>       but the 'make TESTS=test_secmem V=1 test' output (or some dtruss/similar)
>       would be helpful in tracking things down.

The obvious candidate for closer inspection is a few commits previous,

commit 7031ddac94d0ae616d1b0670263a9265ce672cd2
Author: Todd Short <tshort at akamai.com>
Date:   Thu May 11 15:48:10 2017 -0400

    Fix infinite loops in secure memory allocation.
   
    Issue 1:
   
    sh.bittable_size is a size_t but i is and int, which can result in
    freelist == -1 if sh.bittable_size exceeds an int.
   
    This seems to result in an OPENSSL_assert due to invalid allocation
    size, so maybe that is "ok."
   
    Worse, if sh.bittable_size is exactly 1<<31, then this becomes an
    infinite loop (because 1<<31 is a negative int, so it can be shifted
    right forever and sticks at -1).
   
    Issue 2:
   
    CRYPTO_secure_malloc_init() sets secure_mem_initialized=1 even when
    sh_init() returns 0.
   
    If sh_init() fails, we end up with secure_mem_initialized=1 but
    sh.minsize=0. If you then call secure_malloc(), which then calls,
    sh_malloc(), this then enters an infite loop since 0 << anything will
    never be larger than size.
   
    Issue 3:
   
    That same sh_malloc loop will loop forever for a size greater
    than size_t/2 because i will proceed (assuming sh.minsize=16):
    i=16, 32, 64, ..., size_t/8, size_t/4, size_t/2, 0, 0, 0, 0, ....
    This sequence will never be larger than "size".
   
    Reviewed-by: Rich Salz <rsalz at openssl.org>
    Reviewed-by: Richard Levitte <levitte at openssl.org>
    (Merged from https://github.com/openssl/openssl/pull/3449)

which adds test cases intended to trigger the edge cases being fixed.

> Sorry.
>
>       How locked up is the machine?  Can you get memory usage stats or is it completely unresponsive?
>
> Completely unresponsive. Totally. No memory usage. The only thing that works at this point is the power button.

It seems that there should also be a bug report against OS X, as regular
userspace code running as non-root should not be able to hang a machine
like that.

>From just looking at the code, the only question that comes to mind is
whether you have a 32- or 64-bit size_t in the build environment in
question, which is unlikely to cause a eureka moment :(

-Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/88097768/attachment-0001.html>

From bkaduk at akamai.com  Fri May 12 19:26:44 2017
From: bkaduk at akamai.com (Benjamin Kaduk)
Date: Fri, 12 May 2017 14:26:44 -0500
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
Message-ID: <922dcc70-d40a-85bf-58af-fdde9032827b@akamai.com>

On 05/12/2017 02:15 PM, Benjamin Kaduk via openssl-dev wrote:
> From just looking at the code, the only question that comes to mind is
> whether you have a 32- or 64-bit size_t in the build environment in
> question, which is unlikely to cause a eureka moment :(

(The test runs happily on my Ubuntu Trusty-ish machine, FWIW.)

Some other information to check: do you have MAP_ANON defined by your
mman.h?
Do you have a /dev/zero that can be opened read/write?

-Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/0aac7bfd/attachment.html>

From tshort at akamai.com  Fri May 12 19:38:25 2017
From: tshort at akamai.com (Short, Todd)
Date: Fri, 12 May 2017 19:38:25 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <32bfda79569e48deb91d5bd4ee8ab0ad@usma1ex-dag1mb1.msg.corp.akamai.com>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <32bfda79569e48deb91d5bd4ee8ab0ad@usma1ex-dag1mb1.msg.corp.akamai.com>
Message-ID: <F3353F06-037E-44C7-8620-D8C761C6AC3C@akamai.com>

Yes, it?s likely this is due to the amount of memory available in the machine. I tried to use reasonable values, but apparently not reasonable enough.

This is certainly a case where we?re trying to stretch the limits of the hardware; so it may not be an appropriate test for all hardware.

In the case below, the call to mmap() failed, and there?s an OPENSSL_assert() there that?s probably not necessary; the error condition checked by the assert is handled safely.

This is different than the problem <uri at ll.mit.edu<mailto:uri at ll.mit.edu>> was seeing, which is likely a condition where memory is being maxed out.

--
-Todd Short
// tshort at akamai.com<mailto:tshort at akamai.com>
// "One if by land, two if by sea, three if by the Internet."

On May 12, 2017, at 2:51 PM, Salz, Rich via openssl-dev <openssl-dev at openssl.org<mailto:openssl-dev at openssl.org>> wrote:

Current master 1d0f116 fails on my machine.

../test/recipes/90-test_secmem.t ..
1..1
   # Subtest: ./secmemtest
   1..1
   # INFO:  @ test/secmemtest.c:65
   # Possible infinite loop: allocate more than available

   # INFO:  @ test/secmemtest.c:71
   # Possible infinite loop: small arena

   # INFO:  @ test/secmemtest.c:79
   # Possible infinite loop: 1<<31 limit

crypto/mem_sec.c:428: OpenSSL internal error: assertion failed: sh.map_result != MAP_FAILED
../util/shlib_wrap.sh ./secmemtest => 134
not ok 1 - running secmemtest

--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/048de3b5/attachment.html>

From uri at ll.mit.edu  Fri May 12 20:05:10 2017
From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL)
Date: Fri, 12 May 2017 20:05:10 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
Message-ID: <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>

The obvious candidate for closer inspection is a few commits previous,

commit 7031ddac94d0ae616d1b0670263a9265ce672cd2
Author: Todd Short <tshort at akamai.com>
Date:   Thu May 11 15:48:10 2017 -0400

which adds test cases intended to trigger the edge cases being fixed.



Point well-taken. ;-)

 

It seems that there should also be a bug report against OS X, as regular userspace code running as non-root should not be able to hang a machine like that.

I?m not sure. It may well be that it ?simply? takes all the memory over, so there is no way for anything else to start and do the clean-up?

 

>From just looking at the code, the only question that comes to mind is whether you have a 32- or 64-bit size_t in the build environment in question, which is unlikely to cause a eureka moment :(

 

I can tell you that size_t is 64-bit here. It?s certainly not an ?eureka? moment for me.

 

Some other information to check: do you have MAP_ANON defined by your mman.h?


/**

 * @addtogroup apr_platform

 * @ingroup APR 

 * @{

 */

 

#define APR_HAVE_SHMEM_MMAP_TMP     1

#define APR_HAVE_SHMEM_MMAP_SHM     1

#define APR_HAVE_SHMEM_MMAP_ZERO    0

#define APR_HAVE_SHMEM_SHMGET_ANON  1

#define APR_HAVE_SHMEM_SHMGET       1

#define APR_HAVE_SHMEM_MMAP_ANON    1

#define APR_HAVE_SHMEM_BEOS         0

 

#define APR_USE_SHMEM_MMAP_TMP     0

#define APR_USE_SHMEM_MMAP_SHM     0

#define APR_USE_SHMEM_MMAP_ZERO    0

#define APR_USE_SHMEM_SHMGET_ANON  0

#define APR_USE_SHMEM_SHMGET       1

#define APR_USE_SHMEM_MMAP_ANON    1

#define APR_USE_SHMEM_BEOS         0

 

And this system does not seem to have a straight ?mmap.h?. The above is from /usr/include/apr-1/apr_mmap.h file.

 

Do you have a /dev/zero that can be opened read/write?

 

Yep:

 

$ ll /dev/zero

crw-rw-rw-  1 root  wheel    3,   3 May 12 14:14 /dev/zero

$ 

 

Todd> Yes, it?s likely this is due to the amount of memory available in the machine. I tried to use reasonable values, but apparently not reasonable enough

 

Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, besides these tests :).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/60f057b4/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5211 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/60f057b4/attachment-0001.bin>

From bkaduk at akamai.com  Fri May 12 20:36:33 2017
From: bkaduk at akamai.com (Benjamin Kaduk)
Date: Fri, 12 May 2017 15:36:33 -0500
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
Message-ID: <55ded834-6e0a-3831-b5ff-c1e5ad2e0f60@akamai.com>

On 05/12/2017 03:05 PM, Blumenthal, Uri - 0553 - MITLL wrote:
>
> I?m not sure. It may well be that it ?simply? takes all the memory
> over, so there is no way for anything else to start and do the clean-up?
>

Hmm, I wonder if a top(1) started before the tests would keep running,
in the case that it "simply" takes all the memory over.

>  
>
> From just looking at the code, the only question that comes to mind is
> whether you have a 32- or 64-bit size_t in the build environment in
> question, which is unlikely to cause a eureka moment :(
>
>  
>
> I can tell you that size_t is 64-bit here. It?s certainly not an
> ?eureka? moment for me.
>
>  
>
> Some other information to check: do you have MAP_ANON defined by your
> mman.h?
>

mman.h != mmap.h (consult 'man mmap' for authoritative header).

> Todd> Yes, it?s likely this is due to the amount of memory available
> in the machine. I tried to use reasonable values, but apparently not
> reasonable enough
>
>  
>
> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of
> stuff, besides these tests :).
>
>

Thanks.

-Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/341db124/attachment.html>

From tshort at akamai.com  Fri May 12 20:46:13 2017
From: tshort at akamai.com (Short, Todd)
Date: Fri, 12 May 2017 20:46:13 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
Message-ID: <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>

It?s trying to reserve 1<<34 bytes of memory? there goes your 16GB...
--
-Todd Short
// tshort at akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu> wrote:
> 
> Todd> Yes, it?s likely this is due to the amount of memory available in the machine. I tried to use reasonable values, but apparently not reasonable enough
>  
> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, besides these tests :).
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev <https://mta.openssl.org/mailman/listinfo/openssl-dev>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/2702bab7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4407 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/2702bab7/attachment.bin>

From tshort at akamai.com  Fri May 12 20:50:04 2017
From: tshort at akamai.com (Short, Todd)
Date: Fri, 12 May 2017 20:50:04 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
 <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>
Message-ID: <B114BDAF-1F1C-41E9-826B-27775E375AB8@akamai.com>

Uri:

Look at https://github.com/openssl/openssl/pull/3455 <https://github.com/openssl/openssl/pull/3455>

I limited the test that hung your machine to Linux.

Rich: this removes the OpenSSL_assert() you see.

--
-Todd Short
// tshort at akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org> wrote:
> 
> It?s trying to reserve 1<<34 bytes of memory? there goes your 16GB...
> --
> -Todd Short
> // tshort at akamai.com <mailto:tshort at akamai.com>
> // "One if by land, two if by sea, three if by the Internet."
> 
>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu <mailto:uri at ll.mit.edu>> wrote:
>> 
>> Todd> Yes, it?s likely this is due to the amount of memory available in the machine. I tried to use reasonable values, but apparently not reasonable enough
>>  
>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, besides these tests :).
>> -- 
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev <https://mta.openssl.org/mailman/listinfo/openssl-dev>
> 
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/1af3170c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4407 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170512/1af3170c/attachment-0001.bin>

From kurt at roeckx.be  Sun May 14 09:49:36 2017
From: kurt at roeckx.be (Kurt Roeckx)
Date: Sun, 14 May 2017 11:49:36 +0200
Subject: [openssl-dev] Google OSS-Fuzz reward
Message-ID: <20170514094935.vld47z6ipynqj2c5@roeckx.be>

Google awarded us 1000 USD for the initial integration with
OSS-Fuzz. See https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html

I have asked Google to donate it to European Digital Rights (EDRi,
https://edri.org/). Google doubles the amount if you donate it to
a non-profit organization, so they should be receiving 2000 USD.


Kurt


From ben at links.org  Sun May 14 09:57:56 2017
From: ben at links.org (Ben Laurie)
Date: Sun, 14 May 2017 10:57:56 +0100
Subject: [openssl-dev] Google OSS-Fuzz reward
In-Reply-To: <20170514094935.vld47z6ipynqj2c5@roeckx.be>
References: <20170514094935.vld47z6ipynqj2c5@roeckx.be>
Message-ID: <CAG5KPzxRXzTt87+YFBu1iEdLvc=-5dgrursXB=ejuMu-GQY_XQ@mail.gmail.com>

Cool.

On 14 May 2017 at 10:49, Kurt Roeckx <kurt at roeckx.be> wrote:
> Google awarded us 1000 USD for the initial integration with
> OSS-Fuzz. See https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html
>
> I have asked Google to donate it to European Digital Rights (EDRi,
> https://edri.org/). Google doubles the amount if you donate it to
> a non-profit organization, so they should be receiving 2000 USD.
>
>
> Kurt
>
> --
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

From ben at links.org  Sun May 14 09:58:25 2017
From: ben at links.org (Ben Laurie)
Date: Sun, 14 May 2017 10:58:25 +0100
Subject: [openssl-dev] Google OSS-Fuzz reward
In-Reply-To: <CAG5KPzxRXzTt87+YFBu1iEdLvc=-5dgrursXB=ejuMu-GQY_XQ@mail.gmail.com>
References: <20170514094935.vld47z6ipynqj2c5@roeckx.be>
 <CAG5KPzxRXzTt87+YFBu1iEdLvc=-5dgrursXB=ejuMu-GQY_XQ@mail.gmail.com>
Message-ID: <CAG5KPzxdY_0s7EfJDuoXU+FXcBPrGDVraLcQyfeg+SwzpNJx5w@mail.gmail.com>

I should also point out that up to $10k is available for "ideal" integration.

On 14 May 2017 at 10:57, Ben Laurie <ben at links.org> wrote:
> Cool.
>
> On 14 May 2017 at 10:49, Kurt Roeckx <kurt at roeckx.be> wrote:
>> Google awarded us 1000 USD for the initial integration with
>> OSS-Fuzz. See https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html
>>
>> I have asked Google to donate it to European Digital Rights (EDRi,
>> https://edri.org/). Google doubles the amount if you donate it to
>> a non-profit organization, so they should be receiving 2000 USD.
>>
>>
>> Kurt
>>
>> --
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

From rsalz at akamai.com  Sun May 14 13:07:39 2017
From: rsalz at akamai.com (Salz, Rich)
Date: Sun, 14 May 2017 13:07:39 +0000
Subject: [openssl-dev] Google OSS-Fuzz reward
In-Reply-To: <CAG5KPzxRXzTt87+YFBu1iEdLvc=-5dgrursXB=ejuMu-GQY_XQ@mail.gmail.com>
References: <20170514094935.vld47z6ipynqj2c5@roeckx.be>
 <CAG5KPzxRXzTt87+YFBu1iEdLvc=-5dgrursXB=ejuMu-GQY_XQ@mail.gmail.com>
Message-ID: <490e556ce5804912b2ccc1535775980a@usma1ex-dag1mb1.msg.corp.akamai.com>

Nicely done!

> Cool.
> 
> On 14 May 2017 at 10:49, Kurt Roeckx <kurt at roeckx.be> wrote:
> > Google awarded us 1000 USD for the initial integration with OSS-Fuzz.
> > See
> > https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-a
> > nd.html
> >
> > I have asked Google to donate it to European Digital Rights (EDRi,
> > https://edri.org/). Google doubles the amount if you donate it to a
> > non-profit organization, so they should be receiving 2000 USD.
> >
> >
> > Kurt
> >
> > --
> > openssl-dev mailing list
> > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
> --
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

From tshort at akamai.com  Mon May 15 15:47:18 2017
From: tshort at akamai.com (Short, Todd)
Date: Mon, 15 May 2017 15:47:18 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <B114BDAF-1F1C-41E9-826B-27775E375AB8@akamai.com>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
 <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>
 <B114BDAF-1F1C-41E9-826B-27775E375AB8@akamai.com>
Message-ID: <80623024-B59F-4625-AC75-DF157B570136@akamai.com>

We?ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short period of time with the unit-tests, but it eventually recovers.

What MacOS version are you running? I can try 10.12 later today.
--
-Todd Short
// tshort at akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org> wrote:
> 
> Uri:
> 
> Look at https://github.com/openssl/openssl/pull/3455 <https://github.com/openssl/openssl/pull/3455>
> 
> I limited the test that hung your machine to Linux.
> 
> Rich: this removes the OpenSSL_assert() you see.
> 
> --
> -Todd Short
> // tshort at akamai.com <mailto:tshort at akamai.com>
> // "One if by land, two if by sea, three if by the Internet."
> 
>> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org <mailto:openssl-dev at openssl.org>> wrote:
>> 
>> It?s trying to reserve 1<<34 bytes of memory? there goes your 16GB...
>> --
>> -Todd Short
>> // tshort at akamai.com <mailto:tshort at akamai.com>
>> // "One if by land, two if by sea, three if by the Internet."
>> 
>>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu <mailto:uri at ll.mit.edu>> wrote:
>>> 
>>> Todd> Yes, it?s likely this is due to the amount of memory available in the machine. I tried to use reasonable values, but apparently not reasonable enough
>>>  
>>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, besides these tests :).
>>> -- 
>>> openssl-dev mailing list
>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev <https://mta.openssl.org/mailman/listinfo/openssl-dev>
>> 
>> -- 
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev <https://mta.openssl.org/mailman/listinfo/openssl-dev>
> 
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/26c7bf48/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4407 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/26c7bf48/attachment-0001.bin>

From uri at ll.mit.edu  Mon May 15 15:51:41 2017
From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL)
Date: Mon, 15 May 2017 15:51:41 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <80623024-B59F-4625-AC75-DF157B570136@akamai.com>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
 <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>
 <B114BDAF-1F1C-41E9-826B-27775E375AB8@akamai.com>
 <80623024-B59F-4625-AC75-DF157B570136@akamai.com>
Message-ID: <E45710AA-6B29-42BC-BF0D-F4CA6A367A12@ll.mit.edu>

I?m tracking the current OpenSSL master only on El Capitan 10.11.6. I could try it on Sierra 10.12.4, if you really expect it to make a difference.

 

In my case the hang is not for a short time. It lasts for more than 10 minutes, so I?m forced to interfere. For how long did it hang for you?

? 

Regards,

Uri

 

 

On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via openssl-dev" <openssl-dev-bounces at openssl.org on behalf of openssl-dev at openssl.org> wrote:

 

We?ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short period of time with the unit-tests, but it eventually recovers. 

 

What MacOS version are you running? I can try 10.12 later today.

--

-Todd Short

// tshort at akamai.com

// "One if by land, two if by sea, three if by the Internet."

 

On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org> wrote:? 

 

Uri: 

 

Look at https://github.com/openssl/openssl/pull/3455

 

I limited the test that hung your machine to Linux.

 

Rich: this removes the OpenSSL_assert() you see.

 

--

-Todd Short

// tshort at akamai.com

// "One if by land, two if by sea, three if by the Internet."

 

On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org> wrote:

 

It?s trying to reserve 1<<34 bytes of memory? there goes your 16GB...

--

-Todd Short

// tshort at akamai.com

// "One if by land, two if by sea, three if by the Internet."

 

On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu> wrote:

 

Todd> Yes, it?s likely this is due to the amount of memory available in the machine. I tried to use reasonable values, but apparently not reasonable enough

 

Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, besides these tests :).

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

 

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

 

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/d5dd2929/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5211 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/d5dd2929/attachment.bin>

From tshort at akamai.com  Mon May 15 17:08:13 2017
From: tshort at akamai.com (Short, Todd)
Date: Mon, 15 May 2017 17:08:13 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <E45710AA-6B29-42BC-BF0D-F4CA6A367A12@ll.mit.edu>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
 <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>
 <B114BDAF-1F1C-41E9-826B-27775E375AB8@akamai.com>
 <80623024-B59F-4625-AC75-DF157B570136@akamai.com>
 <E45710AA-6B29-42BC-BF0D-F4CA6A367A12@ll.mit.edu>
Message-ID: <80A99926-68B3-4775-9338-FFCDB43A3A7B@akamai.com>

The time of the hang actually seems dependent on the number of applications running and your disk.

Since a large amount of memory becomes wired, there is very little available for programs and the OS to use (in some instances I have seen ~4MB non-wired memory). Things slow down due to swapping, etc.

In my testing:

With almost no additional programs open, the hang-time is short, ~200 seconds.
With a lot of programs open, the hang-time is increased, ~400 seconds; twice as long. And the number of swapins is 25x and the swapouts is ~34x the original test period.

This is on a machine with an SSD (late-2013 MBP)
If you have a spinning HDD, the swapins and swapouts will be significantly more expensive in terms of performance/time.

If you quit all your programs, (other than Terminal), I suspect the hang may eventually recover; but if you have a hard disk that time might be quite long.

--
-Todd Short
// tshort at akamai.com<mailto:tshort at akamai.com>
// "One if by land, two if by sea, three if by the Internet."

On May 15, 2017, at 11:51 AM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu<mailto:uri at ll.mit.edu>> wrote:

I?m tracking the current OpenSSL master only on El Capitan 10.11.6. I could try it on Sierra 10.12.4, if you really expect it to make a difference.

In my case the hang is not for a short time. It lasts for more than 10 minutes, so I?m forced to interfere. For how long did it hang for you?
?
Regards,
Uri


On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via openssl-dev" <openssl-dev-bounces at openssl.org<mailto:openssl-dev-bounces at openssl.org> on behalf of openssl-dev at openssl.org<mailto:openssl-dev at openssl.org>> wrote:

We?ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short period of time with the unit-tests, but it eventually recovers.

What MacOS version are you running? I can try 10.12 later today.
--
-Todd Short
// tshort at akamai.com<mailto:tshort at akamai.com>
// "One if by land, two if by sea, three if by the Internet."

On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org<mailto:openssl-dev at openssl.org>> wrote:

Uri:

Look at https://github.com/openssl/openssl/pull/3455

I limited the test that hung your machine to Linux.

Rich: this removes the OpenSSL_assert() you see.

--
-Todd Short
// tshort at akamai.com<mailto:tshort at akamai.com>
// "One if by land, two if by sea, three if by the Internet."

On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org<mailto:openssl-dev at openssl.org>> wrote:

It?s trying to reserve 1<<34 bytes of memory? there goes your 16GB...
--
-Todd Short
// tshort at akamai.com<mailto:tshort at akamai.com>
// "One if by land, two if by sea, three if by the Internet."

On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu<mailto:uri at ll.mit.edu>> wrote:

Todd> Yes, it?s likely this is due to the amount of memory available in the machine. I tried to use reasonable values, but apparently not reasonable enough

Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, besides these tests :).
--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/d9ac3158/attachment-0001.html>

From uri at ll.mit.edu  Mon May 15 17:15:15 2017
From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL)
Date: Mon, 15 May 2017 17:15:15 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <80A99926-68B3-4775-9338-FFCDB43A3A7B@akamai.com>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
 <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>
 <B114BDAF-1F1C-41E9-826B-27775E375AB8@akamai.com>
 <80623024-B59F-4625-AC75-DF157B570136@akamai.com>
 <E45710AA-6B29-42BC-BF0D-F4CA6A367A12@ll.mit.edu>
 <80A99926-68B3-4775-9338-FFCDB43A3A7B@akamai.com>
Message-ID: <343EF696-58FE-48F9-8643-0C46E956C4D3@ll.mit.edu>

My disk is SSD, but computer load is pretty high. Which probably explains that recovery doesn't take place in 200-400 seconds...

On a semi-related note, I want able to locate mann.h file either.

Regards,
Uri

Sent from my iPhone

> On May 15, 2017, at 13:09, Short, Todd <tshort at akamai.com> wrote:
> 
> The time of the hang actually seems dependent on the number of applications running and your disk.
> 
> Since a large amount of memory becomes wired, there is very little available for programs and the OS to use (in some instances I have seen ~4MB non-wired memory). Things slow down due to swapping, etc. 
> 
> In my testing:
> 
> With almost no additional programs open, the hang-time is short, ~200 seconds.
> With a lot of programs open, the hang-time is increased, ~400 seconds; twice as long. And the number of swapins is 25x and the swapouts is ~34x the original test period.
> 
> This is on a machine with an SSD (late-2013 MBP)
> If you have a spinning HDD, the swapins and swapouts will be significantly more expensive in terms of performance/time.
> 
> If you quit all your programs, (other than Terminal), I suspect the hang may eventually recover; but if you have a hard disk that time might be quite long.
> 
> --
> -Todd Short
> // tshort at akamai.com
> // "One if by land, two if by sea, three if by the Internet."
> 
>> On May 15, 2017, at 11:51 AM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu> wrote:
>> 
>> I?m tracking the current OpenSSL master only on El Capitan 10.11.6. I could try it on Sierra 10.12.4, if you really expect it to make a difference.
>>  
>> In my case the hang is not for a short time. It lasts for more than 10 minutes, so I?m forced to interfere. For how long did it hang for you?
>> ? 
>> Regards,
>> Uri
>>  
>>  
>> On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via openssl-dev" <openssl-dev-bounces at openssl.org on behalf of openssl-dev at openssl.org> wrote:
>>  
>> We?ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short period of time with the unit-tests, but it eventually recovers. 
>>  
>> What MacOS version are you running? I can try 10.12 later today.
>> --
>> -Todd Short
>> // tshort at akamai.com
>> // "One if by land, two if by sea, three if by the Internet."
>>  
>>> On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org> wrote:  
>>>  
>>> Uri: 
>>>  
>>> Look at https://github.com/openssl/openssl/pull/3455
>>>  
>>> I limited the test that hung your machine to Linux.
>>>  
>>> Rich: this removes the OpenSSL_assert() you see.
>>>  
>>> --
>>> -Todd Short
>>> // tshort at akamai.com
>>> // "One if by land, two if by sea, three if by the Internet."
>>>  
>>>> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org> wrote:
>>>>  
>>>> It?s trying to reserve 1<<34 bytes of memory? there goes your 16GB...
>>>> --
>>>> -Todd Short
>>>> // tshort at akamai.com
>>>> // "One if by land, two if by sea, three if by the Internet."
>>>>  
>>>>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu> wrote:
>>>>>  
>>>>> Todd> Yes, it?s likely this is due to the amount of memory available in the machine. I tried to use reasonable values, but apparently not reasonable enough
>>>>>  
>>>>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, besides these tests :).
>>>>> -- 
>>>>> openssl-dev mailing list
>>>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>>>> 
>>>>  
>>>> -- 
>>>> openssl-dev mailing list
>>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>>> 
>>>  
>>> -- 
>>> openssl-dev mailing list
>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>> 
>>  
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/271887c7/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4223 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/271887c7/attachment-0001.bin>

From tshort at akamai.com  Mon May 15 17:17:48 2017
From: tshort at akamai.com (Short, Todd)
Date: Mon, 15 May 2017 17:17:48 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <343EF696-58FE-48F9-8643-0C46E956C4D3@ll.mit.edu>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
 <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>
 <B114BDAF-1F1C-41E9-826B-27775E375AB8@akamai.com>
 <80623024-B59F-4625-AC75-DF157B570136@akamai.com>
 <E45710AA-6B29-42BC-BF0D-F4CA6A367A12@ll.mit.edu>
 <80A99926-68B3-4775-9338-FFCDB43A3A7B@akamai.com>
 <343EF696-58FE-48F9-8643-0C46E956C4D3@ll.mit.edu>
Message-ID: <397B8873-EEA3-4C8E-8C1C-62630FBE4815@akamai.com>

s/want/wasn?t/ ?

So, no MLOCK_ONFAULT for you? That?s only included on Linux systems, which makes sense if you?re on a Mac.
--
-Todd Short
// tshort at akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 15, 2017, at 1:15 PM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu> wrote:
> 
> My disk is SSD, but computer load is pretty high. Which probably explains that recovery doesn't take place in 200-400 seconds...
> 
> On a semi-related note, I want able to locate mann.h file either.
> 
> Regards,
> Uri
> 
> Sent from my iPhone
> 
> On May 15, 2017, at 13:09, Short, Todd <tshort at akamai.com <mailto:tshort at akamai.com>> wrote:
> 
>> The time of the hang actually seems dependent on the number of applications running and your disk.
>> 
>> Since a large amount of memory becomes wired, there is very little available for programs and the OS to use (in some instances I have seen ~4MB non-wired memory). Things slow down due to swapping, etc. 
>> 
>> In my testing:
>> 
>> With almost no additional programs open, the hang-time is short, ~200 seconds.
>> With a lot of programs open, the hang-time is increased, ~400 seconds; twice as long. And the number of swapins is 25x and the swapouts is ~34x the original test period.
>> 
>> This is on a machine with an SSD (late-2013 MBP)
>> If you have a spinning HDD, the swapins and swapouts will be significantly more expensive in terms of performance/time.
>> 
>> If you quit all your programs, (other than Terminal), I suspect the hang may eventually recover; but if you have a hard disk that time might be quite long.
>> 
>> --
>> -Todd Short
>> // tshort at akamai.com <mailto:tshort at akamai.com>
>> // "One if by land, two if by sea, three if by the Internet."
>> 
>>> On May 15, 2017, at 11:51 AM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu <mailto:uri at ll.mit.edu>> wrote:
>>> 
>>> I?m tracking the current OpenSSL master only on El Capitan 10.11.6. I could try it on Sierra 10.12.4, if you really expect it to make a difference.
>>>  
>>> In my case the hang is not for a short time. It lasts for more than 10 minutes, so I?m forced to interfere. For how long did it hang for you?
>>> ? 
>>> Regards,
>>> Uri
>>>  
>>>  
>>> On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via openssl-dev" <openssl-dev-bounces at openssl.org <mailto:openssl-dev-bounces at openssl.org> on behalf of openssl-dev at openssl.org <mailto:openssl-dev at openssl.org>> wrote:
>>>  
>>> We?ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short period of time with the unit-tests, but it eventually recovers. 
>>>  
>>> What MacOS version are you running? I can try 10.12 later today.
>>> --
>>> -Todd Short
>>> // tshort at akamai.com <mailto:tshort at akamai.com>
>>> // "One if by land, two if by sea, three if by the Internet."
>>>  
>>>> On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org <mailto:openssl-dev at openssl.org>> wrote:  
>>>>  
>>>> Uri: 
>>>>  
>>>> Look at https://github.com/openssl/openssl/pull/3455 <https://github.com/openssl/openssl/pull/3455>
>>>>  
>>>> I limited the test that hung your machine to Linux.
>>>>  
>>>> Rich: this removes the OpenSSL_assert() you see.
>>>>  
>>>> --
>>>> -Todd Short
>>>> // tshort at akamai.com <mailto:tshort at akamai.com>
>>>> // "One if by land, two if by sea, three if by the Internet."
>>>>  
>>>>> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org <mailto:openssl-dev at openssl.org>> wrote:
>>>>>  
>>>>> It?s trying to reserve 1<<34 bytes of memory? there goes your 16GB...
>>>>> --
>>>>> -Todd Short
>>>>> // tshort at akamai.com <mailto:tshort at akamai.com>
>>>>> // "One if by land, two if by sea, three if by the Internet."
>>>>>  
>>>>>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu <mailto:uri at ll.mit.edu>> wrote:
>>>>>>  
>>>>>> Todd> Yes, it?s likely this is due to the amount of memory available in the machine. I tried to use reasonable values, but apparently not reasonable enough
>>>>>>  
>>>>>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, besides these tests :).
>>>>>> -- 
>>>>>> openssl-dev mailing list
>>>>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev <https://mta.openssl.org/mailman/listinfo/openssl-dev>
>>>>>  
>>>>> -- 
>>>>> openssl-dev mailing list
>>>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev <https://mta.openssl.org/mailman/listinfo/openssl-dev>
>>>>  
>>>> -- 
>>>> openssl-dev mailing list
>>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev <https://mta.openssl.org/mailman/listinfo/openssl-dev>
>>>  
>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/32352729/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4407 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/32352729/attachment-0001.bin>

From uri at ll.mit.edu  Mon May 15 17:19:29 2017
From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL)
Date: Mon, 15 May 2017 17:19:29 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <397B8873-EEA3-4C8E-8C1C-62630FBE4815@akamai.com>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
 <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>
 <B114BDAF-1F1C-41E9-826B-27775E375AB8@akamai.com>
 <80623024-B59F-4625-AC75-DF157B570136@akamai.com>
 <E45710AA-6B29-42BC-BF0D-F4CA6A367A12@ll.mit.edu>
 <80A99926-68B3-4775-9338-FFCDB43A3A7B@akamai.com>
 <343EF696-58FE-48F9-8643-0C46E956C4D3@ll.mit.edu>
 <397B8873-EEA3-4C8E-8C1C-62630FBE4815@akamai.com>
Message-ID: <5FE08AB9-D6A2-4E99-B205-C6DFFA6A429C@ll.mit.edu>

Yes and yes. :-)

Regards,
Uri

Sent from my iPhone

> On May 15, 2017, at 13:18, Short, Todd <tshort at akamai.com> wrote:
> 
> s/want/wasn?t/ ?
> 
> So, no MLOCK_ONFAULT for you? That?s only included on Linux systems, which makes sense if you?re on a Mac.
> --
> -Todd Short
> // tshort at akamai.com
> // "One if by land, two if by sea, three if by the Internet."
> 
>> On May 15, 2017, at 1:15 PM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu> wrote:
>> 
>> My disk is SSD, but computer load is pretty high. Which probably explains that recovery doesn't take place in 200-400 seconds...
>> 
>> On a semi-related note, I want able to locate mann.h file either.
>> 
>> Regards,
>> Uri
>> 
>> Sent from my iPhone
>> 
>>> On May 15, 2017, at 13:09, Short, Todd <tshort at akamai.com> wrote:
>>> 
>>> The time of the hang actually seems dependent on the number of applications running and your disk.
>>> 
>>> Since a large amount of memory becomes wired, there is very little available for programs and the OS to use (in some instances I have seen ~4MB non-wired memory). Things slow down due to swapping, etc. 
>>> 
>>> In my testing:
>>> 
>>> With almost no additional programs open, the hang-time is short, ~200 seconds.
>>> With a lot of programs open, the hang-time is increased, ~400 seconds; twice as long. And the number of swapins is 25x and the swapouts is ~34x the original test period.
>>> 
>>> This is on a machine with an SSD (late-2013 MBP)
>>> If you have a spinning HDD, the swapins and swapouts will be significantly more expensive in terms of performance/time.
>>> 
>>> If you quit all your programs, (other than Terminal), I suspect the hang may eventually recover; but if you have a hard disk that time might be quite long.
>>> 
>>> --
>>> -Todd Short
>>> // tshort at akamai.com
>>> // "One if by land, two if by sea, three if by the Internet."
>>> 
>>>> On May 15, 2017, at 11:51 AM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu> wrote:
>>>> 
>>>> I?m tracking the current OpenSSL master only on El Capitan 10.11.6. I could try it on Sierra 10.12.4, if you really expect it to make a difference.
>>>>  
>>>> In my case the hang is not for a short time. It lasts for more than 10 minutes, so I?m forced to interfere. For how long did it hang for you?
>>>> ? 
>>>> Regards,
>>>> Uri
>>>>  
>>>>  
>>>> On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via openssl-dev" <openssl-dev-bounces at openssl.org on behalf of openssl-dev at openssl.org> wrote:
>>>>  
>>>> We?ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short period of time with the unit-tests, but it eventually recovers. 
>>>>  
>>>> What MacOS version are you running? I can try 10.12 later today.
>>>> --
>>>> -Todd Short
>>>> // tshort at akamai.com
>>>> // "One if by land, two if by sea, three if by the Internet."
>>>>  
>>>>> On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org> wrote:  
>>>>>  
>>>>> Uri: 
>>>>>  
>>>>> Look at https://github.com/openssl/openssl/pull/3455
>>>>>  
>>>>> I limited the test that hung your machine to Linux.
>>>>>  
>>>>> Rich: this removes the OpenSSL_assert() you see.
>>>>>  
>>>>> --
>>>>> -Todd Short
>>>>> // tshort at akamai.com
>>>>> // "One if by land, two if by sea, three if by the Internet."
>>>>>  
>>>>>> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev <openssl-dev at openssl.org> wrote:
>>>>>>  
>>>>>> It?s trying to reserve 1<<34 bytes of memory? there goes your 16GB...
>>>>>> --
>>>>>> -Todd Short
>>>>>> // tshort at akamai.com
>>>>>> // "One if by land, two if by sea, three if by the Internet."
>>>>>>  
>>>>>>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu> wrote:
>>>>>>>  
>>>>>>> Todd> Yes, it?s likely this is due to the amount of memory available in the machine. I tried to use reasonable values, but apparently not reasonable enough
>>>>>>>  
>>>>>>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, besides these tests :).
>>>>>>> -- 
>>>>>>> openssl-dev mailing list
>>>>>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>>>>>> 
>>>>>>  
>>>>>> -- 
>>>>>> openssl-dev mailing list
>>>>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>>>>> 
>>>>>  
>>>>> -- 
>>>>> openssl-dev mailing list
>>>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>>>> 
>>>>  
>>> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/f4abf1b0/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4223 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/f4abf1b0/attachment-0001.bin>

From bkaduk at akamai.com  Mon May 15 17:20:41 2017
From: bkaduk at akamai.com (Benjamin Kaduk)
Date: Mon, 15 May 2017 12:20:41 -0500
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <343EF696-58FE-48F9-8643-0C46E956C4D3@ll.mit.edu>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
 <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>
 <B114BDAF-1F1C-41E9-826B-27775E375AB8@akamai.com>
 <80623024-B59F-4625-AC75-DF157B570136@akamai.com>
 <E45710AA-6B29-42BC-BF0D-F4CA6A367A12@ll.mit.edu>
 <80A99926-68B3-4775-9338-FFCDB43A3A7B@akamai.com>
 <343EF696-58FE-48F9-8643-0C46E956C4D3@ll.mit.edu>
Message-ID: <221cfda9-7fed-e1b1-cf62-cea2397e84a7@akamai.com>

On 05/15/2017 12:15 PM, Blumenthal, Uri - 0553 - MITLL wrote:
>
> On a semi-related note, I want able to locate mann.h file either.

`man mmap` will list any headers needed for the mmap() declaration and
flag values.
On the random OS X machine I have handy, it claims <sys/mman.h> is
needed, and a /usr/include/sys/mman.h is present.

-Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/6bcbb967/attachment.html>

From uri at ll.mit.edu  Mon May 15 17:51:25 2017
From: uri at ll.mit.edu (Blumenthal, Uri - 0553 - MITLL)
Date: Mon, 15 May 2017 17:51:25 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <221cfda9-7fed-e1b1-cf62-cea2397e84a7@akamai.com>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
 <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>
 <B114BDAF-1F1C-41E9-826B-27775E375AB8@akamai.com>
 <80623024-B59F-4625-AC75-DF157B570136@akamai.com>
 <E45710AA-6B29-42BC-BF0D-F4CA6A367A12@ll.mit.edu>
 <80A99926-68B3-4775-9338-FFCDB43A3A7B@akamai.com>
 <343EF696-58FE-48F9-8643-0C46E956C4D3@ll.mit.edu>
 <221cfda9-7fed-e1b1-cf62-cea2397e84a7@akamai.com>
Message-ID: <ACFA4F9B-6A1E-49E9-BBFD-0B073AF44584@ll.mit.edu>

On 5/15/17, 1:20 PM, "openssl-dev on behalf of Benjamin Kaduk via openssl-dev" <openssl-dev-bounces at openssl.org on behalf of openssl-dev at openssl.org> wrote:

> On a semi-related note, I want able to locate mann.h file either.


`man mmap` will list any headers needed for the mmap() declaration and flag values.
On the random OS X machine I have handy, it claims <sys/mman.h> is needed, 

and a /usr/include/sys/mman.h is present.


Thanks ? I confirm that `man mmap` mentions /usr/include/sys/mman.h, and that file does exist (how could my first `find` miss it?!). 

This file does not contain MLOCK_ONFAULT though.

 

It has MAP_ANON:

 

/*

 * Mapping type

 */

#define MAP_FILE        0x0000  /* map from file (default) */

#define MAP_ANON        0x1000  /* allocated from memory, swap space */

#define MAP_ANONYMOUS   MAP_ANON

 

And as I already mentioned, it has a world-accessible (figuratively speaking :) /dev/zero that can be opened read/write.

 

Also, with #3455 applied the problem disappeared (thankfully :).

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/c55bad9d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5211 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170515/c55bad9d/attachment.bin>

From tshort at akamai.com  Tue May 16 19:42:48 2017
From: tshort at akamai.com (Short, Todd)
Date: Tue, 16 May 2017 19:42:48 +0000
Subject: [openssl-dev] 90-test_secmem.t hangs the machine for good
In-Reply-To: <ACFA4F9B-6A1E-49E9-BBFD-0B073AF44584@ll.mit.edu>
References: <FC66FFD5-3C19-476B-B359-813FB6995EE9@ll.mit.edu>
 <8d9ee7e6-1408-d35e-6c65-3b64e7afd1bd@akamai.com>
 <DC63BA4B-64A7-4619-97B4-9BA6F5314ABA@ll.mit.edu>
 <2370a31d-7977-d9e1-60bf-75a3fc0be613@akamai.com>
 <BA8914AD-A118-470C-A09C-2906096824B3@ll.mit.edu>
 <6B74200D-ABC6-4615-AE93-B06F65C2FE2C@akamai.com>
 <B114BDAF-1F1C-41E9-826B-27775E375AB8@akamai.com>
 <80623024-B59F-4625-AC75-DF157B570136@akamai.com>
 <E45710AA-6B29-42BC-BF0D-F4CA6A367A12@ll.mit.edu>
 <80A99926-68B3-4775-9338-FFCDB43A3A7B@akamai.com>
 <343EF696-58FE-48F9-8643-0C46E956C4D3@ll.mit.edu>
 <221cfda9-7fed-e1b1-cf62-cea2397e84a7@akamai.com>
 <ACFA4F9B-6A1E-49E9-BBFD-0B073AF44584@ll.mit.edu>
Message-ID: <DEF6E859-2342-45F9-8E77-3E4AA0C74E6A@akamai.com>

MLOCK_ONFAULT is a Linux-only feature (hence the need to include <linux/mman.h> wrapped by OPENSSL_SYS_LINUX.

So, you should not be encountering any MLOCK_ONFAULT or <linux/mman.h> issues on MacOS.
--
-Todd Short
// tshort at akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 15, 2017, at 1:51 PM, Blumenthal, Uri - 0553 - MITLL <uri at ll.mit.edu> wrote:
> 
> On 5/15/17, 1:20 PM, "openssl-dev on behalf of Benjamin Kaduk via openssl-dev" <openssl-dev-bounces at openssl.org <mailto:openssl-dev-bounces at openssl.org> on behalf of openssl-dev at openssl.org <mailto:openssl-dev at openssl.org>> wrote:
>> > On a semi-related note, I want able to locate mann.h file either.
> 
> `man mmap` will list any headers needed for the mmap() declaration and flag values.
> On the random OS X machine I have handy, it claims <sys/mman.h> is needed, 
> and a /usr/include/sys/mman.h is present.
> 
> Thanks ? I confirm that `man mmap` mentions /usr/include/sys/mman.h, and that file does exist (how could my first `find` miss it?!). 
> This file does not contain MLOCK_ONFAULT though.
>  
> It has MAP_ANON:
>  
> /*
>  * Mapping type
>  */
> #define MAP_FILE        0x0000  /* map from file (default) */
> #define MAP_ANON        0x1000  /* allocated from memory, swap space */
> #define MAP_ANONYMOUS   MAP_ANON
>  
> And as I already mentioned, it has a world-accessible (figuratively speaking :) /dev/zero that can be opened read/write.
>  
> Also, with #3455 applied the problem disappeared (thankfully :).
>  
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev <https://mta.openssl.org/mailman/listinfo/openssl-dev>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170516/7127b27d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4407 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170516/7127b27d/attachment-0001.bin>

From bhat.jayalakshmi at gmail.com  Thu May 18 05:32:00 2017
From: bhat.jayalakshmi at gmail.com (Jayalakshmi bhat)
Date: Wed, 17 May 2017 23:32:00 -0600
Subject: [openssl-dev] OpenSSL 1.1.1 release timeframe
Message-ID: <CALq8RvKj2FDDK_TOmH012nX69eR+tZa6Du+Bx+9+RbnuvTkXAA@mail.gmail.com>

Hi All,

Please can any one let me know the release date or time line for OpenSSL
1.1.1?

Regards
Jayalakshmi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170517/8fc75338/attachment.html>

From matt at openssl.org  Thu May 18 09:17:29 2017
From: matt at openssl.org (Matt Caswell)
Date: Thu, 18 May 2017 10:17:29 +0100
Subject: [openssl-dev] [openssl-users] OpenSSL 1.1.1 release timeframe
In-Reply-To: <CALq8RvKj2FDDK_TOmH012nX69eR+tZa6Du+Bx+9+RbnuvTkXAA@mail.gmail.com>
References: <CALq8RvKj2FDDK_TOmH012nX69eR+tZa6Du+Bx+9+RbnuvTkXAA@mail.gmail.com>
Message-ID: <9d2c80f2-c375-d4a4-4089-1c7079e5609c@openssl.org>



On 18/05/17 06:32, Jayalakshmi bhat wrote:
> Please can any one let me know the release date or time line for OpenSSL
> 1.1.1? 

We have not set a date as yet. At the very least we will not be able to
release until the IETF takes TLSv1.3 out of draft status - which is not
in our control.

Matt

From bhat.jayalakshmi at gmail.com  Thu May 18 10:00:12 2017
From: bhat.jayalakshmi at gmail.com (Jayalakshmi bhat)
Date: Thu, 18 May 2017 15:30:12 +0530
Subject: [openssl-dev] [openssl-users] OpenSSL 1.1.1 release timeframe
In-Reply-To: <9d2c80f2-c375-d4a4-4089-1c7079e5609c@openssl.org>
References: <CALq8RvKj2FDDK_TOmH012nX69eR+tZa6Du+Bx+9+RbnuvTkXAA@mail.gmail.com>
 <9d2c80f2-c375-d4a4-4089-1c7079e5609c@openssl.org>
Message-ID: <CALq8RvJsfc8OjMJHVANKUi_U03z7Pmz1Do0uKbmx0cehO9JzZA@mail.gmail.com>

Hi Matt,

I do understand. Thanks a lot for the reply.

Regards
Jayalakshmi

On Thu, May 18, 2017 at 2:47 PM, Matt Caswell <matt at openssl.org> wrote:

>
>
> On 18/05/17 06:32, Jayalakshmi bhat wrote:
> > Please can any one let me know the release date or time line for OpenSSL
> > 1.1.1?
>
> We have not set a date as yet. At the very least we will not be able to
> release until the IETF takes TLSv1.3 out of draft status - which is not
> in our control.
>
> Matt
> --
> openssl-users mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170518/7aa782de/attachment.html>

From matt at openssl.org  Mon May 22 16:04:30 2017
From: matt at openssl.org (Matt Caswell)
Date: Mon, 22 May 2017 17:04:30 +0100
Subject: [openssl-dev] Forthcoming OpenSSL releases
Message-ID: <d081c98a-c952-d709-b80a-21e460faaaaf@openssl.org>


Forthcoming OpenSSL releases
============================

The OpenSSL project team would like to announce the forthcoming release
of OpenSSL versions 1.0.2l and 1.1.0f.

These releases will be made available on 25th May 2017 between
approximately 1200-1600 UTC.

Note: These are bug-fix only releases. No security defects are addressed
in these releases.

Please also note that, as per our previous announcements, support for
1.0.1 ended on 31st December 2016.

Yours

The OpenSSL Project Team

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 480 bytes
Desc: OpenPGP digital signature
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170522/deefc73d/attachment.sig>

From rschm2 at unh.newhaven.edu  Tue May 23 15:38:15 2017
From: rschm2 at unh.newhaven.edu (Schmicker, Robert)
Date: Tue, 23 May 2017 15:38:15 +0000
Subject: [openssl-dev] Documentation for Integrating a New Symmetric Cipher
Message-ID: <CAADCE4E-862B-41C7-B184-E80DBD09FEB0@unh.newhaven.edu>

Hello,

After following this as well as the users email chain for the past several months I?ve noticed that once in a blue moon people will ask how to integrate a new cipher. In response I decided to write up some documentation on the matter and just added it to ?Internals and Development? on the front page of the wiki.

https://wiki.openssl.org/index.php/How_to_Integrate_a_Symmetric_Cipher

Please let me know if there?s anything I?ve missed or needs clarification.

Thank you,
Rob Schmicker
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170523/42e83e06/attachment.html>

From openssl at openssl.org  Thu May 25 13:57:56 2017
From: openssl at openssl.org (OpenSSL)
Date: Thu, 25 May 2017 13:57:56 +0000
Subject: [openssl-dev] OpenSSL version 1.0.2l published
Message-ID: <20170525135756.GA25645@openssl.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


   OpenSSL version 1.0.2l released
   ===============================

   OpenSSL - The Open Source toolkit for SSL/TLS
   https://www.openssl.org/

   The OpenSSL project team is pleased to announce the release of
   version 1.0.2l of our open source toolkit for SSL/TLS. For details
   of changes and known issues see the release notes at:

        https://www.openssl.org/news/openssl-1.0.2-notes.html

   OpenSSL 1.0.2l is available for download via HTTP and FTP from the
   following master locations (you can find the various FTP mirrors under
   https://www.openssl.org/source/mirror.html):

     * https://www.openssl.org/source/
     * ftp://ftp.openssl.org/source/

   The distribution file name is:

    o openssl-1.0.2l.tar.gz
      Size: 5365054
      SHA1 checksum: b58d5d0e9cea20e571d903aafa853e2ccd914138
      SHA256 checksum: ce07195b659e75f4e1db43552860070061f156a98bb37b672b101ba6e3ddf30c

   The checksums were calculated using the following commands:

    openssl sha1 openssl-1.0.2l.tar.gz
    openssl sha256 openssl-1.0.2l.tar.gz

   Yours,

   The OpenSSL Project Team.

-----BEGIN PGP SIGNATURE-----

iQEcBAEBCAAGBQJZJtRNAAoJENnE0m0OYESROsEIALuf8f97c3YgUOz+72Cqrd+x
NEDBmDASsRuIlqkXSkN6CunLUb/FQtCMP1n7POsYMAdNqJz+1tOxwxS42j4qsoxj
AdNjf7qn/B0Jhd1A6q6GGxO25tmZne3GEga76ya99+/FRMmUWk/QFdCkaNlRtqf+
+6B3KLCAw/pOsGucS8FIk8Wlr1gR/VTiwlxY63ZhzQg941vVNaOsuz+CNWlTc1pW
E06cEBnbkjo23LcZH3E07TWHJdDayfROsZTkOZ30uXXo4Xk/KK/Mk4lOAMd7UPMh
gxt/jSNcIjf32sGsJRwydlUq7f4OjQQFkFmm8GDY6HgAyRyN4EKCfEWgrCqQs1w=
=F+zf
-----END PGP SIGNATURE-----

From openssl at openssl.org  Thu May 25 13:58:18 2017
From: openssl at openssl.org (OpenSSL)
Date: Thu, 25 May 2017 13:58:18 +0000
Subject: [openssl-dev] OpenSSL version 1.1.0f published
Message-ID: <20170525135818.GA26011@openssl.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


   OpenSSL version 1.1.0f released
   ===============================

   OpenSSL - The Open Source toolkit for SSL/TLS
   https://www.openssl.org/

   The OpenSSL project team is pleased to announce the release of
   version 1.1.0f of our open source toolkit for SSL/TLS. For details
   of changes and known issues see the release notes at:

        https://www.openssl.org/news/openssl-1.1.0-notes.html

   OpenSSL 1.1.0f is available for download via HTTP and FTP from the
   following master locations (you can find the various FTP mirrors under
   https://www.openssl.org/source/mirror.html):

     * https://www.openssl.org/source/
     * ftp://ftp.openssl.org/source/

   The distribution file name is:

    o openssl-1.1.0f.tar.gz
      Size: 5278176
      SHA1 checksum: 9e3e02bc8b4965477a7a1d33be1249299a9deb15
      SHA256 checksum: 12f746f3f2493b2f39da7ecf63d7ee19c6ac9ec6a4fcd8c229da8a522cb12765

   The checksums were calculated using the following commands:

    openssl sha1 openssl-1.1.0f.tar.gz
    openssl sha256 openssl-1.1.0f.tar.gz

   Yours,

   The OpenSSL Project Team.

-----BEGIN PGP SIGNATURE-----

iQEcBAEBCAAGBQJZJtIgAAoJENnE0m0OYESRKjUH/RkGMsp/PM+PxHXgZ0K5nvYP
jbxfoun1iQ27NkFKs/HTueWl5EgVEH4w/qT1SSXUQ8DM/2YY7Z8fDFUh7Xrx5mEM
ud5q4pqbdDRjmF7HYMhbR1D6dvKjkOpHPV6OzD3iHg8ssUQNaZpvrn/1KCUZFxp+
tp/Mt9qAqlEAtFGo+qw7wIKa+8Do1y5L151HBk9jPSWIPPGnRzD8z+M0rbTD+bjx
t/1emoySoRcUjwkq7xqdBix08Sc69RT8ms8AVhINC8gcMdN93UKu4P4JN7qf2Cai
Krx1nkEYQJjp65WB0RGuLaZ0Bs80jJydknvTvFj3azeDfMLCjXg+GX1YSAh6R6k=
=fyH8
-----END PGP SIGNATURE-----

From rainer.jung at kippdata.de  Thu May 25 19:22:17 2017
From: rainer.jung at kippdata.de (Rainer Jung)
Date: Thu, 25 May 2017 21:22:17 +0200
Subject: [openssl-dev] Fix use of "can_load" in run_tests.pl (was GH PR
	#3424)
In-Reply-To: <1494428711.375654.28303.nullmailer@dev.openssl.org>
References: <1494428711.375654.28303.nullmailer@dev.openssl.org>
Message-ID: <c06f08c4-ce04-5f36-36a9-f69375c4dfad@kippdata.de>

The below change doesn't work, because "can_load" must be used differently.

When running on SLES 11 which contains perl 1.10.0 (and thus no 
TAP::Harness module), run_tests.pl gives:

Can't locate object method "new" via package "TAP::Harness" (perhaps you 
forgot to load "TAP::Harness"?) at .././test/run_tests.pl line 64.

Please see below for a patch.

Am 10.05.2017 um 17:05 schrieb Richard Levitte:
> The branch master has been updated
>        via  76e0d0b21cc4e8a879d54f4d78a392826dadb1d1 (commit)
>       from  03d8e9cb43da5c524e5890a5a51e2c77f1fbd789 (commit)
>
>
> - Log -----------------------------------------------------------------
> commit 76e0d0b21cc4e8a879d54f4d78a392826dadb1d1
> Author: Richard Levitte <levitte at openssl.org>
> Date:   Wed May 10 12:58:36 2017 +0200
>
>     Prefer TAP::Harness over Test::Harness
>
>     TAP:Harness came along in perl 5.10.1, and since we claim to support
>     perl 5.10.0 in configuration and testing, we can only load it
>     conditionally.
>
>     The main reason to use TAP::Harness rather than Test::Harness is its
>     capability to merge stdout and stderr output from the test recipes,
>     which Test::Harness can't.  The merge gives much more comprehensible
>     output when testing verbosely.
>
>     Reviewed-by: Rich Salz <rsalz at openssl.org>
>     Reviewed-by: Matt Caswell <matt at openssl.org>
>     (Merged from https://github.com/openssl/openssl/pull/3424)
>
> -----------------------------------------------------------------------
>
> Summary of changes:
>  test/run_tests.pl | 58 ++++++++++++++++++++++++++++++++++++++++++++-----------
>  1 file changed, 47 insertions(+), 11 deletions(-)
>
> diff --git a/test/run_tests.pl b/test/run_tests.pl
> index 9c5ade1..aa1dba0 100644
> --- a/test/run_tests.pl
> +++ b/test/run_tests.pl
> @@ -17,7 +17,10 @@ BEGIN {
>  use File::Spec::Functions qw/catdir catfile curdir abs2rel rel2abs/;
>  use File::Basename;
>  use if $^O ne "VMS", 'File::Glob' => qw/glob/;
> -use Test::Harness qw/runtests $switches/;
> +use Module::Load::Conditional qw(can_load);
> +
> +my $TAP_Harness = can_load({modules => [ 'TAP::Harness' ]})
> +    ? 'TAP::Harness' : 'OpenSSL::TAP::Harness';

The following patch fixes the can_load use:

--- test/run_tests.pl	2017-05-25 16:21:00.000000000 +0200
+++ test/run_tests.pl	2017-05-25 21:07:20.066149000 +0200
@@ -19,7 +19,7 @@
  use if $^O ne "VMS", 'File::Glob' => qw/glob/;
  use Module::Load::Conditional qw(can_load);

-my $TAP_Harness = can_load({modules => [ 'TAP::Harness' ]})
+my $TAP_Harness = can_load(modules => { 'TAP::Harness' => undef })
      ? 'TAP::Harness' : 'OpenSSL::TAP::Harness';

  my $srctop = $ENV{SRCTOP} || $ENV{TOP};

It should be applied to master and the 1.1.0 branch.

Should I open a GH issue or pull request for that trivial change?

Thanks a bunch,

Rainer

From kurt at roeckx.be  Thu May 25 20:10:14 2017
From: kurt at roeckx.be (Kurt Roeckx)
Date: Thu, 25 May 2017 22:10:14 +0200
Subject: [openssl-dev] Fix use of "can_load" in run_tests.pl (was GH PR
 #3424)
In-Reply-To: <c06f08c4-ce04-5f36-36a9-f69375c4dfad@kippdata.de>
References: <1494428711.375654.28303.nullmailer@dev.openssl.org>
 <c06f08c4-ce04-5f36-36a9-f69375c4dfad@kippdata.de>
Message-ID: <20170525201014.yhmbmvpnxf2nijje@roeckx.be>

On Thu, May 25, 2017 at 09:22:17PM +0200, Rainer Jung wrote:
> 
> Should I open a GH issue or pull request for that trivial change?

Yes. Everything is required to be reviewed, and we use github for
that.


Kurt


From vladislav at vm-0.com  Sat May 27 10:03:57 2017
From: vladislav at vm-0.com (Vladislav Yarmak)
Date: Sat, 27 May 2017 13:03:57 +0300
Subject: [openssl-dev] GH PR #3353
Message-ID: <20170527130357.5c16ea08@dt1>

Hello,

Recently I've placed pull request on Github related to RSA key
generation and FIPS conformance. Could someone review proposed change?

Here is link for your convenience:
https://github.com/openssl/openssl/pull/3353

-- 
Best Regards,
Vladislav Yarmak