Subject: bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

From: Ken Brown <[email-masked]>
Date: Fri, 08 Aug 2014 09:09:31 -0400

malloc_enable_thread() in gmalloc.c calls pthread_atfork to set up fork 
handlers.  There are a couple of problems with this, but the immediate 
reason for this bug report is a problem on Cygwin that was reported in 
the thread starting at

   https://cygwin.com/ml/cygwin/2014-07/msg00387.html

and continuing at

   https://cygwin.com/ml/cygwin/2014-08/msg00001.html.

The issue is that the 'prepare' fork handler locks the pthread_mutexes 
prior to forking, and the ensuing processing of the fork command by the 
Cygwin DLL leads to a call to malloc in the same thread, resulting in 
deadlock.  This is a long-standing problem, but it was masked until 
recently by the fact that pthread_mutexes on Cygwin were ERRORCHECK 
mutexes by default.  As of Cygwin 1.7.31, pthread_mutexes are now NORMAL 
mutexes by default, so the problem has shown up.

A simple short-term workaround would be to explicitly set the mutexes to 
be ERRORCHECK or RECURSIVE mutexes on Cygwin, thereby restoring the 
previous behavior.  But this does not seem like the right long-term 
solution, for the reasons explained here:

   https://cygwin.com/ml/cygwin/2014-08/msg00161.html
   https://cygwin.com/ml/cygwin/2014-08/msg00175.html

I know nothing about this other than what I learned from the two 
messages above, so I would appreciate some guidance.

Ken