Spencer Baugh <sbaugh@janestreet.com> writes:
> One theory: clear_waiting_thread_info only clears .waiting_thread up to
> max_desc.

OK, I figured it out.  This is indeed the sole issue.  With this patch:

diff --git a/src/process.c b/src/process.c
index 1a5c99139a6..5db427a4a99 100644
--- a/src/process.c
+++ b/src/process.c
@@ -670,10 +670,15 @@ clear_waiting_thread_info (void)
   int fd;
 
   eassert (max_desc < FD_SETSIZE);
-  for (fd = 0; fd <= max_desc; ++fd)
+  for (fd = 0; fd < FD_SETSIZE; ++fd)
     {
-      if (fd_callback_info[fd].waiting_thread == current_thread)
+      if (fd_callback_info[fd].waiting_thread == current_thread) {
+	if (fd > max_desc) {
+	  fprintf (stderr, "fd too high: %d > max_desc=%d\n", fd, max_desc);
+	  terminate_due_to_signal (SIGABRT, INT_MAX);
+	}
 	fd_callback_info[fd].waiting_thread = NULL;
+      }
     }
 }
 
And my earlier reproducer (but not my earlier patch), we quickly get an
abort, indicating that there's an fd_callback_info slot which would be
left with a non-NULL .waiting_thread.

The cause is that recompute_max_desc runs when a file descriptor is
deleted, and reduces max_desc.  But when it does so, that means
clear_waiting_thread_info will not get the chance to clear the
waiting_thread field in fd_callback_info slots above the new max_desc.

My initial thought was that recompute_max_desc should zero the
fd_callback_info[fd] entry when it reduces max_desc.  But it turns out
to be equivalent, and simpler, to zero fd_callback_info[desc] completely
instead of just setting flags to 0 right before we call
recompute_max_desc.

I do this in the attached patch, which fixes the bug.