GNU bug report logs - #21804
25.0.50; file-notify-tests failure on Cygwin

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 21804 in the body.
You can then email your comments to 21804 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: bug-gnu-emacs <at> gnu.org
Subject: 25.0.50; file-notify-tests failure on Cygwin
Date: Sun, 1 Nov 2015 10:51:19 -0500

[Message part 1 (text/plain, inline)]

The following tests are failing on Cygwin:

file-notify-test02-events
file-notify-test02-events-remote
file-notify-test04-file-validity
file-notify-test04-file-validity-remote
file-notify-test05-dir-validity

The first two have been failing for as long as I can remember, but I 
never got around to reporting it.

The next two failed immediately after they were introduced in the 
following commit:

commit 7a3f3183cd8faff8901ead547711e1c90ea02efe
Author: Tassilo Horn <tsdh <at> gnu.org>
Date: Mon Sep 14 08:03:11 2015 +0200

Test file-notify-valid-p.

* test/automated/file-notify-tests.el
(file-notify-test04-file-validity, file-notify-test05-dir-validity): New
tests.

The last one started failing shortly after that, but I haven’t pinned 
down the exact commit.

I’m attaching the log based on a build from git 1500667.  Please let me 
know what I can do to help track this down.
Ken

[file-notify-tests.log (application/octet-stream, attachment)]

Message #8 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: michael.albinus <at> gmx.de, 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Sun, 01 Nov 2015 18:40:12 +0200

> From: Ken Brown <kbrown <at> cornell.edu>
> Date: Sun, 1 Nov 2015 10:51:19 -0500
> Cc: Michael Albinus <michael.albinus <at> gmx.de>
> 
> The following tests are failing on Cygwin:
> 
> file-notify-test02-events
> file-notify-test02-events-remote
> file-notify-test04-file-validity
> file-notify-test04-file-validity-remote
> file-notify-test05-dir-validity

What back-end do you use? gfilenotify or something else?

Message #11 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: michael.albinus <at> gmx.de, 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Sun, 1 Nov 2015 12:25:35 -0500

On 11/1/2015 11:40 AM, Eli Zaretskii wrote:
>> From: Ken Brown <kbrown <at> cornell.edu>
>> Date: Sun, 1 Nov 2015 10:51:19 -0500
>> Cc: Michael Albinus <michael.albinus <at> gmx.de>
>>
>> The following tests are failing on Cygwin:
>>
>> file-notify-test02-events
>> file-notify-test02-events-remote
>> file-notify-test04-file-validity
>> file-notify-test04-file-validity-remote
>> file-notify-test05-dir-validity
>
> What back-end do you use? gfilenotify or something else?

gfilenotify

Message #14 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Sun, 01 Nov 2015 18:34:23 +0100

Ken Brown <kbrown <at> cornell.edu> writes:

Hi Ken,

> The following tests are failing on Cygwin:
>
> file-notify-test02-events
> file-notify-test02-events-remote
> file-notify-test04-file-validity
> file-notify-test04-file-validity-remote
> file-notify-test05-dir-validity
>
> I’m attaching the log based on a build from git 1500667.  Please let
> me know what I can do to help track this down.

You are using

> Local library: `gfilenotify'
>    passed   1/12  file-notify-test00-availability
> Remote command: `gvfs-monitor-dir'
>    passed   2/12  file-notify-test00-availability-remote

With the same settings, for me it fails for

file-notify-test02-events
file-notify-test04-file-validity

With local gfilenotify, there is a problem to get the `stopped' event
once a watched file or directory has been deleted. Notably, it works for
the remote gvfs-monitor-dir, which shall behave similar.

I will try to fix these cases first, because I could reproduce them
locally. Let's see how it behaves afterwards.

> Ken

Best regards, Michael.

Message #17 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Tue, 03 Nov 2015 18:20:53 +0100

Michael Albinus <michael.albinus <at> gmx.de> writes:

Hi Ken,

> With the same settings, for me it fails for
>
> file-notify-test02-events
> file-notify-test04-file-validity
>
> With local gfilenotify, there is a problem to get the `stopped' event
> once a watched file or directory has been deleted. Notably, it works for
> the remote gvfs-monitor-dir, which shall behave similar.
>
> I will try to fix these cases first, because I could reproduce them
> locally. Let's see how it behaves afterwards.

I've fixed this with commit 436ed2399ade5c41b8ed3cffe177fb5210eff574.
Could you, please, check, whether it improves your tests?

>> Ken

Best regards, Michael.

Message #20 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Tue, 3 Nov 2015 22:54:50 -0500

[Message part 1 (text/plain, inline)]

On 11/3/2015 12:20 PM, Michael Albinus wrote:
> Michael Albinus <michael.albinus <at> gmx.de> writes:
>
> Hi Ken,
>
>> With the same settings, for me it fails for
>>
>> file-notify-test02-events
>> file-notify-test04-file-validity
>>
>> With local gfilenotify, there is a problem to get the `stopped' event
>> once a watched file or directory has been deleted. Notably, it works for
>> the remote gvfs-monitor-dir, which shall behave similar.
>>
>> I will try to fix these cases first, because I could reproduce them
>> locally. Let's see how it behaves afterwards.
>
> I've fixed this with commit 436ed2399ade5c41b8ed3cffe177fb5210eff574.
> Could you, please, check, whether it improves your tests?

Hi Michael,

Yes, it improves the tests slightly; file-notify-test05-dir-validity now 
passes.  But the other four still fail.  The new log is attached.

Ken

[file-notify-tests.log (application/octet-stream, attachment)]

Message #23 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Wed, 04 Nov 2015 09:20:12 +0100

Ken Brown <kbrown <at> cornell.edu> writes:

> Hi Michael,

Hi Ken,

> Yes, it improves the tests slightly; file-notify-test05-dir-validity
> now passes.  But the other four still fail.  The new log is attached.

In all four cases, `file-notify--test-with-events' returned nil, where
it was supposed to show the incoming events. Strange.

In lisp/filenotify.el and test/automated/file-notify-tests.el, I've
prepared trace messages. You'll find them searching for ";;(message".
Could you, please, enable these traces and show the resulting log?

Another idea is to increase the timeouts in `file-notify--test-timeout'.

Btw (and likely you know it already), you can run just this test with

# make -C test/automated file-notify-tests

or

# rm test/automated/file-notify-tests.log
# make -C test/automated file-notify-tests.log

This might save time for your tests.

> Ken

Best regards, Michael.

Message #26 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Wed, 4 Nov 2015 11:55:44 -0500

[Message part 1 (text/plain, inline)]

On 11/4/2015 3:20 AM, Michael Albinus wrote:
> In all four cases, `file-notify--test-with-events' returned nil, where
> it was supposed to show the incoming events. Strange.
>
> In lisp/filenotify.el and test/automated/file-notify-tests.el, I've
> prepared trace messages. You'll find them searching for ";;(message".
> Could you, please, enable these traces and show the resulting log?

Attached.

> Another idea is to increase the timeouts in `file-notify--test-timeout'.

This didn't help.

Ken

[file-notify-tests.log (application/octet-stream, attachment)]

Message #29 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Wed, 04 Nov 2015 19:44:29 +0100

Ken Brown <kbrown <at> cornell.edu> writes:

>> In lisp/filenotify.el and test/automated/file-notify-tests.el, I've
>> prepared trace messages. You'll find them searching for ";;(message".
>> Could you, please, enable these traces and show the resulting log?
>
> Attached.
>
> Running 12 tests (2015-11-04 11:51:58-0500)
> Local library: `gfilenotify'
>    passed   1/12  file-notify-test00-availability
> Remote command: `gvfs-monitor-dir'
>    passed   2/12  file-notify-test00-availability-remote
>    passed   3/12  file-notify-test01-add-watch
>    passed   4/12  file-notify-test01-add-watch-remote
> file-notify--test-event-handler (6443046528 stopped "/tmp/file-notify-test552448U")
> Test file-notify-test02-events backtrace:

It is obvious, that `file-notify--test-with-events' isn't kosher. No
event arrived inside this macro. The stopped event arrived later, in one
of the unwindforms of the surrounding unwind-protect.

Are there special problems with cygwin Emacs related to the
`with-timeout' macro?

Hard to debug, because we are faced with timing issues. So I would need
to do this myself ...

Well, I'll try to find a machine where I could install cygwin. Where
could I find a very recent Emacs, including the patch of gfilenotify.c
I've committed yesterday?

> Ken

Best regards, Michael.

Message #32 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Wed, 4 Nov 2015 22:21:35 -0500

Hi Michael,

On 11/4/2015 1:44 PM, Michael Albinus wrote:
> Ken Brown <kbrown <at> cornell.edu> writes:
>> Running 12 tests (2015-11-04 11:51:58-0500)
>> Local library: `gfilenotify'
>>     passed   1/12  file-notify-test00-availability
>> Remote command: `gvfs-monitor-dir'
>>     passed   2/12  file-notify-test00-availability-remote
>>     passed   3/12  file-notify-test01-add-watch
>>     passed   4/12  file-notify-test01-add-watch-remote
>> file-notify--test-event-handler (6443046528 stopped "/tmp/file-notify-test552448U")
>> Test file-notify-test02-events backtrace:
>
> It is obvious, that `file-notify--test-with-events' isn't kosher. No
> event arrived inside this macro. The stopped event arrived later, in one
> of the unwindforms of the surrounding unwind-protect.
>
> Are there special problems with cygwin Emacs related to the
> `with-timeout' macro?

Not that I know of.  There are other tests that use it, and those aren't 
failing.

> Hard to debug, because we are faced with timing issues. So I would need
> to do this myself ...
>
> Well, I'll try to find a machine where I could install cygwin. Where
> could I find a very recent Emacs, including the patch of gfilenotify.c
> I've committed yesterday?

I've just built one and put it on my private Cygwin repository at

  http://sanibeltranquility.com/cygwin/index.html

There are instructions at that site.  Let me know if anything is 
unclear.  Thanks for your help with this.

Ken

Message #35 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Thu, 05 Nov 2015 20:58:42 +0100

Ken Brown <kbrown <at> cornell.edu> writes:

> Hi Michael,

Hi Ken,

>> Hard to debug, because we are faced with timing issues. So I would need
>> to do this myself ...
>>
>> Well, I'll try to find a machine where I could install cygwin. Where
>> could I find a very recent Emacs, including the patch of gfilenotify.c
>> I've committed yesterday?
>
> I've just built one and put it on my private Cygwin repository at

I've played with this the whole afternoon. Looks like there is no error
in gfilenotify for Cygwin. But it is a hard job to trigger the file
notification events to appear such a way they could be checked for
correctness in the test cases.

Increasing timeouts was necessary. But even this does not make the
events to appear reliably. One would need to write additional code to
get every single event one after the other. Waiting for a series of
events, as the test cases do expect, does not seem to work.

Since I have no further idea how to get those events reliably, I tend to
skip both test cases for cygwin. What do you think?

> Ken

Best regards, Michael.

Message #38 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Thu, 5 Nov 2015 15:14:58 -0500

On 11/5/2015 2:58 PM, Michael Albinus wrote:
> Ken Brown <kbrown <at> cornell.edu> writes:
>
>> Hi Michael,
>
> Hi Ken,
>
>>> Hard to debug, because we are faced with timing issues. So I would need
>>> to do this myself ...
>>>
>>> Well, I'll try to find a machine where I could install cygwin. Where
>>> could I find a very recent Emacs, including the patch of gfilenotify.c
>>> I've committed yesterday?
>>
>> I've just built one and put it on my private Cygwin repository at
>
> I've played with this the whole afternoon. Looks like there is no error
> in gfilenotify for Cygwin. But it is a hard job to trigger the file
> notification events to appear such a way they could be checked for
> correctness in the test cases.
>
> Increasing timeouts was necessary. But even this does not make the
> events to appear reliably. One would need to write additional code to
> get every single event one after the other. Waiting for a series of
> events, as the test cases do expect, does not seem to work.
>
> Since I have no further idea how to get those events reliably, I tend to
> skip both test cases for cygwin. What do you think?

That makes sense to me.  Thanks for your efforts on this.

Ken

Message #43 received at 21804-done <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804-done <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Fri, 06 Nov 2015 07:35:19 +0100

Ken Brown <kbrown <at> cornell.edu> writes:

>> Since I have no further idea how to get those events reliably, I tend to
>> skip both test cases for cygwin. What do you think?
>
> That makes sense to me.  Thanks for your efforts on this.

Done. Closing the bug.

> Ken

Best regards, Michael.

Message #50 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Mon, 26 Dec 2016 12:08:18 -0500

[Message part 1 (text/plain, inline)]

[Resending after unarchiving the bug.]


On 11/5/2015 2:58 PM, Michael Albinus wrote:
> I've played with this the whole afternoon. Looks like there is no error
> in gfilenotify for Cygwin. But it is a hard job to trigger the file
> notification events to appear such a way they could be checked for
> correctness in the test cases.
>
> Increasing timeouts was necessary. But even this does not make the
> events to appear reliably. One would need to write additional code to
> get every single event one after the other. Waiting for a series of
> events, as the test cases do expect, does not seem to work.

Hi Michael,

I've made some progress on understanding why some file notifications 
aren't triggered in the tests on Cygwin.  One issue is that 
file-notify--test-read-event-timeout has to be increased drastically in 
order for `created' and `changed' events to be received.  These can take 
5 seconds or more.

A second issue is that a small delay is needed between the creation of a 
file watch and the beginning of file activity.  This is not normally a 
problem in interactive sessions.  But it can be observed in an 
interactive session by evaluating the following:

(require 'filenotify)
filenotify

(defun my-notify-callback (event)
   (message "Event %S" event))

(progn
   (file-notify-add-watch
    "/tmp/foo" '(change) 'my-notify-callback)
   (write-region "" nil "/tmp/foo"))

(delete-file "/tmp/foo")

;; The above yields `deleted' and `stopped' events, but no `created' 
event.  Now we introduce a delay after adding the watch.

(progn
   (file-notify-add-watch
    "/tmp/foo" '(change) 'my-notify-callback)
   (sleep-for 1)
   (write-region "" nil "/tmp/foo"))

(delete-file "/tmp/foo")

;; This time we get the expected `created' event.

The attached patch takes care of both timing issues.  With this patch, 
all the inexpensive file-notify tests pass on all my Cygwin systems.

Best regards,

Ken

[filenotify-tests.patch (text/plain, attachment)]

Message #53 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Tue, 27 Dec 2016 13:06:13 +0100

Ken Brown <kbrown <at> cornell.edu> writes:

> Hi Michael,

Hi Ken,

> I've made some progress on understanding why some file notifications
> aren't triggered in the tests on Cygwin.  One issue is that
> file-notify--test-read-event-timeout has to be increased drastically
> in order for `created' and `changed' events to be received.  These can
> take 5 seconds or more.

Well, gfilenotify uses a native library like inotify or kqueue when
available. If not, it does polling.

I guess the latter happens in the cygwin case. Maybe we need to
determine the polling interval, and to set
file-notify--test-read-event-timeout larger than that. Will check,
whether information this could be retrieved from gfilenotify.

> A second issue is that a small delay is needed between the creation of
> a file watch and the beginning of file activity.  This is not normally
> a problem in interactive sessions.

I see. I will check further, whether I could modify Fgfile_add_watch
such a way that it returns only when it is able to create events.

> The attached patch takes care of both timing issues.  With this patch,
> all the inexpensive file-notify tests pass on all my Cygwin systems.

LGTM, please install.

I would also like to modify the expensive tests accordingly. Do you know
where I could retrieve a compiled Emacs 26 for cygwin?

> Best regards,
>
> Ken

Best regards, Michael.

Message #56 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Tue, 27 Dec 2016 11:28:56 -0500

Hi Michael,

On 12/27/2016 7:06 AM, Michael Albinus wrote:
> Well, gfilenotify uses a native library like inotify or kqueue when
> available. If not, it does polling.
>
> I guess the latter happens in the cygwin case. Maybe we need to
> determine the polling interval, and to set
> file-notify--test-read-event-timeout larger than that. Will check,
> whether information this could be retrieved from gfilenotify.

That would be helpful.

> I see. I will check further, whether I could modify Fgfile_add_watch
> such a way that it returns only when it is able to create events.

And that.

> LGTM, please install.

Done.

> I would also like to modify the expensive tests accordingly. Do you know
> where I could retrieve a compiled Emacs 26 for cygwin?

I've just built one and put it on my private Cygwin repository at

  http://sanibeltranquility.com/cygwin/index.html

There are instructions at that site.  Let me know if anything is unclear.

Best regards,

Ken

Message #59 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Tue, 27 Dec 2016 12:33:39 -0500

Hi again Michael,

On 12/27/2016 7:06 AM, Michael Albinus wrote:
> Maybe we need to
> determine the polling interval, and to set
> file-notify--test-read-event-timeout larger than that. Will check,
> whether information this could be retrieved from gfilenotify.

Isn't the polling interval set to 100 msec by the following line in 
gfilenotify.c?

  g_file_monitor_set_rate_limit (monitor, 100);

By the way, I tried removing that line and using the default polling 
interval, and it seemed (subjectively) to work better, i.e., the wait 
time for a `created' event in an interactive session appeared to be 2-3 
seconds instead of 5-6 seconds.  But this doesn't seem to be consistent 
enough to allow me to substantially reduce 
file-notify--test-read-event-timeout.

Best regards,

Ken

Message #62 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Tue, 27 Dec 2016 19:35:42 +0100

Ken Brown <kbrown <at> cornell.edu> writes:

> Hi again Michael,

Hi Ken,

>> Maybe we need to determine the polling interval, and to set
>> file-notify--test-read-event-timeout larger than that. Will check,
>> whether information this could be retrieved from gfilenotify.
>
> Isn't the polling interval set to 100 msec by the following line in
> gfilenotify.c?
>
>   g_file_monitor_set_rate_limit (monitor, 100);

Yes. IIRC, I did that, because there were problems with gfilenotify on
GNU/Linux otherwise.

> By the way, I tried removing that line and using the default polling
> interval, and it seemed (subjectively) to work better, i.e., the wait
> time for a `created' event in an interactive session appeared to be
> 2-3 
> seconds instead of 5-6 seconds.  But this doesn't seem to be
> consistent enough to allow me to substantially reduce
> file-notify--test-read-event-timeout.

Good to know, thanks. I'm reading glib's sources now, in order to
understand how the polling works exactly.

Btw, which version of glib do you use for your tests?

> Best regards,
>
> Ken

Best regards, Michael.

Message #65 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Tue, 27 Dec 2016 17:48:34 -0500

On 12/27/2016 1:35 PM, Michael Albinus wrote:
> Btw, which version of glib do you use for your tests?

glib2.0-2.46.2

Message #68 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Wed, 28 Dec 2016 18:16:20 -0500

On 12/27/2016 1:35 PM, Michael Albinus wrote:
> I'm reading glib's sources now, in order to
> understand how the polling works exactly.

Hi Michael,

I just looked at the sources also, and the reason for the roughly 
5-second delays I've been observing is clear.  In gio/gpollfilemonitor.c 
we have the following:

#define POLL_TIME_SECS 5
[...]
schedule_poll_timeout (GPollFileMonitor* poll_monitor)
{
  poll_monitor->timeout = g_timeout_source_new_seconds (POLL_TIME_SECS);
[...]
}

So I guess there's no way to avoid using a value of 
file-notify--test-read-event-timeout bigger than 5.

Best regards,

Ken

Message #71 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Thu, 29 Dec 2016 20:06:59 +0100

Ken Brown <kbrown <at> cornell.edu> writes:

> Hi Michael,

Hi Ken,

> I just looked at the sources also, and the reason for the roughly
> 5-second delays I've been observing is clear.  In
> gio/gpollfilemonitor.c we have the following:
>
> #define POLL_TIME_SECS 5
> [...]
> schedule_poll_timeout (GPollFileMonitor* poll_monitor)
> {
>   poll_monitor->timeout = g_timeout_source_new_seconds (POLL_TIME_SECS);
> [...]
> }
>
> So I guess there's no way to avoid using a value of
> file-notify--test-read-event-timeout bigger than 5.

Yes, for a polling gfilenotify.

I've added the new function Fgfile_monitor_name to gfilenotify.c. It
tells us the name of the used monitor. In my case (GNU/Linux), it is
GInotifyFileMonitor. In your case it is GPollFileMonitor.

Based on this, we could detect better which timeout to use in
filenotify-tests.el. I worked on this, and the non-remote cases seem to
be OK now (although sometimes not all events do arrive). You could play
also with this, just call from Emacs root directory

# make -C test filenotify-tests

I recommend to change the value of n to 10 temporarily in both
file-notify-test06-many-events and
file-notify-test08-watched-file-in-watched-dir; otherwise the tests last
too long. If you want to run just one test case, call (for example)

# make -C test filenotify-tests SELECTOR='\"file-notify-test06-many-events$$\"'

SELECTOR is a regexp over test case names.

Installing the cygwin package "gvfs" brings you gvfs-monitor-dir.exe. If
available, the remote test cases are not skipped anymore.

> Best regards,
>
> Ken

Best regards, Michael.

PS: Could you pls provide on your site a recompiled Emacs? I would like
to use the changed gfilenotify.c. Thanks!

Message #74 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Thu, 29 Dec 2016 15:33:15 -0500

Hi Michael,

On 12/29/2016 2:06 PM, Michael Albinus wrote:
> Ken Brown <kbrown <at> cornell.edu> writes:
> I've added the new function Fgfile_monitor_name to gfilenotify.c. It
> tells us the name of the used monitor. In my case (GNU/Linux), it is
> GInotifyFileMonitor. In your case it is GPollFileMonitor.

Actually it turns out to be GFamFileMonitor on my system, presumably because I have the gamin package installed.  I'm going to see if I can figure out how GFamFileMonitor works.  As a quick check, however, I applied the following patch...

--- a/test/lisp/filenotify-tests.el
+++ b/test/lisp/filenotify-tests.el
@@ -73,7 +73,8 @@ file-notify--test-read-event
     ;; gio/gpollfilemonitor.c declares POLL_TIME_SECS 5. So we must
     ;; wait at least this time.
     ((and (string-equal (file-notify--test-library) "gfilenotify")
-         (string-equal (file-notify--test-monitor) "GPollFileMonitor"))
+          (or (string-equal (file-notify--test-monitor) "GPollFileMonitor")
+              (string-equal (file-notify--test-monitor) "GFamFileMonitor")))
      7)
     ((file-remote-p temporary-file-directory) 0.1)
     (t 0.01))))

...and all the inexpensive tests passed.  I'll keep playing with this.

> PS: Could you pls provide on your site a recompiled Emacs? I would like
> to use the changed gfilenotify.c. Thanks!

It's building right now and should be available shortly.

Best regards,

Ken

Message #77 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: michael.albinus <at> gmx.de, 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Fri, 30 Dec 2016 10:50:02 +0200

> From: Ken Brown <kbrown <at> cornell.edu>
> Date: Thu, 29 Dec 2016 15:33:15 -0500
> Cc: 21804 <at> debbugs.gnu.org
> 
> -         (string-equal (file-notify--test-monitor) "GPollFileMonitor"))
> +          (or (string-equal (file-notify--test-monitor) "GPollFileMonitor")
> +              (string-equal (file-notify--test-monitor) "GFamFileMonitor")))

Wouldn't it be better if gfile-monitor-name returned a symbol instead
of a string?  That's our general convention with functions, I think.

Message #80 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21804 <at> debbugs.gnu.org, Ken Brown <kbrown <at> cornell.edu>
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Fri, 30 Dec 2016 20:07:05 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

>> -         (string-equal (file-notify--test-monitor) "GPollFileMonitor"))
>> +          (or (string-equal (file-notify--test-monitor) "GPollFileMonitor")
>> +              (string-equal (file-notify--test-monitor) "GFamFileMonitor")))
>
> Wouldn't it be better if gfile-monitor-name returned a symbol instead
> of a string?  That's our general convention with functions, I think.

I've changed it accordingly.

Best regards, Michael.

Message #83 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Fri, 30 Dec 2016 20:16:17 +0100

Ken Brown <kbrown <at> cornell.edu> writes:

> Hi Michael,

Hi Ken,

>> Ken Brown <kbrown <at> cornell.edu> writes:
>> I've added the new function Fgfile_monitor_name to gfilenotify.c. It
>> tells us the name of the used monitor. In my case (GNU/Linux), it is
>> GInotifyFileMonitor. In your case it is GPollFileMonitor.
>
> Actually it turns out to be GFamFileMonitor on my system, presumably
> because I have the gamin package installed.  I'm going to see if I can
> figure out how GFamFileMonitor works.

Next time I have access to a cygwin machine I'll check. However, I
wonder whether we need to respect the polling period then. According to
<http://oss.sgi.com/projects/fam/faq.html#what_is_fam>, FAM does not poll.

> As a quick check, however, I applied the following patch...
>
> --- a/test/lisp/filenotify-tests.el
> +++ b/test/lisp/filenotify-tests.el
> @@ -73,7 +73,8 @@ file-notify--test-read-event
>      ;; gio/gpollfilemonitor.c declares POLL_TIME_SECS 5. So we must
>      ;; wait at least this time.
>      ((and (string-equal (file-notify--test-library) "gfilenotify")
> -         (string-equal (file-notify--test-monitor) "GPollFileMonitor"))
> +          (or (string-equal (file-notify--test-monitor) "GPollFileMonitor")
> +              (string-equal (file-notify--test-monitor) "GFamFileMonitor")))
>       7)
>      ((file-remote-p temporary-file-directory) 0.1)
>      (t 0.01))))
>
> ...and all the inexpensive tests passed.  I'll keep playing with this.

It's OK for me if you commit this patch. You know cygwin behaviour much
better then I do.

> Best regards,
>
> Ken

Best regards, Michael.

Message #86 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>, Eli Zaretskii <eliz <at> gnu.org>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Fri, 30 Dec 2016 17:19:03 -0500

On 12/30/2016 2:07 PM, Michael Albinus wrote:
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
>>> -         (string-equal (file-notify--test-monitor) "GPollFileMonitor"))
>>> +          (or (string-equal (file-notify--test-monitor) "GPollFileMonitor")
>>> +              (string-equal (file-notify--test-monitor) "GFamFileMonitor")))
>>
>> Wouldn't it be better if gfile-monitor-name returned a symbol instead
>> of a string?  That's our general convention with functions, I think.
>
> I've changed it accordingly.

But since it returns an uninterned symbol, I think I still need to use 
string-equal above.  Or am I misunderstanding something?

Ken

Message #89 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Fri, 30 Dec 2016 18:15:57 -0500

Hi Michael,

On 12/30/2016 2:16 PM, Michael Albinus wrote:
> Ken Brown <kbrown <at> cornell.edu> writes:
>> Actually it turns out to be GFamFileMonitor on my system, presumably
>> because I have the gamin package installed.

It turns out that this isn't special to my system.  The emacs package on 
Cygwin requires libglib2.0_0, which requires gamin.  So the monitor on 
Cygwin will always be of type GFamFileMonitor.

> Next time I have access to a cygwin machine I'll check. However, I
> wonder whether we need to respect the polling period then. According to
> <http://oss.sgi.com/projects/fam/faq.html#what_is_fam>, FAM does not poll.

It says on that page, "If FAM is running on a system that doesn't have a 
supported kernel monitor, it polls."  And I've checked the gamin sources 
(since Cygwin uses gamin for its implementation of libfam), and the same 
seems to be true there.

On the other hand, the default polling interval seems to be 1 second, so 
I'm not sure why the tests need such a big timeout.  One factor might be 
the following in gio/glocalfilemonitor.c:

  #define VIRTUAL_CHANGES_DONE_DELAY  2 * G_TIME_SPAN_SECOND

If I understand the code correctly, this causes a 2-second delay before 
a CHANGES_DONE_HINT event is sent after a CHANGED event.

>> As a quick check, however, I applied the following patch...
>>
>> --- a/test/lisp/filenotify-tests.el
>> +++ b/test/lisp/filenotify-tests.el
>> @@ -73,7 +73,8 @@ file-notify--test-read-event
>>      ;; gio/gpollfilemonitor.c declares POLL_TIME_SECS 5. So we must
>>      ;; wait at least this time.
>>      ((and (string-equal (file-notify--test-library) "gfilenotify")
>> -         (string-equal (file-notify--test-monitor) "GPollFileMonitor"))
>> +          (or (string-equal (file-notify--test-monitor) "GPollFileMonitor")
>> +              (string-equal (file-notify--test-monitor) "GFamFileMonitor")))
>>       7)
>>      ((file-remote-p temporary-file-directory) 0.1)
>>      (t 0.01))))
>>
>> ...and all the inexpensive tests passed.  I'll keep playing with this.
>
> It's OK for me if you commit this patch. You know cygwin behaviour much
> better then I do.

Let me play with this a little more first.

Best regards,

Ken

Message #92 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: michael.albinus <at> gmx.de, 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Sat, 31 Dec 2016 10:26:53 +0200

> Cc: 21804 <at> debbugs.gnu.org
> From: Ken Brown <kbrown <at> cornell.edu>
> Date: Fri, 30 Dec 2016 17:19:03 -0500
> 
> On 12/30/2016 2:07 PM, Michael Albinus wrote:
> > Eli Zaretskii <eliz <at> gnu.org> writes:
> >
> >>> -         (string-equal (file-notify--test-monitor) "GPollFileMonitor"))
> >>> +          (or (string-equal (file-notify--test-monitor) "GPollFileMonitor")
> >>> +              (string-equal (file-notify--test-monitor) "GFamFileMonitor")))
> >>
> >> Wouldn't it be better if gfile-monitor-name returned a symbol instead
> >> of a string?  That's our general convention with functions, I think.
> >
> > I've changed it accordingly.
> 
> But since it returns an uninterned symbol, I think I still need to use 
> string-equal above.  Or am I misunderstanding something?

I actually don't understand why Michael decided to return an
uninterned symbol here.

Message #95 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21804 <at> debbugs.gnu.org, Ken Brown <kbrown <at> cornell.edu>
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Sat, 31 Dec 2016 10:42:06 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

> I actually don't understand why Michael decided to return an
> uninterned symbol here.

I also don't understand :-(

Fixed.

Best regards, Michael.

Message #98 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Mon, 02 Jan 2017 19:47:03 +0100

[Message part 1 (text/plain, inline)]

Ken Brown <kbrown <at> cornell.edu> writes:

> Hi Michael,

Hi Ken,

> It turns out that this isn't special to my system.  The emacs package
> on Cygwin requires libglib2.0_0, which requires gamin.  So the monitor
> on Cygwin will always be of type GFamFileMonitor.

Indeed, it happens also for me.

> Let me play with this a little more first.

I've tested in parallel today. The applied patch let all non-remote test
cases pass successfully for cygwin. There were some special timing
issues (for example, when a watch was removed), which should be solved
now.

There are other problems in the remote case. Mainly, I don't know a way
to determine, which backend is linked to
gvfs-directory-monitor. According to my tests, it behaves different when
running on a (remote) GNU/Linux machine, or a (remote) cygwin machine.

I've committed a patch to Tramp, which let us detect this program as
"gvfs-monitor-dir.exe" for cygwin. This is hackish, of course; but I
don't know it better. The appended patch let pass all remote test cases
for cygwin too, except file-notify-test06-many-events-remote and
file-notify-test08-watched-file-in-watched-dir-remote. For these two
test cases, there's always a loss of some events. I see no chance to fix
this also.

If you agree, I would commit the patch to the master branch, and I would
close bug#21804 then.

> Best regards,
>
> Ken

Best regards, Michael.

[Message part 2 (text/plain, attachment)]

Message #101 received at 21804 <at> debbugs.gnu.org (full text, mbox):

From: Ken Brown <kbrown <at> cornell.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 21804 <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Mon, 2 Jan 2017 18:15:26 -0500

Hi Michael,

On 1/2/2017 1:47 PM, Michael Albinus wrote:
> If you agree, I would commit the patch to the master branch, and I would
> close bug#21804 then.

Yes, please go ahead.  And thanks for all of your work on this.

Best regards,

Ken

Message #104 received at 21804-done <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Ken Brown <kbrown <at> cornell.edu>
Cc: 21804-done <at> debbugs.gnu.org
Subject: Re: bug#21804: 25.0.50; file-notify-tests failure on Cygwin
Date: Tue, 03 Jan 2017 10:11:25 +0100

Ken Brown <kbrown <at> cornell.edu> writes:

> Hi Michael,

Hi Ken,

>> If you agree, I would commit the patch to the master branch, and I would
>> close bug#21804 then.
>
> Yes, please go ahead.  And thanks for all of your work on this.

I've committed it to master, closing the bug.

> Best regards,
>
> Ken

Best regards, Michael.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.