GNU bug report logs - #23430
25.0.93; iter-defun does not support special form save-current-buffer

Previous Next

To reply to this bug, email your comments to 23430 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: bug-gnu-emacs <at> gnu.org
Subject: 25.0.93; iter-defun does not support special form save-current-buffer
Date: Tue, 03 May 2016 04:31:24 +0300

Example:

;; -*- lexical-binding: t -*-

(iter-defun (str buf)
  (with-current-buffer buf
    (goto-char (point-min)))
  (let (done)
    (while (not done)
      (with-current-buffer buf
        (if (search-forward str)
            (iter-yield (match-beginning 0))
          (setq done 0))))))

...upon evaluation, says: "special form (save-current-buffer (set-buffer
buf) (if (search-forward str) (cps-internal-yield (match-beginning 0))
(setq cps-binding-done- 0))) incorrect or not supported".

The above example seemed to me to be the prime use case for generator.el
(e.g. lazily parsing output from an asynchronous process), so this
is disappointing.

In GNU Emacs 25.0.93.2 (x86_64-unknown-linux-gnu, GTK+ Version 3.18.9)
 of 2016-04-30 built on axl
Repository revision: ffe701cb07cfb3584c4e4894976f0c9487d02c59
Windowing system distributor 'The X.Org Foundation', version 11.0.11803000
System Description:	Ubuntu 16.04 LTS

Message #8 received at 23430 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: 23430 <at> debbugs.gnu.org
Subject: Re: bug#23430: 25.0.93; iter-defun does not support special form
 save-current-buffer
Date: Fri, 6 May 2016 01:46:54 +0300

[Message part 1 (text/plain, inline)]

On 05/03/2016 04:31 AM, Dmitry Gutov wrote:

> The above example seemed to me to be the prime use case for generator.el
> (e.g. lazily parsing output from an asynchronous process), so this
> is disappointing.

Actually, I take this back.

- generator.el doesn't seem useful for parsing output from an 
asynchronous process because it has no way to indicate whether the 
process has new output, or to return control back to Emacs while the 
process is still running but has no new output. It seems to call for a 
different abstraction.

- with-current-buffer turned out not to be so essential. Nor 
save-excursion. I've reimplemented xref--buf-pairs-iterator using 
iter-lambda but couldn't get the full benefit of the package because 
that function also needs a way to reliably clean up resources. The 
result is a bit more comprehensible, but also longer, so I'm hesitant to 
use it (attached).

So sum up, maybe this bug can be closed, but I'd like to see Daniel's 
opinion. Supporting with-current-buffer might turn out to be more 
essential in other cases.

[xref-gen.diff (text/x-patch, attachment)]

Message #11 received at 23430 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 23430 <at> debbugs.gnu.org
Subject: Re: bug#23430: 25.0.93;
 iter-defun does not support special form save-current-buffer
Date: Sat, 07 May 2016 01:25:44 +0200

Dmitry Gutov <dgutov <at> yandex.ru> writes:

> So sum up, maybe this bug can be closed, but I'd like to see Daniel's
> opinion. Supporting with-current-buffer might turn out to be more
> essential in other cases.

FWIW it's not crystal clear to me what the semantics of a supported
`with-current-buffer' would be.  In particular: would `iter-yield' "jump
out" of the `with-current-buffer' and restore the previous buffer, and
would a restart of the generator change the current buffer again?


Regards,

Michael.

Message #14 received at 23430 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: 23430 <at> debbugs.gnu.org
Subject: Re: bug#23430: 25.0.93; iter-defun does not support special form
 save-current-buffer
Date: Sat, 7 May 2016 02:30:12 +0300

On 05/07/2016 02:25 AM, Michael Heerdegen wrote:

> FWIW it's not crystal clear to me what the semantics of a supported
> `with-current-buffer' would be.  In particular: would `iter-yield' "jump
> out" of the `with-current-buffer' and restore the previous buffer, and
> would a restart of the generator change the current buffer again?

I think so, yeah. In the "rewritten" version of code, 
with-current-buffer would surround the function's body, so each time the 
function is called, with-current-buffer would be applied.

The value of point may or may not be saved and restored (this question 
also seems problematic, admittedly).

Message #19 received at 23430 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 23430 <at> debbugs.gnu.org
Subject: Re: bug#23430: 25.0.93;
 iter-defun does not support special form save-current-buffer
Date: Tue, 11 Aug 2020 19:41:39 -0700

Hi Dmitry,

Dmitry Gutov <dgutov <at> yandex.ru> writes:

> On 05/03/2016 04:31 AM, Dmitry Gutov wrote:
>
>> The above example seemed to me to be the prime use case for generator.el
>> (e.g. lazily parsing output from an asynchronous process), so this
>> is disappointing.
>
> Actually, I take this back.
>
> - generator.el doesn't seem useful for parsing output from an asynchronous
>  process because it has no way to indicate whether the process has new output,
> or to return control back to Emacs while the process is still running but has no
> new output. It seems to call for a different abstraction.
>
> - with-current-buffer turned out not to be so essential. Nor
>   save-excursion. I've reimplemented xref--buf-pairs-iterator using
> iter-lambda but couldn't get the full benefit of the package because that
> function also needs a way to reliably clean up resources. The result is a bit
> more comprehensible, but also longer, so I'm hesitant to use it (attached).
>
> So sum up, maybe this bug can be closed, but I'd like to see Daniel's
> opinion. Supporting with-current-buffer might turn out to be more
> essential in other cases.

Is this patch still relevant or should this be closed?

Best regards,
Stefan Kangas

Message #22 received at 23430 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 23430 <at> debbugs.gnu.org
Subject: Re: bug#23430: 25.0.93; iter-defun does not support special form
 save-current-buffer
Date: Wed, 12 Aug 2020 13:06:33 +0300

On 12.08.2020 05:41, Stefan Kangas wrote:
> Is this patch still relevant or should this be closed?

The patch was just an illustration.

But problem is still there, and still seems valid.

Before we close this, I'd like to hear from somebody who understands 
what generator.el is actually for.

Message #25 received at 23430 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 23430 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#23430: 25.0.93; iter-defun does not support special form
 save-current-buffer
Date: Sun, 16 Aug 2020 15:33:38 +0200

Dmitry Gutov <dgutov <at> yandex.ru> writes:

> Before we close this, I'd like to hear from somebody who understands
> what generator.el is actually for.

I think it's just a straightforward implementation of generators as
described here:

  https://en.wikipedia.org/wiki/Generator_%28computer_programming%29

I learned about them in university, and have used them once in a while.

AFAIK handling asynchronous process output is not a good task for them.

I think: The problem of the concept of generators in an editor is that
generators are good for saving the state of a computation, but Emacs as
an editor has a lot of "environment" state (current buffer, value of
point, ...), and when the computation represented by the generator
messes with this state (or has side effects), the concept doesn't fit
that well.

When working with streams, I make the handling of the according part of
the environment explicit saving it in variables (iterators are closures)
and "yield" outside of any xxx-recursion, in your introductory example,
that would look like this:

#+begin_src emacs-lisp
(require 'generator)

(iter-defun my-search (str buf)
  (with-current-buffer buf
    (goto-char (point-min)))
  (let ((pos (point))
        (yield nil)
        (done nil))
    (while (not done)
      (when yield
        (iter-yield yield)
        (setq yield nil))
      (with-current-buffer buf
        (goto-char pos)
        (if (search-forward str)
            (setq yield (match-beginning 0)
                  pos   (point))
          (setq done 0))))))
#+end_src

I guess it would be nice if that could work implicitly in some way, but
that would probably require the introduction of new special forms
adapted to the situation, like `iterator-save-buffer-and-point' or so.

Just extrapolating my university knowledge however...


Michael.

Message #28 received at 23430 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 23430 <at> debbugs.gnu.org, Daniel Colascione <dancol <at> dancol.org>,
 Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#23430: 25.0.93; iter-defun does not support special form
 save-current-buffer
Date: Tue, 18 Aug 2020 14:08:38 +0200

Dmitry Gutov <dgutov <at> yandex.ru> writes:

> But problem is still there, and still seems valid.
>
> Before we close this, I'd like to hear from somebody who understands
> what generator.el is actually for.

I've just CC'd Daniel assuming he may has missed this bug report.

Michael.

Message #31 received at 23430 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: 23430 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#23430: 25.0.93; iter-defun does not support special form
 save-current-buffer
Date: Sat, 22 Aug 2020 03:18:08 +0300

On 16.08.2020 16:33, Michael Heerdegen wrote:
> Dmitry Gutov <dgutov <at> yandex.ru> writes:
> 
>> Before we close this, I'd like to hear from somebody who understands
>> what generator.el is actually for.
> 
> I think it's just a straightforward implementation of generators as
> described here:
> 
>    https://en.wikipedia.org/wiki/Generator_%28computer_programming%29
> 
> I learned about them in university, and have used them once in a while.

Right. I was really wondering, though, whether we can/should use them 
more often.

> AFAIK handling asynchronous process output is not a good task for them.

Perhaps if we also used a separate thread for waiting for the output to 
come...

> I think: The problem of the concept of generators in an editor is that
> generators are good for saving the state of a computation, but Emacs as
> an editor has a lot of "environment" state (current buffer, value of
> point, ...), and when the computation represented by the generator
> messes with this state (or has side effects), the concept doesn't fit
> that well.
> 
> When working with streams, I make the handling of the according part of
> the environment explicit saving it in variables (iterators are closures)
> and "yield" outside of any xxx-recursion, in your introductory example,
> that would look like this:
> 
> #+begin_src emacs-lisp
> (require 'generator)
> 
> (iter-defun my-search (str buf)
>    (with-current-buffer buf
>      (goto-char (point-min)))
>    (let ((pos (point))
>          (yield nil)
>          (done nil))
>      (while (not done)
>        (when yield
>          (iter-yield yield)
>          (setq yield nil))
>        (with-current-buffer buf
>          (goto-char pos)
>          (if (search-forward str)
>              (setq yield (match-beginning 0)
>                    pos   (point))
>            (setq done 0))))))
> #+end_src

Thanks. I think this is fairly close to the approach I showed in the 
patch. Problem is, it's no shorter than an implementation one can write 
using plain functions. Longer, usually.

> I guess it would be nice if that could work implicitly in some way, but
> that would probably require the introduction of new special forms
> adapted to the situation, like `iterator-save-buffer-and-point' or so.

Perhaps.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.