GNU bug report logs -
#76926
31.0.50; Hyperbole tarball bloat
Previous Next
To reply to this bug, email your comments to 76926 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
monnier <at> iro.umontreal.ca, rsw <at> gnu.org, matsl <at> gnu.org, bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Mon, 10 Mar 2025 20:17:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Stefan Monnier <monnier <at> iro.umontreal.ca>
:
New bug report received and forwarded. Copy sent to
monnier <at> iro.umontreal.ca, rsw <at> gnu.org, matsl <at> gnu.org, bug-gnu-emacs <at> gnu.org
.
(Mon, 10 Mar 2025 20:17:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Package: Emacs
Version: 31.0.50
The current Hyperbole code is a bit less than 3MB of ELisp code, but the
resulting tarball in GNU ELPA is more than 20MB.
Skipping the worst offenders
*.eps
*.pdf
*.html
HY-TALK/improve-performance.png
man/im/wgrid4x3.png
would bring it down to about a third of the current size, but I suggest
you think about which of the files in the Git could be skipped in the
tarball (e.g. because they're only useful for the website) or even
completely removed from Git (like `man/im/wgrid[34]*.png` files which
don't seem to be used for anything, or the non-source files like
`hyperbole.pdf` or the `.eps` files which could be replaced by
appropriate Makefile rules).
There's no point skipping small files, so usually I encourage package
maintainers to include pretty much everything, but here I think we went
a bit overboard.
On a related note, have you tried to use `x-export-frames` to generate
screenshots in PDF or SVG format? It might (should?) lead to images
that are more compact than `wgrid4x3.png` while giving a higher quality
rendering.
Also, I see there's a `man/hyperbole.texi` but the current package's
spec doesn't specify any manual. I guess we should add
:doc "man/hyperbole.texi"
right? [ That would auto-generate `hyperbole.info` and `dir` files so
users who install your packages get it in `M-x info`, as well as add an
HTML rendering of that same manual over at
https://elpa.gnu.org/packages/hyperbole.html. You don't need to
include `hyperbole.html` and `dir` and `hyperbole.info` in the Git for that. ]
Stefan
Severity set to 'wishlist' from 'normal'
Request was from
Stefan Kangas <stefankangas <at> gmail.com>
to
control <at> debbugs.gnu.org
.
(Mon, 10 Mar 2025 21:05:01 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Mon, 10 Mar 2025 21:14:01 GMT)
Full text and
rfc822 format available.
Message #10 received at 76926 <at> debbugs.gnu.org (full text, mbox):
Hi Stefan,
> Stefan Monnier writes:
> Package: Emacs
> Version: 31.0.50
>
>
> The current Hyperbole code is a bit less than 3MB of ELisp code, but the
> resulting tarball in GNU ELPA is more than 20MB.
>
> Skipping the worst offenders
>
> *.eps
> *.pdf
> *.html
> HY-TALK/improve-performance.png
> man/im/wgrid4x3.png
>
> would bring it down to about a third of the current size, but I suggest
> you think about which of the files in the Git could be skipped in the
> tarball (e.g. because they're only useful for the website) or even
> completely removed from Git (like `man/im/wgrid[34]*.png` files which
> don't seem to be used for anything, or the non-source files like
> `hyperbole.pdf` or the `.eps` files which could be replaced by
> appropriate Makefile rules).
Point taken. We'll see how we can clean git from big files.
The tarball is built with the idea that users would get everything needed
without having any extra tools nor to be required to run any make
targets. That might be an old fashioned goal these days and users are bettered
served by fetching things from the net. Especially if we already provide those
through different channels.
> There's no point skipping small files, so usually I encourage package
> maintainers to include pretty much everything, but here I think we went
> a bit overboard.
> On a related note, have you tried to use `x-export-frames` to generate
> screenshots in PDF or SVG format? It might (should?) lead to images
> that are more compact than `wgrid4x3.png` while giving a higher quality
> rendering.
No, we have not tried that. I did not know that was available. Thanks for the
suggestion. (So many things in Emacs I learn new stuff every day ;-) I will
take a look what can be done about the images. They might need regeneration
anyway with next release due to things have changed since we produced them. We
can try to make them more compact when we update them.
> Also, I see there's a `man/hyperbole.texi` but the current package's
> spec doesn't specify any manual. I guess we should add
>
> :doc "man/hyperbole.texi"
>
> right? [ That would auto-generate `hyperbole.info` and `dir` files so
> users who install your packages get it in `M-x info`, as well as add an
> HTML rendering of that same manual over at
> https://elpa.gnu.org/packages/hyperbole.html. You don't need to
> include `hyperbole.html` and `dir` and `hyperbole.info` in the Git for that. ]
I think our current solution also works in making Hyperbole available in
info. But if we can get the necessary files generated, using the standard
machinery, there would be less to ship. Thanks for the suggestion.
Yours
--
%% Mats
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Mon, 10 Mar 2025 21:36:02 GMT)
Full text and
rfc822 format available.
Message #13 received at 76926 <at> debbugs.gnu.org (full text, mbox):
Appreciate the tips, Stefan. As Mats said, we’ll work on some of those changes. We want Hyperbole including all the doc formats to work out of the box on all the different platforms Emacs can run on. The pdf is especially hard to set up to generate so we’ll likely want to keep that one. But we’re all for shrinking size without losing functionality out of the box.
-- rsw
> On Mar 10, 2025, at 5:13 PM, Mats Lidell <matsl <at> gnu.org> wrote:
>
> Hi Stefan,
>
>> Stefan Monnier writes:
>> Package: Emacs
>> Version: 31.0.50
>>
>>
>> The current Hyperbole code is a bit less than 3MB of ELisp code, but the
>> resulting tarball in GNU ELPA is more than 20MB.
>>
>> Skipping the worst offenders
>>
>> *.eps
>> *.pdf
>> *.html
>> HY-TALK/improve-performance.png
>> man/im/wgrid4x3.png
>>
>> would bring it down to about a third of the current size, but I suggest
>> you think about which of the files in the Git could be skipped in the
>> tarball (e.g. because they're only useful for the website) or even
>> completely removed from Git (like `man/im/wgrid[34]*.png` files which
>> don't seem to be used for anything, or the non-source files like
>> `hyperbole.pdf` or the `.eps` files which could be replaced by
>> appropriate Makefile rules).
>
> Point taken. We'll see how we can clean git from big files.
>
> The tarball is built with the idea that users would get everything needed
> without having any extra tools nor to be required to run any make
> targets. That might be an old fashioned goal these days and users are bettered
> served by fetching things from the net. Especially if we already provide those
> through different channels.
>
>> There's no point skipping small files, so usually I encourage package
>> maintainers to include pretty much everything, but here I think we went
>> a bit overboard.
>
>> On a related note, have you tried to use `x-export-frames` to generate
>> screenshots in PDF or SVG format? It might (should?) lead to images
>> that are more compact than `wgrid4x3.png` while giving a higher quality
>> rendering.
>
> No, we have not tried that. I did not know that was available. Thanks for the
> suggestion. (So many things in Emacs I learn new stuff every day ;-) I will
> take a look what can be done about the images. They might need regeneration
> anyway with next release due to things have changed since we produced them. We
> can try to make them more compact when we update them.
>
>> Also, I see there's a `man/hyperbole.texi` but the current package's
>> spec doesn't specify any manual. I guess we should add
>>
>> :doc "man/hyperbole.texi"
>>
>> right? [ That would auto-generate `hyperbole.info` and `dir` files so
>> users who install your packages get it in `M-x info`, as well as add an
>> HTML rendering of that same manual over at
>> https://elpa.gnu.org/packages/hyperbole.html. You don't need to
>> include `hyperbole.html` and `dir` and `hyperbole.info` in the Git for that. ]
>
> I think our current solution also works in making Hyperbole available in
> info. But if we can get the necessary files generated, using the standard
> machinery, there would be less to ship. Thanks for the suggestion.
>
> Yours
> --
> %% Mats
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Mon, 24 Mar 2025 20:45:01 GMT)
Full text and
rfc822 format available.
Message #16 received at 76926 <at> debbugs.gnu.org (full text, mbox):
Hi Stefan,
> Stefan Monnier writes:
> Skipping the worst offenders
>
> *.eps
> *.pdf
> *.html
> HY-TALK/improve-performance.png
> man/im/wgrid4x3.png
I noticed that you filter out these files now and that the elpa tar-ball is
reduced in size since a few days.
> [...]
> Also, I see there's a `man/hyperbole.texi` but the current package's
> spec doesn't specify any manual. I guess we should add
>
> :doc "man/hyperbole.texi"
>
> right?
What I can see you did not add the doc: attribute. Can you please add that so
we can evaluate the result? From your description is sounds like it is an
improvement over our current solution so would be good to see it in action.
Yours
--
%% Mats
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Mon, 24 Mar 2025 21:36:02 GMT)
Full text and
rfc822 format available.
Message #19 received at 76926 <at> debbugs.gnu.org (full text, mbox):
>> *.eps
>> *.pdf
>> *.html
>> HY-TALK/improve-performance.png
>> man/im/wgrid4x3.png
>
> I noticed that you filter out these files now and that the elpa tar-ball is
> reduced in size since a few days.
Oops, sorry. I had made the change locally for testing purposes, and it
looks like I pushed it by accident as part of another change.
>> [...]
>
>> Also, I see there's a `man/hyperbole.texi` but the current package's
>> spec doesn't specify any manual. I guess we should add
>>
>> :doc "man/hyperbole.texi"
>>
>> right?
>
> What I can see you did not add the doc: attribute. Can you please add that so
> we can evaluate the result?
OK, I just added it (and removed the `:ignored-files` which is better
replaced by adding entries to `.elpaignore` anyway, so it's all under
your direct control).
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Mon, 24 Mar 2025 22:08:04 GMT)
Full text and
rfc822 format available.
Message #22 received at 76926 <at> debbugs.gnu.org (full text, mbox):
> Stefan Monnier writes:
> Oops, sorry. I had made the change locally for testing purposes, and it
> looks like I pushed it by accident as part of another change.
Oh, back to us then. ;-)
> OK, I just added it
Just came to think of it. The current existent of the info file and dir in git
will not cause any problems? Will they just be overwritten?
> (and removed the `:ignored-files` which is better replaced by adding entries
> to `.elpaignore` anyway, so it's all under your direct control).
Thanks for sharing. I did not know of .elpaignore.
Yours
--
%% Mats
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Tue, 25 Mar 2025 04:47:03 GMT)
Full text and
rfc822 format available.
Message #25 received at 76926 <at> debbugs.gnu.org (full text, mbox):
>> OK, I just added it
> Just came to think of it. The current existent of the info file and dir in git
> will not cause any problems? Will they just be overwritten?
I think they'll be overwritten, yes. But, GNU-devel will tell us. 🙂
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Sun, 06 Apr 2025 21:04:02 GMT)
Full text and
rfc822 format available.
Message #28 received at 76926 <at> debbugs.gnu.org (full text, mbox):
Hi Stefan,
> Stefan Monnier writes:
> I think they'll be overwritten, yes. But, GNU-devel will tell us. 🙂
I had a look at the manual, it is this one right? https://elpa.gnu.org/devel/doc/hyperbole.html
It seems it is missing the illustrations. Can it be a relative folder issue
since the texi file itself refers to a sub folder (im) of the "man" folder for
the pictures?
%% Mats
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Sun, 06 Apr 2025 21:38:01 GMT)
Full text and
rfc822 format available.
Message #31 received at 76926 <at> debbugs.gnu.org (full text, mbox):
>> Stefan Monnier writes:
>> I think they'll be overwritten, yes. But, GNU-devel will tell us. 🙂
> I had a look at the manual, it is this one right? https://elpa.gnu.org/devel/doc/hyperbole.html
> It seems it is missing the illustrations. Can it be a relative folder issue
> since the texi file itself refers to a sub folder (im) of the "man" folder for
> the pictures?
Indeed we still have not made references to "local" files, like
images, work. 🙁
Part of the problem is simply to list the image files that need to be
copied over to the `doc` subdir, but part of the problem is also that
all the manuals share the same https://elpa.gnu.org/devel/doc/ directory
(so that references between Texinfo manuals provided by different
packages work properly).
so we need a way to avoid name clashes between auxiliary files
of manuals of different packages.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Sun, 06 Apr 2025 21:46:02 GMT)
Full text and
rfc822 format available.
Message #34 received at 76926 <at> debbugs.gnu.org (full text, mbox):
If you decide on a standard naming for doc subdirectories, we could adapt to it. But I still don’t understand why all the manual files and directories can’t just stay under the man or a doc subdir below the root Hyperbole directory and just be auto added to Info-directory list, for example.
-- rsw
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Mon, 07 Apr 2025 04:23:02 GMT)
Full text and
rfc822 format available.
Message #37 received at 76926 <at> debbugs.gnu.org (full text, mbox):
> If you decide on a standard naming for doc subdirectories, we could
> adapt to it.
Oh, we need a solution that works without any such change on your side
(there are too many packages in need of this: they won't all make
changes for us).
> But I still don’t understand why all the manual files and directories
> can’t just stay under the man or a doc subdir below the root Hyperbole
> directory and just be auto added to Info-directory list, for example.
AFAIK this is unrelated to the HTML file you see in `elpa.gnu.org` and
furthermore I think you're making invalid assumptions (AFAIK they *can*
stay there just fine). IOW, please provide more details about exactly
what it is you want, what you tried to do to get it, and what
happened instead.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Wed, 09 Apr 2025 21:52:01 GMT)
Full text and
rfc822 format available.
Message #40 received at 76926 <at> debbugs.gnu.org (full text, mbox):
> Stefan Monnier writes:
>
> Oh, we need a solution that works without any such change on your side
> (there are too many packages in need of this: they won't all make
> changes for us).
>
> > But I still don’t understand why all the manual files and directories
> > can’t just stay under the man or a doc subdir below the root Hyperbole
> > directory and just be auto added to Info-directory list, for example.
>
> AFAIK this is unrelated to the HTML file you see in `elpa.gnu.org` and
> furthermore I think you're making invalid assumptions (AFAIK they *can*
> stay there just fine). IOW, please provide more details about exactly
> what it is you want, what you tried to do to get it, and what
> happened instead.
Sorry but I'm confused here. Let me take step back.
This thread or bug is about reducing the bloat in the Hyperbole package and
possibly what files are stored in git. One of the things you suggested was
that we could let ELPA generate the info and dir files plus generate a HTML
file that would be linked to from the ELPA web site.
That sounded like something worth testing so I suggested that we had a go at
it and you helped us by adding the :doc attribute to the ELPA recipe for
Hyperbole. Unfortunately that did not work because the texi file refers to
images stored in a sub folder relative to the info file.
I think here is where we stand today. What we want is that the images are
shown in info and when web browsing. It is not clear to me if this is possible
with changes in the Hyperbole package, or if the recipe needs to be changed as
well or do we even need changes to the ELPA build system to support local
files!?
Adding the images to the :doc attribute and change the texi file to assume a
flat file structure seems plausible but will that work for the web page?
%% Mats
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Wed, 09 Apr 2025 23:53:02 GMT)
Full text and
rfc822 format available.
Message #43 received at 76926 <at> debbugs.gnu.org (full text, mbox):
Why can’t Info installation handle subdirectories from the source directory.
Part of how we build things is trying to reduce the need for installation steps. But I get that elpa has to deal with many package setups.
-- Bob
> On Apr 9, 2025, at 5:51 PM, Mats Lidell <matsl <at> gnu.org> wrote:
>
>
>>
>> Stefan Monnier writes:
>>
>> Oh, we need a solution that works without any such change on your side
>> (there are too many packages in need of this: they won't all make
>> changes for us).
>>
>>> But I still don’t understand why all the manual files and directories
>>> can’t just stay under the man or a doc subdir below the root Hyperbole
>>> directory and just be auto added to Info-directory list, for example.
>>
>> AFAIK this is unrelated to the HTML file you see in `elpa.gnu.org` and
>> furthermore I think you're making invalid assumptions (AFAIK they *can*
>> stay there just fine). IOW, please provide more details about exactly
>> what it is you want, what you tried to do to get it, and what
>> happened instead.
>
> Sorry but I'm confused here. Let me take step back.
>
> This thread or bug is about reducing the bloat in the Hyperbole package and
> possibly what files are stored in git. One of the things you suggested was
> that we could let ELPA generate the info and dir files plus generate a HTML
> file that would be linked to from the ELPA web site.
>
> That sounded like something worth testing so I suggested that we had a go at
> it and you helped us by adding the :doc attribute to the ELPA recipe for
> Hyperbole. Unfortunately that did not work because the texi file refers to
> images stored in a sub folder relative to the info file.
>
> I think here is where we stand today. What we want is that the images are
> shown in info and when web browsing. It is not clear to me if this is possible
> with changes in the Hyperbole package, or if the recipe needs to be changed as
> well or do we even need changes to the ELPA build system to support local
> files!?
>
> Adding the images to the :doc attribute and change the texi file to assume a
> flat file structure seems plausible but will that work for the web page?
>
> %% Mats
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Thu, 10 Apr 2025 02:09:01 GMT)
Full text and
rfc822 format available.
Message #46 received at 76926 <at> debbugs.gnu.org (full text, mbox):
> Sorry but I'm confused here. Let me take step back.
>
> This thread or bug is about reducing the bloat in the Hyperbole package and
> possibly what files are stored in git. One of the things you suggested was
> that we could let ELPA generate the info and dir files plus generate a HTML
> file that would be linked to from the ELPA web site.
>
> That sounded like something worth testing so I suggested that we had a go at
> it and you helped us by adding the :doc attribute to the ELPA recipe for
> Hyperbole. Unfortunately that did not work because the texi file refers to
> images stored in a sub folder relative to the info file.
Hmm... I'm still a bit confused.
AFAIK you reported that the HTML page at `elpa.gnu.org` is failing to
show images (and I was not surprised, because IIRC this is not
supported at all right now by the `elpa.gnu.org` scripts 🙁).
But do I understand correctly that you're also now reporting that the
Info version of that documentation, available within Emacs when the
package is installed, also can't display the images?
That would be a separate problem (and one that should be easier to fix).
> I think here is where we stand today. What we want is that the images
> are shown in info and when web browsing. It is not clear to me if this
> is possible with changes in the Hyperbole package, or if the recipe
> needs to be changed as well or do we even need changes to the ELPA
> build system to support local files!?
For the web docs, changes to the ELPA build system are definitely needed
(and not completely straightforward: someone already took a stab at it
and it go very far).
For the Info docs, I think the best possible solution will require
changes to the ELPA build system as well, but I haven't looked at the
problem yet.
> Adding the images to the :doc attribute and change the texi file to
> assume a flat file structure seems plausible but will that work for
> the web page?
I hope we can arrange so that the list of image files can be provided
only once in the package's spec and that it makes it work for both Info
and web versions of the manual, but the problem for each of the two
cases will have to be implemented separately because they're quite
different in practice.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Thu, 10 Apr 2025 06:29:02 GMT)
Full text and
rfc822 format available.
Message #49 received at submit <at> debbugs.gnu.org (full text, mbox):
Hi,
Stefan Monnier writes:
> AFAIK you reported that the HTML page at `elpa.gnu.org` is failing to
> show images (and I was not surprised, because IIRC this is not
> supported at all right now by the `elpa.gnu.org` scripts 🙁).
For reference, that's the topic of bug#73425, IIUC.
Best,
Eshel
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Thu, 10 Apr 2025 06:29:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Fri, 11 Apr 2025 14:24:02 GMT)
Full text and
rfc822 format available.
Message #55 received at 76926 <at> debbugs.gnu.org (full text, mbox):
> Stefan Monnier writes:
> Hmm... I'm still a bit confused.
>
> AFAIK you reported that the HTML page at `elpa.gnu.org` is failing to
> show images (and I was not surprised, because IIRC this is not
> supported at all right now by the `elpa.gnu.org` scripts 🙁).
>
> But do I understand correctly that you're also now reporting that the
> Info version of that documentation, available within Emacs when the
> package is installed, also can't display the images?
Yes. Sorry if I was unclear but it is related to that the images are
referenced using relative paths both in info and HTML. Your description of the
problem in bug#73425 is correct.
%% Mats
Information forwarded
to
bug-gnu-emacs <at> gnu.org
:
bug#76926
; Package
emacs
.
(Fri, 11 Apr 2025 15:55:01 GMT)
Full text and
rfc822 format available.
Message #58 received at 76926 <at> debbugs.gnu.org (full text, mbox):
> Yes. Sorry if I was unclear but it is related to that the images are
> referenced using relative paths both in info and HTML. Your description of the
> problem in bug#73425 is correct.
Thanks for the confirmation.
So, I think the solution I outlined in bug#73425 shoujld work and
shouldn't be too hard to implement (massaging of HTML and Info
files to change the relative references might be a bit difficult to "do
it right", but I think a crude solution shouldn't be too difficult).
Stefan
This bug report was last modified 64 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.