GNU bug report logs - #37269
[PATCH] build-system/asdf: Add option to compress programs.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 37269 in the body.
You can then email your comments to 37269 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: guix-patches <at> gnu.org
Subject: [PATCH] build-system/asdf: Add option to compress programs.
Date: Mon,  2 Sep 2019 15:50:20 +0200

* guix/build/lisp-utils.scm (build-program): Add `compress?' key argument.
(generate-executable-for-system): Same.
(generate-executable): Same.
---
 guix/build/lisp-utils.scm | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/guix/build/lisp-utils.scm b/guix/build/lisp-utils.scm
index 97bc6197a3..c7a589c902 100644
--- a/guix/build/lisp-utils.scm
+++ b/guix/build/lisp-utils.scm
@@ -220,12 +220,19 @@ Also load TEST-ASD-FILE if necessary."
   "Return a lisp keyword for the concatenation of STRINGS."
   (string->symbol (apply string-append ":" strings)))
 
-(define (generate-executable-for-system type system)
+(define* (generate-executable-for-system type system #:key compress?)
   "Use LISP to generate an executable, whose TYPE can be 'asdf:image-op or
 'asdf:program-op.  The latter will always be standalone.  Depends on having
 created a \"SYSTEM-exec\" system which contains the entry program."
   (lisp-eval-program
    `((require :asdf)
+     ;; Only SBCL supports compression as of 2019-09-02.
+     ,(if (and compress? (string=? (%lisp-type) "sbcl"))
+          '(defmethod asdf:perform ((o asdf:image-op) (c asdf:system))
+                      (uiop:dump-image (asdf:output-file o c)
+                                       :executable t
+                                       :compression t))
+          '())
      (asdf:operate ',type ,(string-append system "-exec")))))
 
 (define (generate-executable-wrapper-system system dependencies)
@@ -339,6 +346,7 @@ which are not nested."
                         (dependency-prefixes (list (library-output outputs)))
                         (dependencies (list (basename program)))
                         entry-program
+                        compress?
                         #:allow-other-keys)
   "Generate an executable program containing all DEPENDENCIES, and which will
 execute ENTRY-PROGRAM.  The result is placed in PROGRAM.  When executed, it
@@ -350,6 +358,7 @@ retained."
                        #:dependencies dependencies
                        #:dependency-prefixes dependency-prefixes
                        #:entry-program entry-program
+                       #:compress? compress?
                        #:type 'asdf:program-op)
   (let* ((name (basename program))
          (bin-directory (dirname program)))
@@ -382,6 +391,7 @@ DEPENDENCY-PREFIXES to ensure references to those libraries are retained."
                               dependency-prefixes
                               entry-program
                               type
+                              compress?
                               #:allow-other-keys)
   "Generate an executable by using asdf operation TYPE, containing whithin the
 image all DEPENDENCIES, and running ENTRY-PROGRAM in the case of an
@@ -405,7 +415,7 @@ references to those libraries are retained."
                `(((,bin-directory :**/ :*.*.*)
                   (,bin-directory :**/ :*.*.*)))))))
 
-    (generate-executable-for-system type name)
+    (generate-executable-for-system type name #:compress? compress?)
 
     (let* ((after-store-prefix-index
             (string-index out-file #\/
-- 
2.23.0

Message #8 received at 37269 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: 37269 <at> debbugs.gnu.org, Andy Patterson <ajpatter <at> uwaterloo.ca>,
 Katherine Cox-Buday <cox.katherine.e <at> gmail.com>
Subject: Re: bug#37269: Acknowledgement ([PATCH] build-system/asdf: Add option
 to compress programs.)
Date: Mon, 02 Sep 2019 15:54:51 +0200

[Message part 1 (text/plain, inline)]

This patch could be used by StumpWM and Next browser to compress the
binary.
For Next, this compresses the binary from ~100 MiB to ~23 MiB.  There is
no real drawback, the startup time is still well under 100 ms.

Andy, Katherine, thoughts?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 37269 <at> debbugs.gnu.org (full text, mbox):

From: Katherine Cox-Buday <cox.katherine.e <at> gmail.com>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: Andy Patterson <ajpatter <at> uwaterloo.ca>, 37269 <at> debbugs.gnu.org
Subject: Re: bug#37269: Acknowledgement ([PATCH] build-system/asdf: Add option
 to compress programs.)
Date: Tue, 03 Sep 2019 09:12:35 -0500

Pierre Neidhardt <mail <at> ambrevar.xyz> writes:

> This patch could be used by StumpWM and Next browser to compress the
> binary. For Next, this compresses the binary from ~100 MiB to ~23 MiB.
> There is no real drawback, the startup time is still well under
> 100 ms.

This is a great idea and something I lost when I moved from
self-compiling SBCL to Guix's version!

I am also not aware of any drawbacks, and I love that you implemented
this as an option in the build-system. Should we default it to true?

Also, as an aside, I am tracking Next; thank you for this project!

-- 
Katherine

Message #14 received at 37269 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Katherine Cox-Buday <cox.katherine.e <at> gmail.com>
Cc: Andy Patterson <ajpatter <at> uwaterloo.ca>, 37269 <at> debbugs.gnu.org
Subject: Re: bug#37269: Acknowledgement ([PATCH] build-system/asdf: Add option
 to compress programs.)
Date: Tue, 03 Sep 2019 16:56:47 +0200

[Message part 1 (text/plain, inline)]

Hi,

> I am also not aware of any drawbacks, and I love that you implemented
> this as an option in the build-system. Should we default it to true?

SBCL does not do it by default, so maybe we should respect the default
behaviour of the compiler instead.

I'll go ahead and apply the patch then.  Update SBCL to 1.5.6 and use
CLISP or ECL to build it instead of CCL which is not bootstrappable.

> Also, as an aside, I am tracking Next; thank you for this project!

Thanks!  Lots of good stuff will come this month!
(1.3.1 released today!)

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 37269 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Katherine Cox-Buday <cox.katherine.e <at> gmail.com>
Cc: 37269 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>,
 Andy Patterson <ajpatter <at> uwaterloo.ca>
Subject: Re: [bug#37269] Acknowledgement ([PATCH] build-system/asdf: Add
 option to compress programs.)
Date: Wed, 04 Sep 2019 14:07:12 +0200

Hello!

Katherine Cox-Buday <cox.katherine.e <at> gmail.com> skribis:

> Pierre Neidhardt <mail <at> ambrevar.xyz> writes:
>
>> This patch could be used by StumpWM and Next browser to compress the
>> binary. For Next, this compresses the binary from ~100 MiB to ~23 MiB.
>> There is no real drawback, the startup time is still well under
>> 100 ms.
>
> This is a great idea and something I lost when I moved from
> self-compiling SBCL to Guix's version!
>
> I am also not aware of any drawbacks, and I love that you implemented
> this as an option in the build-system. Should we default it to true?

Compressed files are opaque to the garbage collector scanner and to
grafting, so we could end up in situations like:

  https://issues.guix.gnu.org/issue/33848

(Which would be nice to address, BTW! :-))

Thus, I think we’ll have keep compression turned off.

Ludo’.

Message #20 received at 37269 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Ludovic Courtès <ludo <at> gnu.org>, Katherine Cox-Buday
 <cox.katherine.e <at> gmail.com>
Cc: 37269 <at> debbugs.gnu.org, Andy Patterson <ajpatter <at> uwaterloo.ca>
Subject: Re: [bug#37269] Acknowledgement ([PATCH] build-system/asdf: Add
 option to compress programs.)
Date: Wed, 04 Sep 2019 14:54:06 +0200

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> writes:

> Compressed files are opaque to the garbage collector scanner and to
> grafting, so we could end up in situations like:
>
>   https://issues.guix.gnu.org/issue/33848
>
> (Which would be nice to address, BTW! :-))
>
> Thus, I think we’ll have keep compression turned off.

Actually, in this case it doesn't.  I tried with Next and the resulting
package keeps all references to the Common Lisp libraries.
I believe that SBCL only compresses the image, not the sources (which are
embedded as well).

Anyways, I won't turn it off for now.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #23 received at 37269 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: Andy Patterson <ajpatter <at> uwaterloo.ca>, 37269 <at> debbugs.gnu.org,
 Katherine Cox-Buday <cox.katherine.e <at> gmail.com>
Subject: Re: [bug#37269] Acknowledgement ([PATCH] build-system/asdf: Add
 option to compress programs.)
Date: Thu, 05 Sep 2019 10:33:56 +0200

Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>> Compressed files are opaque to the garbage collector scanner and to
>> grafting, so we could end up in situations like:
>>
>>   https://issues.guix.gnu.org/issue/33848
>>
>> (Which would be nice to address, BTW! :-))
>>
>> Thus, I think we’ll have keep compression turned off.
>
> Actually, in this case it doesn't.  I tried with Next and the resulting
> package keeps all references to the Common Lisp libraries.
> I believe that SBCL only compresses the image, not the sources (which are
> embedded as well).

OK.  We’d need to see exactly what it does if we decide to enable
it—better safe than sorry!

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.