GNU bug report logs - #64738
[PATCH] gnu: Make Mesa use zstd compression for shader cache instead of zlib.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 64738 in the body.
You can then email your comments to 64738 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sigve Sudland <sigve_sudland <at> hotmail.com>
To: guix-patches <at> gnu.org
Cc: Sigve Sudland <sigve_sudland <at> hotmail.com>
Subject: [PATCH] gnu: Make Mesa use zstd compression for shader cache instead
 of zlib.
Date: Wed, 19 Jul 2023 23:02:34 +0200

* gnu/packages/gl.scm (mesa): Enable zstd compression.
---
 gnu/packages/gl.scm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/gl.scm b/gnu/packages/gl.scm
index 1691086e1a..2f54d8553d 100644
--- a/gnu/packages/gl.scm
+++ b/gnu/packages/gl.scm
@@ -299,7 +299,8 @@ (define-public mesa
            libxvmc
            llvm-for-mesa
            wayland
-           wayland-protocols))
+           wayland-protocols
+           `(,zstd "lib")))
     (native-inputs
      (list bison
            flex
@@ -362,6 +363,9 @@ (define-public mesa
          ;; Enable the codecs that were built by default as part of the
          ;; 21.3.x releases to avoid functionality regressions.
          "-Dvideo-codecs=vc1dec,h264dec,h264enc,h265dec,h265enc"
+
+         ;; Enable ZSTD compression for shader cache
+         "-Dzstd=enabled"

          ;; Also enable the tests.
          "-Dbuild-tests=true"

base-commit: ebb54e6a5fba36a571ed239398ee3e0307a91c26
--
2.41.0

Message #8 received at 64738 <at> debbugs.gnu.org (full text, mbox):

From: 宋文武 <iyzsong <at> envs.net>
To: Sigve Sudland <sigve_sudland <at> hotmail.com>
Cc: 64738 <at> debbugs.gnu.org
Subject: Re: bug#64738: [PATCH] gnu: Make Mesa use zstd compression for
 shader cache instead of zlib.
Date: Thu, 20 Jul 2023 18:22:20 +0800

Sigve Sudland <sigve_sudland <at> hotmail.com> writes:

> * gnu/packages/gl.scm (mesa): Enable zstd compression.

Good idea!  Follow our commit log style, I adjust it to:

    gnu: mesa: Enable zstd compression for shader cache.
    
    * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib.
    [arguments]: Add '-Dzstd=enabled' to configure-flags.


And apply it on the mesa-updates branch locally, wait it building, once
done I'll pushed it to the mesa-updates branch.  I think this issue
should be closed when it got merged into master.

Thank you!

Message #11 received at 64738 <at> debbugs.gnu.org (full text, mbox):

From: 宋文武 <iyzsong <at> envs.net>
To: Sigve Sudland <sigve_sudland <at> hotmail.com>
Cc: 64369 <at> debbugs.gnu.org, 64738 <at> debbugs.gnu.org
Subject: Re: bug#64738: [PATCH] gnu: Make Mesa use zstd compression for
 shader cache instead of zlib.
Date: Thu, 20 Jul 2023 18:58:31 +0800

>
>     gnu: mesa: Enable zstd compression for shader cache.
>     
>     * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib.
>     [arguments]: Add '-Dzstd=enabled' to configure-flags.

Pushed to the mesa-updates branch, merging progress tracked in
https://issues.guix.gnu.org/64369.

Thanks.

Message #14 received at 64738 <at> debbugs.gnu.org (full text, mbox):

From: John Kehayias <john.kehayias <at> protonmail.com>
To: 宋文武 <iyzsong <at> envs.net>
Cc: 64369 <at> debbugs.gnu.org, Sigve Sudland <sigve_sudland <at> hotmail.com>,
 64738 <at> debbugs.gnu.org, Christopher Baines <mail <at> cbaines.net>
Subject: Re: bug#64369: Request for merging "mesa-updates" branch
Date: Thu, 20 Jul 2023 15:54:55 +0000

Hi,

On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote:

>>
>>     gnu: mesa: Enable zstd compression for shader cache.
>>
>>     * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib.
>>     [arguments]: Add '-Dzstd=enabled' to configure-flags.
>
> Pushed to the mesa-updates branch, merging progress tracked in
> https://issues.guix.gnu.org/64369.
>
> Thanks.

I just saw this as the message only went to the bug number. I would not
have merged this yet without checking about build status after the big
texlive updates. I was hoping to avoid a full rebuild so we could merge
to master with substitutes already, after checking post-texlive and the
graft of mesa on master.

However, doesn't seem like Cuirass has picked up this change and hasn't
built yet. Any idea why?

My plan right now was going to be to merge master into mesa-updates, see
what the build status looks like, and then decide from there. If there
were not too many rebuilds (say less than 1000) from the updates on
master, we should be able to go ahead soon. And then start on the next
round of patches/updates for mesa-updates.

On the other hand, if there were lots of rebuilds there are other
patches waiting that should be used also (libdrm, libva if I remember)
as well as ungrafting the change on master. So, preventing waiting and
doing another big rebuild with the changes that have been waiting since
the first build of this branch.

What do people think? My thought right now was to revert this change
(since it rebuilds all of mesa dependents), merge master in, and check
the status. But I'm confused why Cuirass wasn't building..

John

Message #17 received at 64738 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: John Kehayias <john.kehayias <at> protonmail.com>
Cc: 64369 <at> debbugs.gnu.org, Sigve Sudland <sigve_sudland <at> hotmail.com>,
 64738 <at> debbugs.gnu.org, 宋文武 <iyzsong <at> envs.net>
Subject: Re: bug#64369: Request for merging "mesa-updates" branch
Date: Thu, 20 Jul 2023 17:03:28 +0100

[Message part 1 (text/plain, inline)]

John Kehayias <john.kehayias <at> protonmail.com> writes:

> Hi,
>
> On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote:
>
>>>
>>>     gnu: mesa: Enable zstd compression for shader cache.
>>>
>>>     * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib.
>>>     [arguments]: Add '-Dzstd=enabled' to configure-flags.
>>
>> Pushed to the mesa-updates branch, merging progress tracked in
>> https://issues.guix.gnu.org/64369.
>>
>> Thanks.
>
> I just saw this as the message only went to the bug number. I would not
> have merged this yet without checking about build status after the big
> texlive updates. I was hoping to avoid a full rebuild so we could merge
> to master with substitutes already, after checking post-texlive and the
> graft of mesa on master.
>
> However, doesn't seem like Cuirass has picked up this change and hasn't
> built yet. Any idea why?

Cuirass at ci.guix.gnu.org seems to be having problems noticing new
revisions. At least it's about a day behind for master branch revisions.

> My plan right now was going to be to merge master into mesa-updates, see
> what the build status looks like, and then decide from there. If there
> were not too many rebuilds (say less than 1000) from the updates on
> master, we should be able to go ahead soon. And then start on the next
> round of patches/updates for mesa-updates.
>
> On the other hand, if there were lots of rebuilds there are other
> patches waiting that should be used also (libdrm, libva if I remember)
> as well as ungrafting the change on master. So, preventing waiting and
> doing another big rebuild with the changes that have been waiting since
> the first build of this branch.
>
> What do people think? My thought right now was to revert this change
> (since it rebuilds all of mesa dependents), merge master in, and check
> the status. But I'm confused why Cuirass wasn't building.

The mesa package wasn't affected by tex-team-next, so unless there are
other inputs to most/all off the packages to which mesa is an input,
there shouldn't be many rebuilds. I'd still rebase or merge master in to
the branch and see what QA says though.

Whether you want to revert this latest commit and try and merge sooner,
or wait for things to be built and merge later is up to you though.

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 64738 <at> debbugs.gnu.org (full text, mbox):

From: John Kehayias <john.kehayias <at> protonmail.com>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 64369 <at> debbugs.gnu.org, Sigve Sudland <sigve_sudland <at> hotmail.com>,
 64738 <at> debbugs.gnu.org, 宋文武 <iyzsong <at> envs.net>
Subject: Re: bug#64369: Request for merging "mesa-updates" branch
Date: Thu, 20 Jul 2023 16:23:35 +0000

Hi Chris,

On Thu, Jul 20, 2023 at 05:03 PM, Christopher Baines wrote:

>
> John Kehayias <john.kehayias <at> protonmail.com> writes:
>> However, doesn't seem like Cuirass has picked up this change and hasn't
>> built yet. Any idea why?
>
> Cuirass at ci.guix.gnu.org seems to be having problems noticing new
> revisions. At least it's about a day behind for master branch revisions.
>

Okay, thanks for the info. So either way will let things build through
the weekend probably.

>> My plan right now was going to be to merge master into mesa-updates, see
>> what the build status looks like, and then decide from there. If there
>> were not too many rebuilds (say less than 1000) from the updates on
>> master, we should be able to go ahead soon. And then start on the next
>> round of patches/updates for mesa-updates.
>>
>> On the other hand, if there were lots of rebuilds there are other
>> patches waiting that should be used also (libdrm, libva if I remember)
>> as well as ungrafting the change on master. So, preventing waiting and
>> doing another big rebuild with the changes that have been waiting since
>> the first build of this branch.
>>
>> What do people think? My thought right now was to revert this change
>> (since it rebuilds all of mesa dependents), merge master in, and check
>> the status. But I'm confused why Cuirass wasn't building.
>
> The mesa package wasn't affected by tex-team-next, so unless there are
> other inputs to most/all off the packages to which mesa is an input,
> there shouldn't be many rebuilds. I'd still rebase or merge master in to
> the branch and see what QA says though.
>

Right, wasn't sure what the number of rebuilds the texlive updates
caused beyond tex-related stuff, so I didn't want to assume.

Are we able to rebase and then force push on savannah? I thought not,
or is that just for the master branch? If I can force push (I suppose
I could just try later), I would leave off 64738 (zstd in mesa) and
rebase on master. Otherwise I guess revert and merge master to
mesa-updates.

> Whether you want to revert this latest commit and try and merge sooner,
> or wait for things to be built and merge later is up to you though.
>

Then either way see what rebuilding looks like. If there isn't much
I'd opt for going sooner while we have substitutes already and then
gather these other patches for the next merge. Likely waiting for a
new mesa update which is probably due very soon. But if things will be
rebuilding a lot anyway, I might as well get all these other related
changes in to just build once.

Thanks Chris, I'll see what to do and keep an eye on the builds.

John

Message #23 received at 64738 <at> debbugs.gnu.org (full text, mbox):

From: John Kehayias <john.kehayias <at> protonmail.com>
To: 宋文武 <iyzsong <at> envs.net>
Cc: Sigve Sudland <sigve_sudland <at> hotmail.com>, 64738 <at> debbugs.gnu.org
Subject: Re: bug#64738: [PATCH] gnu: Make Mesa use zstd compression for shader
 cache instead of zlib.
Date: Thu, 20 Jul 2023 19:35:00 +0000

On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote:

>>
>>     gnu: mesa: Enable zstd compression for shader cache.
>>
>>     * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib.
>>     [arguments]: Add '-Dzstd=enabled' to configure-flags.
>
> Pushed to the mesa-updates branch, merging progress tracked in
> https://issues.guix.gnu.org/64369.
>
> Thanks.

An update here: I reset mesa-updates to just be the 2 commits from
before this patch (that were already built on Cuirass) but rebased on
master. Based on how many builds may be new, we can see what to do. I
hope it is just a small amount of rebuilds so mesa-updates can be merged
as is, and then include this and other patches for the next round.

Hope that is okay, especially as I think this is a minor change to mesa,
rather than a bug or missing hardware support.

John

Message #26 received at 64738 <at> debbugs.gnu.org (full text, mbox):

From: 宋文武 <iyzsong <at> envs.net>
To: John Kehayias <john.kehayias <at> protonmail.com>
Cc: Sigve Sudland <sigve_sudland <at> hotmail.com>, 64738 <at> debbugs.gnu.org
Subject: Re: bug#64738: [PATCH] gnu: Make Mesa use zstd compression for
 shader cache instead of zlib.
Date: Fri, 21 Jul 2023 18:15:46 +0800

John Kehayias <john.kehayias <at> protonmail.com> writes:

> On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote:
>
>>>
>>>     gnu: mesa: Enable zstd compression for shader cache.
>>>
>>>     * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib.
>>>     [arguments]: Add '-Dzstd=enabled' to configure-flags.
>>
>> Pushed to the mesa-updates branch, merging progress tracked in
>> https://issues.guix.gnu.org/64369.
>>
>> Thanks.
>
> An update here: I reset mesa-updates to just be the 2 commits from
> before this patch (that were already built on Cuirass) but rebased on
> master. Based on how many builds may be new, we can see what to do. I
> hope it is just a small amount of rebuilds so mesa-updates can be merged
> as is, and then include this and other patches for the next round.
>
> Hope that is okay, especially as I think this is a minor change to mesa,
> rather than a bug or missing hardware support.
>

That's fine, thank you for take care of it.

Message #29 received at 64738 <at> debbugs.gnu.org (full text, mbox):

From: John Kehayias <john.kehayias <at> protonmail.com>
To: 宋文武 <iyzsong <at> envs.net>
Cc: Sigve Sudland <sigve_sudland <at> hotmail.com>, 64738 <at> debbugs.gnu.org
Subject: Re: bug#64738: [PATCH] gnu: Make Mesa use zstd compression for shader
 cache instead of zlib.
Date: Tue, 25 Jul 2023 21:27:17 +0000

Hello,

On Fri, Jul 21, 2023 at 06:15 PM, 宋文武 wrote:

> John Kehayias <john.kehayias <at> protonmail.com> writes:
>
>> On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote:
>>
>>>>
>>>>     gnu: mesa: Enable zstd compression for shader cache.
>>>>
>>>>     * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib.
>>>>     [arguments]: Add '-Dzstd=enabled' to configure-flags.
>>>
>>> Pushed to the mesa-updates branch, merging progress tracked in
>>> <https://issues.guix.gnu.org/64369>.
>>>
>>> Thanks.
>>
>> An update here: I reset mesa-updates to just be the 2 commits from
>> before this patch (that were already built on Cuirass) but rebased on
>> master. Based on how many builds may be new, we can see what to do. I
>> hope it is just a small amount of rebuilds so mesa-updates can be merged
>> as is, and then include this and other patches for the next round.
>>
>> Hope that is okay, especially as I think this is a minor change to mesa,
>> rather than a bug or missing hardware support.
>>
>
> That's fine, thank you for take care of it.

I ended up adding this (and other changes) to mesa-updates along with
a mesa version bump and some other patches. Hopefully merging to
master in the next few days once substitutes look good.

Thanks!
John

Message #34 received at 64738-done <at> debbugs.gnu.org (full text, mbox):

From: John Kehayias <john.kehayias <at> protonmail.com>
To: 宋文武 <iyzsong <at> envs.net>
Cc: Sigve Sudland <sigve_sudland <at> hotmail.com>, 64738-done <at> debbugs.gnu.org
Subject: Re: bug#64738: [PATCH] gnu: Make Mesa use zstd compression for shader
 cache instead of zlib.
Date: Mon, 31 Jul 2023 15:27:23 +0000

On Tue, Jul 25, 2023 at 09:27 PM, John Kehayias wrote:

> Hello,
>
> On Fri, Jul 21, 2023 at 06:15 PM, 宋文武 wrote:
>
>> John Kehayias <john.kehayias <at> protonmail.com> writes:
>>
>>> On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote:
>>>
>>>>>
>>>>>     gnu: mesa: Enable zstd compression for shader cache.
>>>>>
>>>>>     * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib.
>>>>>     [arguments]: Add '-Dzstd=enabled' to configure-flags.
>>>>
>>>> Pushed to the mesa-updates branch, merging progress tracked in
>>>> <https://issues.guix.gnu.org/64369>.
>>>>
>>>> Thanks.
>>>
>>> An update here: I reset mesa-updates to just be the 2 commits from
>>> before this patch (that were already built on Cuirass) but rebased on
>>> master. Based on how many builds may be new, we can see what to do. I
>>> hope it is just a small amount of rebuilds so mesa-updates can be merged
>>> as is, and then include this and other patches for the next round.
>>>
>>> Hope that is okay, especially as I think this is a minor change to mesa,
>>> rather than a bug or missing hardware support.
>>>
>>
>> That's fine, thank you for take care of it.
>
> I ended up adding this (and other changes) to mesa-updates along with
> a mesa version bump and some other patches. Hopefully merging to
> master in the next few days once substitutes look good.
>
> Thanks!
> John

Pushed as 090c254fe7713db330636dae2c204c8282207cc8. Thanks!

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.