Package: guix;
Reported by: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Date: Wed, 15 Jul 2020 16:14:02 UTC
Severity: normal
To reply to this bug, email your comments to 42371 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
bug-guix <at> gnu.org:bug#42371; Package guix.
(Wed, 15 Jul 2020 16:14:02 GMT) Full text and rfc822 format available.Maxim Cournoyer <maxim.cournoyer <at> gmail.com>:bug-guix <at> gnu.org.
(Wed, 15 Jul 2020 16:14:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> To: bug-guix <bug-guix <at> gnu.org> Subject: Grafts max out build slots without being queued Date: Wed, 15 Jul 2020 12:13:13 -0400
Hello, While working on ruby-asciidoctor-pdf, the following command failed because of maxing out build users: ./pre-inst-env guix build --quiet --keep-going --check ruby_version <at> 1.0.2 gnome-shell-extension-gsconnect <at> 33 numix-gtk-theme <at> 2.6.7 arc-theme <at> 20190917 gnome <at> 3.34.2 ruby-fakefs <at> 1.2.2 ruby-protobuf <at> 3.10.3 ruby-fast-gettext <at> 2.0.3 ruby-sass-rails <at> 5.0.7 mailcatcher <at> 0.7.1 ruby-rspec-its <at> 1.3.0 warzone2100 <at> 3.4.0 keepassxc <at> 2.6.0 wsjtx <at> 2.2.2 js8call <at> 2.2.0 ruby-yard-with-tests <at> 0.9.25 ruby-asciidoctor-pdf <at> 1.5.3-1.f25ea78 ruby-web-console <at> 3.7.0 ruby-autoprefixer-rails <at> 9.4.7 ruby-jekyll-paginate-v2 <at> 2.0.0 ruby-solargraph <at> 0.36.0 --max-jobs=1 The following derivations will be built: /gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv /gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv /gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv /gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv /gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv /gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv /gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv /gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv /gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv /gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drv The following grafts will be made: /gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv /gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv /gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv /gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv /gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv /gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv /gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv /gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv /gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv /gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv /gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drv guix build: error: all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group It seems that grafts trigger the problem, as passing the --no-grafts option works around it. Could it be that grafts can't somehow be queued like normal builds, adapting to fit to the available build slots? Maxim
bug-guix <at> gnu.org:bug#42371; Package guix.
(Wed, 15 Jul 2020 17:14:01 GMT) Full text and rfc822 format available.Message #8 received at 42371 <at> debbugs.gnu.org (full text, mbox):
From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> To: 42371 <at> debbugs.gnu.org Subject: Re: bug#42371: Grafts max out build slots without being queued Date: Wed, 15 Jul 2020 13:13:46 -0400
Hello, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes: > It seems that grafts trigger the problem, as passing the --no-grafts > option works around it. Could it be that grafts can't somehow be queued > like normal builds, adapting to fit to the available build slots? I sent this message too early; the '--no-grafts' option does *not* work around the issue... I wonder what triggers this here? I commonly build things (sometimes hundreds of packages at the same time) and never ran into this issue. Maxim
Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
to control <at> debbugs.gnu.org.
(Wed, 15 Jul 2020 17:16:02 GMT) Full text and rfc822 format available.bug-guix <at> gnu.org:bug#42371; Package guix.
(Thu, 16 Jul 2020 10:19:01 GMT) Full text and rfc822 format available.Message #13 received at 42371 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Cc: 42371 <at> debbugs.gnu.org Subject: Re: bug#42371: Grafts max out build slots without being queued Date: Thu, 16 Jul 2020 12:18:11 +0200
Hi, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis: > While working on ruby-asciidoctor-pdf, the following command failed > because of maxing out build users: > > ./pre-inst-env guix build --quiet --keep-going --check ruby_version <at> 1.0.2 gnome-shell-extension-gsconnect <at> 33 numix-gtk-theme <at> 2.6.7 arc-theme <at> 20190917 gnome <at> 3.34.2 ruby-fakefs <at> 1.2.2 ruby-protobuf <at> 3.10.3 ruby-fast-gettext <at> 2.0.3 ruby-sass-rails <at> 5.0.7 mailcatcher <at> 0.7.1 ruby-rspec-its <at> 1.3.0 warzone2100 <at> 3.4.0 keepassxc <at> 2.6.0 wsjtx <at> 2.2.2 js8call <at> 2.2.0 ruby-yard-with-tests <at> 0.9.25 ruby-asciidoctor-pdf <at> 1.5.3-1.f25ea78 ruby-web-console <at> 3.7.0 ruby-autoprefixer-rails <at> 9.4.7 ruby-jekyll-paginate-v2 <at> 2.0.0 ruby-solargraph <at> 0.36.0 --max-jobs=1 > The following derivations will be built: > /gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv > /gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv > /gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv > /gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv > /gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv > /gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv > /gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv > /gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv > /gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv > /gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drv > The following grafts will be made: > /gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv > /gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv > /gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv > /gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv > /gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv > /gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv > /gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv > /gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv > /gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv > /gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv > /gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drv > guix build: error: all build users are currently in use; consider > creating additional users and adding them to the `guixbuild' group Presumably here it’s running one job at a time, so I don’t think it can end up using more than one build user. What does “guix processes” return when that happens? Could it be that some other Guix clients are running? Thanks, Ludo’.
bug-guix <at> gnu.org:bug#42371; Package guix.
(Thu, 16 Jul 2020 11:05:01 GMT) Full text and rfc822 format available.Message #16 received at 42371 <at> debbugs.gnu.org (full text, mbox):
From: zimoun <zimon.toutoune <at> gmail.com> To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, 42371 <at> debbugs.gnu.org Subject: Re: bug#42371: Grafts max out build slots without being queued Date: Thu, 16 Jul 2020 13:04:26 +0200
Dear Maxim, On Wed, 15 Jul 2020 at 12:13, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> wrote: > ./pre-inst-env guix build --quiet --keep-going --check > ruby_version <at> 1.0.2 gnome-shell-extension-gsconnect <at> 33 > numix-gtk-theme <at> 2.6.7 arc-theme <at> 20190917 gnome <at> 3.34.2 > ruby-fakefs <at> 1.2.2 ruby-protobuf <at> 3.10.3 ruby-fast-gettext <at> 2.0.3 > ruby-sass-rails <at> 5.0.7 mailcatcher <at> 0.7.1 ruby-rspec-its <at> 1.3.0 > warzone2100 <at> 3.4.0 keepassxc <at> 2.6.0 wsjtx <at> 2.2.2 js8call <at> 2.2.0 > ruby-yard-with-tests <at> 0.9.25 ruby-asciidoctor-pdf <at> 1.5.3-1.f25ea78 > ruby-web-console <at> 3.7.0 ruby-autoprefixer-rails <at> 9.4.7 > ruby-jekyll-paginate-v2 <at> 2.0.0 ruby-solargraph <at> 0.36.0 --max-jobs=1 Instead, I have run: --8<---------------cut here---------------start------------->8--- guix build --keep-going --check ruby_version gnome-shell-extension-gsconnect numix-gtk-theme arc-theme gnome ruby-fakefs git-minimal ruby-fast-gettext ruby-sass-rails mailcatcher ruby-rspec-its warzone2100 keepassxc wsjtx js8call ruby-web-console hello ruby-autoprefixer-rails ruby-jekyll-paginate-v2 ruby-solargraph --max-jobs=1 --8<---------------cut here---------------end--------------->8--- replacing the 2 unknown packages ruby-yard-with-tests and ruby-protobuf by git-minimal and hello. And it passes. I have 10 guixbuilder users, as it is by default. :-) > The following derivations will be built: > /gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv > /gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv > /gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv > /gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv > /gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv > /gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv > /gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv > /gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv > /gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv > /gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drv > The following grafts will be made: > /gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv > /gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv > /gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv > /gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv > /gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv > /gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv > /gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv > /gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv > /gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv > /gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv > /gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drv > guix build: error: all build users are currently in use; consider > creating additional users and adding them to the `guixbuild' group How many users do you have? What are they doing? All the best, simon
bug-guix <at> gnu.org:bug#42371; Package guix.
(Thu, 16 Jul 2020 11:10:01 GMT) Full text and rfc822 format available.Message #19 received at 42371 <at> debbugs.gnu.org (full text, mbox):
From: zimoun <zimon.toutoune <at> gmail.com> To: Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Cc: 42371 <at> debbugs.gnu.org Subject: Re: bug#42371: Grafts max out build slots without being queued Date: Thu, 16 Jul 2020 13:09:20 +0200
On Thu, 16 Jul 2020 at 12:18, Ludovic Courtès <ludo <at> gnu.org> wrote: > Presumably here it’s running one job at a time, so I don’t think it can > end up using more than one build user. > > What does “guix processes” return when that happens? Could it be that > some other Guix clients are running? Is it possible that the "lockFile()" is busy for another reason? Cheers, simon
bug-guix <at> gnu.org:bug#42371; Package guix.
(Thu, 16 Jul 2020 22:07:02 GMT) Full text and rfc822 format available.Message #22 received at 42371 <at> debbugs.gnu.org (full text, mbox):
From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> To: Ludovic Courtès <ludo <at> gnu.org> Cc: 42371 <at> debbugs.gnu.org Subject: Re: bug#42371: Grafts max out build slots without being queued Date: Thu, 16 Jul 2020 18:06:24 -0400
Hi Ludo :-) Ludovic Courtès <ludo <at> gnu.org> writes: > Hi, > > Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis: > >> While working on ruby-asciidoctor-pdf, the following command failed >> because of maxing out build users: >> >> ./pre-inst-env guix build --quiet --keep-going --check >> ruby_version <at> 1.0.2 gnome-shell-extension-gsconnect <at> 33 >> numix-gtk-theme <at> 2.6.7 arc-theme <at> 20190917 gnome <at> 3.34.2 >> ruby-fakefs <at> 1.2.2 ruby-protobuf <at> 3.10.3 ruby-fast-gettext <at> 2.0.3 >> ruby-sass-rails <at> 5.0.7 mailcatcher <at> 0.7.1 ruby-rspec-its <at> 1.3.0 >> warzone2100 <at> 3.4.0 keepassxc <at> 2.6.0 wsjtx <at> 2.2.2 js8call <at> 2.2.0 >> ruby-yard-with-tests <at> 0.9.25 ruby-asciidoctor-pdf <at> 1.5.3-1.f25ea78 >> ruby-web-console <at> 3.7.0 ruby-autoprefixer-rails <at> 9.4.7 >> ruby-jekyll-paginate-v2 <at> 2.0.0 ruby-solargraph <at> 0.36.0 --max-jobs=1 >> The following derivations will be built: >> /gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv >> /gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv >> /gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv >> /gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv >> /gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv >> /gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv >> /gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv >> /gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv >> /gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv >> /gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drv >> The following grafts will be made: >> /gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv >> /gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv >> /gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv >> /gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv >> /gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv >> /gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv >> /gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv >> /gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv >> /gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv >> /gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv >> /gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drv >> guix build: error: all build users are currently in use; consider >> creating additional users and adding them to the `guixbuild' group > > Presumably here it’s running one job at a time, so I don’t think it can > end up using more than one build user. > > What does “guix processes” return when that happens? Could it be that > some other Guix clients are running? > > Thanks, > Ludo’. So, I'm afraid this isn't very useful, but: for the most part of the process, I had something like: --8<---------------cut here---------------start------------->8--- SessionPID: 12549 ClientPID: 12540 ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment --pure gcc-toolchain <at> 10 --ad-hoc coreutils strace the-silver-searcher findutils SessionPID: 1876 ClientPID: 1868 ClientCommand: /gnu/store/6ms9q743slbyx4zj7p97sqrwx8x3dgkm-profile/bin/guile \ /home/maxim/src/guix/scripts/guix build --no-grafts --max-jobs=1 --quiet --keep-going --check ruby_version <at> 1.0.2 gnome-shell-extension-gsconnect <at> 33 numix-gtk-theme <at> 2.6.7 arc-theme <at> 20190917 gnome <at> 3.34.2 ruby-fakefs <at> 1.2.2 ruby-protobuf <at> 3.10.3 ruby-fast-gettext <at> 2.0.3 ruby-sass-rails <at> 5.0.7 mailcatcher <at> 0.7.1 ruby-rspec-its <at> 1.3.0 warzone2100 <at> 3.4.0 keepassxc <at> 2.6.0 wsjtx <at> 2.2.2 js8call <at> 2.2.0 ruby-yard-with-tests <at> 0.9.25 ruby-asciidoctor-pdf <at> 1.5.3-1.f25ea78 ruby-web-console <at> 3.7.0 ruby-autoprefixer-rails <at> 9.4.7 ruby-jekyll-paginate-v2 <at> 2.0.0 ruby-solargraph <at> 0.36.0 SessionPID: 447 ClientPID: 441 ClientCommand: /gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2/bin/guile \ /gnu/store/3f4xln2mii6cmc4c6x9138wl0bsh0fs7-guix-1.1.0-7.d445c30/bin/.guix-real publish -u guix-publish -p 80 -C gzip:3 --nar-path=nar --listen=0.0.0.0 SessionPID: 7060 ClientPID: 7039 ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment guix --8<---------------cut here---------------end--------------->8--- And after the same error was printed (guix build: error: all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group), then I had: --8<---------------cut here---------------start------------->8--- SessionPID: 12549 ClientPID: 12540 ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment --pure gcc-toolchain <at> 10 --ad-hoc coreutils strace the-silver-searcher findutils SessionPID: 447 ClientPID: 441 ClientCommand: /gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2/bin/guile \ /gnu/store/3f4xln2mii6cmc4c6x9138wl0bsh0fs7-guix-1.1.0-7.d445c30/bin/.guix-real publish -u guix-publish -p 80 -C gzip:3 --nar-path=nar --listen=0.0.0.0 SessionPID: 7060 ClientPID: 7039 ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment guix --8<---------------cut here---------------end--------------->8--- I didn't see the amount of build process going higher than than, monitoring it every second. I've pushed the branch I'm using to test locally here [0], hoping it'll help to reproduce the issue somewhere else. The command to run on this branch is: ./pre-inst-env guix build --no-grafts --max-jobs=1 --quiet --keep-going --check ruby_version <at> 1.0.2 gnome-shell-extension-gsconnect <at> 33 numix-gtk-theme <at> 2.6.7 arc-theme <at> 20190917 gnome <at> 3.34.2 ruby-fakefs <at> 1.2.2 ruby-protobuf <at> 3.10.3 ruby-fast-gettext <at> 2.0.3 ruby-sass-rails <at> 5.0.7 mailcatcher <at> 0.7.1 ruby-rspec-its <at> 1.3.0 warzone2100 <at> 3.4.0 keepassxc <at> 2.6.0 wsjtx <at> 2.2.2 js8call <at> 2.2.0 ruby-yard-with-tests <at> 0.9.25 ruby-asciidoctor-pdf <at> 1.5.3-1.f25ea78 ruby-web-console <at> 3.7.0 ruby-autoprefixer-rails <at> 9.4.7 ruby-jekyll-paginate-v2 <at> 2.0.0 ruby-solargraph <at> 0.36.0 The bug is only triggered when I also pass the "--check" option. So you'll probably have to build the stack once without "--check", then a second time with "--check" passed to trigger the issue (if it's reproducible). Thank you! Maxim [0] https://gitlab.com/Apteryks/guix/-/tree/add-ruby-asciidoctor-pdf
bug-guix <at> gnu.org:bug#42371; Package guix.
(Fri, 17 Jul 2020 02:39:01 GMT) Full text and rfc822 format available.Message #25 received at 42371 <at> debbugs.gnu.org (full text, mbox):
From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> To: zimoun <zimon.toutoune <at> gmail.com> Cc: 42371 <at> debbugs.gnu.org Subject: Re: bug#42371: Grafts max out build slots without being queued Date: Thu, 16 Jul 2020 22:38:12 -0400
Hey Simon, zimoun <zimon.toutoune <at> gmail.com> writes: > Dear Maxim, > > > On Wed, 15 Jul 2020 at 12:13, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> wrote: > >> ./pre-inst-env guix build --quiet --keep-going --check >> ruby_version <at> 1.0.2 gnome-shell-extension-gsconnect <at> 33 >> numix-gtk-theme <at> 2.6.7 arc-theme <at> 20190917 gnome <at> 3.34.2 >> ruby-fakefs <at> 1.2.2 ruby-protobuf <at> 3.10.3 ruby-fast-gettext <at> 2.0.3 >> ruby-sass-rails <at> 5.0.7 mailcatcher <at> 0.7.1 ruby-rspec-its <at> 1.3.0 >> warzone2100 <at> 3.4.0 keepassxc <at> 2.6.0 wsjtx <at> 2.2.2 js8call <at> 2.2.0 >> ruby-yard-with-tests <at> 0.9.25 ruby-asciidoctor-pdf <at> 1.5.3-1.f25ea78 >> ruby-web-console <at> 3.7.0 ruby-autoprefixer-rails <at> 9.4.7 >> ruby-jekyll-paginate-v2 <at> 2.0.0 ruby-solargraph <at> 0.36.0 --max-jobs=1 > > Instead, I have run: > > guix build --keep-going --check ruby_version > gnome-shell-extension-gsconnect numix-gtk-theme arc-theme gnome > ruby-fakefs git-minimal ruby-fast-gettext ruby-sass-rails mailcatcher > ruby-rspec-its warzone2100 keepassxc wsjtx js8call ruby-web-console > hello ruby-autoprefixer-rails ruby-jekyll-paginate-v2 ruby-solargraph > --max-jobs=1 > > replacing the 2 unknown packages ruby-yard-with-tests and ruby-protobuf > by git-minimal and hello. Thank you for your tests! This makes me wonder whether these are the ones causing the problem... I've uploaded the branch I'm working on so that you could try to reproduce [0] if you are so inclined :-). > [...] I have 10 guixbuilder users, as it is by default. :-) Same here. [...] >> guix build: error: all build users are currently in use; consider >> creating additional users and adding them to the `guixbuild' group > > How many users do you have? What are they doing? 10 users, not doing anything special, as I found out looking at guix processes, so I don't know what is causing the exhaustion of build users. Thanks, Maxim [0] https://gitlab.com/Apteryks/guix/-/tree/add-ruby-asciidoctor-pdf
bug-guix <at> gnu.org:bug#42371; Package guix.
(Fri, 17 Jul 2020 03:07:01 GMT) Full text and rfc822 format available.Message #28 received at 42371 <at> debbugs.gnu.org (full text, mbox):
From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> To: zimoun <zimon.toutoune <at> gmail.com> Cc: 42371 <at> debbugs.gnu.org, ludo <at> gnu.org Subject: Re: bug#42371: Grafts max out build slots without being queued Date: Thu, 16 Jul 2020 23:06:41 -0400
Hello everyone, I did a couple more tests, and there's no need to get fancy with the reproduction steps; any number greater of packages greater than your number of build slots/users should suffice: --8<---------------cut here---------------start------------->8--- guix build --quiet --check $(guix package -A | head -n 11 | cut -f1) The following derivations will be built: /gnu/store/l7hrbi8cikf4c0calvqwd2n99v51ljzb-aalib-1.4rc5.drv /gnu/store/bijqg7w67rq1lr20hqbk18rq73963xfr-a2ps-4.14.drv /gnu/store/vh283nmxk08sl0m89s1fn07m432c9dg0-4ti2-1.6.9.drv /gnu/store/davmjim4nfvbhimpim329p642p9mg50d-0xffff-0.8.drv /gnu/store/7qaf91fmf42c8vp3fcysvqpkg4zahl1j-0ad-data-0.0.23b-alpha.drv The following grafts will be made: /gnu/store/hjr33zqgx46ldswbqlmvgz7g37l7hrxl-abbaye-2.0.1.drv /gnu/store/71rdqs3hldmy1d92742camwc97l3pb3w-7kaa-2.15.3.drv /gnu/store/0b92jff2kfmd6l23y8v9xcprk8l7casw-4store-1.1.6.drv /gnu/store/82pj69v420fjmkqjxjxix3cckwcfdwg2-4dtris-0.4.3.drv /gnu/store/gamc676fdg1p7qzvzwfq7rp6qn4agss1-389-ds-base-1.4.0.21.drv /gnu/store/p310ysfwix0ci2594x78vv4x5isqcxix-0ad-0.0.23b-alpha.drv guix build: error: all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group --8<---------------cut here---------------end--------------->8--- The key is passing more than 10 packages (I have 10 build users) as an argument when using the '--check' argument. I'll update the title to reflect the actual bug. Maxim
Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
to control <at> debbugs.gnu.org.
(Fri, 17 Jul 2020 03:09:02 GMT) Full text and rfc822 format available.bug-guix <at> gnu.org:bug#42371; Package guix.
(Fri, 17 Jul 2020 09:24:01 GMT) Full text and rfc822 format available.Message #33 received at 42371 <at> debbugs.gnu.org (full text, mbox):
From: zimoun <zimon.toutoune <at> gmail.com> To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Cc: 42371 <at> debbugs.gnu.org, ludo <at> gnu.org Subject: Re: bug#42371: Grafts max out build slots without being queued Date: Fri, 17 Jul 2020 11:23:04 +0200
Hi Maxim,
> --8<---------------cut here---------------start------------->8---
> guix build --quiet --check $(guix package -A | head -n 11 | cut -f1)
[...]
> guix build: error: all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group
> --8<---------------cut here---------------end--------------->8---
In my case, it starts to download substitutes (but I have remove
--quiet), something like:
--8<---------------cut here---------------start------------->8---
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
11.3 MB will be downloaded:
/gnu/store/…-jansson-2.12
/gnu/store/…-jemalloc-5.2.1
/gnu/store/…-libev-4.31
/gnu/store/…-glibc-2.31
/gnu/store/…-glibc-2.31-static
/gnu/store/…-c-ares-1.16.0
/gnu/store/…-nghttp2-1.41.0-lib
/gnu/store/…-nghttp2-1.41.0
substituting /gnu/store/…-c-ares-1.16.0...
downloading from https://ci.guix.gnu.org/nar/lzip/84hjhzjypgp5pgh6r5y7m561lk625\
pn0-c-ares-1.16.0 ...
c-ares-1.16.0 123KiB 1.6MiB/s 00:00 [##################] 100.0%
substituting /gnu/store/…-jansson-2.12...
downloading from https://ci.guix.gnu.org/nar/lzip/v5rgf8v6cjxjbngkzjcznj98dkxj8\
svg-jansson-2.12 ...
jansson-2.12 27KiB 4.3MiB/s 00:00 [##################] 100.0%
[…]
13.2 MB will be downloaded:
/gnu/store/…-cyrus-sasl-2.1.27
/gnu/store/…-glibc-2.31
/gnu/store/…-glibc-2.31-static
/gnu/store/…-gnutls-3.6.12
/gnu/store/…-openldap-2.4.50
substituting /gnu/store/…-cyrus-sasl-2.1.27...
downloading from https://ci.guix.gnu.org/nar/lzip/xp8545c74xkkj12q41d1zl5wngb47\
7gj-cyrus-sasl-2.1.27 ...
cyrus-sasl-2.1.27 234K
[…]
28.0 MB will be downloaded:
/gnu/store/…-pcre-8.44-bin
/gnu/store/…-pcre-8.44
/gnu/store/…-mpfr-4.0.2
/gnu/store/…-rasqal-0.9.33
/gnu/store/…-libxslt-1.1.34
/gnu/store/…-raptor2-2.0.15
/gnu/store/…-glib-2.62.6
/gnu/store/…-dbus-1.12.16
/gnu/store/…-glib-2.62.6-bin
/gnu/store/…-glib-2.62.6
/gnu/store/…-avahi-0.8
/gnu/store/…-4store-1.1.6
/gnu/store/…-a2ps-4.14
/gnu/store/…-mesa-20.0.8-bin
/gnu/store/…-mesa-20.0.8
[…]
The following derivations will be built:
/gnu/store/…-abbaye-2.0.1.drv
/gnu/store/…-sdl-union-1.2.15.drv
/gnu/store/…-0ad-0.0.23b-alpha.drv
899.1 MB will be downloaded:
/gnu/store/…-libsodium-1.0.18
/gnu/store/…-mozjs-38.2.1.rc0
[…]
/gnu/store/…-ed-1.16
/gnu/store/…-patch-2.7.6
/gnu/store/…-diffutils-3.7
[…]
successfully built /gnu/store/…-sdl-union-1.2.15.drv
building /gnu/store/…-0ad-0.0.23b-alpha.drv...
WARNING: (guile-user): imported module (guix build utils) overrides core bindin\
g `delete'
starting phase `set-SOURCE-DATE-EPOCH'
phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
starting phase `set-paths'
environment variable `PATH' …
[…]
--8<---------------cut here---------------end--------------->8---
And I even tried with "-n21" and it seems ok.
I am running Guix on the top of Debian with ext4.
> The key is passing more than 10 packages (I have 10 build users) as an
> argument when using the '--check' argument.
Well, I am not sure to understand, but in nix/libstore/build.cc,
"void UserLock::acquire()", there is:
--8<---------------cut here---------------start------------->8---
foreach (Strings::iterator, i, users) {
[…]
if (lockFile(fd, ltWrite, false)) {
[…]
return;
}
}
throw Error(format("all build users are currently in use; "
"consider creating additional users and adding them to the `%1%' group")
% settings.buildUsersGroup);
--8<---------------cut here---------------end--------------->8---
So maybe the file is locked for whatever other reason on your machine?
All the best,
simon
bug-guix <at> gnu.org:bug#42371; Package guix.
(Tue, 21 Jul 2020 10:42:01 GMT) Full text and rfc822 format available.Message #36 received at 42371 <at> debbugs.gnu.org (full text, mbox):
From: zimoun <zimon.toutoune <at> gmail.com> To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Cc: 42371 <at> debbugs.gnu.org, ludo <at> gnu.org Subject: Re: bug#42371: Grafts max out build slots without being queued Date: Tue, 21 Jul 2020 12:40:59 +0200
Dear,
On my laptop and my desktop, both “cat /proc/cpuinfo |grep processor|wc
-l” returning 4, and with 10 guixbuild users, I get these results.
On Thu, 16 Jul 2020 at 23:06, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> wrote:
> --8<---------------cut here---------------start------------->8---
> guix build --quiet --check $(guix package -A | head -n 11 | cut -f1)
[…]
guix build: error: derivation `/gnu/store/…-4ti2-1.6.9.drv' may not be determin\
istic: output `/gnu/store/…-4ti2-1.6.9' differs
> --8<---------------cut here---------------end--------------->8---
Then, I run this tiny script fixing --max-jobs and --cores on my desktop
machine only:
--8<---------------cut here---------------start------------->8---
for npkg in 4 5 6 7 8 9 10 11 12
do
for JOBS in 1 2 3 4
do
for CORES in 1 2 4 8 16 32
do
echo "# num.of pkg=$npkg"
echo "# --max-jobs=$JOBS"
echo "# --cores=$CORES"
guix build --quiet --check -k \
--max-jobs=$JOBS \
--cores=$CORES \
$(guix package -A | head -n $npkg | cut -f1)
echo " "
sleep 2
done
done
done
--8<---------------cut here---------------end--------------->8---
And it hits the non-deterministic error for all the cases until the case
num.of pkg=11 --max-jobs=2 --cores=1 where it errors "all build users
are currently in use; consider creating additional users and adding them
to the `guixbuild' group".
Then npkg=11 --max-jobs=2 --cores=2 fails with non-deterministic.
(I reproduce on my laptop the result for npkg=11 max-jobs=2 cores=1,2.)
The next case, npkg=11 --max-jobs=2 --cores=4 does not seem reproducible
on my machine; similarly for cores=8,16,32.
Then I have not checked the reproducibility of the cases below.
Interestingly, npkg=11 --max-jobs=3 --cores=1,2,4,8,16 fails with not
enough builders. But not for --cores=32 which fails with
non-deterministic.
All the values for cores fails with not enough builder for npkg=11 and
--max-jobs=4.
Another surprise, npkg=12 --max-jobs=1 --cores=1,2,4,8,16,32 fails with
non-deterministic. And npkg=12 --max-jobs=2 --cores=2,4,8,16,32 too.
All the best,
simon
bug-guix <at> gnu.org:bug#42371; Package guix.
(Tue, 08 Dec 2020 18:25:01 GMT) Full text and rfc822 format available.Message #39 received at 42371 <at> debbugs.gnu.org (full text, mbox):
From: zimoun <zimon.toutoune <at> gmail.com> To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Cc: 42371 <at> debbugs.gnu.org Subject: Re: bug#42371: Grafts max out build slots without being queued Date: Fri, 04 Dec 2020 15:18:35 +0100
Hi,
As a reminder about the annoyance of the bug, I get:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build -m ~/tmp/Guix/list-bioconductor.scm -k -v0 --check
guix build: error: all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group
--8<---------------cut here---------------end--------------->8---
where list-bioconductor.scm is below. It is annoying because the bug
prevents massive rebuild; for instance here the ones from the
Bioconductor archive, useful when massive update or to track
reproducibility issue.
All the best,
simon
--8<---------------cut here---------------start------------->8---
(use-modules (guix) (gnu)
(guix download)
(guix svn-download)
(guix git-download)
(ice-9 match)
(srfi srfi-1)
(srfi srfi-26))
(define (bioconductor? package)
(define (bioc-string? str)
(string-contains str "bioconductor.org"))
(match (package-source package)
((? origin? o)
(match (origin-uri o)
((? string? url)
(bioc-string? url))
(((? string? urls) ...)
(any bioc-string? urls)) ;or 'find'
(_ #f)))
(_ #f)))
(define packages-from-bioconductor
(fold-packages (lambda (package result)
(if (bioconductor? package)
(cons package result)
result))
'()))
;; (map (lambda (pkg)
;; (begin
;; (format #t "~A~%"(package-name pkg))))
;; packages-from-bioconductor)
(specifications->manifest
(map package-name packages-from-bioconductor))
--8<---------------cut here---------------end--------------->8---
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.