GNU bug report logs - #50312
Recurring timeout error while using offload

Previous Next

To reply to this bug, email your comments to 50312 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: bug-guix <bug-guix <at> gnu.org>
Subject: Recurring timeout eror while using offload
Date: Wed, 01 Sep 2021 01:35:11 -0400

Hello Guix,

I'm encountered timeout error that seem to be caused by a particular
payload rather than networking issues.   With a specific payload (a
bunch of rust package to build on the core-updates-frozen branch), it
seems to be triggered every time.

Here's the derivations to be built:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build rust -n
The following derivations would be built:
   /gnu/store/nv3l1lyx4x33gx80pxk8033g01b6bv0m-rust-1.54.0.drv
   /gnu/store/bmc82kpr11gpk1xy7d3l6hy37cs58i7d-rustc-1.54.0-src.tar.xz.drv
   /gnu/store/x1v25nclqps677j1g5dvyck2njaxy0ya-rust-1.53.0.drv
   /gnu/store/9v5yl0fg4dj1n82pkznksasmdjgi541w-rust-1.52.1.drv
   /gnu/store/72572pjayh2j5frr187i7axjzfd01kp6-rust-1.51.0.drv
   /gnu/store/0z13dkldzsxf87f4kgi3b2545bm39nnh-rustc-1.51.0-src.tar.xz.drv
   /gnu/store/q9mbicnmrpd1c98vmi5y0gyjnpr3gqbn-rust-1.50.0.drv
   /gnu/store/h5959ysczr862ldwh6f85h10895c4csp-rust-1.49.0.drv
   /gnu/store/izgl0l21r5lwd3d3glxqwmigl0315b26-rustc-1.49.0-src.tar.xz.drv
   /gnu/store/n0dpka76b9q0w4r2k912qq5p2a57bz6c-rust-1.48.0.drv
   /gnu/store/a9qwa9cl90xrm2dpa3hnd3ak54nqss8n-rust-1.47.0.drv
   /gnu/store/7ib14fx50651av5bq5g195cqf2azsvhc-rust-1.46.0.drv
   /gnu/store/h3mxv9zlx8q3x0m0s5gvin2n1ndliivw-rust-1.45.2.drv
   /gnu/store/4i5xm0wcasilh55f9gfcrzp2sjdkd507-rust-1.44.1.drv
   /gnu/store/m7z7jdd2my4p9p5mjwa0cna4fzhd1v8j-rust-1.43.0.drv
   /gnu/store/pw4ynvgbnk1j6fxlh37a9wd8fyi7bjnc-rustc-1.50.0-src.tar.xz.drv
   /gnu/store/g9izbhyhc6jcalxgbxc6aas3aa66h2aa-rustc-1.52.1-src.tar.xz.drv
   /gnu/store/jbr3f7360x3ww5rn6nnb803yxq7ppck3-rustc-1.53.0-src.tar.xz.drv
--8<---------------cut here---------------end--------------->8---

And here is what the error looks like:

--8<---------------cut here---------------start------------->8---
rustc-1.52.1-src/src/llvm-project/clang/test/Misc/message-length.c
rustc-1.52.1-src/src/llvm-project/clang/test/Driver/fmessage-length.c
rustc-1.52.1-src/src/llvm-project/compiler-rt/test/hwasan/TestCases/stack-history-length.c
rustc-1.52.1-src/src/llvm-project/clang/test/Sema/vector_swizzle_length.c
rustc-1.52.1-src/src/llvm-project/clang/test/Sema/array-bounds-ptr-arith.c
rustc-1.52.1-src/src/llvm-project/clang/test/Analysis/ptr-arith.c
rustc-1.52.1-src/vendor/openssl-src/openssl/crypto/engine/tb_asnmth.c
rustc-1.52.1-src/src/llvm-project/clang/test/CodeGen/aarch64-sve-intrinsics/acle_sve_cnth.c
rustc-1.52.1-src/vendor/openssl-src/openssl/crypto/pem/pem_oth.c
rustc-1.52.1-src/vendor/libgit2-sys/libgit2/src/transports/auth.c
guix offload: error: failed to connect to '127.0.0.1': Timeout connecting to 127.0.0.1
cannot build derivation `/gnu/store/4i5xm0wcasilh55f9gfcrzp2sjdkd507-rust-1.44.1.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/h3mxv9zlx8q3x0m0s5gvin2n1ndliivw-rust-1.45.2.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/7ib14fx50651av5bq5g195cqf2azsvhc-rust-1.46.0.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/a9qwa9cl90xrm2dpa3hnd3ak54nqss8n-rust-1.47.0.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/n0dpka76b9q0w4r2k912qq5p2a57bz6c-rust-1.48.0.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/h5959ysczr862ldwh6f85h10895c4csp-rust-1.49.0.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/q9mbicnmrpd1c98vmi5y0gyjnpr3gqbn-rust-1.50.0.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/72572pjayh2j5frr187i7axjzfd01kp6-rust-1.51.0.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/9v5yl0fg4dj1n82pkznksasmdjgi541w-rust-1.52.1.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/x1v25nclqps677j1g5dvyck2njaxy0ya-rust-1.53.0.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/nv3l1lyx4x33gx80pxk8033g01b6bv0m-rust-1.54.0.drv': 1 dependencies couldn't be built
guix build: error: build of `/gnu/store/nv3l1lyx4x33gx80pxk8033g01b6bv0m-rust-1.54.0.drv' failed

real    2m50.854s
user    0m11.302s
sys     0m0.661s
--8<---------------cut here---------------end--------------->8---

It occurs shortly after heavy scrolling starts.

I've verified that it's not my network link at least on my local
network; any ideas what I should look into for debugging?  Perhaps the
logs/strace of the remote SSH server?

Thanks,

Maxim

Message #8 received at 50312 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 50312 <at> debbugs.gnu.org
Subject: Re: bug#50312: Recurring timeout eror while using offload
Date: Wed, 01 Sep 2021 12:57:38 -0400

Hello,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:

[...]

> And here is what the error looks like:
>
> rustc-1.52.1-src/src/llvm-project/clang/test/Misc/message-length.c
> rustc-1.52.1-src/src/llvm-project/clang/test/Driver/fmessage-length.c
> rustc-1.52.1-src/src/llvm-project/compiler-rt/test/hwasan/TestCases/stack-history-length.c
> rustc-1.52.1-src/src/llvm-project/clang/test/Sema/vector_swizzle_length.c
> rustc-1.52.1-src/src/llvm-project/clang/test/Sema/array-bounds-ptr-arith.c
> rustc-1.52.1-src/src/llvm-project/clang/test/Analysis/ptr-arith.c
> rustc-1.52.1-src/vendor/openssl-src/openssl/crypto/engine/tb_asnmth.c
> rustc-1.52.1-src/src/llvm-project/clang/test/CodeGen/aarch64-sve-intrinsics/acle_sve_cnth.c
> rustc-1.52.1-src/vendor/openssl-src/openssl/crypto/pem/pem_oth.c
> rustc-1.52.1-src/vendor/libgit2-sys/libgit2/src/transports/auth.c
> guix offload: error: failed to connect to '127.0.0.1': Timeout connecting to 127.0.0.1
> cannot build derivation `/gnu/store/4i5xm0wcasilh55f9gfcrzp2sjdkd507-rust-1.44.1.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/h3mxv9zlx8q3x0m0s5gvin2n1ndliivw-rust-1.45.2.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/7ib14fx50651av5bq5g195cqf2azsvhc-rust-1.46.0.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/a9qwa9cl90xrm2dpa3hnd3ak54nqss8n-rust-1.47.0.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/n0dpka76b9q0w4r2k912qq5p2a57bz6c-rust-1.48.0.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/h5959ysczr862ldwh6f85h10895c4csp-rust-1.49.0.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/q9mbicnmrpd1c98vmi5y0gyjnpr3gqbn-rust-1.50.0.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/72572pjayh2j5frr187i7axjzfd01kp6-rust-1.51.0.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/9v5yl0fg4dj1n82pkznksasmdjgi541w-rust-1.52.1.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/x1v25nclqps677j1g5dvyck2njaxy0ya-rust-1.53.0.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/nv3l1lyx4x33gx80pxk8033g01b6bv0m-rust-1.54.0.drv': 1 dependencies couldn't be built
> guix build: error: build of `/gnu/store/nv3l1lyx4x33gx80pxk8033g01b6bv0m-rust-1.54.0.drv' failed
>
> real    2m50.854s
> user    0m11.302s
> sys     0m0.661s
>
> It occurs shortly after heavy scrolling starts.
>
> I've verified that it's not my network link at least on my local
> network; any ideas what I should look into for debugging?  Perhaps the
> logs/strace of the remote SSH server?
>
> Thanks,
>
> Maxim

Now that I've manually built them by small batches of 4 (that would not
fail), I reached the last item rust-1.54, which failed like so:

--8<---------------cut here---------------start------------->8---
rustc-1.54.0-src/vendor/lzma-sys/xz-5.2/tests/files/good-0cat-empty.xz
source is at 'rustc-1.54.0-src'
Backtrace:
          12 (primitive-load "/gnu/store/j2hl7fmsbh7jmixpy8dy07finqp…")
In ice-9/eval.scm:
    619:8 11 (_ #(#(#<directory (guile-user) 7ffff5fdbc80> "rus…") #))
In ice-9/boot-9.scm:
    142:2 10 (dynamic-wind _ _ #<procedure 7ffff5fc8e00 at ice-9/eva…>)
In system/base/compile.scm:
   352:28  9 (compile _ #:from _ #:to _ #:env _ #:optimization-level …)
   265:44  8 (_ _ _)
   265:44  7 (_ _ _)
   265:44  6 (_ _ _)
   261:27  5 (_ _ _)
In ice-9/boot-9.scm:
   2835:4  4 (save-module-excursion _)
In language/bytecode/spec.scm:
    35:19  3 (_)
In ice-9/boot-9.scm:
   260:13  2 (for-each #<procedure delete-file-recursively (dir #:k…> …)
In guix/build/utils.scm:
   446:23  1 (delete-file-recursively "vendor/jemalloc-sys/jemalloc" …)
In unknown file:
           0 (lstat "vendor/jemalloc-sys/jemalloc")

ERROR: In procedure lstat:
In procedure lstat: No such file or directory: "vendor/jemalloc-sys/jemalloc"
note: keeping build directory `/tmp/guix-build-rustc-1.54.0-src.tar.xz.drv-0'
builder for `/gnu/store/bmc82kpr11gpk1xy7d3l6hy37cs58i7d-rustc-1.54.0-src.tar.xz.drv' failed with exit code 1
build of /gnu/store/bmc82kpr11gpk1xy7d3l6hy37cs58i7d-rustc-1.54.0-src.tar.xz.drv failed
View build log at '/var/log/guix/drvs/bm/c82kpr11gpk1xy7d3l6hy37cs58i7d-rustc-1.54.0-src.tar.xz.drv'.
cannot build derivation `/gnu/store/nv3l1lyx4x33gx80pxk8033g01b6bv0m-rust-1.54.0.drv': 1 dependencies couldn't be built
guix build: error: build of `/gnu/store/nv3l1lyx4x33gx80pxk8033g01b6bv0m-rust-1.54.0.drv' failed
--8<---------------cut here---------------end--------------->8---

Perhaps this failure was badly handled in the parallel build an reported
as a timeout?  I'm just guessing, but reporting it as it may be at
cause.

Thanks,

Maxim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.