GNU bug report logs - #75269
Crash of shepherd service, of GNU Artanis app, only happening in the server, not on my machine

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 75269 in the body.
You can then email your comments to 75269 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Josep Bigorra <jjbigorra <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: Crash of shepherd service, of GNU Artanis app, only happening in the
 server, not on my machine
Date: Wed, 1 Jan 2025 22:47:55 +0100

[Message part 1 (text/plain, inline)]

Hey all , happy new year 🥳

I am very happy to announce that my Guile Scheme CI/CD project,
byggsteg (https://codeberg.org/jjba23/byggsteg) on SQLite + base64 +
Artanis is a reality, and it's got an amazing performance boost, all
also thanks to your contributions, help and hinting, and my hours 😄
Byggsteg works real nicely on all my computers.

I was having a bit of problems deploying the new version to my Guix
server (https://codeberg.org/jjba23/wolk-jjba), specially due to the
addition of guile-dbd and guile-dbi-sqlite but I have resolved this. I
also managed to now configure everything properly about hostnames and
redirections of HTTP to HTTPS.

Byggsteg almost works 100% on the server, but somehow the Shepherd
service crashes and restarts when i try to start a new job.

I was suspicious of threading capabilities of my server, but all works
as expected there, also checked from the Guix REPL, and hardware is
powerful and up to date enough. So now I suspect that my database
operations somehow are making things crash.

Could you please help me figure this out, I am at a loss cause there
is no useful error being displayed 😢

I attached the logs from my server, maybe they tell you more than they tell me.

The code where things go BOOM 💣 I think? is at
lib/byggsteg/job/pipeline.scm, also attached to this thread.

Find below also the Shepherd service definition I use (in byggsteg
readme as well)

Thank you in advance!

```
  (define (wolk-jjba-byggsteg-service config)
    (list (shepherd-service (documentation "Run byggsteg as a daemon")
                            (provision '(byggsteg))
                            (requirement '())
                            (start #~(make-forkexec-constructor
'("make" "production-server")
                                      #:directory

"/var/log/byggsteg/job-clone/byggsteg/trunk"
                                      #:environment-variables
                                      (list

"GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/3.0"

"GUILE_DBD_PATH=/run/current-system/profile/lib"

"C_INCLUDE_PATH=/run/current-system/profile/include"

"GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guile/site/3.0"

"LIBRARY_PATH=/run/current-system/profile/lib"
                                       "GIT_SSL_NO_VERIFY=1"
                                       "LANG=nl_NL.UTF-8"
                                       "GUILE_AUTO_COMPILE=0"

"PWD=/var/log/byggsteg/job-clone/byggsteg/trunk"

"PATH=/run/privileged/bin:/run/current-system/profile/bin:/run/current-system/profile/sbin")))
                            (stop #~(make-kill-destructor))
                            (auto-start? #t)
                            (respawn? #t))))

  (define wolk-jjba-byggsteg-service-type
    (service-type (name 'byggsteg)
                  (description "Run byggsteg as a daemon")
                  (extensions (list (service-extension
                                     shepherd-root-service-type
                                     wolk-jjba-byggsteg-service)))
                  (default-value '())))

```

[byggsteg-crash-log.txt (text/plain, attachment)]

[pipeline.scm (text/x-scheme, attachment)]

Message #8 received at 75269 <at> debbugs.gnu.org (full text, mbox):

From: Josep Bigorra <jjbigorra <at> gmail.com>
To: 75269 <at> debbugs.gnu.org
Subject: found the problem!
Date: Thu, 2 Jan 2025 15:49:38 +0100

So it turns out running Guile code as Shepherd service with
GUILE_AUTOCOMPILE=0 makes things crash real bad.. For future
reference, never use this in prod
Please close this ticket

Message #13 received at 75269-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Josep Bigorra <jjbigorra <at> gmail.com>
Cc: 75269-done <at> debbugs.gnu.org
Subject: Re: bug#75269: Crash of shepherd service, of GNU Artanis app, only
 happening in the server, not on my machine
Date: Wed, 08 Jan 2025 09:38:43 +0100

Hola,

Josep Bigorra <jjbigorra <at> gmail.com> skribis:

> Jan  1 22:20:17 localhost shepherd[1]: make[911] ERROR: In procedure apply-smob/0: 
> Jan  1 22:20:17 localhost shepherd[1]: make[911] In procedure dbi-close: Wrong type argument in position 1: #<finalized smob 7f80244f41b0> 

For the record, this sounds like a memory management bug in guile-dbi:
‘dbi-close’ is passed a “SMOB” (a Scheme wrapper for a C object) that
has been freed in the meantime.

Closing!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.