Package: guix-patches;
Reported by: Ludovic Courtès <ludo <at> gnu.org>
Date: Fri, 26 Oct 2018 09:26:01 UTC
Severity: normal
Tags: patch
Done: ludo <at> gnu.org (Ludovic Courtès)
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 33161 in the body.
You can then email your comments to 33161 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
guix-patches <at> gnu.org:bug#33161; Package guix-patches.
(Fri, 26 Oct 2018 09:26:02 GMT) Full text and rfc822 format available.Ludovic Courtès <ludo <at> gnu.org>:guix-patches <at> gnu.org.
(Fri, 26 Oct 2018 09:26:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: guix-patches <at> gnu.org Cc: Ludovic Courtès <ludo <at> gnu.org> Subject: [PATCH 0/1] Add 'guix processes' Date: Fri, 26 Oct 2018 11:25:12 +0200
Hello Guix! This patch adds a ‘guix processes’ command that I find handy to inspect/debug Guix activity. In particular, I realized I had been doing this by hand on berlin when trying to understand what’s going on, so this tool will be helpful there. :-) Note that it’s low-tech: it works by browsing the processes in /proc and extracting the relevant info from there. Thoughts? Ludo’. Ludovic Courtès (1): Add 'guix processes'. Makefile.am | 2 + doc/guix.texi | 56 ++++++++++ guix/scripts/processes.scm | 223 +++++++++++++++++++++++++++++++++++++ po/guix/POTFILES.in | 1 + tests/processes.scm | 86 ++++++++++++++ 5 files changed, 368 insertions(+) create mode 100644 guix/scripts/processes.scm create mode 100644 tests/processes.scm -- 2.19.1
guix-patches <at> gnu.org:bug#33161; Package guix-patches.
(Fri, 26 Oct 2018 09:43:01 GMT) Full text and rfc822 format available.Message #8 received at 33161 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: 33161 <at> debbugs.gnu.org Cc: Ludovic Courtès <ludo <at> gnu.org> Subject: [PATCH 1/1] Add 'guix processes'. Date: Fri, 26 Oct 2018 11:42:22 +0200
* guix/scripts/processes.scm, tests/processes.scm: New files.
* Makefile.am (MODULES): Add the former.
(SCM_TESTS): Add the latter.
* po/guix/POTFILES.in: Add guix/scripts/processes.scm.
* doc/guix.texi (Invoking guix processes): New node.
---
Makefile.am | 2 +
doc/guix.texi | 56 ++++++++++
guix/scripts/processes.scm | 223 +++++++++++++++++++++++++++++++++++++
po/guix/POTFILES.in | 1 +
tests/processes.scm | 86 ++++++++++++++
5 files changed, 368 insertions(+)
create mode 100644 guix/scripts/processes.scm
create mode 100644 tests/processes.scm
diff --git a/Makefile.am b/Makefile.am
index 7fd29b90a8..c7171d8f5d 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -201,6 +201,7 @@ MODULES = \
guix/scripts/hash.scm \
guix/scripts/pack.scm \
guix/scripts/pull.scm \
+ guix/scripts/processes.scm \
guix/scripts/substitute.scm \
guix/scripts/authenticate.scm \
guix/scripts/refresh.scm \
@@ -343,6 +344,7 @@ SCM_TESTS = \
tests/ui.scm \
tests/status.scm \
tests/records.scm \
+ tests/processes.scm \
tests/upstream.scm \
tests/combinators.scm \
tests/discovery.scm \
diff --git a/doc/guix.texi b/doc/guix.texi
index 12346c4b8e..214e65a07e 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -193,6 +193,7 @@ Utilities
* Invoking guix copy:: Copying to and from a remote store.
* Invoking guix container:: Process isolation.
* Invoking guix weather:: Assessing substitute availability.
+* Invoking guix processes:: Listing client processes.
Invoking @command{guix build}
@@ -6051,6 +6052,7 @@ the Scheme programming interface of Guix in a convenient way.
* Invoking guix copy:: Copying to and from a remote store.
* Invoking guix container:: Process isolation.
* Invoking guix weather:: Assessing substitute availability.
+* Invoking guix processes:: Listing client processes.
@end menu
@node Invoking guix build
@@ -8751,6 +8753,60 @@ with the @code{-m} option of @command{guix package} (@pxref{Invoking
guix package}).
@end table
+@node Invoking guix processes
+@section Invoking @command{guix processes}
+
+The @command{guix processes} command can be useful to developers and system
+administrators, especially on multi-user machines and on build farms: it lists
+the current Guix sessions (connections to the daemon), as well as information
+about the processes involved. Here's an example of the information it
+returns:
+
+@example
+$ sudo guix processes
+SessionPID: 19002
+ClientPID: 19090
+ClientCommand: guix environment --ad-hoc python
+
+SessionPID: 19402
+ClientPID: 19367
+ClientCommand: guix publish -u guix-publish -p 3000 -C 9 @dots{}
+
+SessionPID: 19444
+ClientPID: 19419
+ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
+LockHeld: /gnu/store/@dots{}-perl-ipc-cmd-0.96.lock
+LockHeld: /gnu/store/@dots{}-python-six-bootstrap-1.11.0.lock
+LockHeld: /gnu/store/@dots{}-libjpeg-turbo-2.0.0.lock
+ChildProcess: 20495: guix offload x86_64-linux 7200 1 28800
+ChildProcess: 27733: guix offload x86_64-linux 7200 1 28800
+ChildProcess: 27793: guix offload x86_64-linux 7200 1 28800
+@end example
+
+In this example we see that @command{guix-daemon} has three clients:
+@command{guix environment}, @command{guix publish}, and the Cuirass continuous
+integration tool; their process identifier (PID) is given by the
+@code{ClientPID} field. The @code{SessionPID} field gives the PID of the
+@command{guix-daemon} sub-process of this particular session.
+
+The @code{LockHeld} fields show which store items are currently locked by this
+session, which corresponds to store items being built or substituted (the
+@code{LockHeld} field is not displayed when @command{guix processes} is not
+running as root.) Last, by looking at the @code{ChildProcess} field, we
+understand that these three builds are being offloaded (@pxref{Daemon Offload
+Setup}).
+
+The output is in Recutils format so we can use the handy @command{recsel}
+command to select sessions of interest (@pxref{Selection Expressions,,,
+recutils, GNU recutils manual}). As an example, the command shows the command
+line and PID of the client that triggered the build of a Perl package:
+
+@example
+$ sudo guix processes | \
+ recsel -p ClientPID,ClientCommand -e 'LockHeld ~ "perl"'
+ClientPID: 19419
+ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
+@end example
@c *********************************************************************
@node GNU Distribution
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
new file mode 100644
index 0000000000..6a2f603599
--- /dev/null
+++ b/guix/scripts/processes.scm
@@ -0,0 +1,223 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <ludo <at> gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix scripts processes)
+ #:use-module ((guix store) #:select (%store-prefix))
+ #:use-module (guix scripts)
+ #:use-module (guix ui)
+ #:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-9)
+ #:use-module (srfi srfi-9 gnu)
+ #:use-module (srfi srfi-37)
+ #:use-module (ice-9 ftw)
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 rdelim)
+ #:use-module (ice-9 format)
+ #:export (process?
+ process-id
+ process-parent-id
+ process-command
+ processes
+
+ daemon-session?
+ daemon-session-process
+ daemon-session-client
+ daemon-session-children
+ daemon-session-locks-held
+ daemon-sessions
+
+ guix-processes))
+
+;; Process as can be found in /proc on GNU/Linux.
+(define-record-type <process>
+ (process id parent command)
+ process?
+ (id process-id) ;integer
+ (parent process-parent-id) ;integer | #f
+ (command process-command)) ;list of strings
+
+(define (write-process process port)
+ (format port "#<process ~a>" (process-id process)))
+
+(set-record-type-printer! <process> write-process)
+
+(define (read-status-ppid port)
+ "Read the PPID from PORT, an input port on a /proc/PID/status file. Return
+#f for PID 1 and kernel pseudo-processes."
+ (let loop ()
+ (match (read-line port)
+ ((? eof-object?) #f)
+ (line
+ (if (string-prefix? "PPid:" line)
+ (string->number (string-trim-both (string-drop line 5)))
+ (loop))))))
+
+(define %not-nul
+ (char-set-complement (char-set #\nul)))
+
+(define (read-command-line port)
+ "Read the zero-split command line from PORT, a /proc/PID/cmdline file, and
+return it as a list."
+ (string-tokenize (read-string port) %not-nul))
+
+(define (processes)
+ "Return a list of process records representing the currently alive
+processes."
+ ;; This assumes a Linux-compatible /proc file system. There exists one for
+ ;; GNU/Hurd.
+ (filter-map (lambda (pid)
+ ;; There's a TOCTTOU race here. If we get ENOENT, simply
+ ;; ignore PID.
+ (catch 'system-error
+ (lambda ()
+ (define ppid
+ (call-with-input-file (string-append "/proc/" pid "/status")
+ read-status-ppid))
+ (define command
+ (call-with-input-file (string-append "/proc/" pid "/cmdline")
+ read-command-line))
+ (process (string->number pid) ppid command))
+ (lambda args
+ (if (= ENOENT (system-error-errno args))
+ #f
+ (apply throw args)))))
+ (scandir "/proc" string->number)))
+
+(define (process-open-files process)
+ "Return the list of files currently open by PROCESS."
+ (let ((directory (string-append "/proc/"
+ (number->string (process-id process))
+ "/fd")))
+ (map (lambda (fd)
+ (readlink (string-append directory "/" fd)))
+ (or (scandir directory string->number) '()))))
+
+;; Daemon session.
+(define-record-type <daemon-session>
+ (daemon-session process client children locks)
+ daemon-session?
+ (process daemon-session-process) ;<process>
+ (client daemon-session-client) ;<process>
+ (children daemon-session-children) ;list of <process>
+ (locks daemon-session-locks-held)) ;list of strings
+
+(define (daemon-sessions)
+ "Return two values: the list of <daemon-session> denoting the currently
+active sessions, and the master 'guix-daemon' process."
+ (define (lock-file? file)
+ (and (string-prefix? (%store-prefix) file)
+ (string-suffix? ".lock" file)))
+
+ (let* ((processes (processes))
+ (daemons (filter (lambda (process)
+ (match (process-command process)
+ ((argv0 _ ...)
+ (string=? (basename argv0) "guix-daemon"))
+ (_ #f)))
+ processes))
+ (children (filter (lambda (process)
+ (match (process-command process)
+ ((argv0 (= string->number argv1) _ ...)
+ (integer? argv1))
+ (_ #f)))
+ daemons))
+ (master (remove (lambda (process)
+ (memq process children))
+ daemons)))
+ (define (lookup-process pid)
+ (find (lambda (process)
+ (and (process-id process)
+ (= pid (process-id process))))
+ processes))
+
+ (define (lookup-children pid)
+ (filter (lambda (process)
+ (and (process-parent-id process)
+ (= pid (process-parent-id process))))
+ processes))
+
+ (values (map (lambda (process)
+ (match (process-command process)
+ ((argv0 (= string->number client) _ ...)
+ (let ((files (process-open-files process)))
+ (daemon-session process
+ (lookup-process client)
+ (lookup-children (process-id process))
+ (filter lock-file? files))))))
+ children)
+ master)))
+
+(define (daemon-session->recutils session port)
+ "Display SESSION information in recutils format on PORT."
+ (format port "SessionPID: ~a~%"
+ (process-id (daemon-session-process session)))
+ (format port "ClientPID: ~a~%"
+ (process-id (daemon-session-client session)))
+ (format port "ClientCommand:~{ ~a~}~%"
+ (process-command (daemon-session-client session)))
+ (for-each (lambda (lock)
+ (format port "LockHeld: ~a~%" lock))
+ (daemon-session-locks-held session))
+ (for-each (lambda (process)
+ (format port "ChildProcess: ~a:~{ ~a~}~%"
+ (process-id process)
+ (process-command process)))
+ (daemon-session-children session)))
+
+
+;;;
+;;; Options.
+;;;
+
+(define %options
+ (list (option '(#\h "help") #f #f
+ (lambda args
+ (show-help)
+ (exit 0)))
+ (option '(#\V "version") #f #f
+ (lambda args
+ (show-version-and-exit "guix processes")))))
+
+(define (show-help)
+ (display (G_ "Usage: guix processes
+List the current Guix sessions and their processes."))
+ (newline)
+ (display (G_ "
+ -h, --help display this help and exit"))
+ (display (G_ "
+ -V, --version display version information and exit"))
+ (newline)
+ (show-bug-report-information))
+
+
+;;;
+;;; Entry point.
+;;;
+
+(define (guix-processes . args)
+ (define options
+ (args-fold* args %options
+ (lambda (opt name arg result)
+ (leave (G_ "~A: unrecognized option~%") name))
+ cons
+ '()))
+
+ (for-each (lambda (session)
+ (daemon-session->recutils session (current-output-port))
+ (newline))
+ (daemon-sessions)))
diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.in
index 2e37a19407..74c223b283 100644
--- a/po/guix/POTFILES.in
+++ b/po/guix/POTFILES.in
@@ -32,6 +32,7 @@ guix/scripts/copy.scm
guix/scripts/pack.scm
guix/scripts/weather.scm
guix/scripts/describe.scm
+guix/scripts/processes.scm
guix/gnu-maintenance.scm
guix/scripts/container.scm
guix/scripts/container/exec.scm
diff --git a/tests/processes.scm b/tests/processes.scm
new file mode 100644
index 0000000000..40454bcbc7
--- /dev/null
+++ b/tests/processes.scm
@@ -0,0 +1,86 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <ludo <at> gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (test-processes)
+ #:use-module (guix scripts processes)
+ #:use-module (guix store)
+ #:use-module (guix derivations)
+ #:use-module (guix packages)
+ #:use-module (guix gexp)
+ #:use-module ((guix utils) #:select (call-with-temporary-directory))
+ #:use-module (gnu packages bootstrap)
+ #:use-module (guix tests)
+ #:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-64)
+ #:use-module (rnrs bytevectors)
+ #:use-module (rnrs io ports)
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 threads))
+
+(test-begin "processes")
+
+(test-assert "not a client"
+ (not (find (lambda (session)
+ (= (getpid)
+ (process-id (daemon-session-client session))))
+ (daemon-sessions))))
+
+(test-assert "client"
+ (with-store store
+ (let* ((session (find (lambda (session)
+ (= (getpid)
+ (process-id (daemon-session-client session))))
+ (daemon-sessions)))
+ (daemon (daemon-session-process session)))
+ (and (kill (process-id daemon) 0)
+ (string-suffix? "guix-daemon" (first (process-command daemon)))))))
+
+(test-assert "client + lock"
+ (with-store store
+ (call-with-temporary-directory
+ (lambda (directory)
+ (let* ((token1 (string-append directory "/token1"))
+ (token2 (string-append directory "/token2"))
+ (exp #~(begin #$(random-text)
+ (mkdir #$token1)
+ (let loop ()
+ (unless (file-exists? #$token2)
+ (sleep 1)
+ (loop)))
+ (mkdir #$output)))
+ (guile (package-derivation store %bootstrap-guile))
+ (drv (run-with-store store
+ (gexp->derivation "foo" exp
+ #:guile-for-build guile)))
+ (thread (call-with-new-thread
+ (lambda ()
+ (build-derivations store (list drv)))))
+ (_ (let loop ()
+ (unless (file-exists? token1)
+ (usleep 200)
+ (loop))))
+ (session (find (lambda (session)
+ (= (getpid)
+ (process-id (daemon-session-client session))))
+ (daemon-sessions)))
+ (locks (daemon-session-locks-held (pk 'session session))))
+ (call-with-output-file token2 (const #t))
+ (equal? (list (string-append (derivation->output-path drv) ".lock"))
+ locks))))))
+
+(test-end "processes")
--
2.19.1
ludo <at> gnu.org (Ludovic Courtès):Ludovic Courtès <ludo <at> gnu.org>:Message #13 received at 33161-done <at> debbugs.gnu.org (full text, mbox):
From: ludo <at> gnu.org (Ludovic Courtès) To: 33161-done <at> debbugs.gnu.org Subject: Re: [bug#33161] [PATCH 0/1] Add 'guix processes' Date: Mon, 29 Oct 2018 00:15:09 +0100
Ludovic Courtès <ludo <at> gnu.org> skribis: > This patch adds a ‘guix processes’ command that I find handy to > inspect/debug Guix activity. In particular, I realized I had been > doing this by hand on berlin when trying to understand what’s going > on, so this tool will be helpful there. :-) Pushed as 63eb2b899be7ac857454442e09d0ebd23fe4f871 with small improvements to the doc! Ludo’.
Debbugs Internal Request <help-debbugs <at> gnu.org>
to internal_control <at> debbugs.gnu.org.
(Mon, 26 Nov 2018 12:24:07 GMT) Full text and rfc822 format available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.