GNU bug report logs - #39021
[PATCH] Add Keybase

Previous Next

To reply to this bug, email your comments to 39021 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jakub Kądziołka <kuba <at> kadziolka.net>
To: guix-patches <at> gnu.org
Subject: [PATCH] Add Keybase
Date: Tue, 7 Jan 2020 21:00:39 +0100

[Message part 1 (text/plain, inline)]

Hi Guix,

please find attached a set of patches that adds the non-GUI parts of
Keybase (the GUI is an Electron application - as far as I am aware,
there are no similar packages in Guix, so packaging the GUI requires a
significant amount of further work).

Unresolved question: in other distributions, the package provides user
units for systemd. Should the Guix package provide Shepherd services to
be ran as the user? If so, are there any other examples I could look at
and draw inspiration from? Also, I haven't been able to find any
documentation on how to set up user shepherd, apart from a passing
mention of that being possible in the documentation. I would appreciate
any pointers on the topic.

Regards,
Jakub Kądziołka

[0001-build-system-go-Allow-providing-additional-build-fla.patch (text/x-diff, attachment)]

[0002-gnu-Add-keybase.patch (text/x-diff, attachment)]

Message #8 received at 39021 <at> debbugs.gnu.org (full text, mbox):

From: Jakub Kądziołka <kuba <at> kadziolka.net>
To: 39021 <at> debbugs.gnu.org
Cc: me <at> tobias.gr
Subject: go package rebuilds
Date: Fri, 24 Jan 2020 19:34:28 +0100

With regards to the modification of go-build-system in this patchstack,
it seems that this would cause "only" <180 packages to be rebuilt:

~/guix$ rg -w go-build-system gnu/packages | wc -l
172
~/guix$ guix refresh --list-dependent go
Building the following 52 packages would ensure 176 dependent packages
are rebuilt: [...]

This has been discussed on IRC, and since some time has since passed, I
want to put the relevant link here, for future reference:
http://logs.guix.gnu.org/guix/2020-01-19.log#215122

Regards,
Jakub Kądziołka

Message #11 received at 39021 <at> debbugs.gnu.org (full text, mbox):

From: Jakub Kądziołka <kuba <at> kadziolka.net>
To: 39021 <at> debbugs.gnu.org
Subject: [PATCH 1/2 v2] build-system/go: Allow providing additional build flags
Date: Tue, 28 Jan 2020 22:54:05 +0100

* guix/build-system/go.scm (build-flags): New argument.
* guix/build/go-build-system.scm (build): Use apply to pass the
  additional arguments to invoke.
---
I have discovered a bug when testing v1 of this patch locally - some Go
packages have been failing to build due to a quoting mishap. This is now
fixed.

Diff between patches:
-                   (build-flags '())
+                   (build-flags ''())
[this note should be automatically removed by git am]
---
 guix/build-system/go.scm       | 3 +++
 guix/build/go-build-system.scm | 7 ++++---
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/guix/build-system/go.scm b/guix/build-system/go.scm
index 1b916af8f9..f8ebaefb27 100644
--- a/guix/build-system/go.scm
+++ b/guix/build-system/go.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 Petter <petter <at> mykolab.ch>
 ;;; Copyright © 2017 Leo Famulari <leo <at> famulari.name>
+;;; Copyright © 2020 Jakub Kądziołka <kuba <at> kadziolka.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -82,6 +83,7 @@
                    (install-source? #t)
                    (import-path "")
                    (unpack-path "")
+                   (build-flags ''())
                    (tests? #t)
                    (allow-go-reference? #f)
                    (system (%current-system))
@@ -109,6 +111,7 @@
                 #:install-source? ,install-source?
                 #:import-path ,import-path
                 #:unpack-path ,unpack-path
+                #:build-flags ,build-flags
                 #:tests? ,tests?
                 #:allow-go-reference? ,allow-go-reference?
                 #:inputs %build-inputs)))
diff --git a/guix/build/go-build-system.scm b/guix/build/go-build-system.scm
index 4bc0156a88..49887848f6 100644
--- a/guix/build/go-build-system.scm
+++ b/guix/build/go-build-system.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2016 Petter <petter <at> mykolab.ch>
 ;;; Copyright © 2017, 2019 Leo Famulari <leo <at> famulari.name>
 ;;; Copyright © 2019 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
+;;; Copyright © 2020 Jakub Kądziołka <kuba <at> kadziolka.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -209,18 +210,18 @@ unpacking."
                 (_ #f))
               inputs))))
 
-(define* (build #:key import-path #:allow-other-keys)
+(define* (build #:key import-path build-flags #:allow-other-keys)
   "Build the package named by IMPORT-PATH."
   (with-throw-handler
     #t
     (lambda _
-      (invoke "go" "install"
+      (apply invoke "go" "install"
               "-v" ; print the name of packages as they are compiled
               "-x" ; print each command as it is invoked
               ;; Respectively, strip the symbol table and debug
               ;; information, and the DWARF symbol table.
               "-ldflags=-s -w"
-              import-path))
+              `(,@build-flags ,import-path)))
     (lambda (key . args)
       (display (string-append "Building '" import-path "' failed.\n"
                               "Here are the results of `go env`:\n"))
-- 
2.25.0

Message #14 received at 39021 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Jakub Kądziołka <kuba <at> kadziolka.net>
Cc: 39021 <at> debbugs.gnu.org
Subject: Re: [bug#39021] [PATCH] Add Keybase
Date: Fri, 7 Feb 2020 19:20:20 -0500

On Tue, Jan 07, 2020 at 09:00:39PM +0100, Jakub Kądziołka wrote:
> please find attached a set of patches that adds the non-GUI parts of
> Keybase (the GUI is an Electron application - as far as I am aware,
> there are no similar packages in Guix, so packaging the GUI requires a
> significant amount of further work).

I don't *think* we have any Electron packages, but we do have Chromium
(and qtwebengine) and Node.js, so I hope it's possible.

> Unresolved question: in other distributions, the package provides user
> units for systemd. Should the Guix package provide Shepherd services to
> be ran as the user? If so, are there any other examples I could look at
> and draw inspiration from? Also, I haven't been able to find any
> documentation on how to set up user shepherd, apart from a passing
> mention of that being possible in the documentation. I would appreciate
> any pointers on the topic.

We don't really do this kind of thing — adding features or service
manager files to upstream packages.

As for user shepherd, there's an example here:

https://git.dthompson.us/dotfiles.git/blob/HEAD:/dotfiles/.config/shepherd/init.scm

There's a lot going on there but the important parts are
(register-services ...), (make <service> ...), (action ...), and
(for-each start ...)

Then you just invoke `shepherd`.

> From 0d744787c48c7184a70dda9fc1bb2c0d334fc080 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Jakub=20K=C4=85dzio=C5=82ka?= <kuba <at> kadziolka.net>
> Date: Sun, 5 Jan 2020 17:13:20 +0100
> Subject: [PATCH 1/2] build-system/go: Allow providing additional build flags
> 
> * guix/build-system/go.scm (build-flags): New argument.
> * guix/build/go-build-system.scm (build): Use apply to pass the
>   additional arguments to invoke.

Nice, LGTM!

> From 3de233a2d8e6bdb4723844337b69b6612616c9c5 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Jakub=20K=C4=85dzio=C5=82ka?= <kuba <at> kadziolka.net>
> Date: Tue, 7 Jan 2020 20:29:21 +0100
> Subject: [PATCH 2/2] gnu: Add keybase.
> 
> * gnu/packages/crypto.scm
>   (keybase-component): New function.
>   (keybase, git-remote-keybase, kbfs): New variables.

This is enough of it's own thing that we can make a new (gnu packages
keybase) module.

> +(define* (keybase-component #:key name repo-path synopsis description)

We avoid abbreviations, so maybe "repository-path"? Bonus points if we
can make it more descriptive.

Can you take a look at the bundled ("vendored") dependencies:

https://github.com/keybase/client/tree/master/go/vendor

We strive to avoid using these, but sometimes we do, as in the Docker
package. It's not really idiomatic to unbundle things in Go. But we need
to at least make sure all the bundled dependencies are freely licensed.

Also, please run `guix lint` on these packages and make sure the
descriptions are written in complete sentences.

Message #17 received at 39021 <at> debbugs.gnu.org (full text, mbox):

From: Jakub Kądziołka <kuba <at> kadziolka.net>
To: Leo Famulari <leo <at> famulari.name>
Cc: 39021 <at> debbugs.gnu.org
Subject: Re: [bug#39021] [PATCH] Add Keybase
Date: Tue, 11 Feb 2020 17:36:54 +0100

[Message part 1 (text/plain, inline)]

> > From 3de233a2d8e6bdb4723844337b69b6612616c9c5 Mon Sep 17 00:00:00 2001
> > From: =?UTF-8?q?Jakub=20K=C4=85dzio=C5=82ka?= <kuba <at> kadziolka.net>
> > Date: Tue, 7 Jan 2020 20:29:21 +0100
> > Subject: [PATCH 2/2] gnu: Add keybase.
> > 
> > * gnu/packages/crypto.scm
> >   (keybase-component): New function.
> >   (keybase, git-remote-keybase, kbfs): New variables.
> 
> This is enough of it's own thing that we can make a new (gnu packages
> keybase) module.

Sure, will do.

> > +(define* (keybase-component #:key name repo-path synopsis description)
> 
> We avoid abbreviations, so maybe "repository-path"? Bonus points if we
> can make it more descriptive.

I can't think of anything more descriptive, as it's literally the path
in the repository the component is at.

> Can you take a look at the bundled ("vendored") dependencies:
> 
> https://github.com/keybase/client/tree/master/go/vendor
> 
> We strive to avoid using these, but sometimes we do, as in the Docker
> package. It's not really idiomatic to unbundle things in Go. But we need
> to at least make sure all the bundled dependencies are freely licensed.

Apart from licensing concerns, what are the arguments for splitting this
into separate packages? I feel like this is just busywork...

> Also, please run `guix lint` on these packages and make sure the
> descriptions are written in complete sentences.

Ah, sure, somehow I forgot to do this before.

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 39021 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Jakub Kądziołka <kuba <at> kadziolka.net>
Cc: 39021 <at> debbugs.gnu.org
Subject: Re: [bug#39021] [PATCH] Add Keybase
Date: Tue, 11 Feb 2020 12:36:34 -0500

On Tue, Feb 11, 2020 at 05:36:54PM +0100, Jakub Kądziołka wrote:
> > We strive to avoid using these, but sometimes we do, as in the Docker
> > package. It's not really idiomatic to unbundle things in Go. But we need
> > to at least make sure all the bundled dependencies are freely licensed.
> 
> Apart from licensing concerns, what are the arguments for splitting this
> into separate packages? I feel like this is just busywork...

The question of licensing is unrelated to bundling, sorry if that wasn't
clear. The only thing you have to do here is make sure they are all
freely licensed.

To clarify, those bundled dependencies *are* separate packages,
developed by different organizations.

It's the standard in Guix (and every major GNU/Linux distro) to not
allow bundled dependencies because they make the graph of software
basically uninspectable and unmaintainable using the distro's normal
tools, as well as having the potential to waste time and space building
multiple versions of a package if it is bundled in more than one place
or already present as its own package. It negates all the advantages of
creating a distrubtion, especially for Go binaries, which can be
trivially deployed on any system, including Guix, without any extra
work.

But like I said, it's normal to bundle things in Go land, where there is
really no principled concept of dependency management or versioned
releases, and as time goes by changes to the Go compiler make it harder
and harder to unbundle. I did do it for Syncthing and I can confirm it
was a lot of work for no clear benefit. Excepting the standard library,
Go libraries do not even get security updates because nobody is looking
closely at them.

Message #23 received at 39021 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Leo Famulari <leo <at> famulari.name>
Cc: Jakub Kądziołka <kuba <at> kadziolka.net>,
 39021 <at> debbugs.gnu.org
Subject: Re: [bug#39021] [PATCH] Add Keybase
Date: Sun, 31 May 2020 10:09:38 +0300

[Message part 1 (text/plain, inline)]

I have a working version of keybase, although I haven't gotten around to
unbundling all the go dependencies yet. I figured I'd post my WIP stuff
here.


-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[keybase.scm (text/plain, attachment)]

[keybase.scm (text/plain, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #26 received at 39021 <at> debbugs.gnu.org (full text, mbox):

From: Jakub Kądziołka <kuba <at> kadziolka.net>
To: Efraim Flashner <efraim <at> flashner.co.il>
Cc: 39021 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: [bug#39021] [PATCH] Add Keybase
Date: Sun, 31 May 2020 23:47:38 +0200

[Message part 1 (text/plain, inline)]

On Sun, May 31, 2020 at 10:09:38AM +0300, Efraim Flashner wrote:
> I have a working version of keybase, although I haven't gotten around to
> unbundling all the go dependencies yet. I figured I'd post my WIP stuff
> here.

Hi Efraim,

thanks for your message! I like how you consolidated the binaries into a
single package with a custom phase. I was going to ask whether you got
the GUI working, but after taking another look at the unbundling part,
it's clear that this is still not there.

When it comes to my own efforts for packaging keybase, I have
categorized all the vendored dependencies according to their licenses.
I have found 3 libraries without a license. For 2 of them, I prodded the
authors a bit and they now include a license. However, I haven't
received a response on github.com/keybase/golang-ico (note that despite
the /keybase/ in the URL, this is just a fork, the original author has
no relation to keybase).

I am considering learning enough Go to replace that library with a
freely-licensed alternative and submitting a PR upstream, however it is
quite low on my TODOs.

Regards,
Jakub Kądziołka

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.