Package: guix-patches;
Reported by: Ludovic Courtès <ludo <at> gnu.org>
Date: Sun, 23 Feb 2025 15:21:02 UTC
Severity: normal
Done: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: guix-patches <at> gnu.org Subject: [GCD] Migrating repositories, issues, and patches to Codeberg Date: Sun, 23 Feb 2025 16:20:34 +0100
[Message part 1 (text/plain, inline)]
Hello Guix! This is the formal submission of “Migrating repositories, issues, and patches to Codeberg” (GCD 002), a preliminary draft of which I posted before the Guix Days⁰. In accordance with the GCD Process, discussion will end on April 23rd at the latest. I would like to remind everyone that you can try out Codeberg either by contributing to one of the Guix-Science repositories¹, or by reviewing or making a pull request for a trivial packaging change (and nothing more!) in my Guix clone at Codeberg: https://lists.gnu.org/archive/html/guix-devel/2025-02/msg00313.html Please do try it especially if you feel reluctant or wonder what the workflow would be like. Ludo’. ⁰ https://lists.gnu.org/archive/html/guix-devel/2025-01/msg00218.html ¹ https://codeberg.org/guix-science
[002-codeberg.md (text/plain, inline)]
title: Migrating repositories, issues, and patches to Codeberg
id: 002
status: submitted
discussion: https://issues.guix.gnu.org/<number assigned by issue tracker>
authors: Ludovic Courtès
sponsors: Tobias Geerinckx-Rice, Ricardo Wurmus
date-submitted: 2025-02-23
date: 2025-02-23
SPDX-License-Identifier: CC-BY-SA-4.0 OR GFDL-1.3-no-invariants-or-later
---
# Summary
The contribution workflow in Guix has been facing several challenges:
difficult onboarding, lack of legibility, complex, unreliable, and
labor-intensive infrastructure, and lack of automation. All these lead
to an experience that contributors often find frustrating and hinders
quality assurance efforts. We propose to address these limitations by
migrating repositories, issue tracking, and patch tracking to Codeberg,
a “modern” forge hosted by a non-profit.
# Motivation
To keep track of bug reports and patches, Guix historically chose tools
that were *simple* in their design:
- bug reports and patches can be sent by plain email, without having
to create an account or even subscribe to a mailing list;
- discussion and patch review happen naturally by email, without
requiring special tools;
- the Debbugs instance at https://bugs.gnu.org keeps track of bug
reports and patches by assigning them an identifier and creating a
mailing list specifically for each bug or patch.
However, to overcome several limitations, the project developed
processes and tools, which can be characterized as *incidental
complexity*:
- because the Debbugs web interface is crude by today’s standards and
hard to search and navigate, the project developed
[mumi](https://git.savannah.gnu.org/cgit/guix/mumi.git/), the web
interface running at https://issues.guix.gnu.org;
- to navigate bugs and patches more conveniently than what an email
client supports, contributors were
[encouraged](https://guix.gnu.org/manual/devel/en/html_node/Debbugs-User-Interfaces.html)
to use interfaces like `debbugs.el` or `b4`;
- sending patch series by email does not play well with Debbugs’
automatic identifier assignment, so [contributors were told to send
their “cover letter”, wait for an identifier to be assigned, and
then send the
rest](https://guix.gnu.org/manual/devel/en/html_node/Sending-a-Patch-Series.html#Multiple-Patches-1);
- to help sending and applying patch series, mumi was extended to
provide a command line interface;
- to build patch series submitted by email, the [QA
service](https://qa.guix.gnu.org) has to rely on a [Patchwork
instance](https://patches.guix-patches.cbaines.net/project/guix-patches/list/)
that is subscribed to the `guix-patches` mailing list, coupled with
its own [parsing of incoming
email](https://git.savannah.gnu.org/gitweb/?p=guix/data-service.git;a=blob;f=guix-data-service/branch-updated-emails.scm;h=aeb1570dfda725864a77780d0541f26c090b0e55;hb=c886685e9284da4bbed9377f70dd70da9e7ca29f);
- the project added a commit hook to create add unique `Change-Id`
headers in commit messages in an attempt to correlate commits in the
repository with messages send to `guix-patches`; none of the
existing tools takes advantage of it though, and it is up to
contributors to manually close entries in the bug/patch tracker once
they have been fixed/applied.
Developing and maintaining this software and infrastructure is
time-consuming. Worse, it leaves contributors largely dissatisfied for
a variety of reasons:
- the process is unfamiliar to most newcomers;
- the tools and infrastructure in Guix have become a maze;
- apart from the happy few using `debbugs.el` in Emacs, navigating
open issues and patches is hard; filtering incoming messages is
equally hard, even for those with 10+ years of experience with
advanced email tools (Gnus, mu4e, notmuch, b4, etc.);
- because the various parts of the development process (repository,
issue tracking, QA automation, `etc/teams.scm`) are largely
disconnected, even long-time contributors can hardly follow issues
relevant to them; issues may remain open after they’ve been fixed,
new activity on an issue may go unnoticed, cross-references among
issues are not visible in any of the interfaces, etc.
All this contributes to a [poor
experience](https://guix.gnu.org/en/blog/2025/guix-user-and-contributor-survey-2024-the-results-part-3/)
for those who choose to contribute despite the barrier to entry,
probably discourages many to even start contributing, and adds to the
load of committers and infrastructure maintainers.
# Detailed Design
This section explains the chosen solution among the available options,
the scope of the proposed migration, a migration path, and an outlook on
automation.
## Choice of a Forge
We set out to choose a “modern forge” that supports a pull-request style
workflow and provides good integration between the repository, the issue
tracker, and the merge request tracker. Such a system is necessarily
more *complex* at first glance than the email-based tools we have but
(1) the increase in complexity is reasonable once we consider the
incidental complexity of the existing services, as mentioned above, and
(2) we think the added usage benefits outweigh this increase in
complexity.
The software behind the forge has to be free software that is
*plausibly* self-hosted on Guix System—this probably rules out GitLab
Community Edition and makes [Forgejo](https://forgejo.org/) the main
contender.
[SourceHut](https://sourcehut.org/), the other interesting option, does
not offer the same convenience when it comes to dealing with patches and
runs the risk of reproducing onboarding and integration issues
surrounding an email-based workflow and “read-only” web interface that
Guix is already experiencing.
Forgejo has several features to support collaboration among a large
number of people and on a large code base, including
[teams](https://docs.codeberg.org/collaborating/create-organization/#teams)
and [issue and pull request
templates](https://forgejo.org/docs/latest/user/issue-pull-request-templates/).
Support for
[federation](https://forgejo.org/2023-01-10-answering-forgejo-federation-questions/)
is also under development and is a promising way to avoid
centralization.
Instead of self-hosting, this GCD suggests using the Forgejo instance on
codeberg.org, run by the [Codeberg e.V.](https://codeberg.org/about)
non-profit, registered in Germany. The non-profit has a good track
record of running codeberg.org with minimal downtime, is [committed to
supporting free software
development](https://codeberg.org/Codeberg/org/src/branch/main/en/bylaws.md#preamble),
[transparent](https://codeberg.org/Codeberg/org), and has governance set
up to achieve its mission.
The Guix-Science umbrella project [has been using Codeberg for several
months
now](https://hpc.guix.info/blog/2025/01/join-the-guix-science-community/),
which has allowed us to gain confidence in its suitability for a project
like Guix.
## Rights and Privileges
Migration should preserve rights and privileges regarding access to the
repositories. To that end, we propose the following rules:
- Committers to several of the repositories listed above and [Savannah
“group admins”](https://savannah.gnu.org/projects/guix) can request
membership in the [“Owners”
team](https://docs.codeberg.org/collaborating/create-organization/#teams)
of the [Guix *organization*](https://codeberg.org/guix). As of this
writing, only three people are members.
- Anyone listed the `.guix-authorizations` file of Guix can request
membership of the https://codeberg.org/guix/guix once it is created.
- Committers to one of the other repositories can request membership
of that repository.
In the future, we should extend the [“Commit
Rights”](https://guix.gnu.org/manual/devel/en/html_node/Commit-Access.html)
section of the manual to clarify the distinction between being a member
of the organization and being a member of a specific repository, in a
specific team.
## Repository Migration Path
The Guix project at Savannah contains the following repositories:
- [Guix itself](https://git.savannah.gnu.org/git/guix.git);
- [the bootstrappable.org web
site](https://git.savannah.gnu.org/git/guix/bootstrappable.git);
- [the DHCP client in
Guile](https://git.savannah.gnu.org/git/guix/dhcp.git) (forgotten
2015 Google Summer of Code project);
- [Guile bindings to
GNUnet](https://git.savannah.gnu.org/git/guix/gnunet.git) (forgotten
2015 Google Summer of Code project);
- [Guix artwork and web
site](https://git.savannah.gnu.org/git/guix/guix-artwork.git);
- [Cuirass](https://git.savannah.gnu.org/git/guix/guix-cuirass.git);
- [“maintenance”
repository](https://git.savannah.gnu.org/git/guix/maintenance.git)
(includes Guix System infrastructure configuration, talks, and other
documents);
- [scripts for videos presenting
Guix](https://git.savannah.gnu.org/git/guix/videos.git);
- [Guix Data
Service](https://git.savannah.gnu.org/git/guix/data-service.git);
- [Emacs-Guix](https://git.savannah.gnu.org/git/guix/emacs-guix.git);
- [Guix Build
Coordinator](https://git.savannah.gnu.org/git/guix/build-coordinator.git);
- [nar-herder](https://git.savannah.gnu.org/git/guix/nar-herder.git);
- [QA
Frontpage](https://git.savannah.gnu.org/git/guix/qa-frontpage.git);
- [mumi](https://git.savannah.gnu.org/cgit/guix/mumi.git);
- [Guix Consensus
Documents](https://git.savannah.gnu.org/git/guix/guix-consensus-documents.git).
Within **30 days** following acceptance of this GCD, committers would
migrate all these repositories to https://codeberg.org/guix.
For Guix itself, we would decide on a **flag day** 14 days after
acceptance of this GCD at the earliest, and 30 days at the latest. On
that day, the official URL of the Guix repository would become
https://codeberg.org/guix/guix.git. A commit would reflect that by
updating:
1. the `url` field in `.guix-channel`;
2. the `%default-channel-url` variable in `(guix channels)`;
3. any other reference to the URL that may appear in the repository,
in particular in the manual.
To ease this migration and possibly future migration, we may add a new
`git.guix.gnu.org` DNS entry with HTTP redirects to
`git.savannah.gnu.org` (before migration) and `codeberg.org` (after
migration); a [patch](https://issues.guix.gnu.org/76296) implementing
this has been submitted. The `%default-channel-url` variable would
refer to `https://git.guix.gnu.org/guix.git`.
Following this commit, an entry in `etc/news.scm` would explain the
migration. See [this entry in
Guix-Science](https://codeberg.org/guix-science/guix-science/commit/fd1b2dacd8d37c9d1939f9dc5a5b74256171ccbd)
for an example.
The Savannah `guix.git` repository would become a mirror of the one at
Codeberg, with a script periodically updating it for **at least one
year** after the switch, as a way to ease migration to the new
repository for users. Other repositories would be deleted from Savannah
once migrated, to avoid confusion.
## Issue Tracker Migration Path
Importing all the issues and patches from Debbugs/mumi into Codeberg
would be impractical: it would require the development of specific
tools, would be a lossy process due to the fundamental mismatch between
plain text email threads and Forgejo issues and pull requests, and would
bring little in return.
Our proposal is the following:
- https://issues.guix.gnu.org will remain up and running for at least
**two years** following acceptance of this GCD. Note that once
issues.guix.gnu.org is down, issues will remain visible at
https://bugs.gnu.org and email archives will remain visible at
https://mail.gnu.org.
- Within **30 days** after acceptance of this GCD, mailing list
administrators will set up the `bug-guix` and `guix-patches` mailing
lists in “Emergency Moderation” mode in the Mailman
interface—meaning that messages will not get through anymore. It
will still be possible to interact on individual issues via
`NNN <at> debbugs.gnu.org`.
- The switchover will be advertised before it takes place with a post
to `info-guix <at> gnu.org`, to `guix-devel <at> gnu.org`, as well as through
a blog post.
- The “Contributing” section of the manual will be updated accordingly
at that time.
## User Interfaces
For many contributors, a strength of the email-based workflow is that it
works out of the browser, possibly offline; we want to preserve that
comfort as much as possible.
Everything that can be done through Forgejo’s web interface can be done
*via* its [HTTP
interface](https://forgejo.org/docs/latest/user/api-usage/). This has
given rise to several Emacs and command-line interfaces that existing
contributors may find convenient.
[forgejo-cli](https://codeberg.org/Cyborus/forgejo-cli/) and
[codeberg-cli](https://codeberg.org/Aviac/codeberg-cli) provide rather
comprehensive command-line interfaces, as [reported by
Efraim](https://lists.gnu.org/archive/html/guix-devel/2025-02/msg00057.html).
[fj.el](https://codeberg.org/martianh/fj.el/) is an Emacs interface
similar to `mastodon.el` that lets you view and comment on issues and
pull requests, list repositories, view notifications, and so on. It
does not support off-line access. It can be set up with something like:
```lisp
(with-eval-after-load 'fj
(setq fj-host "https://codeberg.org")
(setq fj-user "civodul")
(setq fj-token
(funcall (plist-get
(car
(auth-source-search :host "codeberg.org/api/v1"
:user fj-user
:type 'netrc))
:secret))))
```
… and a line like this one to `~/.authinfo.gpg`:
```
machine codeberg.org/api/v1 login civodul password TOKEN
```
… where `TOKEN` is the [token obtained from
Codeberg](https://docs.codeberg.org/advanced/access-token/).
[Magit-Forge](https://github.com/magit/forge/) is another Emacs
interface to forges, with Magit integration and support for working
off-line. However, Forgejo support is currently next to nonexistent:
only `forge-browse` is supported (allowing users to open a browser on a
forge page).
Besides these interfaces, there is a couple of tricks that can simplify
the life of contributors and reviewers, out of the browser.
As a contributor, you can create pull requests without first creating a
fork and then a merge request thanks to the [AGit
workflow](https://forgejo.org/docs/latest/user/agit-support/). This
works by passing `git push` the relevant *options*, as in this example:
```
git push origin HEAD:refs/for/main \
-o topic="update-hello" \
-o title="gnu: hello: Update to 42." \
-o description='This updates the `hello` package."
```
As a reviewer, it is possible to pull references of pending pull
requests by adding something like this to `.git/config`:
```
[remote "pulls"]
url = git <at> codeberg.org:org/guix-science/guix-science.git
fetch = +refs/pull/*/head:refs/remotes/pulls/pr/*
```
Running `git fetch pulls` then retrieves references to branches
corresponding to all the pull requests.
## Teams
All the teams currently defined in `etc/teams.scm` will be reified as
[teams](https://docs.codeberg.org/collaborating/create-organization/#teams)
in the [Guix organization](https://codeberg.org/guix).
All these teams would have read-only access to the repositories, with
the exception of a new *Committers* team, with read-write access to the
repository, which would contain all the people who already have [commit
rights on
Savannah](https://savannah.gnu.org/project/memberlist.php?group=guix)
(“on-duty members”).
Team scopes in `etc/teams.scm` will be converted to a `CODEOWNERS` file
similar to [that found in
Guix-Science](https://codeberg.org/guix-science/guix-science/src/branch/master/CODEOWNERS).
That way, pull requests will automatically have them suggested as
reviewers for changes in their scope.
## Continuous Integration
Forgejo supports
[*webhooks*](https://forgejo.org/docs/latest/user/webhooks/), `POST`
requests that are sent to the server of one’s choice upon events such as
pull request creation. Cuirass (running at ci.guix.gnu.org) already
[supports](https://hpc.guix.info/blog/2025/01/join-the-guix-science-community/)
them and automatically creates a *jobset* when a pull request is made.
The [QA frontpage](https://qa.guix.gnu.org) and its [Data
Service](https://data.qa.guix.gnu.org) does not support Forgejo webhooks
yet but can be extended to do so without too much effort, possibly
sharing or reusing the Forgejo interface code from Cuirass.
In the Guix repository, we will set up webhooks to trigger the creation
of a new jobset at ci.guix.gnu.org (Cuirass) as soon as migration is
complete. While this has been successfully used for several months for
[Guix-Science](https://codeberg.org/guix-science), scalability will be
the major concern here; additional developments may be needed to
consolidate this support. Eventually the QA frontpage will also support
those webhooks.
We will arrange so that the build status of a pull request is clearly
visible right from that pull request.
Eventually, the QA service or a [Forgejo
*action*](https://forgejo.org/docs/latest/user/actions/) may
automatically provide feedback from `guix lint` as a reply to pull
requests.
## Workflow
Once continuous integration (CI) is fully operational, pull requests may
be merged if and only if they successfully built. “World-rebuild” pull
requests would still follow the [existing branching
process](https://guix.gnu.org/manual/devel/en/html_node/Managing-Patches-and-Branches.html).
Note that since Guix requires signed commits by people listed in
`.guix-authorizations`, we will *not* be able to click the “Merge”
button nor to enable auto-merge on build success.
If and when the project migrates, we will incrementally adjust our
workflow to ensure it scales better.
## Translation
We may eventually consider migrating translation work from [Fedora’s
Weblate](https://translate.fedoraproject.org/projects/guix/) to
[Codeberg’s](https://docs.codeberg.org/codeberg-translate/), as a way to
make it more discoverable and better integrated.
# Cost of Reverting
While the project *could* migrate back from Codeberg to bugs.gnu.org
(Debbugs), migrating issues and pull requests from Codeberg to Debbugs
would be practically infeasible. It is unlikely that anyone would want
this.
A more interesting question is: what would it take to migrate to a
different Forgejo instance or to a different forge?
Migrating to a different Forgejo instance would be rather simple since
Forgejo is able to *import* entire repositories with their settings
(including teams) and issues and pull requests from other instances.
Users would have to create accounts on the new forge instance though.
However, if federation support matures in Forgejo, one may be able to
operate on a repository from distinct but *federated* instances. That
would make any move much easier.
Forgejo appears to support
[“mirroring”](https://forgejo.org/docs/latest/user/repo-mirror/) to
GitLab instances, for instance, which could help migrating to a GitLab
instance. Migrating to a sourcehut instance would probably be more
difficult because of the feature mismatch.
Note that Forgejo offers a [rich HTTP
interface](https://forgejo.org/docs/latest/user/api-usage/) that
essentially allows users to get the raw data behind issues, pull
requests, and more, meaning that it is theoretically always possible to
grab the data.
# Drawbacks and Open Issues
Leaving it up to an external organization to manage critical
infrastructure of our project comes with risks.
First, everyone will have to create an account and accept [Codeberg’s
Terms of
Use](https://codeberg.org/Codeberg/org/src/branch/main/TermsOfUse.md)
before they can contribute, which can be seen as a step back compared to
the email-based workflow.
Codeberg e.V., the non-profit that runs Codeberg, could always go
bankrupt, suffer from governance issues, or run into problems that any
non-profits face and which could lead to service discontinuation. This
could be mitigated by weaving close ties with Codeberg e.V., for
instance by financially supporting it, by setting a joint team to
monitor resource consumption induced by Guix and discuss any issues
encountered by either parties proactively, or by [joining
it](https://join.codeberg.org/) as an active member with voting rights.
The self-hosting option has its appeal for a project with the size and
values of Guix—it is not uncommon for similar projects to do that, an
example being the [Lix project](https://git.lix.systems/); there even
exists a [preliminary Forgejo service for
Guix](https://git.boiledscript.com/hako/Rosenthal/src/commit/7a6a28e872b3168f9b6513ccf797e247cd8a366d/rosenthal/services/web.scm#L32).
However the author thinks that, as it stands, Guix system administrators
have more than enough on their plate and are perhaps not up to the task
of providing the availability guarantees we expect from such a service.
As of this writing, Forgejo integration in Cuirass is functional but
partial (useful configuration options and hardening mechanisms are
missing) and missing from QA-Frontpage. This will have to be addressed
to fully take advantage of the new pull-request workflow.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.