Package: guix-patches;
Reported by: Ludovic Courtès <ludo <at> gnu.org>
Date: Sun, 23 Feb 2025 15:21:02 UTC
Severity: normal
Done: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
View this message in rfc822 format
From: Trev <trev <at> trevdev.ca> To: Ludovic Courtès <ludo <at> gnu.org>, info-guix <at> gnu.org, Guix Devel <guix-devel <at> gnu.org> Cc: 76503 <at> debbugs.gnu.org Subject: [bug#76503] [GCD] Deliberation on: Migrating repositories, issues, and patches to Codeberg Date: Fri, 25 Apr 2025 11:30:44 -0700
I support Ludovic Courtès <ludo <at> gnu.org> writes: > Hello Guix, > > It’s been two months since we started discussing GCD 002, entitled > “Migrating repositories, issues, and patches to Codeberg”. Its final > version is attached below. > > In accordance with the GCD process, team members have until May 6th to > participate in deliberation by sending one of the following replies > (quoting the GCD process): > > - “I support”, meaning that one supports the proposal; > - “I accept”, meaning that one consents to the implementation of the > proposal; > - “I disapprove”, meaning that one opposes the implementation of the > proposal. A team member sending this reply should have made constructive > comments during the discussion period. > > Thanks in advance, > Ludo’. > > title: Migrating repositories, issues, and patches to Codeberg > id: 002 > status: submitted > discussion: https://issues.guix.gnu.org/76503 > authors: Ludovic Courtès > sponsors: Tobias Geerinckx-Rice, Ricardo Wurmus > date-submitted: 2025-02-23 > date: 2025-02-23 > SPDX-License-Identifier: CC-BY-SA-4.0 OR GFDL-1.3-no-invariants-or-later > --- > > # Summary > > The contribution workflow in Guix has been facing several challenges: > difficult onboarding, lack of legibility, complex, unreliable, and > labor-intensive infrastructure, and lack of automation. All these lead > to an experience that contributors often find frustrating and hinders > quality assurance efforts. We propose to address these limitations by > migrating repositories, issue tracking, and patch tracking to Codeberg, > an integrated forge with a Web interface that is hosted by a non-profit. > > # Motivation > > To keep track of bug reports and patches, Guix historically chose tools > that were *simple* in their design: > > - bug reports and patches can be sent by plain email, without having > to create an account or even subscribe to a mailing list; > - discussion and patch review happen naturally by email, without > requiring special tools; > - the Debbugs instance at https://bugs.gnu.org keeps track of bug > reports and patches by assigning them an identifier and creating a > mailing list specifically for each bug or patch. > > However, to overcome several limitations, the project developed > processes and tools, which can be characterized as *incidental > complexity*: > > - because the Debbugs web interface is crude by today’s standards and > hard to search and navigate, the project developed > [mumi](https://git.savannah.gnu.org/cgit/guix/mumi.git/), the web > interface running at https://issues.guix.gnu.org; > - to navigate bugs and patches more conveniently than what an email > client supports, contributors were > [encouraged](https://guix.gnu.org/manual/devel/en/html_node/Debbugs-User-Interfaces.html) > to use interfaces like `debbugs.el` or `b4`; > - sending patch series by email does not play well with Debbugs’ > automatic identifier assignment, so [contributors were told to send > their “cover letter”, wait for an identifier to be assigned, and > then send the > rest](https://guix.gnu.org/manual/devel/en/html_node/Sending-a-Patch-Series.html#Multiple-Patches-1); > - to help sending and applying patch series, mumi was extended to > provide a command line interface; > - to build patch series submitted by email, the [QA > service](https://qa.guix.gnu.org) has to rely on a [Patchwork > instance](https://patches.guix-patches.cbaines.net/project/guix-patches/list/) > that is subscribed to the `guix-patches` mailing list, coupled with > its own [parsing of incoming > email](https://git.savannah.gnu.org/gitweb/?p=guix/data-service.git;a=blob;f=guix-data-service/branch-updated-emails.scm;h=aeb1570dfda725864a77780d0541f26c090b0e55;hb=c886685e9284da4bbed9377f70dd70da9e7ca29f); > - the project added a commit hook to add unique `Change-Id` > headers in commit messages in an attempt to correlate commits in the > repository with messages send to `guix-patches`; none of the > existing tools takes advantage of it though, and it is up to > contributors to manually close entries in the bug/patch tracker once > they have been fixed/applied. > > Developing and maintaining this software and infrastructure is > time-consuming. Worse, it leaves contributors largely dissatisfied for > a variety of reasons: > > - the process is unfamiliar to most newcomers; > - the tools and infrastructure in Guix have become a maze; > - apart from the happy few using `debbugs.el` in Emacs, navigating > open issues and patches is hard; filtering incoming messages is > equally hard, even for those with 10+ years of experience with > advanced email tools (Gnus, mu4e, notmuch, b4, etc.); > - because the various parts of the development process (repository, > issue tracking, QA automation, `etc/teams.scm`) are largely > disconnected, even long-time contributors can hardly follow issues > relevant to them; issues may remain open after they’ve been fixed, > new activity on an issue may go unnoticed, cross-references among > issues are not visible in any of the interfaces, etc. > > All this contributes to a [poor > experience](https://guix.gnu.org/en/blog/2025/guix-user-and-contributor-survey-2024-the-results-part-3/) > for those who choose to contribute despite the barrier to entry, > probably discourages many to even start contributing, and adds to the > load of committers and infrastructure maintainers. > > # Detailed Design > > This section explains the chosen solution among the available options, > the scope of the proposed migration, a migration path, and an outlook on > automation. > > ## Choice of a Forge > > We set out to choose a “modern forge” that supports a pull-request style > workflow and provides good integration between the repository, the issue > tracker, and the merge request tracker. Such a system is necessarily > more *complex* at first glance than the email-based tools we have but > (1) the increase in complexity is reasonable once we consider the > incidental complexity of the existing services, as mentioned above, and > (2) we think the added usage benefits outweigh this increase in > complexity. > > The software behind the forge has to be free software that is > *plausibly* self-hosted on Guix System—this probably rules out GitLab > Community Edition and makes [Forgejo](https://forgejo.org/) the main > contender. > > [SourceHut](https://sourcehut.org/), the other interesting option, does > not offer the same convenience when it comes to dealing with patches and > runs the risk of reproducing onboarding and integration issues > surrounding an email-based workflow and “read-only” web interface that > Guix is already experiencing. > > Forgejo has several features to support collaboration among a large > number of people and on a large code base, including > [teams](https://docs.codeberg.org/collaborating/create-organization/#teams) > and [issue and pull request > templates](https://forgejo.org/docs/latest/user/issue-pull-request-templates/). > Support for > [federation](https://forgejo.org/2023-01-10-answering-forgejo-federation-questions/) > is also under development and is a promising way to avoid > centralization. > > Instead of self-hosting, this GCD suggests using the Forgejo instance on > codeberg.org, run by the [Codeberg e.V.](https://codeberg.org/about) > non-profit, registered in Germany. The non-profit has a good track > record of running codeberg.org with minimal downtime, is [committed to > supporting free software > development](https://codeberg.org/Codeberg/org/src/branch/main/en/bylaws.md#preamble), > [transparent](https://codeberg.org/Codeberg/org), and has governance set > up to achieve its mission. > > The Guix-Science umbrella project [has been using Codeberg for several > months > now](https://hpc.guix.info/blog/2025/01/join-the-guix-science-community/), > which has allowed us to gain confidence in its suitability for a project > like Guix. > > ## Rights and Privileges > > Migration should preserve rights and privileges regarding access to the > repositories. > > Currently, [committers in the “guix” group on > Savannah](https://savannah.gnu.org/project/memberlist.php?group=guix) > have write access to [all the repositories of that > group](https://savannah.gnu.org/git/?group=guix). Since Codeberg allows > for per-repository rules, we propose the following rules to determine > who can access each repository: > > - Committers to *several* of the repositories as well as people marked > as [“group admins” on > Savannah](https://savannah.gnu.org/projects/guix) can request > membership in the [“Owners” > team](https://docs.codeberg.org/collaborating/create-organization/#teams) > of the [Guix *organization*](https://codeberg.org/guix). As of this > writing, only three people are members. > - Anyone listed in the `.guix-authorizations` file of Guix can request > membership of the https://codeberg.org/guix/guix repository once it > is created. > - Committers to one of the other repositories can request membership > of that repository. > > The [“Commit > Rights”](https://guix.gnu.org/manual/devel/en/html_node/Commit-Access.html) > section of the manual will be extended to clarify the distinction > between being a member of the organization and being a member of a > specific repository, in a specific team. > > ## Repository Migration Path > > The Guix project at Savannah contains the following repositories: > > - [Guix itself](https://git.savannah.gnu.org/git/guix.git); > - [the bootstrappable.org web > site](https://git.savannah.gnu.org/git/guix/bootstrappable.git); > - [the DHCP client in > Guile](https://git.savannah.gnu.org/git/guix/dhcp.git) (forgotten > 2015 Google Summer of Code project); > - [Guile bindings to > GNUnet](https://git.savannah.gnu.org/git/guix/gnunet.git) (forgotten > 2015 Google Summer of Code project); > - [Guix artwork and web > site](https://git.savannah.gnu.org/git/guix/guix-artwork.git); > - [Cuirass](https://git.savannah.gnu.org/git/guix/guix-cuirass.git); > - [“maintenance” > repository](https://git.savannah.gnu.org/git/guix/maintenance.git) > (includes Guix System infrastructure configuration, talks, and other > documents); > - [scripts for videos presenting > Guix](https://git.savannah.gnu.org/git/guix/videos.git); > - [Guix Data > Service](https://git.savannah.gnu.org/git/guix/data-service.git); > - [Emacs-Guix](https://git.savannah.gnu.org/git/guix/emacs-guix.git); > - [Guix Build > Coordinator](https://git.savannah.gnu.org/git/guix/build-coordinator.git); > - [nar-herder](https://git.savannah.gnu.org/git/guix/nar-herder.git); > - [QA > Frontpage](https://git.savannah.gnu.org/git/guix/qa-frontpage.git); > - [mumi](https://git.savannah.gnu.org/cgit/guix/mumi.git); > - [Guix Consensus > Documents](https://git.savannah.gnu.org/git/guix/guix-consensus-documents.git). > > Within **30 days** following acceptance of this GCD, committers would > migrate all these repositories to https://codeberg.org/guix. > > For Guix itself, we would decide on a **flag day** 14 days after > acceptance of this GCD at the earliest, and 30 days at the latest. On > that day, the official URL of the Guix repository would become > https://codeberg.org/guix/guix.git (this flag day is *only* about the > canonical repository; see the next section for bug reports and patch > tracking.) > > After this switch, committers will have to push exclusively to > https://codeberg.org/guix/guix.git. > The Savannah `guix.git` repository will become a mirror of the one at > Codeberg, with a script periodically updating it for **at least one > year** after the switch, as a way to ease migration to the new > repository for users. Other repositories would be deleted from Savannah > once migrated, to avoid confusion. > > A commit will reflect the change of Guix’s repository URL by updating: > > 1. the `url` field in `.guix-channel`; > 2. the `%default-channel-url` variable in `(guix channels)`; > 3. any other reference to the URL that may appear in the repository, > in particular in the manual. > > To ease this migration and possibly future migration, we may add a new > `git.guix.gnu.org` DNS entry with HTTP redirects to > `git.savannah.gnu.org` (before migration) and `codeberg.org` (after > migration); a [patch](https://issues.guix.gnu.org/76296) implementing > this has been submitted. The `%default-channel-url` variable would > refer to `https://git.guix.gnu.org/guix.git`. > > Following this commit, an entry in `etc/news.scm` would explain the > migration, accompanied by a blog post on > <https://guix.gnu.org/en/blog/>. See [this entry in > Guix-Science](https://codeberg.org/guix-science/guix-science/commit/fd1b2dacd8d37c9d1939f9dc5a5b74256171ccbd) > for an example. > > ## Issue Tracker Migration Path > > Importing all the issues and patches from Debbugs/mumi into Codeberg > would be impractical: it would require the development of specific > tools, would be a lossy process due to the fundamental mismatch between > plain text email threads and Forgejo issues and pull requests, and would > bring little in return. > > We propose a gradual migration process, where both systems would be used > simultaneously for several months: > > - https://issues.guix.gnu.org will remain up and running for at least > **two years** following acceptance of this GCD. Note that once > issues.guix.gnu.org is down, issues will remain visible at > https://bugs.gnu.org and email archives will remain visible at > https://mail.gnu.org. > - Until **December 31st, 2025**, bug reports and patches will be accepted both by > email (`bug-guix` and `guix-patches`) and on Codeberg (issues and > pull requests); documentation will encourage people to choose > Codeberg, but contributors will keep paying attention to both > systems. > - Soon after **January 1st, 2026**, mailing list administrators will > set up the `bug-guix` and `guix-patches` mailing lists in “Emergency > Moderation” mode in the Mailman interface—meaning that messages to > open *new* issues will not get through anymore. > - To better serve those for whom Codeberg is inaccessible, > occasionally patches sent by email to `guix-devel` will be accepted. > - Interaction on individual issues that were opened via Debbugs before > this deadline will continue on Debbugs for as long as needed, *via* > `NNN <at> debbugs.gnu.org`. > - The switchover will be advertised before it takes place with a post > to `info-guix <at> gnu.org`, to `guix-devel <at> gnu.org`, as well as through > a blog post. > - The “Contributing” section of the manual will be updated accordingly > at that time. > > ## User Interfaces > > For many contributors, a strength of the email-based workflow is that it > works out of the browser, possibly offline; we want to preserve that > comfort as much as possible. > > Everything that can be done through Forgejo’s web interface can be done > *via* its [HTTP > interface](https://forgejo.org/docs/latest/user/api-usage/). This has > given rise to several Emacs and command-line interfaces that existing > contributors may find convenient. > > [forgejo-cli](https://codeberg.org/Cyborus/forgejo-cli/) and > [codeberg-cli](https://codeberg.org/Aviac/codeberg-cli) provide rather > comprehensive command-line interfaces, as [reported by > Efraim](https://lists.gnu.org/archive/html/guix-devel/2025-02/msg00057.html). > > [fj.el](https://codeberg.org/martianh/fj.el/) is an Emacs interface > similar to `mastodon.el` that lets you view and comment on issues and > pull requests, list repositories, view notifications, and so on. It > does not support off-line access. It can be set up with something like: > > ```lisp > (with-eval-after-load 'fj > (setq fj-host "https://codeberg.org") > (setq fj-user "charlie") > (setq fj-token > (funcall (plist-get > (car > (auth-source-search :host "codeberg.org/api/v1" > :user fj-user > :type 'netrc)) > :secret)))) > ``` > > … and a line like this one to `~/.authinfo.gpg`: > > ``` > machine codeberg.org/api/v1 login charlie password TOKEN > ``` > > … where `TOKEN` is the [token obtained from > Codeberg](https://docs.codeberg.org/advanced/access-token/). > > [Magit-Forge](https://github.com/magit/forge/) is another Emacs > interface to forges, with Magit integration and support for working > off-line. However, Forgejo support is currently next to nonexistent: > only `forge-browse` is supported (allowing users to open a browser on a > forge page). > > Besides these interfaces, there is a couple of tricks that can simplify > the life of contributors and reviewers, out of the browser. > > As a contributor, you can create pull requests without first creating a > fork and then a merge request thanks to the [AGit > workflow](https://forgejo.org/docs/latest/user/agit-support/). This > works by passing `git push` the relevant *options*, as in this example: > > ``` > git push origin HEAD:refs/for/main \ > -o topic="update-hello" \ > -o title="gnu: hello: Update to 42." \ > -o description='This updates the `hello` package." > ``` > > As a reviewer, it is possible to pull references of pull > requests by adding something like this to `.git/config`: > > ``` > [remote "pulls"] > url = git <at> codeberg.org:org/guix-science/guix-science.git > fetch = +refs/pull/*/head:refs/remotes/pulls/pr/* > ``` > > Running `git fetch pulls` then retrieves references to branches > corresponding to _all_ the pull requests. > > In addition, one may fetch the list of [open pull requests as > JSON](https://codeberg.org/api/v1/repos/guix-science/guix-science/pulls?state=open), > extract the number of a pull request of interest, and retrieve just this > pull request with: > > ``` > git fetch -u origin pull/123/head:pull/123/head > ``` > > … where `123` is the pull request number. The `.git/config` snippet > below combines these to define an alias to fetch branches for open pull > requests (it requires `wget` and `jq`): > > ``` > [alias] > fetch-open-pull-requests="!for pr in \ > $(wget -qO - 'https://codeberg.org/api/v1/repos/guix-science/guix-science/pulls?state=open' \ > | jq '.[] | .number'); \ > do \ > echo Fetching PR $pr...; \ > git fetch -f -u codeberg pull/$pr/head:pull/$pr/head; \ > done" \ > ``` > > It allows you to run `git fetch-open-pull-requests` to retrieve > references for open pull requests only (it assumes the remote is called > `codeberg` and the project is `guix-science/guix-science`; adapt to your > needs!). > > Last, one may similarly fetch the list of [closed pull > requests](https://codeberg.org/api/v1/repos/guix-science/guix-science/pulls?state=closed), > extract the number of a pull request of interest, and clean just this > pull request with: > > ``` > git update-ref -d refs/heads/pull/123/head > ``` > > … where `123` is the pull request number. > > ## Teams > > All the teams currently defined in `etc/teams.scm` will be reified as > [teams](https://docs.codeberg.org/collaborating/create-organization/#teams) > in the [Guix organization](https://codeberg.org/guix). > > All these teams would have read-only access to the repositories, with > the exception of a new *Committers* team, with read-write access to the > repository, which would contain all the people who already have [commit > rights on > Savannah](https://savannah.gnu.org/project/memberlist.php?group=guix) > (“on-duty members”). > > Team scopes in `etc/teams.scm` will be converted to a `CODEOWNERS` file > similar to [that found in > Guix-Science](https://codeberg.org/guix-science/guix-science/src/branch/master/CODEOWNERS). > That way, pull requests will automatically have them suggested as > reviewers for changes in their scope. > > ## Continuous Integration > > Forgejo supports > [*webhooks*](https://forgejo.org/docs/latest/user/webhooks/), `POST` > requests that are sent to the server of one’s choice upon events such as > pull request creation. Cuirass (running at ci.guix.gnu.org) already > [supports](https://hpc.guix.info/blog/2025/01/join-the-guix-science-community/) > them and automatically creates a *jobset* when a pull request is made. > The [QA frontpage](https://qa.guix.gnu.org) and its [Data > Service](https://data.qa.guix.gnu.org) does not support Forgejo webhooks > yet but can be extended to do so without too much effort, possibly > sharing or reusing the Forgejo interface code from Cuirass. > > In the Guix repository, we will set up webhooks to trigger the creation > of a new jobset at ci.guix.gnu.org (Cuirass) as soon as migration is > complete. While this has been successfully used for several months for > [Guix-Science](https://codeberg.org/guix-science), scalability will be > the major concern here; additional developments may be needed to > consolidate this support. Eventually the QA frontpage will also support > those webhooks. > > We will arrange so that the build status of a pull request is clearly > visible right from that pull request. > > Eventually, the QA service or a [Forgejo > *action*](https://forgejo.org/docs/latest/user/actions/) may > automatically provide feedback from `guix lint` as a reply to pull > requests. > > ## Workflow > > Once continuous integration (CI) is fully operational, pull requests may > be merged if and only if they successfully built. “World-rebuild” pull > requests would still follow the [existing branching > process](https://guix.gnu.org/manual/devel/en/html_node/Managing-Patches-and-Branches.html). > > Note that since Guix requires signed commits by people listed in > `.guix-authorizations`, we will *not* be able to click the “Merge” > button nor to enable auto-merge on build success. We will disable it > to avoid mistakes. Manual merge is supported by Forgejo. > > If and when the project migrates, we will incrementally adjust our > workflow to ensure it scales better. > > ## Translation > > We may eventually consider migrating translation work from [Fedora’s > Weblate](https://translate.fedoraproject.org/projects/guix/) to > [Codeberg’s](https://docs.codeberg.org/codeberg-translate/), as a way to > make it more discoverable and better integrated. > > # Cost of Reverting > > While the project *could* migrate back from Codeberg to bugs.gnu.org > (Debbugs), migrating issues and pull requests from Codeberg to Debbugs > would be practically infeasible. It is unlikely that anyone would want > this. > > A more interesting question is: what would it take to migrate to a > different Forgejo instance or to a different forge? > > Migrating to a different Forgejo instance would be rather simple since > Forgejo is able to *import* entire repositories with their settings > (including teams) and issues and pull requests from other instances. > Users would have to create accounts on the new forge instance though. > However, if federation support matures in Forgejo, one may be able to > operate on a repository from distinct but *federated* instances. That > would make any move much easier. > > Forgejo appears to support > [“mirroring”](https://forgejo.org/docs/latest/user/repo-mirror/) to > GitLab instances, for instance, which could help migrating to a GitLab > instance. Migrating to a sourcehut instance would probably be more > difficult because of the feature mismatch. > > Note that Forgejo offers a [rich HTTP > interface](https://forgejo.org/docs/latest/user/api-usage/) that > essentially allows users to get the raw data behind issues, pull > requests, and more, meaning that it is theoretically always possible to > grab the data. > > # Drawbacks and Open Issues > > Leaving it up to an external organization to manage critical > infrastructure of our project comes with risks. > > First, everyone will have to create an account and accept [Codeberg’s > Terms of > Use](https://codeberg.org/Codeberg/org/src/branch/main/TermsOfUse.md) > before they can contribute, which can be seen as a step back compared to > the email-based workflow. > > Codeberg e.V., the non-profit that runs Codeberg, could always go > bankrupt, suffer from governance issues, or run into problems that any > non-profits face and which could lead to service discontinuation. This > could be mitigated by weaving close ties with Codeberg e.V., for > instance by financially supporting it, by setting a joint team to > monitor resource consumption induced by Guix and discuss any issues > encountered by either parties proactively, or by [joining > it](https://join.codeberg.org/) as an active member with voting rights. > > Backup (for issues and discussions in particular) remains an open issue. > We could set up a Forgejo instance for this purpose or find an existing > one willing to mirror the project. > > The self-hosting option has its appeal for a project with the size and > values of Guix—it is not uncommon for similar projects to do that, an > example being the [Lix project](https://git.lix.systems/); there even > exists a [preliminary Forgejo service for > Guix](https://git.boiledscript.com/hako/Rosenthal/src/commit/7a6a28e872b3168f9b6513ccf797e247cd8a366d/rosenthal/services/web.scm#L32). > However, at the moment, Guix system administrators > have more than enough on their plate and are perhaps not up to the task > of providing the availability guarantees we expect from such a service. > Whether to self-host is something that could be revisited in the future. > > As of this writing, Forgejo integration in Cuirass is functional but > partial (useful configuration options and hardening mechanisms are > missing) and missing from QA-Frontpage. This will have to be addressed > to fully take advantage of the new pull-request workflow. -- Trev : 0FB7 D06B 4A2A F07E AD5B 1169 183B 6306 8AA1 D206
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.