GNU bug report logs - #74413
[PATCH] Allow to store and read repository information of VCS builds

Previous Next

Full log

Message #62 received at 74413 <at> debbugs.gnu.org (full text, mbox):

From: Björn Bidar <bjorn.bidar <at> thaodan.de>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: luangruo <at> yahoo.com, 74413 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#74413: [PATCH] Allow to store and read repository
 information of VCS builds
Date: Tue, 26 Nov 2024 17:35:58 +0200

Stefan Kangas <stefankangas <at> gmail.com> writes:

> Björn Bidar <bjorn.bidar <at> thaodan.de> writes:
>
>>> I don't fully understand how you can have a situation where you can get
>>> this information in the Makefile, but you can't also get it when dumping
>>> using `emacs-repository-get-version` and `emacs-repository-get-branch`
>>> (lisp/loadup.el:474).  Could you please elaborate on this?
>>
>> I added the Makefile part so it would work outside of my environment if
>> one builds Emacs and then distributes/install it without the sources or
>> without them containing the VCS information while git is also installed
>> in their build environment.
>
> I don't think I understand what you're saying here.  Is it possible that
> there are some typos above?  I'm thinking of the part that contains the
> tautology "without the sources or without them".

I meant to say that if a user builds with Git installed and Emacs sources 
checked out from git. Then packages Emacs but only puts the checkout
sources without the VCS information into the source package/tarball.

> There are also some words where the meaning is not entirely clear to me,
> such as "distributes" and "installs".

Distributed in this case meant to be the package build put into a
repository and then installed using a package manager.

> Maybe it will be clarified by the answers to my below questions.
>
>>> Do you mean that you have one containerized process with Git that clones
>>> emacs.git into a directory, and then an entirely separate containerized
>>> process, without Git, builds Emacs from that very same directory?  Or
>>> something along those lines?
>>
>> Yes more or less.
>> I'm using an obs source service that generates a tarball from VCS's and
>> then creates a tarball. [1]
>
> I took a look at the linked documentation, but it seems to be on a very
> detailed level, and does not provide a good overview.  So I don't think
> I understood much from it.
>
> I'm not familiar with the Open Build System, so this is quite hard for
> me to follow.  Please be patient with me.
>
> In your description above, could you explain what is the difference
> between the two steps "generates a tarball from VCS" and "creates a
> tarball"?

It should have be more detailed or used the proper word.
The program that I use to generate the sources fetches the sources from
a VCS such as git and then creates a cpio archive (for storing the
sources more efficiently but that's beyond this topic).

The cpio archive is then converted to a tar archive and compressed.

> Do you mean to say that it:
> 1) generates a tarball from emacs.git, excluding the .git directory
> 2) copies that tarball to a completely different environment
> 3) builds Emacs

Essentially yes. The point of this is that the environment that builds
the packages is disconnected from internet or other external factors for
security reasons.

The tarball is copied into the build environment while it prepares it
so that a minimal linux vm or chroot can start to build software.

> The part that I do not understand is in which step you run `make`, and
> how.  Is it run in the first step, the third, the first and the third,
> or something else?  Is it run automatically by OBS, or do you have to
> specify it?

I run make in in the 3 step right after the environment is setup up
inside rpmbuild.

Before make is called I parse the file that contains the reference hash
and the file that contains the configuration for the program that
generates the sources which contains the branch that the source where
generated from into the file that Emacs then can read during the build
process and after that when the package was installed.

The only part that's special here is that I generate the version
file from the information gathered while generating the source tarball
as opposed to using git to gather those.


> Basically, I still don't understand the answer to my below question:
>
>>> In this very particular scenario, isn't it enough to add this
>>> additional step to your CI pipeline:
>>>
>>>     ./src/emacs -Q --batch --eval '(princ (format "%s:%s" \
>>>     emacs-repository-version emacs-repository-branch))' > version.info
>> The obs source service already contains that information see the
>> emacs.obsinfo file I mentioned earlier.
>
> Hmm.  Would an alternative solution be to read the version information
> from that file then, if it exists?
>
>>> In other words, is it really necessary for us to support this use case
>>> in our Makefile?  Do we expect that building Emacs in such CI pipelines
>>> using non-released development version of Emacs will be very common?
>>
>> The Makefile target is there already for the Android builds all I did I
>> added it to other platforms too. If the target would be shared then the
>> hole process is reused.
>
> If we want this for other builds, then reusing what we do for the
> Android build sounds prudent.  It is clear that this is needed for
> Android.  What remains to clarify is why it is needed elsewhere.
>
>> I think it would become more common to build from VCS sources would be
>> become more common where the information is useful.
>
> Building from VCS sources is already quite common.  I'm asking something
> more specific: will it be increasingly common to build using an
> environment very similar to OBS?

I think so. It's already quite common to build software in a much
simpler CI such as Gitlab CI. Fedora has their own CI similar to OBS for example.

> If it is not going to be very common, this should perhaps be fixed in
> the OBS pipeline directly, instead of complicating our Makefile.  But
> I'm not yet clear on whether that is true.

The makefile only has the rule add or moved for the version file.
To me this just looks like refactoring so the logic that was added for
Android can used on other platforms.

>> To get more users to test earlier development builds to have faster
>> general feedback and bug reports reports it is very useful to have
>> non-released development builds. There are other instances outside my
>> own usecase where this is frequent such as for example the AUR emacs-git, guix
>> emacs-next or the Android Fdroid builds.
>
> Are you saying that AUR emacs-git, guix emacs-next, and the Android
> Fdroid builds also have no way to know which revision Emacs was built
> from without your patch?

The Android builds obviously have the revision already integrated
as the logic that I'm trying to reuse here have been already
implemented for Android.
I don't know about the Guix build process but for Arch git isn't
into the change-root when building Emacs from the tarball that has been
generated before hand.

The OBS can build packages for all kinds of Linux distributions such as
Flatpak, Debian, Fedora, SUSE, Ubuntu etc.

So this change can help building test builds for all these distributions
in theory. Personally just building for SUSE.

>>> BTW, what is the name of that CI system that you're using here?  For
>>> what purpose are you building Emacs: to test Elisp packages, to test
>>> Emacs, or something else?  Finally, why are you not using officially
>>> tagged versions, either from us or from some distro, in this context?
>>
>> I'm build Emacs on the open build service. I'm building Emacs on the obs
>> to provide development builds that can be used to test Emacs and submit
>> bug reports.
>> The package is build using the official RPM packaging just
>> with the newer sources from git.
>>
>> The initial reason reason was that I noticed that even if you build from
>> git directly without a CI/or something similar to the source service
>> mentioned earlier that the functions to retrieve the repository
>> information don't work since the source path either doesn't exist on any
>> the machine that installs the final package or that it doesn't contain
>> the VCS information anymore.
>> The setup is very similar to the Android builds.
>
> Thanks.

No problem.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.