GNU bug report logs - #36747
Official MesCC bootstrap binaries differ from my locally built ones

Previous Next

Full log

Message #64 received at 36747 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 36747 <at> debbugs.gnu.org,
 Jan Nieuwenhuizen <janneke <at> gnu.org>
Subject: Re: bug#36747: Official MesCC bootstrap binaries differ from my
 locally built ones
Date: Wed, 14 Aug 2019 14:35:47 -0400

Hi Marius,

Marius Bakke <mbakke <at> fastmail.com> writes:

> Marius Bakke <mbakke <at> fastmail.com> writes:
>
>> Jan Nieuwenhuizen <janneke <at> gnu.org> writes:
>>
>>> Mark H Weaver writes:
>>>
>>> Hi Mark,
>>>
>>>>> I called that `wip-binaries', @master from three weeks ago.
>>>>
>>>> Thank you, that was a good start.  I found that some additional patches
>>>> were needed to match the bootstrap binaries that 'core-updates' is
>>>> currently based on.
>>>>
>>>> I ended up deleting and repushing a revised 'wip-binaries' to Savannah.
>>>> It includes slightly modified versions of the two commits you had
>>>> included, as well as some additional cherry-picked commits of yours to
>>>> update mescc-tools and add linux-libre-headers-bootstrap-tarball, and a
>>>> few of my own.
>>>
>>> Very nice.
>>>
>>>> I built the new bootstrap tarballs at the new 'wip-binaries', commit
>>>> c67becb31c30a5cd7685f166970ac4793e3a34a9, and here's what I got:
>>>>
>>>> mhw <at> jojen ~/guix-wip-binaries$ git describe
>>>> v1.0.1-2404-gc67becb31c
>>>> mhw <at> jojen ~/guix-wip-binaries$ ./pre-inst-env guix build --system=i686-linux bootstrap-tarballs
>>>> /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0
>>>> mhw <at> jojen ~/guix-wip-binaries$ cd /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0
>>>> mhw <at> jojen /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0$ sha256sum *
>>>> 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b  guile-static-stripped-2.2.4-i686-linux.tar.xz
>>>> 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a  linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>>>> 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330  mescc-tools-static-stripped-0.5.2-0.bb062b0-i686-linux.tar.xz
>>>> fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0  mes-minimal-stripped-0.19-i686-linux.tar.xz
>>>> c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7  static-binaries-0-i686-linux.tar.xz
>>>
>>>> Can you try "guix build --system=i686-linux bootstrap-tarballs" at the
>>>> new 'wip-binaries' branch and see if you get the same results?
>>>
>>> Yes, on c67becb31c30a5cd7685f166970ac4793e3a34a9 running
>>> "./pre-inst-env guix build --system=i686-linux bootstrap-tarballs" gives me exactly this,
>>> also for guile-static-stripped! \o/
>>>
>>>> Also, I have a question: One of the changes I made to 'wip-binaries' was
>>>> to update mescc-tools to 0.5.2-0.bb062b0, to match the
>>>> %bootstrap-mescc-tools that's currently being used in 'core-updates'.
>>>>
>>>> However, I noticed that you have also apparently built the official
>>>> release of mescc-tools-0.5.2, which is on your site:
>>>>
>>>>   http://lilypond.org/janneke/guix/20190722/mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz
>>>>
>>>> and that this tarball is identical to the build output of the later git
>>>> commit: mescc-tools-static-stripped-0.5.2-0.bb062b0-i686-linux.tar.xz.
>>>>
>>>> With this in mind, could we just use 0.5.2?  What changed between 0.5.2
>>>> and 0.5.2-0.bb062b0, and what was the rationale for updating to bb062b0?
>>>
>>> Good catch.  We probably can, we might try that.
>>>
>>> I think the need for updating to bb062b0 has been removed during the
>>> review of the integration of the reduced binary seed bootstrap into
>>> core-updates by Ludovic.
>>>
>>> For historical reasons, I think this mescc-tools commit
>>>
>>> --8<---------------cut here---------------start------------->8---
>>> commit c184e95096881a13f29ebd7fc507fe305d3d8de5 (gitlab/janneke, janneke)
>>> Author: Jan Nieuwenhuizen <janneke <at> gnu.org>
>>> Date:   Thu Oct 4 22:03:31 2018 +0200
>>>
>>>     build.sh: Update for mes 0.18.
>>> --8<---------------cut here---------------end--------------->8---
>>>
>>> was needed at a time that we did not have mescc-tools or mes in
>>> bootstrap tarballs.  We built bootstrap variants of mescc-tools and mes
>>> using a externally (outside fo Guix) built mescc-tools-seed and
>>> (an almost pure ASCII) mes-seed.
>>
>> I tried building the i686 bootstrap tarballs from wip-binaries with this
>> additional patch:
>>
>> diff --git a/gnu/packages/mes.scm b/gnu/packages/mes.scm
>> index e298cb05c1..380cac6c88 100644
>> --- a/gnu/packages/mes.scm
>> +++ b/gnu/packages/mes.scm
>> @@ -139,33 +139,31 @@ Guile.")
>>      (license gpl3+)))
>>  
>>  (define-public mescc-tools
>> -  (let ((commit "bb062b0da7bf2724ca40f9002b121579898d4ef7")
>> -        (revision "0")
>> -        (version "0.5.2"))
>> -    (package
>> -      (name "mescc-tools")
>> -      (version (string-append version "-" revision "." (string-take commit 7)))
>> -      (source (origin
>> -                (method url-fetch)
>> -                (uri (string-append
>> -                      "https://git.savannah.nongnu.org/cgit/mescc-tools.git/snapshot/"
>> -                      name "-" commit
>> -                      ".tar.gz"))
>> -                (sha256
>> -                 (base32
>> -                  "1h6j57wyf91i42b26f8msbv6451cw3nm4nmpl1fckp9c7vi8mwkh"))))
>> -      (build-system gnu-build-system)
>> -      (supported-systems '("i686-linux" "x86_64-linux"))
>> -      (arguments
>> -       `(#:make-flags (list (string-append "PREFIX=" (assoc-ref %outputs "out")))
>> -         #:test-target "test"
>> -         #:phases (modify-phases %standard-phases
>> -                    (delete 'configure))))
>> -      (synopsis "Tools for the full source bootstrapping process")
>> -      (description
>> -       "Mescc-tools is a collection of tools for use in a full source
>> +  (package
>> +    (name "mescc-tools")
>> +    (version "0.5.2")
>> +    (source (origin
>> +              (method url-fetch)
>> +              (uri (string-append
>> +                    "https://git.savannah.nongnu.org/cgit/mescc-tools.git/snapshot/"
>> +                    name "-Release_" version
>> +                    ".tar.gz"))
>> +              (file-name (string-append name "-" version ".tar.gz"))
>> +              (sha256
>> +               (base32
>> +                "01x7bhmgwyf6mc2g1hcvibhps98nllacqm4f0j5l51b1mbi18pc2"))))
>> +    (build-system gnu-build-system)
>> +    (supported-systems '("i686-linux" "x86_64-linux"))
>> +    (arguments
>> +     `(#:make-flags (list (string-append "PREFIX=" (assoc-ref %outputs "out")))
>> +       #:test-target "test"
>> +       #:phases (modify-phases %standard-phases
>> +                  (delete 'configure))))
>> +    (synopsis "Tools for the full source bootstrapping process")
>> +    (description
>> +     "Mescc-tools is a collection of tools for use in a full source
>>  bootstrapping process.  It consists of the M1 macro assembler, the hex2
>>  linker, the blood-elf symbol table generator, the kaem shell, exec_enable and
>>  get_machine.")
>>      (home-page "https://savannah.nongnu.org/projects/mescc-tools")
>> -    (license gpl3+))))
>> +    (license gpl3+)))

I guess this is equivalent to reverting commit
78ced7975b0665e810834391d826c9f0ef7277e1 on the 'wip-binaries' branch,
yes?

>> And got this result:
>>
>> $ cd /gnu/store/rdwyr8mh7dvhfkb5g4cws6q40hp23rbi-bootstrap-tarballs-0
>> $ sha256sum *
>> 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b  guile-static-stripped-2.2.4-i686-linux.tar.xz
>> 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a  linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>> 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330  mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz
>> fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0  mes-minimal-stripped-0.19-i686-linux.tar.xz
>> c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7  static-binaries-0-i686-linux.tar.xz

Great, looks good to me!

>> I also merged the branch to core-updates and reverted the bash patch,

I think that 'bash-4.4-linux-pgrp-pipe.patch', adapted to 5.0, should be
applied to all 'bash' packages in 'core-updates'.  To be more precise,
the patch should be applied to the main 'bash' package in bash.scm,
inherited from all other bash packages, so that PGRP_PIPE is set
unconditionally set on systems based on Linux (the kernel), regardless
of the kernel version running on the build machine.

>> which produced this derivation for "guix build -d -s i686-linux
>> bootstrap-tarballs":
>>
>> /gnu/store/ld82vli1msfrlimjaryznrqcwm0jc5ii-bootstrap-tarballs-0.drv
>
> Here are the hashes from this derivation:
>
> $ cd /gnu/store/srsqilb3g70r8c7vma0gpam21z1zmg2w-bootstrap-tarballs-0
> $ sha256sum *
> 4e5b219be4d9ad4d125f17b8a8a991e78be3908aadc8d22d1a115e96ec856279  guile-static-stripped-2.2.6-i686-linux.tar.xz
> 9e34b26526b184912b0d97df96e316151fd97ede24aec2f22dd13ed33438b2e4  linux-libre-headers-stripped-4.19.56-i686-linux.tar.xz
> 7db07a7097a7920e17f6f1794098b9f29bc3522c36a33cf797123f64581cdc34  mescc-tools-static-0.5.2-i686-linux.tar.xz
> 3a9b050c1a2b61bb1cbed16ee07aecdc83e9097ce4a914623aa1b9808abe57f4  mes-minimal-stripped-0.19-i686-linux.tar.xz
> 1ccb4f39656eb977706db7a4aa811461785200b0a11e847035bb89fef91b85ab  static-binaries-0-i686-linux.tar.xz

What was the purpose of building these?  'core-updates' is built upon on
the earlier, unverified, reduced binary seed bootstrap binaries.

> @Mark: do you think we are ready to merge this now?  Can you do it?

I think what needs to be done is the following:

(1) commit 78ced7975b0665e810834391d826c9f0ef7277e1 on 'wip-binaries'
    should be reverted, to downgrade mescc-tools to the 0.5.2 release.

(2) The 'wip-binaries' tarballs should be uploaded to a new subdirectory
    of <https://alpha.gnu.org/gnu/guix/bootstrap/i686-linux/>, along
    with digital signatures, of course.  I'm talking about these in
    particular:

    3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b  guile-static-stripped-2.2.4-i686-linux.tar.xz
    1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a  linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
    021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330  mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz
    fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0  mes-minimal-stripped-0.19-i686-linux.tar.xz
    c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7  static-binaries-0-i686-linux.tar.xz

(3) The following bootstrap packages in 'core-updates' bootstrap.scm
    should be updated to use the new binaries above:

     (a) %bootstrap-linux-libre-headers
     (b) %bootstrap-mescc-tools
     (c) %bootstrap-mes

(4) Berlin should start rebuilding 'core-updates'.

If desired, steps (3) and (4) could come before (2) if someone
temporarily uploads the new binaries somewhere else, and adjusts
'%bootstrap-base-urls' accordingly.  The key is for the hashes and file
names to match what we've agreed on here, as I listed in (2) above.

What do you think?

It will be a couple of days before I can do more solid work on this, but
I'd be grateful if others could carry it forward in the meantime.

     Thanks!
       Mark

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.