Package: guix-patches;
Reported by: Jan Nieuwenhuizen <janneke <at> gnu.org>
Date: Tue, 18 Feb 2020 12:45:02 UTC
Severity: normal
Tags: patch
Done: Jan Nieuwenhuizen <janneke <at> gnu.org>
Bug is archived. No further changes may be made.
Message #11 received at 39655 <at> debbugs.gnu.org (full text, mbox):
From: Jan Nieuwenhuizen <janneke <at> gnu.org> To: Ludovic Courtès <ludo <at> gnu.org> Cc: Timothy Sample <samplet <at> ngyro.com>, 39655 <at> debbugs.gnu.org Subject: Re: [bug#39655] [PATCH core-updates] doc: Add 'Scheme-only Bootstrap' node. Date: Sun, 17 May 2020 14:43:05 +0200
[Message part 1 (text/plain, inline)]
Ludovic Courtès writes:
Hello Ludo, Timothy!
Finally another version of the new bootstrap situation. Timothy, please
have a look and feel free to change/add some things!
It's also getting time to write a blog post on this, would you like to
help out with that?
>> * doc/guix.texi (@detailmenu): Add Bootstrapping menu.
>> (Scheme-only Bootstrap): New node.
>> * doc/images/gcc-core-mesboot0-scheme-only-graph.dot: New image.
>> * doc/local.mk (DOT_FILES): Add it.
>
> Great! I wonder if we need a new section though. Do you think we could
> just keep it all in a single bootstrapping section that would describe
> the current state of affairs, possibly with a “Bootstrapping History”
> section or similar that would describe the milestones that got us there?
[..]
> That way, the main bootstrapping section would simply describe the
> current status in a standalone fashion.
Thanks for your thoughts on this. I started to rework this by adding a
History section but in the end I removed it again, describing the
current situation while mentioning the gcc-triplet reduction in Guix
v1.1.
>> +@c
>> +packages <at> footnote{@c
>> +gash-boot,
>> +gash-utils-boot,
>> +%bootstrap-mes-rewired,
>> +gzip-mesboot,
>
> I wouldn’t list them all here because (1) it’s hard to understand what
> these names refer to, unless reading commencement.scm is your favorite
> hobby ;-), and (2) it’s bound to become outdated.
>
> WDYT?
Yes, instead of adding more of these, I also removed the previous list.
>> +As Gash and Gash Utils mature, and GNU packages become more
@Timothy: this is possibly touching very lightly on what Gash has
brought us; do you want to share some more here?
>> +bootstrappable (e.g., new releases of GNU Sed will also ship in
[..]
>> +@image{images/gcc-core-mesboot0-scheme-only-graph,6in,,Dependency graph of gcc-core-mesboot0}
>
> It would be nice to have a couple of lines commenting the graph, like:
Sure! Updated version attached.
Greetings,
Janneke
[0001-doc-Update-Bootstrapping-for-further-binary-seed-red.patch (text/x-patch, inline)]
From b1341ecfc77345f1a423bbc66b6255a35613425e Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke <at> gnu.org>
Date: Tue, 18 Feb 2020 11:36:37 +0100
Subject: [PATCH] doc: Update 'Bootstrapping' for further binary seed
reduction.
* doc/images/gcc-core-mesboot0-graph.dot: New image, replacing
* doc/images/gcc-mesboot0-bag-graph.dot: ... remove file.
* doc/local.mk (DOT_FILES): Update for new image file.
* doc/guix.texi (Reduced Binary Seed Bootstrap): Use it in updated description
of further reduction of the trusted computing base.
---
doc/guix.texi | 114 +++++++++++++------------
doc/images/gcc-core-mesboot0-graph.dot | 114 +++++++++++++++++++++++++
doc/local.mk | 2 +-
3 files changed, 175 insertions(+), 55 deletions(-)
create mode 100644 doc/images/gcc-core-mesboot0-graph.dot
diff --git a/doc/guix.texi b/doc/guix.texi
index 90324ce291..833509860c 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -35,7 +35,7 @@ Copyright @copyright{} 2016, 2017, 2018 Chris Marusich@*
Copyright @copyright{} 2016, 2017, 2018, 2019, 2020 Efraim Flashner@*
Copyright @copyright{} 2016 John Darrington@*
Copyright @copyright{} 2016, 2017 Nikita Gillmann@*
-Copyright @copyright{} 2016, 2017, 2018, 2019 Jan Nieuwenhuizen@*
+Copyright @copyright{} 2016, 2017, 2018, 2019, 2020 Jan Nieuwenhuizen@*
Copyright @copyright{} 2016 Julien Lepiller@*
Copyright @copyright{} 2016 Alex ter Weele@*
Copyright @copyright{} 2016, 2017, 2018, 2019 Christopher Baines@*
@@ -28606,10 +28606,8 @@ Binutils, libc, and the other packages mentioned above---the
@dfn{bootstrap binaries}.
These bootstrap binaries are ``taken for granted'', though we can also
-re-create them if needed (more on that later).
-
-For @code{i686-linux} and @code{x86_64-linux} the Guix bootstrap process is
-more elaborate, @pxref{Reduced Binary Seed Bootstrap}.
+re-create them if needed (see @pxref{Preparing to Use the Bootstrap
+Binaries}).
@menu
* Reduced Binary Seed Bootstrap:: A Bootstrap worthy of GNU.
@@ -28625,60 +28623,68 @@ Coreutils, Awk, Findutils, `sed', and `grep' and Guile, GCC, Binutils, and the
GNU C Library (@pxref{Bootstrapping}). Usually, these bootstrap binaries are
``taken for granted.''
-Taking these binaries for granted means that we consider them to be a correct
-and trustworthy `seed' for building the complete system. Therein lies a
-problem: the current combined size of these bootstrap binaries is about 250MB
-(@pxref{Bootstrappable Builds,,, mes, GNU Mes}). Auditing or even inspecting
-these is next to impossible.
+Taking the bootstrap binaries for granted means that we consider them to
+be a correct and trustworthy `seed' for building the complete system.
+Therein lies a problem: the combined size of these bootstrap binaries is
+about 250MB (@pxref{Bootstrappable Builds,,, mes, GNU Mes}). Auditing
+or even inspecting these is next to impossible.
-For @code{i686-linux} and @code{x86_64-linux}, Guix now features a ``Reduced
-Binary Seed'' bootstrap @footnote{We would like to say: ``Full Source
-Bootstrap'' and while we are working towards that goal it would be hyperbole
-to use that term for what we do now.}.
+For @code{i686-linux} and @code{x86_64-linux}, Guix now features a
+``Reduced Binary Seed'' bootstrap @footnote{We would like to say: ``Full
+Source Bootstrap'' and while we are working towards that goal it would
+be hyperbole to use that term for what we do now.}.
The Reduced Binary Seed bootstrap removes the most critical tools---from a
trust perspective---from the bootstrap binaries: GCC, Binutils and the GNU C
Library are replaced by: @code{bootstrap-mescc-tools} (a tiny assembler and
linker) and @code{bootstrap-mes} (a small Scheme Interpreter and a C compiler
-written in Scheme and the Mes C Library, built for TinyCC and for GCC). Using
-these new binary seeds and a new set of
-@c
-packages <at> footnote{@c
-nyacc-boot,
-mes-boot,
-tcc-boot0,
-tcc-boot,
-make-mesboot0,
-diffutils-mesboot,
-binutils-mesboot0,
-gcc-core-mesboot,
-mesboot-headers,
-glibc-mesboot0,
-gcc-mesboot0,
-binutils-mesboot,
-make-mesboot,
-gcc-mesboot1,
-gcc-mesboot1-wrapper,
-glibc-headers-mesboot,
-glibc-mesboot,
-gcc-mesboot,
-and
-gcc-mesboot-wrapper.
-}
-@c
-the ``missing'' Binutils, GCC, and the GNU C Library are built from source.
-From here on the more traditional bootstrap process resumes. This approach
-has reduced the bootstrap binaries in size to about 130MB. Work is ongoing to
-reduce this further. If you are interested, join us on @code{#bootstrappable}
-on the Freenode IRC network.
-
-@c ./pre-inst-env guix graph --type=bag -e '(begin (use-modules (guix packages)) (%current-system "i686-linux") (@@ (gnu packages commencement) gcc-mesboot))' > doc/images/gcc-mesboot-bag-graph.dot
-@c dot -T png doc/images/gcc-mesboot-bag-graph.dot > doc/images/gcc-mesboot-bag-graph.png
-
-Below is the generated dependency graph for @code{gcc-mesboot}, the bootstrap
-compiler used to build the rest of GuixSD.
-
-@image{images/gcc-mesboot-bag-graph,6in,,Dependency graph of the gcc-mesboot}
+written in Scheme and the Mes C Library, built for TinyCC and for GCC).
+
+Using these new binary seeds the ``missing'' Binutils, GCC, and the GNU
+C Library are built from source. From here on the more traditional
+bootstrap process resumes. This approach has reduced the bootstrap
+binaries in size to about 145MB in Guix v1.1.
+
+The next step that Guix has taken is to replace the shell and all its
+utilities with implementations in Guile Scheme, the @emph{Scheme-only
+bootstrap}. Gash (@pxref{Gash,,, gash, The Gash manual}) is a
+POSIX-compatible shell that replaces Bash, and it comes with Gash Utils
+which has minimalist replacements for Awk, the GNU Core Utilities, Grep,
+Gzip, Sed, and Tar. The rest of the bootstrap binary seeds that were
+removed are now built from source.
+
+Building the GNU System from source is currently only possibly by adding
+some historical GNU packages as intermediate steps <at> footnote{Packages
+such as @code{gcc-2.95.3}, @code{binutils-2.14}, @code{glibc-2.2.5},
+@code{gzip-1.2.4}, @code{tar-1.22}, and some others for details see
+@file{gnu/packages/commencement.scm}}. As Gash and Gash Utils mature,
+and GNU packages become more bootstrappable again (e.g., new releases of
+GNU Sed will also ship in @code{tar.gz} format), this set of added
+packages can hopefully be reduced again.
+
+The graph below shows the resulting dependency graph for
+@code{gcc-core-mesboot0}, the bootstrap compiler used for the
+traditional bootstrap of the rest of the Guix System.
+
+@c ./pre-inst-env guix graph -e '(@@ (gnu packages commencement) gcc-core-mesboot0)' | sed -re 's,((bootstrap-mescc-tools|bootstrap-mes|guile-bootstrap).*shape =) box,\1 ellipse,' > doc/images/gcc-core-mesboot0-graph.dot
+@image{images/gcc-core-mesboot0-graph,6in,,Dependency graph of gcc-core-mesboot0}
+
+The only significant binary bootstrap seeds that remain <at> footnote{
+Ignoring the 68KB @code{mescc-tools}; that will be removed later,
+together with @code{mes}.} are a Scheme intepreter and a Scheme
+compiler: GNU Mes and GNU Guile.
+
+This further reduction has brought down the size of the binary seed to
+about 60MB for @code{i686-linux} and @code{x86_64-linux}.
+
+Work is ongoing to remove all binary blobs from our free software
+bootstrap stack, working towards a Full Source Bootstrap. Also ongoing
+is work to bring these bootstraps to the @code{arm-linux} and
+@code{aarch64-linux} architectures and to the Hurd.
+
+If you are interested, join us on @code{#bootstrappable} on the Freenode
+IRC network or discuss on @code{bug-mes@@gnu.org} or
+@code{gash-devel@@nongnu.org}.
@node Preparing to Use the Bootstrap Binaries
@section Preparing to Use the Bootstrap Binaries
@@ -28698,7 +28704,7 @@ guix graph -t derivation \
| dot -Tps > gcc.ps
@end example
-or, for the Reduced Binary Seed bootstrap
+or, for the further Reduced Binary Seed bootstrap
@example
guix graph -t derivation \
diff --git a/doc/images/gcc-core-mesboot0-graph.dot b/doc/images/gcc-core-mesboot0-graph.dot
new file mode 100644
index 0000000000..191582ed4c
--- /dev/null
+++ b/doc/images/gcc-core-mesboot0-graph.dot
@@ -0,0 +1,114 @@
+digraph "Guix package" {
+ "139781154519456" [label = "gcc-core-mesboot0 <at> 2.95.3", shape = box, fontname = Helvetica];
+ "139781154519456" -> "139781154519632" [color = darkgoldenrod];
+ "139781154519456" -> "139781154520688" [color = darkgoldenrod];
+ "139781154519456" -> "139781034971312" [color = darkgoldenrod];
+ "139781154519456" -> "139781154520160" [color = darkgoldenrod];
+ "139781154519456" -> "139781034971664" [color = darkgoldenrod];
+ "139781154519456" -> "139781154519984" [color = darkgoldenrod];
+ "139781154519456" -> "139781154519808" [color = darkgoldenrod];
+ "139781154519456" -> "139781154520512" [color = darkgoldenrod];
+ "139781154519456" -> "139781034971488" [color = darkgoldenrod];
+ "139781154519456" -> "139781034972896" [color = darkgoldenrod];
+ "139781154519456" -> "139781034972368" [color = darkgoldenrod];
+ "139781154519456" -> "139781034973072" [color = darkgoldenrod];
+ "139781154519456" -> "139781131267872" [color = darkgoldenrod];
+ "139781154519632" [label = "binutils-mesboot0 <at> 2.14", shape = box, fontname = Helvetica];
+ "139781154519632" -> "139781154520688" [color = peachpuff4];
+ "139781154519632" -> "139781034971312" [color = peachpuff4];
+ "139781154519632" -> "139781154520160" [color = peachpuff4];
+ "139781154519632" -> "139781034971664" [color = peachpuff4];
+ "139781154519632" -> "139781154519984" [color = peachpuff4];
+ "139781154519632" -> "139781154519808" [color = peachpuff4];
+ "139781154519632" -> "139781154520512" [color = peachpuff4];
+ "139781154519632" -> "139781034971488" [color = peachpuff4];
+ "139781154519632" -> "139781034972896" [color = peachpuff4];
+ "139781154519632" -> "139781034972368" [color = peachpuff4];
+ "139781154519632" -> "139781034973072" [color = peachpuff4];
+ "139781154519632" -> "139781131267872" [color = peachpuff4];
+ "139781154520688" [label = "bash-mesboot0 <at> 2.05b", shape = box, fontname = Helvetica];
+ "139781154520688" -> "139781034971488" [color = magenta];
+ "139781154520688" -> "139781034971840" [color = magenta];
+ "139781154520688" -> "139781034972896" [color = magenta];
+ "139781154520688" -> "139781034972368" [color = magenta];
+ "139781154520688" -> "139781034973072" [color = magenta];
+ "139781154520688" -> "139781131267872" [color = magenta];
+ "139781034971488" [label = "make-mesboot0 <at> 3.80", shape = box, fontname = Helvetica];
+ "139781034971488" -> "139781034971840" [color = dimgrey];
+ "139781034971488" -> "139781034972896" [color = dimgrey];
+ "139781034971488" -> "139781034972368" [color = dimgrey];
+ "139781034971488" -> "139781034973072" [color = dimgrey];
+ "139781034971488" -> "139781131267872" [color = dimgrey];
+ "139781034971840" [label = "tcc-boot0 <at> 0.9.26-1103-g6e62e0e", shape = box, fontname = Helvetica];
+ "139781034971840" -> "139781034972016" [color = magenta];
+ "139781034971840" -> "139781131266816" [color = magenta];
+ "139781034971840" -> "139781034972896" [color = magenta];
+ "139781034971840" -> "139781034972368" [color = magenta];
+ "139781034971840" -> "139781034973072" [color = magenta];
+ "139781034971840" -> "139781131267872" [color = magenta];
+ "139781034972016" [label = "mes-boot <at> 0.22", shape = box, fontname = Helvetica];
+ "139781034972016" -> "139781034972192" [color = cyan3];
+ "139781034972016" -> "139781131266816" [color = cyan3];
+ "139781034972016" -> "139781034972896" [color = cyan3];
+ "139781034972016" -> "139781034972368" [color = cyan3];
+ "139781034972016" -> "139781034973072" [color = cyan3];
+ "139781034972016" -> "139781131267872" [color = cyan3];
+ "139781034972192" [label = "bootstrap-mes-rewired <at> 0.19", shape = ellipse, fontname = Helvetica];
+ "139781034972192" -> "139781131266640" [color = darkseagreen];
+ "139781034972192" -> "139781034972896" [color = darkseagreen];
+ "139781131266640" [label = "bootstrap-mes <at> 0", shape = ellipse, fontname = Helvetica];
+ "139781034972896" [label = "gash-boot <at> 0.2.0", shape = box, fontname = Helvetica];
+ "139781034972896" -> "139781034973072" [color = darkviolet];
+ "139781034972896" -> "139781131267872" [color = darkviolet];
+ "139781034973072" [label = "bootar <at> 1", shape = box, fontname = Helvetica];
+ "139781034973072" -> "139781131267872" [color = darkviolet];
+ "139781131267872" [label = "guile-bootstrap <at> 2.0", shape = ellipse, fontname = Helvetica];
+ "139781131266816" [label = "bootstrap-mescc-tools <at> 0.5.2", shape = ellipse, fontname = Helvetica];
+ "139781034972368" [label = "gash-utils-boot <at> 0.1.0", shape = box, fontname = Helvetica];
+ "139781034972368" -> "139781034973072" [color = darkseagreen];
+ "139781034972368" -> "139781034972896" [color = darkseagreen];
+ "139781034972368" -> "139781131267872" [color = darkseagreen];
+ "139781034971312" [label = "bzip2-mesboot <at> 1.0.8", shape = box, fontname = Helvetica];
+ "139781034971312" -> "139781034971488" [color = cyan3];
+ "139781034971312" -> "139781034971840" [color = cyan3];
+ "139781034971312" -> "139781034972896" [color = cyan3];
+ "139781034971312" -> "139781034972368" [color = cyan3];
+ "139781034971312" -> "139781034973072" [color = cyan3];
+ "139781034971312" -> "139781131267872" [color = cyan3];
+ "139781154520160" [label = "diffutils-mesboot <at> 2.7", shape = box, fontname = Helvetica];
+ "139781154520160" -> "139781034971488" [color = red];
+ "139781154520160" -> "139781034971840" [color = red];
+ "139781154520160" -> "139781034972896" [color = red];
+ "139781154520160" -> "139781034972368" [color = red];
+ "139781154520160" -> "139781034973072" [color = red];
+ "139781154520160" -> "139781131267872" [color = red];
+ "139781034971664" [label = "gzip-mesboot <at> 1.2.4", shape = box, fontname = Helvetica];
+ "139781034971664" -> "139781034971840" [color = red];
+ "139781034971664" -> "139781034972896" [color = red];
+ "139781034971664" -> "139781034972368" [color = red];
+ "139781034971664" -> "139781034973072" [color = red];
+ "139781034971664" -> "139781131267872" [color = red];
+ "139781154519984" [label = "patch-mesboot <at> 2.5.9", shape = box, fontname = Helvetica];
+ "139781154519984" -> "139781034971488" [color = blue];
+ "139781154519984" -> "139781034971840" [color = blue];
+ "139781154519984" -> "139781034972896" [color = blue];
+ "139781154519984" -> "139781034972368" [color = blue];
+ "139781154519984" -> "139781034973072" [color = blue];
+ "139781154519984" -> "139781131267872" [color = blue];
+ "139781154519808" [label = "sed-mesboot0 <at> 1.18", shape = box, fontname = Helvetica];
+ "139781154519808" -> "139781034971488" [color = red];
+ "139781154519808" -> "139781034971840" [color = red];
+ "139781154519808" -> "139781034972896" [color = red];
+ "139781154519808" -> "139781034972368" [color = red];
+ "139781154519808" -> "139781034973072" [color = red];
+ "139781154519808" -> "139781131267872" [color = red];
+ "139781154520512" [label = "tcc-boot <at> 0.9.27", shape = box, fontname = Helvetica];
+ "139781154520512" -> "139781034971312" [color = darkviolet];
+ "139781154520512" -> "139781034971488" [color = darkviolet];
+ "139781154520512" -> "139781034971840" [color = darkviolet];
+ "139781154520512" -> "139781034972896" [color = darkviolet];
+ "139781154520512" -> "139781034972368" [color = darkviolet];
+ "139781154520512" -> "139781034973072" [color = darkviolet];
+ "139781154520512" -> "139781131267872" [color = darkviolet];
+
+}
diff --git a/doc/local.mk b/doc/local.mk
index f8709bb3a4..aca1958edd 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -39,7 +39,7 @@ DOT_FILES = \
%D%/images/bootstrap-packages.dot \
%D%/images/coreutils-graph.dot \
%D%/images/coreutils-bag-graph.dot \
- %D%/images/gcc-mesboot-bag-graph.dot \
+ %D%/images/gcc-core-mesboot0-graph.dot \
%D%/images/service-graph.dot \
%D%/images/shepherd-graph.dot
--
2.26.0
[Message part 3 (text/plain, inline)]
-- Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.