GNU bug report logs - #29100
[PATCH 0/3] Use overlayfs instead of unionfs.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 29100 in the body.
You can then email your comments to 29100 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
To: guix-patches <at> gnu.org
Subject: [PATCH 0/3] Use overlayfs instead of unionfs.
Date: Wed,  1 Nov 2017 09:33:07 +0100

These patches replace the FUSE-based unionfs by the kernel-based overlayfs.
For the insallation image this reduces the size of the initrd by ca. 4.3%
(487K).

I verified this is working by successfiully running
   make check-system TESTS=installed-os.

One thing I did not change: In gnu/system/install.scm
(installation-os)[file-systems], /tmp is still mounted to tmpfs.  This was
introduced in ee03b75dfb3399f41002c38ac512473ab94afa74 for unionfs because
FUSE creates .fuse_hiddenXYZ' files for each open file, and this confuses
Guix's test suite, for instance (see <http://bugs.gnu.org/23056>).  This
should no longer be necessary when using overlayfs, since overlayfs uses a
separate workdir.  However I was not able to trigger bug 23056 while still
using unionfs, so I could not verify whether this mount is still needed with
overlayfs and I decided to keep it.

Hartmut Goebel (3):
  build: Use overlayfs instead of unionfs.
  install: Use overlayfs instead of unionfs.
  gnu: service: Update comment.

 gnu/build/linux-boot.scm    | 43 ++++++++++++---------------------------
 gnu/services/base.scm       |  2 +-
 gnu/system/install.scm      | 49 ++++++++++++++++++---------------------------
 gnu/system/linux-initrd.scm |  4 ++--
 4 files changed, 35 insertions(+), 63 deletions(-)

-- 
2.13.5

Message #8 received at 29100 <at> debbugs.gnu.org (full text, mbox):

From: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
To: 29100 <at> debbugs.gnu.org
Subject: [PATCH 1/3] build: Use overlayfs instead of unionfs.
Date: Wed,  1 Nov 2017 09:39:50 +0100

Overlayfs is part of the kernel, while unionfs needs FUSE.  This also reduces
the size of the initrd by ca. 4.3% (487K).

* gnu/build/linux-boot.scm (mount-root-file-system): Remove optional parameter
  "unionfs"; mount using overlayfs instead of unionfs; new directory layout
  requied by overlayfs; update documentation;
  [mark-as-not-killable]: remove now unused function

* gnu/system/linux-intrd.scm (file-system-packages): Remove now unused
  packages "unionfs-fuse/static".
  (linux-modules): Replace "fuse" by "overlay".
---
 gnu/build/linux-boot.scm    | 43 +++++++++++++------------------------------
 gnu/system/linux-initrd.scm |  4 ++--
 2 files changed, 15 insertions(+), 32 deletions(-)

diff --git a/gnu/build/linux-boot.scm b/gnu/build/linux-boot.scm
index 3712abe91..7f07e8038 100644
--- a/gnu/build/linux-boot.scm
+++ b/gnu/build/linux-boot.scm
@@ -241,20 +241,10 @@ the last argument of `mknod'."
           (filter-map string->number (scandir "/proc")))))
 
 (define* (mount-root-file-system root type
-                                 #:key volatile-root? (unionfs "unionfs"))
+                                 #:key volatile-root?)
   "Mount the root file system of type TYPE at device ROOT.  If VOLATILE-ROOT?
-is true, mount ROOT read-only and make it a union with a writable tmpfs using
-UNIONFS."
-  (define (mark-as-not-killable pid)
-    ;; Tell the 'user-processes' shepherd service that PID must be kept alive
-    ;; when shutting down.
-    (mkdir-p "/root/etc/shepherd")
-    (let ((port (open-file "/root/etc/shepherd/do-not-kill" "a")))
-      (chmod port #o600)
-      (write pid port)
-      (newline port)
-      (close-port port)))
-
+is true, mount ROOT read-only and make it a overlay with a writable tmpfs
+using the kernel build-in overlayfs."
   (if volatile-root?
       (begin
         (mkdir-p "/real-root")
@@ -262,24 +252,17 @@ UNIONFS."
         (mkdir-p "/rw-root")
         (mount "none" "/rw-root" "tmpfs")
 
+        ;; Create the upperdir and the workdir of the overlayfs
+        (mkdir-p "/rw-root/upper")
+        (mkdir-p "/rw-root/work")
+
         ;; We want read-write /dev nodes.
-        (mkdir-p "/rw-root/dev")
-        (mount "none" "/rw-root/dev" "devtmpfs")
-
-        ;; Make /root a union of the tmpfs and the actual root.  Use
-        ;; 'max_files' to set a high RLIMIT_NOFILE for the unionfs process
-        ;; itself.  Failing to do that, we quickly run out of file
-        ;; descriptors; see <http://bugs.gnu.org/17827>.
-        (unless (zero? (system* unionfs "-o"
-                                "cow,allow_other,use_ino,suid,dev,max_files=65536"
-                                "/rw-root=RW:/real-root=RO"
-                                "/root"))
-          (error "unionfs failed"))
-
-        ;; Make sure unionfs remains alive till the end.  Because
-        ;; 'fuse_daemonize' doesn't tell the PID of the forked daemon, we
-        ;; have to resort to 'pidof' here.
-        (mark-as-not-killable (pidof unionfs)))
+        (mkdir-p "/rw-root/upper/dev")
+        (mount "none" "/rw-root/upper/dev" "devtmpfs")
+
+        ;; Make /root a overlay of the tmpfs and the actual root.
+        (mount "none" "/root" "overlay" 0
+               "lowerdir=/real-root,upperdir=/rw-root/upper,workdir=/rw-root/work"))
       (begin
         (check-file-system root type)
         (mount root "/root" type)))
diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm
index 948c543a1..4168582c4 100644
--- a/gnu/system/linux-initrd.scm
+++ b/gnu/system/linux-initrd.scm
@@ -236,7 +236,7 @@ FILE-SYSTEMS."
           (list btrfs-progs/static)
           '())
     ,@(if volatile-root?
-          (list unionfs-fuse/static)
+          (list ) ;; nothing requierd in this case
           '())))
 
 (define* (base-initrd file-systems
@@ -308,7 +308,7 @@ loaded at boot time in the order in which they appear."
             '("isofs")
             '())
       ,@(if volatile-root?
-            '("fuse")
+            '("overlay")
             '())
       ,@extra-modules))
 
-- 
2.13.5

Message #11 received at 29100 <at> debbugs.gnu.org (full text, mbox):

From: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
To: 29100 <at> debbugs.gnu.org
Subject: [PATCH 2/3] install: Use overlayfs instead of unionfs.
Date: Wed,  1 Nov 2017 09:39:51 +0100

I dediced to keep adding /tmp as tmpfs since I was not able to trigger bug
while still using unionfs, so I could not verify whether this mount is still
needed with overlayfs.  Mapping /tmp to tmpfs does not harm, so we are on the
save side.

* gnu/system/install.scm (make-cow-store): Mount /gnu/store without additional
  read-only bind-mount, since in overlayfs the "lower" level is always
  read-only. Add work-dir required by overlayfs. No need to sleep anymore
  since now using the mount syscall. [unionfs]: remove now unused function.
  (%installation-services): Update comment.
  (installation-os)[file-systems]: Update comment.
---
 gnu/system/install.scm | 49 +++++++++++++++++++------------------------------
 1 file changed, 19 insertions(+), 30 deletions(-)

diff --git a/gnu/system/install.scm b/gnu/system/install.scm
index eb362f91a..0c771e81f 100644
--- a/gnu/system/install.scm
+++ b/gnu/system/install.scm
@@ -71,19 +71,6 @@ manual."
   "Return a gexp that makes the store copy-on-write, using TARGET as the
 backing store.  This is useful when TARGET is on a hard disk, whereas the
 current store is on a RAM disk."
-  (define (unionfs read-only read-write mount-point)
-    ;; Make MOUNT-POINT the union of READ-ONLY and READ-WRITE.
-
-    ;; Note: in the command below, READ-WRITE appears before READ-ONLY so that
-    ;; it is considered a "higher-level branch", as per unionfs-fuse(8),
-    ;; thereby allowing files existing on READ-ONLY to be copied over to
-    ;; READ-WRITE.
-    #~(fork+exec-command
-       (list (string-append #$unionfs-fuse "/bin/unionfs")
-             "-o"
-             "cow,allow_other,use_ino,max_files=65536,nonempty"
-             (string-append #$read-write "=RW:" #$read-only "=RO")
-             #$mount-point)))
 
   (define (set-store-permissions directory)
     ;; Set the right perms on DIRECTORY to use it as the store.
@@ -97,23 +84,23 @@ current store is on a RAM disk."
         (mkdir-p tmpdir)
         (mount tmpdir "/tmp" "none" MS_BIND))
 
-      (unless (file-exists? "/.ro-store")
-        (mkdir "/.ro-store")
-        (mount #$(%store-prefix) "/.ro-store" "none"
-               (logior MS_BIND MS_RDONLY)))
-
-      (let ((rw-dir (string-append target #$%backing-directory)))
+      (let* ((rw-dir (string-append target #$%backing-directory))
+             ;; FIXME: calculate work-dir from backing-directory:
+             ;; normpath(backing-directory + "../.overlayfs-workdir")
+             (work-dir (string-append target "/tmp/.overlayfs-workdir")))
         (mkdir-p rw-dir)
+        (mkdir-p work-dir)
         (mkdir-p "/.rw-store")
         #$(set-store-permissions #~rw-dir)
         #$(set-store-permissions "/.rw-store")
 
-        ;; Mount the union, then atomically make it the store.
-        (and #$(unionfs "/.ro-store" #~rw-dir "/.rw-store")
-             (begin
-               (sleep 1) ;XXX: wait for unionfs to be ready
-               (mount "/.rw-store" #$(%store-prefix) "" MS_MOVE)
-               (rmdir "/.rw-store"))))))
+        ;; Mount the overlay, then atomically make it the store.
+        (mount "none" "/.rw-store" "overlay" 0
+               (string-append "lowerdir=" #$(%store-prefix) ","
+                              "upperdir=" rw-dir ","
+                              "workdir=" work-dir))
+        (mount "/.rw-store" #$(%store-prefix) "" MS_MOVE)
+        (rmdir "/.rw-store"))))
 
 (define cow-store-service-type
   (shepherd-service-type
@@ -278,7 +265,7 @@ You have been warned.  Thanks for being so brave.
                     (allow-empty-passwords? #f)
                     (password-authentication? #t)))
 
-          ;; Since this is running on a USB stick with a unionfs as the root
+          ;; Since this is running on a USB stick with a overlayfs as the root
           ;; file system, use an appropriate cache configuration.
           (nscd-service (nscd-configuration
                          (caches %nscd-minimal-caches)))
@@ -317,10 +304,12 @@ Use Alt-F2 for documentation.
               (title 'label)
               (type "ext4"))
 
-            ;; Make /tmp a tmpfs instead of keeping the unionfs.  This is
-            ;; because FUSE creates '.fuse_hiddenXYZ' files for each open file,
-            ;; and this confuses Guix's test suite, for instance.  See
-            ;; <http://bugs.gnu.org/23056>.
+            ;; Make /tmp a tmpfs instead of keeping the overlayfs.  This
+            ;; originally was used for unionfs because FUSE creates
+            ;; '.fuse_hiddenXYZ' files for each open file, and this confuses
+            ;; Guix's test suite, for instance (see
+            ;; <http://bugs.gnu.org/23056>).  We keep this for overlayfs to be
+            ;; on the save side.
             (file-system
               (mount-point "/tmp")
               (device "none")
-- 
2.13.5

Message #14 received at 29100 <at> debbugs.gnu.org (full text, mbox):

From: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
To: 29100 <at> debbugs.gnu.org
Subject: [PATCH 3/3] gnu: service: Update comment.
Date: Wed,  1 Nov 2017 09:39:52 +0100

* gnu/services/base.scm(guix-activation): Update comment.
---
 gnu/services/base.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index b605614ab..2ba655d94 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -1441,7 +1441,7 @@ failed to register hydra.gnu.org public key: ~a~%" status))))))))
   (match config
     (($ <guix-configuration> guix build-group build-accounts authorize-key? keys)
      ;; Assume that the store has BUILD-GROUP as its group.  We could
-     ;; otherwise call 'chown' here, but the problem is that on a COW unionfs,
+     ;; otherwise call 'chown' here, but the problem is that on a COW overlayfs,
      ;; chown leads to an entire copy of the tree, which is a bad idea.
 
      ;; Optionally authorize hydra.gnu.org's key.
-- 
2.13.5

Message #17 received at 29100 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
Cc: 29100 <at> debbugs.gnu.org
Subject: Re: [bug#29100] [PATCH 2/3] install: Use overlayfs instead of unionfs.
Date: Thu, 2 Nov 2017 18:44:05 +0100

[...]
> -      (let ((rw-dir (string-append target #$%backing-directory)))
> +      (let* ((rw-dir (string-append target #$%backing-directory))
> +             ;; FIXME: calculate work-dir from backing-directory:
> +             ;; normpath(backing-directory + "../.overlayfs-workdir")
> +             (work-dir (string-append target "/tmp/.overlayfs-workdir")))

What about the FIXME?

Message #20 received at 29100 <at> debbugs.gnu.org (full text, mbox):

From: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 29100 <at> debbugs.gnu.org
Subject: Re: [bug#29100] [PATCH 2/3] install: Use overlayfs instead of unionfs.
Date: Thu, 2 Nov 2017 19:20:51 +0100

Am 02.11.2017 um 18:44 schrieb Danny Milosavljevic:
> [...]
>> -      (let ((rw-dir (string-append target #$%backing-directory)))
>> +      (let* ((rw-dir (string-append target #$%backing-directory))
>> +             ;; FIXME: calculate work-dir from backing-directory:
>> +             ;; normpath(backing-directory + "../.overlayfs-workdir")
>> +             (work-dir (string-append target "/tmp/.overlayfs-workdir")))
> What about the FIXME?

I'll change it into a TODO. Somebody with more knowledge in scheme as to
implement this.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel <at> crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Message #23 received at 29100 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>, 29100 <at> debbugs.gnu.org
Subject: Re: [bug#29100] [PATCH 2/3] install: Use overlayfs instead of unionfs.
Date: Sun, 05 Nov 2017 22:05:52 +0100

Hi Hartmut,

Hartmut Goebel <h.goebel <at> crazy-compilers.com> skribis:

> Am 02.11.2017 um 18:44 schrieb Danny Milosavljevic:
>> [...]
>>> -      (let ((rw-dir (string-append target #$%backing-directory)))
>>> +      (let* ((rw-dir (string-append target #$%backing-directory))
>>> +             ;; FIXME: calculate work-dir from backing-directory:
>>> +             ;; normpath(backing-directory + "../.overlayfs-workdir")
>>> +             (work-dir (string-append target "/tmp/.overlayfs-workdir")))
>> What about the FIXME?
>
> I'll change it into a TODO. Somebody with more knowledge in scheme as to
> implement this.

What does this comment mean exactly?  The “has to” suggests that it’s
important.  :-)

Perhaps you mean:

  (canonicalize-path (string-append rw-dir "/../.overlayfs-workdir"))

?

Ludo’.

Message #26 received at 29100 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
Cc: 29100 <at> debbugs.gnu.org
Subject: Re: [bug#29100] [PATCH 1/3] build: Use overlayfs instead of unionfs.
Date: Sun, 05 Nov 2017 22:08:49 +0100

Heya,

Hartmut Goebel <h.goebel <at> crazy-compilers.com> skribis:

> Overlayfs is part of the kernel, while unionfs needs FUSE.  This also reduces
> the size of the initrd by ca. 4.3% (487K).

Awesome!  It’s definitely an improvement, and I’m happy that it fixes
<http://bugs.gnu.org/23056>.

> * gnu/build/linux-boot.scm (mount-root-file-system): Remove optional parameter
>   "unionfs"; mount using overlayfs instead of unionfs; new directory layout
>   requied by overlayfs; update documentation;
>   [mark-as-not-killable]: remove now unused function
>
> * gnu/system/linux-intrd.scm (file-system-packages): Remove now unused
                     ^^^
Typo.

>   packages "unionfs-fuse/static".
>   (linux-modules): Replace "fuse" by "overlay".

Nitpick: please capitalize sentences, add missing periods.

> +        ;; Make /root a overlay of the tmpfs and the actual root.

s/a/an/

> --- a/gnu/system/linux-initrd.scm
> +++ b/gnu/system/linux-initrd.scm
> @@ -236,7 +236,7 @@ FILE-SYSTEMS."
>            (list btrfs-progs/static)
>            '())
>      ,@(if volatile-root?
> -          (list unionfs-fuse/static)
> +          (list ) ;; nothing requierd in this case
>            '())))

You can remove the whole ‘if’ here.

OK with these changes, thank you!

Ludo’.

Message #29 received at 29100 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
Cc: 29100 <at> debbugs.gnu.org
Subject: Re: [bug#29100] [PATCH 3/3] gnu: service: Update comment.
Date: Sun, 05 Nov 2017 22:09:20 +0100

Hartmut Goebel <h.goebel <at> crazy-compilers.com> skribis:

> * gnu/services/base.scm(guix-activation): Update comment.
                         ^
Missing space.

Otherwise OK!

Ludo’.

Message #32 received at 29100 <at> debbugs.gnu.org (full text, mbox):

From: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>, 29100 <at> debbugs.gnu.org
Subject: Re: [bug#29100] [PATCH 2/3] install: Use overlayfs instead of unionfs.
Date: Mon, 6 Nov 2017 10:33:23 +0100

Am 05.11.2017 um 22:05 schrieb Ludovic Courtès:
> Perhaps you mean:
>
>   (canonicalize-path (string-append rw-dir "/../.overlayfs-workdir"))

Exactly :-) Thanks.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel <at> crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Message #35 received at 29100 <at> debbugs.gnu.org (full text, mbox):

From: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 29100 <at> debbugs.gnu.org
Subject: Re: [bug#29100] [PATCH 1/3] build: Use overlayfs instead of unionfs.
Date: Mon, 6 Nov 2017 10:40:34 +0100

Am 05.11.2017 um 22:08 schrieb Ludovic Courtès:
>>      ,@(if volatile-root?
>> -          (list unionfs-fuse/static)
>> +          (list ) ;; nothing requierd in this case
>>            '())))
> You can remove the whole ‘if’ here.

Okay. should I remove the then unused #:key volatile-root?, too? This
would change the function's signature and may require more changes at
other places.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel <at> crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Message #38 received at 29100 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
Cc: 29100 <at> debbugs.gnu.org
Subject: Re: [bug#29100] [PATCH 1/3] build: Use overlayfs instead of unionfs.
Date: Mon, 06 Nov 2017 11:30:06 +0100

Hartmut Goebel <h.goebel <at> crazy-compilers.com> skribis:

> Am 05.11.2017 um 22:08 schrieb Ludovic Courtès:
>>>      ,@(if volatile-root?
>>> -          (list unionfs-fuse/static)
>>> +          (list ) ;; nothing requierd in this case
>>>            '())))
>> You can remove the whole ‘if’ here.
>
> Okay. should I remove the then unused #:key volatile-root?, too?

No, I think you can keep it (and also the overlay.ko is added
conditionally based on that IIRC.)

Thanks,
Ludo’.

Message #43 received at 29100-close <at> debbugs.gnu.org (full text, mbox):

From: Hartmut Goebel <h.goebel <at> crazy-compilers.com>
To: 29100-close <at> debbugs.gnu.org
Subject: Updated, tested again and merged
Date: Wed, 8 Nov 2017 18:26:42 +0100

Addressed all concerns and merged as
0af94ad52f65a5a62febd203e431f17642f07577.

I re-testet it, based on cd5c3979d (@master) using make check-system
TESTS=installed-os

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.