GNU bug report logs - #28635
[PATCH] Allow configuring the disk image size of a virtual machine

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 28635 in the body.
You can then email your comments to 28635 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: guix-patches <at> gnu.org
Subject: [PATCH] Allow configuring the disk image size of a virtual machine
Date: Thu, 28 Sep 2017 20:54:16 +0100

[Message part 1 (text/plain, inline)]

Allow configuring the size of the virtual machine disk when using the
virtual-matchine record type.

Christopher Baines (2):
  vm: Add a minimum root size.
  vm: Add disk-image-size to <virtual-machine>.

 gnu/system/vm.scm | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

[Message part 2 (application/pgp-signature, inline)]

Message #8 received at 28635 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28635 <at> debbugs.gnu.org
Subject: [PATCH 1/2] vm: Add a minimum root size.
Date: Thu, 28 Sep 2017 20:57:53 +0100

* gnu/system/vm.scm (qemu-image): When guessing the root-size, use a lower
  bound of 20 MiB, otherwise the root file system size is sometimes 0 MiB in
  size.
---
 gnu/system/vm.scm | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index 78143e4f7..d340a8563 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -304,9 +304,12 @@ the image."
                                #:register-closures? #$register-closures?
                                #:system-directory #$os-drv))
                   (root-size  #$(if (eq? 'guess disk-image-size)
-                                    #~(estimated-partition-size
-                                       (map (cut string-append "/xchg/" <>)
-                                            graphs))
+                                    #~(max
+                                       ;; Minimum 20 MiB root size
+                                       (* 20 (expt 2 20))
+                                       (estimated-partition-size
+                                        (map (cut string-append "/xchg/" <>)
+                                             graphs)))
                                     (- disk-image-size
                                        (* 50 (expt 2 20)))))
                   (partitions (list (partition
-- 
2.14.1

Message #11 received at 28635 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28635 <at> debbugs.gnu.org
Subject: [PATCH 2/2] vm: Add disk-image-size to <virtual-machine>.
Date: Thu, 28 Sep 2017 20:57:54 +0100

* gnu/system/vm.scm (<virtual-machine>): Add disk-image-size.
  (port-forwardings->qemu-options): Use disk-image-size from
  <virtual-machine>.
---
 gnu/system/vm.scm | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index d340a8563..2f31a615b 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -709,6 +709,8 @@ it is mostly useful when FULL-BOOT?  is true."
                     (default #f))
   (memory-size      virtual-machine-memory-size   ;integer (MiB)
                     (default 256))
+  (disk-image-size  virtual-machine-disk-image-size   ;integer (bytes)
+                    (default 'guess))
   (port-forwardings virtual-machine-port-forwardings ;list of integer pairs
                     (default '())))
 
@@ -737,12 +739,15 @@ FORWARDINGS is a list of host-port/guest-port pairs."
                                                 system target)
   ;; XXX: SYSTEM and TARGET are ignored.
   (match vm
-    (($ <virtual-machine> os qemu graphic? memory-size ())
+    (($ <virtual-machine> os qemu graphic? disk-image-size memory-size ())
      (system-qemu-image/shared-store-script os
                                             #:qemu qemu
                                             #:graphic? graphic?
-                                            #:memory-size memory-size))
-    (($ <virtual-machine> os qemu graphic? memory-size forwardings)
+                                            #:memory-size memory-size
+                                            #:disk-image-size
+                                            disk-image-size))
+    (($ <virtual-machine> os qemu graphic? memory-size disk-image-size
+                          forwardings)
      (let ((options
             `("-net" ,(string-append
                        "user,"
@@ -751,6 +756,8 @@ FORWARDINGS is a list of host-port/guest-port pairs."
                                               #:qemu qemu
                                               #:graphic? graphic?
                                               #:memory-size memory-size
+                                              #:disk-image-size
+                                              disk-image-size
                                               #:options options)))))
 
 ;;; vm.scm ends here
-- 
2.14.1

Message #14 received at 28635 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28635 <at> debbugs.gnu.org, 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 1/4] vm: Add disk-image-size to
 <virtual-machine>.
Date: Fri, 29 Sep 2017 07:48:21 +0100

[Message part 1 (text/plain, inline)]

On Tue, 26 Sep 2017 09:14:28 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Hi Chris!
> 
> Christopher Baines <mail <at> cbaines.net> skribis:
> 
> > On Thu, 31 Aug 2017 14:29:56 +0200
> > ludo <at> gnu.org (Ludovic Courtès) wrote:
> >  
> >> Hi Chris,  
> >
> > Thanks for reviewing these patches Ludo, unfortunately its taken me
> > nearly a month to get around to replying.
> >
> > Thankfully, I've made some progress in that time.
> >  
> >> Christopher Baines <mail <at> cbaines.net> skribis:
> >>   
> >> > * gnu/system/vm.scm (<virtual-machine>): Add
> >> >   disk-image-size.
> >> >   (port-forwardings->qemu-options): Use disk-image-size from
> >> >   <virtual-machine>.    
> >> 
> >> In which case is it useful?  Perhaps if you want to create lots of
> >> data on the root file system in the MongoDB test?
> >> 
> >> Currently <virtual-machine> builds a shared-store VM (like ‘guix
> >> system vm’) in which the root file system has a fixed size that’s
> >> usually good enough.  
> >
> > The best answer I have at the moment is that I think MongoDB
> > creates a rather large file, even if it has no data to store in it.
> > I'll do some more investigation to confirm this though.  
> 
> OK, that’s a good reason anyway.  :-)
> 
> >> > --- a/gnu/system/vm.scm
> >> > +++ b/gnu/system/vm.scm
> >> > @@ -653,6 +653,8 @@ it is mostly useful when FULL-BOOT?  is
> >> > true." (default #f))
> >> >    (memory-size      virtual-machine-memory-size   ;integer (MiB)
> >> >                      (default 256))
> >> > +  (disk-image-size  virtual-machine-disk-image-size   ;integer
> >> > (bytes)
> >> > +                    (default (* 70 (expt 2 20))))    
> >> 
> >> I think we can use 'guess here as the default value (and we should
> >> do the same in places where #:disk-image-size has an arbitrary
> >> default.)  
> >
> > I've been looking at this in the last few days. Making 'guess the
> > default value here does mean something, and I think it's a good
> > idea.
> >
> > As far as I can tell, for this specific test, these are the 3
> > functions that are called on the way to using the actual size, and
> > the defaults they have:
> >
> > system-qemu-image/shared-store-script default:
> >   (* (if full-boot? 500 70)
> >      (expt 2 20)))
> >
> > system-qemu-image/shared-store default: 
> >   (* (if full-boot? 500 30) 
> >      (expt 2 20)))
> >
> > qemu-image default:
> >   'guess
> >
> > From my tests, if the default in the <virtual-machine> is set to
> > 'guess, then for the mongodb test the guess seems to be 0 MiB at the
> > moment, which doesn't work. I've started looking at setting a
> > sensible default in qemu-image, so that the root filesystem size
> > isn't 0 MiB.  
> 
> Indeed.  Maybe we can go with your patch as-is and investigate the
> problem with 'guess separately.  Thoughts?

That's fine with me. I've also now created a separate bug with this
patch, and another to add a default value [1].

I tested it with the memcached system test, and the disk image size was
the same plus the test still passed. I haven't checked the other tests
yet.

1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28635

[Message part 2 (application/pgp-signature, inline)]

Message #17 received at 28635 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28635 <at> debbugs.gnu.org
Subject: Re: [bug#28635] [PATCH 1/2] vm: Add a minimum root size.
Date: Tue, 03 Oct 2017 15:11:16 +0200

Christopher Baines <mail <at> cbaines.net> skribis:

> * gnu/system/vm.scm (qemu-image): When guessing the root-size, use a lower
>   bound of 20 MiB, otherwise the root file system size is sometimes 0 MiB in
>   size.

LGTM, thanks!

Ludo’.

Message #20 received at 28635 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28635 <at> debbugs.gnu.org
Subject: Re: [bug#28635] [PATCH 2/2] vm: Add disk-image-size to
 <virtual-machine>.
Date: Tue, 03 Oct 2017 15:12:04 +0200

Christopher Baines <mail <at> cbaines.net> skribis:

> * gnu/system/vm.scm (<virtual-machine>): Add disk-image-size.
>   (port-forwardings->qemu-options): Use disk-image-size from
>   <virtual-machine>.

As long as “make check-system TESTS=basic” passes, fine with me!

Thanks,
Ludo’.

Message #25 received at 28635-done <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28635-done <at> debbugs.gnu.org
Subject: Re: [bug#28635] [PATCH 2/2] vm: Add disk-image-size to
 <virtual-machine>.
Date: Wed, 4 Oct 2017 07:25:36 +0100

[Message part 1 (text/plain, inline)]

On Tue, 03 Oct 2017 15:12:04 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Christopher Baines <mail <at> cbaines.net> skribis:
> 
> > * gnu/system/vm.scm (<virtual-machine>): Add disk-image-size.
> >   (port-forwardings->qemu-options): Use disk-image-size from
> >   <virtual-machine>.  
> 
> As long as “make check-system TESTS=basic” passes, fine with me!

It failed at first, as I had a the memory-size and disk-image-size
fields the wrong way around in a match statement, but after I fixed
that it worked fine.

I've now pushed these changes, thanks for your review :)

[Message part 2 (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.