GNU bug report logs - #76081
OCI provisioning service

Previous Next

Full log

View this message in rfc822 format

From: paul <goodoldpaul <at> autistici.org>
To: 76081 <at> debbugs.gnu.org
Subject: [bug#76081] OCI provisioning service
Date: Wed, 5 Feb 2025 22:59:57 +0100

[Message part 1 (text/plain, inline)]

Hi Guix,

for a while now, we have been able to run Docker/OCI images as Shepherd 
services with the oci-container-service-type. This was useful especially 
to run software that is not packaged yet (or sometimes not packageable 
at all, due to JS and other bootstrapping issues). It allows to declare 
a list of oci-container-configuration records which would map to Docker 
backed Shepherd services:

(service oci-container-service-type
         (list
          (oci-container-configuration
           (image "prom/prometheus")
           (network "host")
           (ports
             '(("9000" . "9000")
               ("9090" . "9090"))))
          (oci-container-configuration
           (image "grafana/grafana:10.0.1")
           (network "host")
           (volumes
             '("/var/lib/grafana:/var/lib/grafana")))))

This allows us to have containerized but apparently native services 
running on the Guix System. The downside is that unless some isolation 
mechanism are disabled, or unless the running services are very simple 
and don't have to interact with the world outside them, we lose some of 
the nice virtualization features of containers. Above all is the need to 
have all containers connected to the host network to be able to interact 
with other containers. This is effectively the default behavior of most 
if not all Guix native services but with the downside that usually OCI 
images lack all the provenance information that is typical of Guix 
packages, hence are less trustable than a native Guix package so in some 
cases users do prefer to have them running in an isolated environment.

Another shortcoming of the oci-container-service-type is that it only 
supports Docker as an OCI runtime which must have a running daemon with 
full root privileges to be able to execute containers. Since some weeks 
now, with the help of subids/subgids and unprivileged namespaces, we are 
able to run completely rootless containers with the 
rootless-podman-service-type.

This patch implements a generalization of the 
oci-container-service-type, which consequently is made deprecated.  The 
oci-service-type, in addition to all the features from the 
oci-container-service-type, can now provision OCI networks and volumes:

(service iptables-service-type)
(service rootless-podman-service-type)
(service oci-service-type
         (oci-configuration
          (runtime 'podman)))
(simple-service 'oci-provisioning
                oci-service-type
                (oci-extension
                  (volumes
                    (list
                      (oci-network-configuration (name "grafana"))))
                  (networks
                    (list
                      (oci-network-configuration (name "monitoring"))))
                  (containers
                   (list
                    (oci-container-configuration
                      (image "docker.io/grafana/grafana:10.1.5")
                      (network "host")
                      (volumes
                       `(,( . "/opt/bitnami/grafana/conf/grafana.ini")
                         ("grafana" . "/var/lib/grafana"))))))))

Please mind that this is only an example and it probably won't work on 
your system without some more thought. The oci-service-type currently 
only handles OCI objects creation, the user is supposed to handle state 
once the objects are provsioned. It currently supports two different OCI 
runtimes: Docker and rootless Podman.  Both runtimes are tested to make 
sure provisioned containers can connect to each other through 
provisioned networks and can read/write data with provisioned volumes. 
Compared to the oci-container-service-type I added some utility Shepherd 
actions that are supposed to help with debugging. The above 
configuration would yield the following

$ sudo herd command-line podman-networks
/run/current-system/profile/bin/podman network create monitoring

$ sudo herd command-line podman-volumes
/run/current-system/profile/bin/podman volume create grafana

$ sudo herd doc podman-grafana list-actions

command-line:
  Prints podman-grafana's OCI runtime command line invokation.

pull:
  Pull podman-grafana's image (docker.io/grafana/grafana:10.1.5).

$ sudo herd command-line podman-grafana
/run/current-system/profile/bin/podman run --rm --name podman-grafana --network host -v grafana:/var/lib/grafana -v /gnu/store/yqfvhvf8j4008ykr52zh2dmc1d2mjxih-grafana.ini:/opt/bitnami/grafana/conf/grafana.ini docker.io/bitnami/grafana:10.1.5


At last the Scheme API is thought to facilitate the implementation of a 
Guix Home service in the future (you can find an untested version of 
that at [0]). I tested my changes with:

guix shell -D guix -CPW -- make check-system TESTS="oci-service-docker oci-container oci-service-rootless-podman docker docker-system rootless-podman"


Please let me know your thoughts about this!

Thank you for all your work,

giacomo


[0]: 
https://github.com/fishinthecalculator/gocix/blob/main/modules/oci/home/services/containers.scm

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.