GNU bug report logs - #30708
[PATCH] utils: Add helper method to list subdirectories.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 30708 in the body.
You can then email your comments to 30708 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: guix-patches <guix-patches <at> gnu.org>
Subject: [PATCH] utils: Add helper method to list subdirectories.
Date: Sun, 04 Mar 2018 23:15:03 -0500

[Message part 1 (text/plain, inline)]

Hello Guix!

This adds a method useful to list subdirectories, which I am using to
list bundled copies of libraries (and delete them), for example.

Thank you,

Maxim

[0001-utils-Add-helper-method-to-list-subdirectories.patch (text/x-patch, attachment)]

Message #8 received at 30708 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 30708 <at> debbugs.gnu.org
Subject: Re: [bug#30708] [PATCH] utils: Add helper method to list
 subdirectories.
Date: Mon, 05 Mar 2018 18:12:40 +0100

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:

> From b4b607800d770c4cf77f92c247276c368357e94f Mon Sep 17 00:00:00 2001
> From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
> Date: Sun, 25 Feb 2018 17:49:06 -0500
> Subject: [PATCH] utils: Add helper method to list subdirectories.
>
> * guix/build/utils.scm (find-subdirectories): New procedure.
> * tests/build-utils.scm: Rename module so that it can be used with Geiser.
> (%test-dir-hierarchy): New variable.
> (make-test-dir-hierarchy): New test procedure.
> ("find-subdirectories"): New test.

[...]

> +(define* (find-subdirectories dir #:key fail-on-error?)
> +  "Return the list of the immediate subdirectories of DIR."
> +  ;; Strip the trailing '/' DIR is '/'.
> +  (let ((dir (if (and (> 1 (string-length dir))
> +                      (eq? (string-take-right dir 1) #\/))
> +                 (string-drop-right dir 1)
> +                 dir)))
> +    (define (pred filename stat)
> +      (and (eq? (stat:type stat) 'directory)
> +           (string-match (string-append dir "/[^/]*$") filename)))
> +    (find-files dir pred
> +                #:directories? #t
> +                #:fail-on-error? fail-on-error?)))

‘find-files’ recurses in subdirectories, so the above implementation is
not as efficient as it could be.

I would instead suggest using ‘scandir’ (or ‘file-system-fold’) from
Guile’s (ice-9 ftw) module.

That said… is this a common enough operation?

Thanks,
Ludo’.

Message #11 received at 30708 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 30708 <at> debbugs.gnu.org
Subject: Re: [bug#30708] [PATCH] utils: Add helper method to list
 subdirectories.
Date: Mon, 05 Mar 2018 21:18:12 -0500

Hi Ludovic,

ludo <at> gnu.org (Ludovic Courtès) writes:

> Hi Maxim,
>
> Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:
>
>> From b4b607800d770c4cf77f92c247276c368357e94f Mon Sep 17 00:00:00 2001
>> From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
>> Date: Sun, 25 Feb 2018 17:49:06 -0500
>> Subject: [PATCH] utils: Add helper method to list subdirectories.
>>
>> * guix/build/utils.scm (find-subdirectories): New procedure.
>> * tests/build-utils.scm: Rename module so that it can be used with Geiser.
>> (%test-dir-hierarchy): New variable.
>> (make-test-dir-hierarchy): New test procedure.
>> ("find-subdirectories"): New test.
>
> [...]
>
>> +(define* (find-subdirectories dir #:key fail-on-error?)
>> +  "Return the list of the immediate subdirectories of DIR."
>> +  ;; Strip the trailing '/' DIR is '/'.
>> +  (let ((dir (if (and (> 1 (string-length dir))
>> +                      (eq? (string-take-right dir 1) #\/))
>> +                 (string-drop-right dir 1)
>> +                 dir)))
>> +    (define (pred filename stat)
>> +      (and (eq? (stat:type stat) 'directory)
>> +           (string-match (string-append dir "/[^/]*$") filename)))
>> +    (find-files dir pred
>> +                #:directories? #t
>> +                #:fail-on-error? fail-on-error?)))
>
> ‘find-files’ recurses in subdirectories, so the above implementation is
> not as efficient as it could be.
>
> I would instead suggest using ‘scandir’ (or ‘file-system-fold’) from
> Guile’s (ice-9 ftw) module.

Thanks! See the new patched attached. The test still passes.

> That said… is this a common enough operation?

I'm using it in a forthcoming new Guix package (SuperCollider) where it
allows me to explicitly list the bundled dependencies that are to be
*kept* rather than the ones to be removed, as is more commonly done. Without a
list of the subdirectories the contrib/vendor/whatever bundled
libraries directory I would not be able to do the following:

--8<---------------cut here---------------start------------->8---
+             ;; The build system doesn't allow us to unbundle the
+             ;; following libraries.
+             (let* ((all-dirs (find-subdirectories "./external_libraries"))
+                    (keep-dirs '("nova-simd" "nova-tt" "hidapi" "TLSF-2.4.6"
+                                 "oscpack_1_1_0"))
+                    (remove-dirs
+                     (remove (lambda (x)
+                               (member (basename x) keep-dirs))
+                             all-dirs)))
+               (format #t "Removing bundled libraries: ~s\n" remove-dirs)
+               (for-each delete-file-recursively remove-dirs)))))))
--8<---------------cut here---------------end--------------->8---

Although now that you've made me see the light (scandir), I could
rewrite the whole thing using:

--8<---------------cut here---------------start------------->8---
          (lambda _
             ;; The build system doesn't allow us to unbundle the following
             ;; libraries.
             (let ((keep-dirs '("nova-simd" "nova-tt" "hidapi" "TLSF-2.4.6"
                                "oscpack_1_1_0" "." "..")))
               (with-directory-excursion "./external_libraries"
                 (for-each
                  delete-file-recursively
                  (scandir "."
                           (lambda (x)
                             (and (eq? (stat:type (stat x)) 'directory)
                                  (not (member (basename x) keep-dirs))))))))
--8<---------------cut here---------------end--------------->8---

So, this patch can go to the recycle bin. Thanks! :)

Maxim

Message #16 received at 30708-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 30708-done <at> debbugs.gnu.org
Subject: Re: [bug#30708] [PATCH] utils: Add helper method to list
 subdirectories.
Date: Tue, 06 Mar 2018 11:33:44 +0100

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:

> Although now that you've made me see the light (scandir), I could
> rewrite the whole thing using:
>
>           (lambda _
>              ;; The build system doesn't allow us to unbundle the following
>              ;; libraries.
>              (let ((keep-dirs '("nova-simd" "nova-tt" "hidapi" "TLSF-2.4.6"
>                                 "oscpack_1_1_0" "." "..")))
>                (with-directory-excursion "./external_libraries"
>                  (for-each
>                   delete-file-recursively
>                   (scandir "."
>                            (lambda (x)
>                              (and (eq? (stat:type (stat x)) 'directory)
>                                   (not (member (basename x) keep-dirs))))))))
>
> So, this patch can go to the recycle bin. Thanks! :)

Well, I’m glad that it works for you.  :-)

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.