GNU bug report logs - #19540
minor: module path picking up ./././...

Previous Next

Package: guile;

Reported by: Matt Wette <mwette <at> alumni.caltech.edu>

Date: Fri, 9 Jan 2015 04:14:01 UTC

Severity: normal

Done: Andy Wingo <wingo <at> pobox.com>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19540 in the body.
You can then email your comments to 19540 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guile <at> gnu.org:
bug#19540; Package guile. (Fri, 09 Jan 2015 04:14:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Matt Wette <mwette <at> alumni.caltech.edu>:
New bug report received and forwarded. Copy sent to bug-guile <at> gnu.org. (Fri, 09 Jan 2015 04:14:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <mwette <at> alumni.caltech.edu>
To: bug-guile <at> gnu.org
Subject: minor: module path picking up ./././...
Date: Thu, 08 Jan 2015 20:13:18 -0800
[Message part 1 (text/plain, inline)]
guile 2.0.11 on macos (darwin) 10.9.5.

This seems very minor to but I thought I'd report it.  If I ",reload" a module in the current directory it seems to pick up extra "./" parts in the path.

...
;;; compiling ././././././././././././lalr1.scm
;;; compiled /Users/mwette/.cache/guile/ccache/2.0-LE-8-2.0/Users/mwette/proj/scheme/myproj/lalr1/lalr1.scm.go
scheme@(guile-user)> (system "touch lalr1.scm")
$5 = 0
scheme@(guile-user)> ,reload (lalr1)
;;; note: source file ./././././././././././././lalr1.scm
;;;       newer than compiled /Users/mwette/.cache/guile/ccache/2.0-LE-8-2.0/Users/mwette/proj/scheme/myproj/lalr1/lalr1.scm.go
;;; compiling ./././././././././././././lalr1.scm
;;; compiled /Users/mwette/.cache/guile/ccache/2.0-LE-8-2.0/Users/mwette/proj/scheme/myproj/lalr1/lalr1.scm.go
scheme@(guile-user)> (system "touch lalr1.scm")
$6 = 0
scheme@(guile-user)> ,reload (lalr1)
;;; note: source file ././././././././././././././lalr1.scm
;;;       newer than compiled /Users/mwette/.cache/guile/ccache/2.0-LE-8-2.0/Users/mwette/proj/scheme/myproj/lalr1/lalr1.scm.go
;;; compiling ././././././././././././././lalr1.scm
;;; compiled /Users/mwette/.cache/guile/ccache/2.0-LE-8-2.0/Users/mwette/proj/scheme/myproj/lalr1/lalr1.scm.go
scheme@(guile-user)> 

[Message part 2 (text/html, inline)]

Information forwarded to bug-guile <at> gnu.org:
bug#19540; Package guile. (Sat, 10 Jan 2015 22:46:02 GMT) Full text and rfc822 format available.

Message #8 received at 19540 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <mwette <at> alumni.caltech.edu>
To: 19540 <at> debbugs.gnu.org
Subject: repeated ./././ in compiled modules
Date: Sat, 10 Jan 2015 14:45:00 -0800
Some more info:

1) I found that this problem persists across restarts of guile.  I have been debugging a module in current dir and I am seeing the path extend an extra "./" every time I type ",reload (lalr1).

2) My environment includes
   GUILE_LOAD_PATH=.:/Users/mwette/opt/guile

3) If I change the above to the following, the problem goes away (but the following solution not satisfactory IMO).
  GUILE_LOAD_PATH=:/Users/mwette/opt/guile   # dot replaced with blank

Take care,
Matt





Information forwarded to bug-guile <at> gnu.org:
bug#19540; Package guile. (Sun, 18 Jan 2015 16:10:02 GMT) Full text and rfc822 format available.

Message #11 received at 19540 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <mwette <at> alumni.caltech.edu>
To: 19540 <at> debbugs.gnu.org
Subject: also generates problem for debugger tracebacks
Date: Sun, 18 Jan 2015 08:09:20 -0800
I had another problem with the "." in my GUILE_LOAD_PATH, I believe.
If my code encountered an error, the debugger traceback would not list file/line information.



Information forwarded to bug-guile <at> gnu.org:
bug#19540; Package guile. (Mon, 19 Jan 2015 20:29:01 GMT) Full text and rfc822 format available.

Message #14 received at 19540 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Matt Wette <mwette <at> alumni.caltech.edu>
Cc: 19540 <at> debbugs.gnu.org
Subject: Re: bug#19540: repeated ./././ in compiled modules
Date: Mon, 19 Jan 2015 21:28:05 +0100
Matt Wette <mwette <at> alumni.caltech.edu> skribis:

> 1) I found that this problem persists across restarts of guile.  I have been debugging a module in current dir and I am seeing the path extend an extra "./" every time I type ",reload (lalr1).
>
> 2) My environment includes
>    GUILE_LOAD_PATH=.:/Users/mwette/opt/guile

The problem stems from the ‘.’ entry in the search path.  On one hand
this is perfectly valid; on the other hand, it’s usually frowned upon,
because you may end up executing possibly malicious code that just
happens to be in $PWD.

All in all, I recommend using only absolute directory names in the
search paths, which will also solve the initial problem.

Can you confirm?

Thanks,
Ludo’.




Information forwarded to bug-guile <at> gnu.org:
bug#19540; Package guile. (Mon, 19 Jan 2015 22:20:01 GMT) Full text and rfc822 format available.

Message #17 received at 19540 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <mwette <at> alumni.caltech.edu>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 19540 <at> debbugs.gnu.org
Subject: Re: bug#19540: repeated ./././ in compiled modules
Date: Mon, 19 Jan 2015 14:19:22 -0800
[Message part 1 (text/plain, inline)]
Absolute paths work.  But this is really unsatisfactory IMO.  I develop code in modules and do so in many directories.  It would be quite painful to just use absolute paths.

I don't see what the big security thread is.   If really a problem, why does guile allow relative paths?

For comparison, Python will load modules in the current directory.

Nonetheless, I think "." is breaking the traceback with the "./" being added on reload.  When I was using "." I wasn't getting file, line information in tracebacks.  Now, without GUILE_LOAD_PATH set to ":$HOME/opt/guile" I get traceback info for modules in my current directory.

I think the following may be a candidate fix?

code is from guile-2.0.11, in file boot-9.scm, near line 1683:

(define (in-vicinity vicinity file)
  (let ((tail (let ((len (string-length vicinity)))
                (if (zero? len)
                    #f
                    (string-ref vicinity (- len 1))))))
    (string-append vicinity
                   (if (or (not tail) (file-name-separator? tail))
                       ""
                       file-name-separator-string)
                   file)))
;; FIX?
(define (new-in-vicinity vicinity file)
  (let ((tail (let ((len (string-length vicinity)))
                (if (or (zero? len) (string=? "." vicinity))
                    #f
                    (string-ref vicinity (- len 1))))))
    (string-append vicinity
                   (if (or (not tail) (file-name-separator? tail))
                       ""
                       file-name-separator-string)
                   file)))


On Jan 19, 2015, at 12:28 PM, Ludovic Courtès <ludo <at> gnu.org> wrote:

> Matt Wette <mwette <at> alumni.caltech.edu> skribis:
> 
>> 1) I found that this problem persists across restarts of guile.  I have been debugging a module in current dir and I am seeing the path extend an extra "./" every time I type ",reload (lalr1).
>> 
>> 2) My environment includes
>>   GUILE_LOAD_PATH=.:/Users/mwette/opt/guile
> 
> The problem stems from the ‘.’ entry in the search path.  On one hand
> this is perfectly valid; on the other hand, it’s usually frowned upon,
> because you may end up executing possibly malicious code that just
> happens to be in $PWD.
> 
> All in all, I recommend using only absolute directory names in the
> search paths, which will also solve the initial problem.
> 
> Can you confirm?
> 
> Thanks,
> Ludo’.

[Message part 2 (text/html, inline)]

Information forwarded to bug-guile <at> gnu.org:
bug#19540; Package guile. (Tue, 20 Jan 2015 21:13:02 GMT) Full text and rfc822 format available.

Message #20 received at 19540 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Matt Wette <mwette <at> alumni.caltech.edu>
Cc: 19540 <at> debbugs.gnu.org
Subject: Re: bug#19540: repeated ./././ in compiled modules
Date: Tue, 20 Jan 2015 22:12:11 +0100
Matt Wette <mwette <at> alumni.caltech.edu> skribis:

> Absolute paths work.  But this is really unsatisfactory IMO.  I develop code in modules and do so in many directories.  It would be quite painful to just use absolute paths.
>
> I don't see what the big security thread is.

Suppose we’re working on the same machine and I install
/tmp/ice-9/regex.scm (say), which just does:

  (system "rm -rf /")

If I can tweak you into running Guile with /tmp as its current working
directory, you lose.

> If really a problem, why does guile allow relative paths?
>
> For comparison, Python will load modules in the current directory.
>
> Nonetheless, I think "." is breaking the traceback with the "./" being
> added on reload.  When I was using "." I wasn't getting file, line
> information in tracebacks.  Now, without GUILE_LOAD_PATH set to
> ":$HOME/opt/guile" I get traceback info for modules in my current
> directory.

Actually the problem stems from the “file name relative
canonicalization”, the process by which Guile attaches a file name
relative to the search path to an open port:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (add-to-load-path ".")
scheme@(guile-user)> %load-path
$2 = ("." [...])
scheme@(guile-user)> (getcwd)
$3 = "/home/ludo/src/guile/module/ice-9"
scheme@(guile-user)> (open-input-file (search-path %load-path "boot-9.scm"))
$4 = #<input: ./boot-9.scm 11>
scheme@(guile-user)> (open-input-file (search-path %load-path "./boot-9.scm"))
$5 = #<input: ././boot-9.scm 12>
scheme@(guile-user)> (open-input-file (search-path %load-path "././boot-9.scm"))
$6 = #<input: ./././boot-9.scm 13>
--8<---------------cut here---------------end--------------->8---

This could be fixed either in ‘search-path’ or in
‘scm_i_relativize_path’, but the former sounds like a better place.

However, I’m unsure of the effect of changing the result of
‘search-path’ in those cases, so I’d rather leave things unchanged.

Thoughts?

Ludo’.




Reply sent to Andy Wingo <wingo <at> pobox.com>:
You have taken responsibility. (Thu, 23 Jun 2016 08:37:02 GMT) Full text and rfc822 format available.

Notification sent to Matt Wette <mwette <at> alumni.caltech.edu>:
bug acknowledged by developer. (Thu, 23 Jun 2016 08:37:02 GMT) Full text and rfc822 format available.

Message #25 received at 19540-done <at> debbugs.gnu.org (full text, mbox):

From: Andy Wingo <wingo <at> pobox.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 19540-done <at> debbugs.gnu.org, Matt Wette <mwette <at> alumni.caltech.edu>
Subject: Re: bug#19540: repeated ./././ in compiled modules
Date: Thu, 23 Jun 2016 10:36:41 +0200
Hi!

On Tue 20 Jan 2015 22:12, ludo <at> gnu.org (Ludovic Courtès) writes:

> scheme@(guile-user)> (add-to-load-path ".")
> scheme@(guile-user)> %load-path
> $2 = ("." [...])
> scheme@(guile-user)> (getcwd)
> $3 = "/home/ludo/src/guile/module/ice-9"
> scheme@(guile-user)> (open-input-file (search-path %load-path "boot-9.scm"))
> $4 = #<input: ./boot-9.scm 11>
> scheme@(guile-user)> (open-input-file (search-path %load-path "./boot-9.scm"))
> $5 = #<input: ././boot-9.scm 12>
> scheme@(guile-user)> (open-input-file (search-path %load-path "././boot-9.scm"))
> $6 = #<input: ./././boot-9.scm 13>
>
> This could be fixed either in ‘search-path’ or in
> ‘scm_i_relativize_path’, but the former sounds like a better place.

I think the latter is actually better.  `relativize-path' tries to find
an element of %load-path which is a prefix of *the canonicalized version
of* a file name.  We can't expect that to work unless we also
canonicalize the elements of %load-path, for the purposes of
relativizing.  We don't want to canonicalize them eagerly or
persistently, as that would access symlinks at the wrong time; just as
needed inside relativize-path.  I pushed the attached fix to master.  If
it sounds good to you let's push it to stable-2.0 as well.  WDYT?

Andy

commit 9a951678713557b548415d32eae6d63d039bf652
Author: Andy Wingo <wingo <at> pobox.com>
Date:   Thu Jun 23 10:03:10 2016 +0200

    Fix relative file name canonicalization on paths with "."
    
    * libguile/filesys.c (scm_i_relativize_path): Canonicalize the file
      names elements that we will be using as prefixes.  Fixes the case
      where a load path contains a relative file name: #19540.
    * test-suite/tests/ports.test ("%file-port-name-canonicalization"): Add
      tests that elements of the load path are canonicalized.

diff --git a/libguile/filesys.c b/libguile/filesys.c
index 7674498..25501ef 100644
--- a/libguile/filesys.c
+++ b/libguile/filesys.c
@@ -1614,22 +1614,40 @@ SCM_DEFINE (scm_canonicalize_path, "canonicalize-path", 1, 0, 0,
 SCM
 scm_i_relativize_path (SCM path, SCM in_path)
 {
-  char *str, *canon;
   SCM scanon;
   
-  str = scm_to_locale_string (path);
-  canon = canonicalize_file_name (str);
-  free (str);
-  
-  if (!canon)
-    return SCM_BOOL_F;
+  {
+    char *str, *canon;
 
-  scanon = scm_take_locale_string (canon);
+    str = scm_to_locale_string (path);
+    canon = canonicalize_file_name (str);
+    free (str);
 
+    if (!canon)
+      return SCM_BOOL_F;
+
+    scanon = scm_take_locale_string (canon);
+  }
+  
   for (; scm_is_pair (in_path); in_path = scm_cdr (in_path))
     {
       SCM dir = scm_car (in_path);
-      size_t len = scm_c_string_length (dir);
+      size_t len;
+
+      /* Try to canonicalize DIR, since we have canonicalized PATH.  */
+      {
+        char *str, *canon;
+
+        str = scm_to_locale_string (dir);
+        canon = canonicalize_file_name (str);
+        free (str);
+  
+        if (canon)
+          dir = scm_from_locale_string (canon);
+        free (canon);
+      }
+
+      len = scm_c_string_length (dir);
 
       /* When DIR is empty, it means "current working directory".  We
 	 could set DIR to (getcwd) in that case, but then the
diff --git a/test-suite/tests/ports.test b/test-suite/tests/ports.test
index dfa430e..ea8eaa7 100644
--- a/test-suite/tests/ports.test
+++ b/test-suite/tests/ports.test
@@ -1865,14 +1865,15 @@
       (with-fluids ((%file-port-name-canonicalization 'relative))
         (port-filename (open-input-file "/dev/null")))))
 
+  (pass-if-equal "relative canonicalization with /dev/.." "dev/null"
+    (with-load-path (cons "/dev/.." %load-path)
+      (with-fluids ((%file-port-name-canonicalization 'relative))
+        (port-filename (open-input-file "/dev/null")))))
+
   (pass-if-equal "relative canonicalization from ice-9" "ice-9/q.scm"
-    ;; If an entry in %LOAD-PATH is not canonical, then
-    ;; `scm_i_relativize_path' is unable to do its job.
-    (if (equal? (map canonicalize-path %load-path) %load-path)
-        (with-fluids ((%file-port-name-canonicalization 'relative))
-          (port-filename
-           (open-input-file (%search-load-path "ice-9/q.scm"))))
-        (throw 'unresolved)))
+    (with-fluids ((%file-port-name-canonicalization 'relative))
+      (port-filename
+       (open-input-file (%search-load-path "ice-9/q.scm")))))
 
   (pass-if-equal "absolute canonicalization from ice-9"
       (canonicalize-path




Information forwarded to bug-guile <at> gnu.org:
bug#19540; Package guile. (Thu, 23 Jun 2016 13:07:02 GMT) Full text and rfc822 format available.

Message #28 received at 19540-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Andy Wingo <wingo <at> pobox.com>
Cc: 19540-done <at> debbugs.gnu.org, Matt Wette <mwette <at> alumni.caltech.edu>
Subject: Re: bug#19540: repeated ./././ in compiled modules
Date: Thu, 23 Jun 2016 15:06:27 +0200
Andy Wingo <wingo <at> pobox.com> skribis:

> commit 9a951678713557b548415d32eae6d63d039bf652
> Author: Andy Wingo <wingo <at> pobox.com>
> Date:   Thu Jun 23 10:03:10 2016 +0200
>
>     Fix relative file name canonicalization on paths with "."
>     
>     * libguile/filesys.c (scm_i_relativize_path): Canonicalize the file
>       names elements that we will be using as prefixes.  Fixes the case
>       where a load path contains a relative file name: #19540.
>     * test-suite/tests/ports.test ("%file-port-name-canonicalization"): Add
>       tests that elements of the load path are canonicalized.
>
> diff --git a/libguile/filesys.c b/libguile/filesys.c
> index 7674498..25501ef 100644
> --- a/libguile/filesys.c
> +++ b/libguile/filesys.c
> @@ -1614,22 +1614,40 @@ SCM_DEFINE (scm_canonicalize_path, "canonicalize-path", 1, 0, 0,
>  SCM
>  scm_i_relativize_path (SCM path, SCM in_path)

[...]

>    for (; scm_is_pair (in_path); in_path = scm_cdr (in_path))
>      {
>        SCM dir = scm_car (in_path);
> -      size_t len = scm_c_string_length (dir);
> +      size_t len;
> +
> +      /* Try to canonicalize DIR, since we have canonicalized PATH.  */
> +      {
> +        char *str, *canon;
> +
> +        str = scm_to_locale_string (dir);
> +        canon = canonicalize_file_name (str);
> +        free (str);
> +  
> +        if (canon)
> +          dir = scm_from_locale_string (canon);
> +        free (canon);
> +      }
> +
> +      len = scm_c_string_length (dir);
>  
>        /* When DIR is empty, it means "current working directory".  We
>  	 could set DIR to (getcwd) in that case, but then the

‘canonicalize_file_name’ is costly: roughly one syscall per file name
component.

IIUC, ‘canonicalize_file_name’ is now called once for each ‘%load-path’
entry and file name that we canonicalize.  Is this correct?

Ludo’.




Information forwarded to bug-guile <at> gnu.org:
bug#19540; Package guile. (Thu, 23 Jun 2016 16:04:02 GMT) Full text and rfc822 format available.

Message #31 received at 19540-done <at> debbugs.gnu.org (full text, mbox):

From: Andy Wingo <wingo <at> pobox.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 19540-done <at> debbugs.gnu.org, Matt Wette <mwette <at> alumni.caltech.edu>
Subject: Re: bug#19540: repeated ./././ in compiled modules
Date: Thu, 23 Jun 2016 18:03:15 +0200
Hi :)

On Thu 23 Jun 2016 15:06, ludo <at> gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo <at> pobox.com> skribis:
>
>> commit 9a951678713557b548415d32eae6d63d039bf652
>> Author: Andy Wingo <wingo <at> pobox.com>
>> Date:   Thu Jun 23 10:03:10 2016 +0200
>>
>>     Fix relative file name canonicalization on paths with "."
>>     
>>     * libguile/filesys.c (scm_i_relativize_path): Canonicalize the file
>>       names elements that we will be using as prefixes.  Fixes the case
>>       where a load path contains a relative file name: #19540.
>>     * test-suite/tests/ports.test ("%file-port-name-canonicalization"): Add
>>       tests that elements of the load path are canonicalized.
>>
> ‘canonicalize_file_name’ is costly: roughly one syscall per file name
> component.
>
> IIUC, ‘canonicalize_file_name’ is now called once for each ‘%load-path’
> entry and file name that we canonicalize.  Is this correct?

That's correct, but only for relative canonicalization, which is in
practive only when loading Scheme files from source.  Seems out of the
hot path; what do you think?

Andy




Information forwarded to bug-guile <at> gnu.org:
bug#19540; Package guile. (Fri, 24 Jun 2016 08:29:02 GMT) Full text and rfc822 format available.

Message #34 received at 19540-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Andy Wingo <wingo <at> pobox.com>
Cc: 19540-done <at> debbugs.gnu.org, Matt Wette <mwette <at> alumni.caltech.edu>
Subject: Re: bug#19540: repeated ./././ in compiled modules
Date: Fri, 24 Jun 2016 10:28:41 +0200
Hello!

Andy Wingo <wingo <at> pobox.com> skribis:

> On Thu 23 Jun 2016 15:06, ludo <at> gnu.org (Ludovic Courtès) writes:
>
>> Andy Wingo <wingo <at> pobox.com> skribis:
>>
>>> commit 9a951678713557b548415d32eae6d63d039bf652
>>> Author: Andy Wingo <wingo <at> pobox.com>
>>> Date:   Thu Jun 23 10:03:10 2016 +0200
>>>
>>>     Fix relative file name canonicalization on paths with "."
>>>     
>>>     * libguile/filesys.c (scm_i_relativize_path): Canonicalize the file
>>>       names elements that we will be using as prefixes.  Fixes the case
>>>       where a load path contains a relative file name: #19540.
>>>     * test-suite/tests/ports.test ("%file-port-name-canonicalization"): Add
>>>       tests that elements of the load path are canonicalized.
>>>
>> ‘canonicalize_file_name’ is costly: roughly one syscall per file name
>> component.
>>
>> IIUC, ‘canonicalize_file_name’ is now called once for each ‘%load-path’
>> entry and file name that we canonicalize.  Is this correct?
>
> That's correct, but only for relative canonicalization, which is in
> practive only when loading Scheme files from source.  Seems out of the
> hot path; what do you think?

I think it’s likely to have a noticeable impact on startup time for
projects with a large number of modules like Guix.

For instance, commands like ‘guix package -A’ or ‘guix build foo’ load
all the modules.  The impact will be smaller on a laptop with an SSD
than on an NFS mount, where it’s likely going to be terrrible (this
could be tested using the ‘delay’ device mapper.)

WDYT?

Ludo’.




Information forwarded to bug-guile <at> gnu.org:
bug#19540; Package guile. (Fri, 24 Jun 2016 08:50:01 GMT) Full text and rfc822 format available.

Message #37 received at 19540-done <at> debbugs.gnu.org (full text, mbox):

From: Andy Wingo <wingo <at> pobox.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 19540-done <at> debbugs.gnu.org, Matt Wette <mwette <at> alumni.caltech.edu>
Subject: Re: bug#19540: repeated ./././ in compiled modules
Date: Fri, 24 Jun 2016 10:49:12 +0200
Hi :)

On Fri 24 Jun 2016 10:28, ludo <at> gnu.org (Ludovic Courtès) writes:

> Hello!
>
> Andy Wingo <wingo <at> pobox.com> skribis:
>
>> On Thu 23 Jun 2016 15:06, ludo <at> gnu.org (Ludovic Courtès) writes:
>>
>>> ‘canonicalize_file_name’ is costly: roughly one syscall per file name
>>> component.
>>>
>>> IIUC, ‘canonicalize_file_name’ is now called once for each ‘%load-path’
>>> entry and file name that we canonicalize.  Is this correct?
>>
>> That's correct, but only for relative canonicalization, which is in
>> practive only when loading Scheme files from source.  Seems out of the
>> hot path; what do you think?
>
> I think it’s likely to have a noticeable impact on startup time for
> projects with a large number of modules like Guix.

Aren't they usually compiled?  Loading compiled files will not
go through this path AFAIU.

> For instance, commands like ‘guix package -A’ or ‘guix build foo’ load
> all the modules.  The impact will be smaller on a laptop with an SSD
> than on an NFS mount, where it’s likely going to be terrrible (this
> could be tested using the ‘delay’ device mapper.)

Oh I'm with you that we need to be careful here.  I am under the
impression though that there's no additional impact here because this is
just something that happens at compile-time.  Or if you load a source
file, but in that case you're evaluating and expecting a perf loss is
not the end of the world.

Andy




Information forwarded to bug-guile <at> gnu.org:
bug#19540; Package guile. (Fri, 24 Jun 2016 09:42:02 GMT) Full text and rfc822 format available.

Message #40 received at 19540-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Andy Wingo <wingo <at> pobox.com>
Cc: 19540-done <at> debbugs.gnu.org, Matt Wette <mwette <at> alumni.caltech.edu>
Subject: Re: bug#19540: repeated ./././ in compiled modules
Date: Fri, 24 Jun 2016 11:41:06 +0200
Andy Wingo <wingo <at> pobox.com> skribis:

> On Fri 24 Jun 2016 10:28, ludo <at> gnu.org (Ludovic Courtès) writes:
>
>> Hello!
>>
>> Andy Wingo <wingo <at> pobox.com> skribis:
>>
>>> On Thu 23 Jun 2016 15:06, ludo <at> gnu.org (Ludovic Courtès) writes:
>>>
>>>> ‘canonicalize_file_name’ is costly: roughly one syscall per file name
>>>> component.
>>>>
>>>> IIUC, ‘canonicalize_file_name’ is now called once for each ‘%load-path’
>>>> entry and file name that we canonicalize.  Is this correct?
>>>
>>> That's correct, but only for relative canonicalization, which is in
>>> practive only when loading Scheme files from source.  Seems out of the
>>> hot path; what do you think?
>>
>> I think it’s likely to have a noticeable impact on startup time for
>> projects with a large number of modules like Guix.
>
> Aren't they usually compiled?  Loading compiled files will not
> go through this path AFAIU.

Good point, I had overlooked that.

>> For instance, commands like ‘guix package -A’ or ‘guix build foo’ load
>> all the modules.  The impact will be smaller on a laptop with an SSD
>> than on an NFS mount, where it’s likely going to be terrrible (this
>> could be tested using the ‘delay’ device mapper.)
>
> Oh I'm with you that we need to be careful here.  I am under the
> impression though that there's no additional impact here because this is
> just something that happens at compile-time.  Or if you load a source
> file, but in that case you're evaluating and expecting a perf loss is
> not the end of the world.

Indeed, this makes sense to me now; sorry for the confusion!

Ludo’.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Fri, 22 Jul 2016 11:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 8 years and 330 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.