GNU bug report logs - #15368
HTTP client is slow [2.0.9]

Previous Next

Full log

Message #13 received at 15368 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mark H Weaver <mhw <at> netris.org>
Cc: 15368 <at> debbugs.gnu.org
Subject: Re: bug#15368: HTTP client is slow [2.0.9]
Date: Fri, 13 Sep 2013 23:14:07 +0200

[Message part 1 (text/plain, inline)]

Mark H Weaver <mhw <at> netris.org> skribis:

> ludo <at> gnu.org (Ludovic Courtès) writes:
>
>> I just noticed that our HTTP client is very slow.  Consider this:
>>
>> (use-modules (web client)
>>              (rnrs io ports)
>>              (rnrs bytevectors)
>>              (srfi srfi-11)
>>              (ice-9 format))
>>
>> (define %uri
>>   "http://ftp.gnu.org/gnu/idutils/idutils-4.6.tar.xz")
>>
>> (with-fluids ((%default-port-encoding #f))
>>   (let*-values (((start)
>>                  (gettimeofday))
>>                 ((p)
>>                  (let ((s (open-socket-for-uri %uri)))
>>                    (setvbuf s _IONBF)
>
> Why are you using an unbuffered port?  On my system, changing this to
> _IOFBF increases throughput from 326 KiB/s to 489.0 KiB/s.

Arf, that’s because I was also forcing the ‘scm_c_read’ hack (which
is currently never used, and this is a bug):

[Message part 2 (text/x-patch, inline)]

diff --git a/libguile/ports.c b/libguile/ports.c
index 9068c5c..c217712 100644
--- a/libguile/ports.c
+++ b/libguile/ports.c
@@ -1657,7 +1657,8 @@ scm_c_read (SCM port, void *buffer, size_t size)
      requested number of bytes.  (Note that a single scm_i_fill_input
      call does not guarantee to fill the whole of the port's read
      buffer.) */
-  if (pt->read_buf_size <= 1 && pt->encoding == NULL)
+  if (pt->read_buf_size <= 1
+      && (pt->encoding == NULL || strcmp (pt->encoding, "ISO-8859-1") == 0))
     {
       /* The port that we are reading from is unbuffered - i.e. does
 	 not have its own persistent buffer - but we have a buffer,

[Message part 3 (text/plain, inline)]

So in practice it was reading several KiB at a time, doing zero-copy.

> Also, the fact that my throughput is so much higher than yours (on a
> several-year-old computer) is interesting.  Obviously I have a faster
> net connection (wget reports 1.19M/s),

So for you wget is ~2.5 times faster than Guile, right?

[...]

>> Looking at the strace output reveals no real difference: they all make
>> one syscall for each chunk of 1410 bytes.
>>
>> ‘time’ reports that Guile spends 0.2 s. in user and 0.8 s. in system,
>> both of which are an order of magnitude higher than wget/curl.
>
> If they make essentially the same syscalls, then why would the system
> time be an order of magnitude higher?  Something doesn't sound right
> here.

I concur.

I’ve tried Linux perf and OProfile but failed to get useful info.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.