Subject: bug#39800: closed (Re: bug#39800: gnutls guile bug receiving https data)

From: [email-masked] (GNU bug Tracking System)
Date: Fri, 06 Mar 2020 22:47:02 +0000

Your bug report

#39800: gnutls guile bug receiving https data

which was filed against the guile package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
39800: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=39800
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#39800: gnutls guile bug receiving https data
From: Ludovic Courtès <[email-masked]>
To: "franco.rcr\@gmail.com" <[email-masked]>
Cc: [email-masked]
Date: Fri, 06 Mar 2020 23:46:35 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi,

"[email-masked]" <[email-masked]> skribis:

> ;;Now, submitting this simple https request, you get an exception
> (http-request "https://www.google.com")
> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
> Throw to key `gnutls-error' with args `(#<gnutls-error-enum La
> connessione TLS non Ã¨ stata terminata in modo corretto.>
> read_from_session_record_port)'.

That happens when passing #:keep-alive? #f (the default), specifically
when reading the response body:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (http-request "https://www.google.com" #:keep-alive? #f #:streaming? #t)
$1 = #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((date . #<date nanosecond: 0 second: 56 minute: 45 hour: 21 day: 6 month: 3 year: 2020 zone-offset: 0>) (expires . #<date nanosecond: 0 second: 0 minute: 0 hour: 0 day: 1 month: 1 year: 1970 zone-offset: 0>) (cache-control private (max-age . 0)) (content-type text/html (charset . "ISO-8859-1")) (p3p . "CP=\"This is not a P3P policy! See g.co/p3phelp for more info.\"") (server . "gws") (x-xss-protection . "0") (x-frame-options . "SAMEORIGIN") (set-cookie . "1P_JAR=2020-03-06-21; expires=Sun, 05-Apr-2020 21:45:56 GMT; path=/; domain=.google.com; Secure") (set-cookie . "NID=199=yXgE_KAGvxJbZAIGEXLt8CsEe3pre-RRLm1Jqap3b3iJRqZZq_PJ9wCT798mfDZ2TC5_3mKnM5KABSh8CguI64SsNoWHIc9EsW2osFltsIJnMswXhrtjQFDpfm_fUb6RDrWrqKHkOuvkG7Izp5im1Ys1TzGdztrFmOQV4FOraJk; expires=Sat, 05-Sep-2020 21:45:56 GMT; path=/; domain=.google.com; HttpOnly") (alt-svc . "quic=\":443\"; ma=2592000; v=\"46,43\",h3-Q050=\":443\"; ma=2592000,h3-Q049=\":443\"; ma=2592000,h3-Q048=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000") (accept-ranges none) (vary accept-encoding) (connection close)) port: #<input-output: file 7f3ae7dd3540>>
$2 = #<input-output: file 7f3ae7dd3540>
scheme@(guile-user)> (define bv (get-bytevector-all $2))
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Throw to key `gnutls-error' with args `(#<gnutls-error-enum The TLS connection was non-properly terminated.> read_from_session_record_port)'.
--8<---------------cut here---------------end--------------->8---

The reason for this is that google.com closes the connection right away,
without sending a proper TLS “bye” message as is conventionally done.

Fixed in commit 076276c4f580368b4106316a77752d69c8f1494a, which will be
in 3.0.1.

Thanks,
Ludo’.

Subject: gnutls guile bug receiving https data
From: "[email-masked]" <[email-masked]>
To: [email-masked]
Date: Wed, 26 Feb 2020 16:20:09 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

Hello,
I installed gnutls for guile and checked the gnutls module with this 
simple code:

;;Guile version 3.0 and gnutls  from git

,show version
GNU Guile 3.0.0.15-ff14b7

(gnutls-version)
$6 = "3.6.12"

;;Now, submitting this simple https request, you get an exception
(http-request "https://www.google.com")
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Throw to key `gnutls-error' with args `(#<gnutls-error-enum La 
connessione TLS non Ã¨ stata terminata in modo corretto.> 
read_from_session_record_port)'.

;;instead, without https there are no errors
(http-request "http://www.google.com") ;;works fine.

The error happens only on https://www.google.com and does not throw with 
a lot of other https web sites.
Furthermore the error is not throw if the method is HEAD, so it is 
related to the data part of the https answer.
I tried to enter in the internal implementation of the http web client 
but after some tests I decided to do some simple tests at application level.
I rewrote the get-bytevector-all, with a loop that reads one byte per 
time and the error was thrown anyway.
I catched the error and I've got the complete answer from the google web 
server.
In the following there is my applicative solution, where I rewrote the 
get-bytevector-all by adding the error checking and specifiyng to 
http-request that the data has to be returned as a port (#:streaming? #t).

;;A macro to catch errors
(define-syntax my-noerr
   (syntax-rules ()
     ((_ __error-return exp ...)
      (let
          ((__st #f))
        (catch #t
          (lambda() exp ...)
          (lambda (k . p) __error-return))))))

;;The rewriting of get-bytevector-all
(defun get-bytevector-all (port)
   (u8-list->bytevector (let loop ((port port))
      (let ((v (my-noerr (eof-object) (get-u8 port))))
        (if (eof-object? v)
        #nil
        (cons v (loop port)))))))

;;the piece of code that now gives the correct result
   (let-values (((a b)(http-request "https://www.google.com" 
#:streaming? #t)))
     (bytevector->string (get-bytevector-all b) "ISO-8859-1"))

As conclusion I can say that web modules read correctly the http answers 
and, with some (one for me, the google web site) https sites there is a 
misinterpretation of EOF in the layer between http and https.

Franco.