Package: emacs;
Reported by: "tenspd137 ." <dcday137 <at> gmail.com>
Date: Tue, 28 May 2019 22:26:02 UTC
Severity: normal
Found in version 26.2
Done: Thomas Fitzsimmons <fitzsim <at> fitzsim.org>
Bug is archived. No further changes may be made.
View this message in rfc822 format
From: Collin Day <dcday137 <at> gmail.com> To: Thomas Fitzsimmons <fitzsim <at> fitzsim.org> Cc: Andreas Schwab <schwab <at> linux-m68k.org>, 35969 <at> debbugs.gnu.org Subject: bug#35969: 26.2, Excorporate Date: Wed, 10 Jul 2019 09:35:34 -0600
When I add your patch, there is still an error:
Contacting host: outlook.office365.com:443
error in process filter: exco--parse-xml-in-current-buffer: Server
response is not an XML document
error in process filter: Server response is not an XML document
* http outlook,office365.com:443*
HTTP/1.1 401 Unauthorized
Server: Microsoft-IIS/10.0
request-id: 33a1707c-bed3-4edd-98c6-bd1443827acf
X-WSSecurity-Enabled: True
X-WSSecurity-For: Logon
X-FederationTrustTokenIssuerUri: urn:federation:MicrosoftOnline
X-WSSecurity-SymmetricKey-Enabled: True
X-WSSecurity-X509Cert-Enabled: True
X-OAuth-Enabled: True
X-Powered-By: ASP.NET
X-FEServer: SN4PR0601CA0013
WWW-Authenticate: Basic Realm=""
Date: Wed, 10 Jul 2019 15:27:49 GMT
Content-Length: 0
URL-DEBUG
http -> Contacting host: outlook.office365.com:443
http -> Marking connection as busy: outlook.office365.com:443
#<process outlook.office365.com>
http -> getting referer from buffer: buffer:#<buffer *scratch*>
target-url:#s(url "https" nil nil "outlook.office365.com" nil
"/EWS/Services.wsdl" nil nil t nil t t) lastloc:nil
http -> Calling after change function
`url-https-proxy-after-change-function' for `#<process
outlook.office365.com>'
http -> url-http-parse-response called in ( *http outlook.office365.com:443*)
http -> Request is:
GET /EWS/Services.wsdl HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: outlook.office365.com
Accept-encoding: gzip
Accept: */*
User-Agent: URL/Emacs Emacs/27.0.50 (X11; x86_64-pc-linux-gnu)
http -> Calling after change function
`url-http-wait-for-headers-change-function' for `#<process
outlook.office365.com>'
http -> url-http-wait-for-headers-change-function ( *http
outlook.office365.com:443*)
http -> Saw end of headers... ( *http outlook.office365.com:443*)
http -> url-http-parse-response called in ( *http outlook.office365.com:443*)
http -> Got a content-length, being smart about document end.
http -> Got 0-length content-length, activating callback immediately.
http -> Marking connection as free: outlook.office365.com:443
#<process outlook.office365.com>
http -> url-http-parse-headers called in ( *http outlook.office365.com:443*)
http -> url-http-parse-response called in ( *http outlook.office365.com:443*)
http -> Parsed HTTP headers: class=4 status=401
http -> Handling normal authentication
http -> Found existing connection: outlook.office365.com:443 #<process
outlook.office365.com>
http -> Reusing existing connection: outlook.office365.com:443
http -> Marking connection as busy: outlook.office365.com:443
#<process outlook.office365.com>
http -> getting referer from buffer: buffer:#<buffer *http
outlook.office365.com:443*> target-url:#s(url "https" nil nil
"outlook.office365.com" nil "/EWS/Services.wsdl" nil nil t nil t t)
lastloc:nil
http -> Finished parsing HTTP headers: nil
http -> Spinning waiting for headers...
http -> Calling after change function
`url-https-proxy-after-change-function' for `#<process
outlook.office365.com>'
http -> url-http-parse-response called in ( *http
outlook.office365.com:443*-293116)
http -> error response: 400
http -> Marking connection as free: outlook.office365.com:443
#<process outlook.office365.com>
http -> Activating callback in buffer ( *http
outlook.office365.com:443*-293116): #[257
"p\303\304\305\306\307 !\310\"\311$\216
@\312=\203$\0\313\301\314\315\316\302\"#\210\317\300\320\"\202/\0\313\301\321\322
#\210\317\300\323\")\207" [fsm-exco--fsm-0 (:identifier ("my email" .
"https://outlook.office365.com/EWS/Exchange.asmx") :mail-address "my
email" :retrying nil :autodiscovery-urls nil :service-url
"https://outlook.office365.com/EWS/Exchange.asmx" :service-xml nil
:service-wsdl nil :next-state-after-success nil :failure-message nil
:server-version nil) "https://outlook.office365.com/EWS/Services.wsdl"
make-byte-code 0 "\301\300!\205 \0\302\300!\207" vconcat vector
[buffer-live-p kill-buffer] 2 :error plist-put :failure-message format
"Failed to retrieve %s" fsm-send :unrecoverable-error :service-xml
exco--parse-xml-in-current-buffer :success] 8 "
(fn STATUS)"] ((:peer (:certificates ((:version 3 :serial-number
"0b:be:0b:85:09:74:2b:7b:f8:7f:80:dc:14:67:6d:54" :issuer
"C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :valid-from
"2018-11-19" :valid-to "2020-11-19" :subject
"C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,CN=outlook.com"
:public-key-algorithm "RSA" :certificate-security-level "Medium"
:signature-algorithm "RSA-SHA256" :public-key-id
"sha1:35:1a:fe:e2:6c:d9:35:08:3c:95:a2:c1:12:a1:2a:8e:7f:87:fa:07"
:certificate-id
"sha1:9a:44:16:e2:84:8e:3d:26:86:f0:23:a2:ff:f2:d0:24:28:2f:18:8f")
(:version 3 :serial-number
"01:9e:c1:c6:bd:3f:59:7b:b2:0c:33:38:e5:51:d8:77" :issuer
"C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert Global Root CA"
:valid-from "2015-08-04" :valid-to "2030-08-04" :subject
"C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1"
:public-key-algorithm "RSA" :certificate-security-level "Medium"
:signature-algorithm "RSA-SHA256" :public-key-id
"sha1:ac:13:d5:79:03:66:f3:cc:ff:fe:7c:23:71:a7:61:5d:39:20:89:6e"
:certificate-id
"sha1:81:b6:8d:6c:d2:f2:21:f8:f5:34:e6:77:52:3b:b2:36:bb:a1:dc:56"))
:certificate (:version 3 :serial-number
"0b:be:0b:85:09:74:2b:7b:f8:7f:80:dc:14:67:6d:54" :issuer
"C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :valid-from
"2018-11-19" :valid-to "2020-11-19" :subject
"C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,CN=outlook.com"
:public-key-algorithm "RSA" :certificate-security-level "Medium"
:signature-algorithm "RSA-SHA256" :public-key-id
"sha1:35:1a:fe:e2:6c:d9:35:08:3c:95:a2:c1:12:a1:2a:8e:7f:87:fa:07"
:certificate-id
"sha1:9a:44:16:e2:84:8e:3d:26:86:f0:23:a2:ff:f2:d0:24:28:2f:18:8f")
:key-exchange "ECDHE-RSA" :protocol "TLS1.2" :cipher "AES-256-GCM"
:mac "AEAD")))
url-http.el with patch from describe function, just to make sure patch
is applied:
...
(setq request
(concat
;; The request
(or url-http-method "GET") " "
(url-http--encode-string
;; (if using-proxy (url-recreate-url url-http-target-url)
real-fname))
(if (and using-proxy
(not (equal "https" (url-type url-http-target-url))))
(url-recreate-url url-http-target-url)
real-fname))
" HTTP/" url-http-version "\r\n"
...
output from make:
make[2]: Entering directory '/home/dayd/projects/emacs/lisp'
ELC url/url-http.elc
Sorry if the extra info is extraneous, just trying to grab as much as
I can for you. I think there is something else wrong at this point
though - like I mentioned it seems strange I had to run
package-install 4 times to get excorporate to install in the first
place.
Thanks!
-C
On Tue, Jul 9, 2019 at 6:11 PM Thomas Fitzsimmons <fitzsim <at> fitzsim.org> wrote:
>
> I think there should be no need to remove test-home, so you won't have
> to re-run the M-x package-install steps.
>
> Thomas
>
> Thomas Fitzsimmons <fitzsim <at> fitzsim.org> writes:
>
> > OK, can you apply my patch, then run "make" again and confirm it works?
> >
> > Thanks,
> > Thomas
> >
> > Collin Day <dcday137 <at> gmail.com> writes:
> >
> >> Did a git pull and followed instructions above.
> >>
> >> There are some issues. First, I need to run M-x package-install <RET>
> >> excorporate <RET> four times because I see (each line after each
> >> invocation)
> >>
> >> package--with-response-buffer-1:
> >> https://elpa.gnu.org/packages/url-http-ntlm-2.0.4.el: Method not
> >> allowed
> >> package--with-response-buffer-1:
> >> https://elpa.gnu.org/packages/fsm-0.2.1.el: Method not allowed
> >> package--with-response-buffer-1:
> >> https://elpa.gnu.org/packages/excorporate-0.8.3.tar: Method not
> >> allowed
> >>
> >> After the 4th call it compiles and becomes available.
> >>
> >> M-x customize-group <RET> excorporate, set up for no autoconfig right
> >> now, hit apply for current sessions, save not availiable....
> >>
> >> M-x excorporate <RET>
> >>
> >> enter uname and password
> >>
> >> Contacting host: outlook.office365.com:443
> >> error in process filter: exco--parse-xml-in-current-buffer: Server
> >> response is not an XML document
> >> error in process filter: Server response is not an XML documen
> >>
> >> *http outlook.office.365.com:443*
> >
> > [...]
> >
> >> So it appears not to work, at least on the current head as of today....
> >>
> >> Thanks!
> >>
> >> -C
> >>
> >> On Tue, Jul 9, 2019 at 4:09 PM Collin Day <dcday137 <at> gmail.com> wrote:
> >>>
> >>> Sorry, I have not had the chance to. A lot has been going on at my
> >>> place of employment. It has crossed my mind, and as soon as I have
> >>> a chance, I will try what you suggested above. Thanks, sorry for
> >>> the inconvenience.
> >>>
> >>> On Tue, Jul 9, 2019, 3:53 PM Thomas Fitzsimmons <fitzsim <at> fitzsim.org> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> Were you able to complete building Emacs? I'd like to know if my patch
> >>>> is needed on top of Andreas's to make Excorporate work through your
> >>>> proxy.
> >>>>
> >>>> I'm not sure why the built version would interfere with the current
> >>>> installation. If you were doing "make install" to a common prefix, that
> >>>> might explain it. Instead you can try something like this:
> >>>>
> >>>> cd emacs-master [your github.com checkout]
> >>>> make
> >>>> mkdir test-home
> >>>> HOME=`pwd`/test-home ./src/emacs -Q
> >>>>
> >>>> That will ensure you're only running the built Emacs, and completely
> >>>> ignoring the packages installed in your home directory.
> >>>>
> >>>> Thomas
> >>>>
> >>>> "tenspd137 ." <dcday137 <at> gmail.com> writes:
> >>>>
> >>>> > I haven't tried the first item on the list yet - ie reloading Andreas'
> >>>> > function. i have tried to build emacs from
> >>>> > https://github.com/emacs-mirror/emacs.git because I can't reach the
> >>>> > official repo from my firewall for whatever reason. I am sure it is
> >>>> > something here. Anyway, I was having problems because my built
> >>>> > version (27.0.x) was getting seemingly mixed up with my current
> >>>> > installation. I can try again later, but right now things are pretty
> >>>> > busy at work.
> >>>> >
> >>>> > Thanks.
> >>>> >
> >>>> > On Tue, Jun 18, 2019 at 10:26 PM Thomas Fitzsimmons <fitzsim <at> fitzsim.org> wrote:
> >>>> >>
> >>>> >> Hi,
> >>>> >>
> >>>> >> "tenspd137 ." <dcday137 <at> gmail.com> writes:
> >>>> >>
> >>>> >> > So - I am not sure if I did it correctly, but I copied this function
> >>>> >> > with Andreas' changes into a file:
> >>>> >> >
> >>>> >> > (defun url-https-proxy-after-change-function (_st _nd _length)
> >>>> >> > (let* ((process-buffer (current-buffer))
> >>>> >> > (proc (get-buffer-process process-buffer)))
> >>>> >> > (goto-char (point-min))
> >>>> >> > (when (re-search-forward "^\r?\n" nil t)
> >>>> >> > (backward-char 1)
> >>>> >> > ;; Saw the end of the headers
> >>>> >> > (setq url-http-end-of-headers (set-marker (make-marker) (point)))
> >>>> >> > (url-http-parse-response)
> >>>> >> > (cond
> >>>> >> > ((null url-http-response-status)
> >>>> >> > ;; We got back a headerless malformed response from the
> >>>> >> > ;; server.
> >>>> >> > (url-http-activate-callback)
> >>>> >> > (error "Malformed response from proxy, fail!"))
> >>>> >> > ((= url-http-response-status 200)
> >>>> >> > (if (gnutls-available-p)
> >>>> >> > (condition-case e
> >>>> >> > (let ((tls-connection (gnutls-negotiate
> >>>> >> > :process proc
> >>>> >> > :hostname (url-host url-current-object)
> >>>> >> > :verify-error nil)))
> >>>> >> > ;; check certificate validity
> >>>> >> > (setq tls-connection
> >>>> >> > (nsm-verify-connection tls-connection
> >>>> >> > (url-host url-current-object)
> >>>> >> > (url-port url-current-object)))
> >>>> >> > (with-current-buffer process-buffer (erase-buffer))
> >>>> >> > (set-process-buffer tls-connection process-buffer)
> >>>> >> > (setq url-http-after-change-function
> >>>> >> > 'url-http-wait-for-headers-change-function)
> >>>> >> > (set-process-filter tls-connection 'url-http-generic-filter)
> >>>> >> > (process-send-string tls-connection
> >>>> >> > ;; Use the non-proxy form of the request
> >>>> >> > (let (url-http-proxy)
> >>>> >> > (url-http-create-request))))
> >>>> >> > (gnutls-error
> >>>> >> > (url-http-activate-callback)
> >>>> >> > (error "gnutls-error: %s" e))
> >>>> >> > (error
> >>>> >> > (url-http-activate-callback)
> >>>> >> > (error "error: %s" e)))
> >>>> >> > (error "error: gnutls support needed!")))
> >>>> >> > (t
> >>>> >> > (url-http-debug "error response: %d" url-http-response-status)
> >>>> >> > (url-http-activate-callback))))))
> >>>> >> >
> >>>> >> > and then loaded it before running excorporate. After that, I did M-x
> >>>> >> > excorporate, and the minibuffer returns: error in process filter:
> >>>> >> > Server response is not an XML document
> >>>> >>
> >>>> >> In this scenario, if you immediately (without restarting Emacs/reloading
> >>>> >> anything) re-run M-x excorporate does it still fail? I just want to
> >>>> >> make sure that's not a transient failure. If it does fail the second
> >>>> >> time, can you post the HTTP response from the server?
> >>>> >>
> >>>> >> > When I do the similar test by loading the url-http-create-request with
> >>>> >> > Thomas's changes, I can get a connection and grab my schedule
> >>>> >> > through the proxy.
> >>>> >>
> >>>> >> OK.
> >>>> >>
> >>>> >> > Let me know if I need to try something different.
> >>>> >>
> >>>> >> Are you in a position to build Emacs master tip and retry the experiment
> >>>> >> without patching anything?
> >>>> >>
> >>>> >> Thanks,
> >>>> >> Thomas
> >>>> >>
> >>>> >> > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons <fitzsim <at> fitzsim.org> wrote:
> >>>> >> >>
> >>>> >> >> Hi,
> >>>> >> >>
> >>>> >> >> Good to hear that the patch I posted worked!
> >>>> >> >>
> >>>> >> >> Yes, that's the patch that Andreas's commit
> >>>> >> >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can tell,
> >>>> >> >> you will not have that in Emacs 26.2.
> >>>> >> >>
> >>>> >> >> Can you try reverting my patch and applying Andreas's patch, and see if
> >>>> >> >> Excorporate still works through the proxy?
> >>>> >> >>
> >>>> >> >> Thanks,
> >>>> >> >> Thomas
> >>>> >> >>
> >>>> >> >> "tenspd137 ." <dcday137 <at> gmail.com> writes:
> >>>> >> >>
> >>>> >> >> > The patch Thomas seems to work from behind the proxy. My current
> >>>> >> >> > emacs version is 26.2, so I would think it would include the commit
> >>>> >> >> > Andreas is talking about.... I went and looked it up - is this the
> >>>> >> >> > correct commit?
> >>>> >> >> >
> >>>> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el
> >>>> >> >> > index 53798f7..817c5ce 100644
> >>>> >> >> > --- a/lisp/url/url-http.el
> >>>> >> >> > +++ b/lisp/url/url-http.el
> >>>> >> >> > @@ -1412,7 +1412,9 @@ The return value of this function is the
> >>>> >> >> > retrieval buffer."
> >>>> >> >> > 'url-http-wait-for-headers-change-function)
> >>>> >> >> > (set-process-filter tls-connection 'url-http-generic-filter)
> >>>> >> >> > (process-send-string tls-connection
> >>>> >> >> > - (url-http-create-request)))
> >>>> >> >> > + ;; Use the non-proxy form of the request
> >>>> >> >> > + (let (url-http-proxy)
> >>>> >> >> > + (url-http-create-request))))
> >>>> >> >> > (gnutls-error
> >>>> >> >> > (url-http-activate-callback)
> >>>> >> >> > (error "gnutls-error: %s" e))
> >>>> >> >> >
> >>>> >> >> > Thanks!
> >>>> >> >> >
> >>>> >> >> > -C
> >>>> >> >> >
> >>>> >> >> >
> >>>> >> >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab <at> linux-m68k.org> wrote:
> >>>> >> >> >>
> >>>> >> >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim <at> fitzsim.org> wrote:
> >>>> >> >> >>
> >>>> >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el
> >>>> >> >> >> > index 00803a103a..723d111d58 100644
> >>>> >> >> >> > --- a/lisp/url/url-http.el
> >>>> >> >> >> > +++ b/lisp/url/url-http.el
> >>>> >> >> >> > @@ -329,7 +329,10 @@ url-http-create-request
> >>>> >> >> >> > ;; The request
> >>>> >> >> >> > (or url-http-method "GET") " "
> >>>> >> >> >> > (url-http--encode-string
> >>>> >> >> >> > - (if using-proxy (url-recreate-url url-http-target-url) real-fname))
> >>>> >> >> >> > + (if (and using-proxy
> >>>> >> >> >> > + (not (equal "https" (url-type url-http-target-url))))
> >>>> >> >> >> > + (url-recreate-url url-http-target-url)
> >>>> >> >> >> > + real-fname))
> >>>> >> >> >>
> >>>> >> >> >> That should already be handled by commit 84613dae5c.
> >>>> >> >> >>
> >>>> >> >> >> Andreas.
> >>>> >> >> >>
> >>>> >> >> >> --
> >>>> >> >> >> Andreas Schwab, schwab <at> linux-m68k.org
> >>>> >> >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
> >>>> >> >> >> "And now for something completely different."
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.