GNU bug report logs - #35969
proxy + excorporate -> Failed: Failed to retrieve https://outlook.office365.com/EWS/Services.wsdl

Previous Next

Package: emacs;

Reported by: "tenspd137 ." <dcday137 <at> gmail.com>

Date: Tue, 28 May 2019 22:26:02 UTC

Severity: normal

Found in version 26.2

Done: Thomas Fitzsimmons <fitzsim <at> fitzsim.org>

Bug is archived. No further changes may be made.

Full log

Message #70 received at 35969 <at> debbugs.gnu.org (full text, mbox):

From: Thomas Fitzsimmons <fitzsim <at> fitzsim.org>
To: "tenspd137 ." <dcday137 <at> gmail.com>
Cc: 35969 <at> debbugs.gnu.org
Subject: Re: bug#35969: 26.2, Excorporate
Date: Fri, 14 Jun 2019 20:14:21 -0400

Hi,

Yes, this seems really promising.  If you just replace (thefile) with
real-fname (and re-eval your url-http-create-request), does Excorporate
then work through the proxy?

Thomas

"tenspd137 ." <dcday137 <at> gmail.com> writes:

> Sorry - I was working fast:
>
> *I created a function that changed the line
> in  url-http-create-request so that the request looks like:*
>
> In a file I did:
>
> (defun thefile() "/EWS/Exchange.asmx")
>
> (copied from url-http library)
> (defun url-http-create-request (&optional ref-url)
> .....
>    ;; This was done with a call to `format'.  Concatenating parts has
>     ;; the advantage of keeping the parts of each header together and
>     ;; allows us to elide null lines directly, at the cost of making
>     ;; the layout less clear.
>     (setq request
>           (concat
>              ;; The request
>              (or url-http-method "GET") " "
>              (url-http--encode-string
>               (if using-proxy (thefile) real-fname))
> <------------Changed this to "hardcode" the request as proof of
> concept
>              " HTTP/" url-http-version "\r\n"
>              ;; Version of MIME we speak
>              "MIME-Version: 1.0\r\n"
>              ;; (maybe) Try to keep the connection open
>              "Connection: " (if (or using-proxy .....
> .......
> .......
>
> Loaded the file and then ran (url-retriev-synchronously "https://......")
>
> Thanks!
>
> -C
>
> On Fri, Jun 14, 2019 at 5:07 PM tenspd137 . <dcday137 <at> gmail.com> wrote:
>>
>> I think I figured it out.  I created a function that changed the line
>> in  url-http-create-request so that the request looks like:
>> GET /EWS/Exchange.asmx HTTP/1.1 <-----  *no server / host*
>> MIME-Version: 1.0
>> Connection: close
>> Extension: Security/Digest Security/SSL
>> Host: outlook.office365.com
>> Accept-encoding: gzip
>> Accept: */*
>> User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu)
>> Cookie: OIDC=1; ClientId=8998C5691CD143E784857A0D01537963
>> Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOlMxa3kzbGk3bmUwNzMxJSU=
>>
>> instead of
>>
>> GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1
>> MIME-Version: 1.0
>> Connection: close
>> Extension: Security/Digest Security/SSL
>> Host: outlook.office365.com
>> Accept-encoding: gzip
>> Accept: */*
>> User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu)
>> Cookie: OIDC=1; ClientId=8998C5691CD143E784857A0D01537963
>> Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOlMxa3kzbGk3bmUwNzMxJSU=
>>
>> and the results buffer gives: *http....*-#####
>> HTTP/1.1 200 OK
>> Cache-Control: private
>> Content-Length: 1213
>> Content-Type: text/html; charset=UTF-8
>> Content-Encoding: gzip
>> Vary: Accept-Encoding
>> Server: Microsoft-IIS/10.0
>> request-id: bcd31568-29fd-44e4-935f-35b54d697f33
>> X-CalculatedFETarget: CY4PR18CU003.internal.outlook.com
>> X-BackEndHttpStatus: 200
>> Set-Cookie: exchangecookie=6c731ced364846f2a979bfaa84496f1e;
>> expires=Sun, 14-Jun-2020 23:02:57 GMT; path=/; secure; HttpOnly
>> X-FEProxyInfo: CY4PR18CA0059.NAMPRD18.PROD.OUTLOOK.COM
>> X-CalculatedBETarget: CS1PR8401MB1223.NAMPRD84.PROD.OUTLOOK.COM
>> X-BackEndHttpStatus: 200
>> X-RUM-Validated: 1
>> X-AspNet-Version: 4.0.30319
>> X-BeSku: Gen9
>> X-DiagInfo: CS1PR8401MB1223
>> X-BEServer: CS1PR8401MB1223
>> X-FEServer: CY4PR18CA0059
>> X-Powered-By: ASP.NET
>> X-FEServer: SN4PR0501CA0003
>> Date: Fri, 14 Jun 2019 23:02:56 GMT
>> Connection: close
>>
>> <HTML><HEAD><link rel="alternate" type="text/xml"
>> href="https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Exchange.asmx?disco"/><STYLE
>> type="text/css">#content{ FONT-SIZE: 0.7em; PADDING-BOTTOM: 2em;
>> MARGIN-LEFT: 30px}BODY{MARGIN-TOP: 0px; MARGIN-LEFT: 0px; COLOR:
>> #000000; FONT-FAMILY: Verdana; BACKGROUND-COLOR: white}P{MARGIN-TOP:
>> 0px; MARGIN-BOTTOM: 12px; COLOR: #000000; FONT-FAMILY:
>> Verdana}PRE{BORDER-RIGHT: #f0f0e0 1px solid; PADDING-RIGHT: 5px;
>> BORDER-TOP: #f0f0e0 1px solid; MARGIN-TOP: -5px; PADDING-LEFT: 5px;
>> FONT-SIZE: 1.2em; PADDING-BOTTOM: 5px; BORDER-LEFT: #f0f0e0 1px solid;
>> PADDING-TOP: 5px; BORDER-BOTTOM: #f0f0e0 1px solid; FONT-FAMILY:
>> Courier New; BACKGROUND-COLOR: #e5e5cc}.heading1{MARGIN-TOP: 0px;
>> PADDING-LEFT: 15px; FONT-WEIGHT: normal; FONT-SIZE: 26px;
>> MARGIN-BOTTOM: 0px; PADDING-BOTTOM: 3px; MARGIN-LEFT: -30px; WIDTH:
>> 100%; COLOR: #ffffff; PADDING-TOP: 10px; FONT-FAMILY: Tahoma;
>> BACKGROUND-COLOR: #003366}.intro{MARGIN-LEFT:
>> -15px}</STYLE><TITLE>Service</TITLE></HEAD><BODY><DIV id="content"><P
>> class="heading1">Service</P><BR/><P class="intro">You have created a
>> service.<P class='intro'>To test this service, you will need to create
>> a client and use it to call the service. You can do this using the
>> svcutil.exe tool from the command line with the following syntax:</P>
>> <BR/><PRE>svcutil.exe <A
>> HREF="https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Services.wsdl">https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Services.wsdl</A></PRE></P><P
>> class="intro"/>This will generate a configuration file and a code file
>> that contains the client class. Add the two files to your client
>> application and use the generated client class to call the Service.
>> For example:<BR/><P class='intro'><B>C#</B></P><PRE><font
>> color="blue">class </font><font color="teal">Test
>> </font>{
>> <font color="blue">    static void </font>Main()
>>     {
>>         <font color="teal">HelloClient</font> client = <font
>> color="blue">new </font><font color="teal">HelloClient</font>();
>>
>> <font color="green">        // Use the 'client' variable to call
>> operations on the service.
>>
>> </font><font color="green">        // Always close the client.
>> </font>        client.Close();
>>     }
>> }
>> </PRE><BR/><P class='intro'><B>Visual Basic</B></P><PRE><font
>> color="blue">Class </font><font color="teal">Test
>> </font><font color="blue">    Shared Sub </font>Main()
>> <font color="blue">        Dim </font>client As <font
>> color="teal">HelloClient</font> = <font color="blue">New </font><font
>> color="teal">HelloClient</font>()
>> <font color="green">        ' Use the 'client' variable to call
>> operations on the service.
>>
>> </font><font color="green">        ' Always close the client.
>> </font>        client.Close()
>> <font color="blue">    End Sub
>> </font><font color="blue">End Class</font></PRE></DIV></BODY></HTML>
>>
>> which matches the file saved with wget.
>>
>> That has to be worth something....
>>
>> Thanks!
>>
>> -C
>>
>> On Fri, Jun 14, 2019 at 4:07 PM tenspd137 . <dcday137 <at> gmail.com> wrote:
>> >
>> > Just some more things I noticed from the wget log vs. the emacs buffers:
>> >
>> > emacs: Accept-encoding:gzip, wget:Accept-encoding: identity
>> >
>> > emacs: GET  https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1,
>> > wget: GET /EWS/Exchange.asmx HTTP/1.1
>> >
>> > Don't know if any of that is helpful, but there it is.
>> >
>> > Thanks!
>> >
>> > -C
>> >
>> > On Fri, Jun 14, 2019 at 3:48 PM tenspd137 . <dcday137 <at> gmail.com> wrote:
>> > >
>> > > Sorry - I am not in control of my proxy.  So, before I give up on this
>> > > for today, I was going over the wget logs from trying the same thing.
>> > > One thing I noticed is:
>> > >
>> > > Connection: Keep-Alive
>> > > Proxy-Connection: Keep-Alive
>> > >
>> > > I see stuff in the emacs buffers mentioning Connection, and it says
>> > > closed (unless marking the connection as busy is the same as keeping
>> > > it open), but I don't see anything mentioning the proxy
>> > > connection....?  Maybe another place to look?
>> > >
>> > > Just some ideas...
>> > >
>> > > Thanks!
>> > >
>> > > -C
>> > >
>> > > On Fri, Jun 14, 2019 at 3:02 PM Thomas Fitzsimmons <fitzsim <at> fitzsim.org> wrote:
>> > > >
>> > > > Hi,
>> > > >
>> > > > Thanks for following up with these further test results.
>> > > >
>> > > > "tenspd137 ." <dcday137 <at> gmail.com> writes:
>> > > >
>> > > > > I was able to try stepping through an Emacs/proxy/Exchange test in an
>> > > > > emacs -Q session.  After setting the proxy and configuring the
>> > > > > debugger to step through url-http and url-http-async-sentilnel, the
>> > > > > only thing I noticed is that it appears url-http-async-sentinel is not
>> > > > > being called.
>> > > >
>> > > > OK, that's probably expected.  I listed it for completeness (all the
>> > > > call sites of url-https-proxy-connect), but I guess it would only be
>> > > > called under url-retrieve, not under url-retrieve-synchronously.
>> > > >
>> > > > > I also put the url-https-proxy-connect override you gave me earlier
>> > > > > into a file, loaded it and set the debugger to run through that as
>> > > > > well as set up proxies.  A broken down list of steps:
>> > > > >
>> > > > > 1.  Load file containing proxy and altered url-https-proxyconnect, set
>> > > > > debugger to run through it when hit.
>> > > > > 2.  set up url-http and url-http-async-sentinel to be picked up by debugger
>> > > > > 3.  eval (url-retrieve-synchronously
>> > > > > "https://outlook.office365.com/EWS/Exchange.asmx"), step through
>> > > > > url-http
>> > > > > 4.  Input username and password when asked
>> > > > > 5.  Continue stepping until end
>> > > > >
>> > > > > url-http-async-sentinel is never called. " *http* ... -####" has text
>> > > > > indicating failure. url-https-proxy-connect is indeed called.
>> > > >
>> > > > OK, it's good to know that url-http ultimately calls
>> > > > url-https-proxy-connect.  Unfortunately, despite this, proxying does not
>> > > > work.
>> > > >
>> > > > > I don't know how to look at the actual <process>, if I try to C-x C-e
>> > > > > or M-: "connection", it goes into the debugger.  The url's, etc look
>> > > > > good as far as I can tell.  Not sure what else I can do.  If there are
>> > > > > certain pieces of url-http you want me to look at, just let me know,
>> > > > > but not really knowing what has to happen under the hood, I am not
>> > > > > going to be able to do much else.
>> > > >
>> > > > I think I'll probably have to set up my own test environment.  Are you
>> > > > in control of the proxy and its configuration?  If so, can you provide
>> > > > rough configuration instructions (proxy software, version, relevant
>> > > > config settings)?  If not, that's OK, I can try setting up generic proxy
>> > > > software.
>> > > >
>> > > > Thanks,
>> > > > Thomas
This bug report was last modified 5 years and 364 days ago.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.