From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 20 08:06:26 2017 Received: (at submit) by debbugs.gnu.org; 20 Aug 2017 12:06:26 +0000 Received: from localhost ([127.0.0.1]:46160 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1djOzl-00031b-J6 for submit@debbugs.gnu.org; Sun, 20 Aug 2017 08:06:26 -0400 Received: from eggs.gnu.org ([208.118.235.92]:37184) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1djOzj-00031O-Su for submit@debbugs.gnu.org; Sun, 20 Aug 2017 08:06:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1djOzd-0008Qf-Pk for submit@debbugs.gnu.org; Sun, 20 Aug 2017 08:06:14 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: * X-Spam-Status: No, score=1.3 required=5.0 tests=BAYES_50,RCVD_IN_SORBS_SPAM autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:37101) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1djOzd-0008QO-Ke for submit@debbugs.gnu.org; Sun, 20 Aug 2017 08:06:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56387) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1djOzc-0002h4-9y for bug-guix@gnu.org; Sun, 20 Aug 2017 08:06:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1djOzZ-0008IR-41 for bug-guix@gnu.org; Sun, 20 Aug 2017 08:06:12 -0400 Received: from mail-out.m-online.net ([212.18.0.10]:60305) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1djOzY-0008Hm-Nc for bug-guix@gnu.org; Sun, 20 Aug 2017 08:06:09 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 3xZwWx2cXDz1qrfQ for ; Sun, 20 Aug 2017 14:06:05 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 3xZwWx217Mz3jgYP for ; Sun, 20 Aug 2017 14:06:05 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id I6g6L2J-dpAL for ; Sun, 20 Aug 2017 14:06:03 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-93-104-74-212.dynamic.mnet-online.de [93.104.74.212]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS for ; Sun, 20 Aug 2017 14:06:03 +0200 (CEST) Received: from [192.168.110.2] (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id D430D601D8 for ; Sun, 20 Aug 2017 14:06:02 +0200 (CEST) To: bug-guix From: Hartmut Goebel Subject: Updater needs to support HTTP(S) servers Organization: crazy-compilers.com Message-ID: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> Date: Sun, 20 Aug 2017 14:06:02 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="------------EFF2D8968916C5A88B89D48D" Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.5 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.5 (/) This is a multi-part message in MIME format. --------------EFF2D8968916C5A88B89D48D Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, our updater currently only supports FTP servers, but more and more projects shutdown the FTP service and provide HTTP(S) servers only (e.g the Linux kernel). For other projects, the main distribution point has changed to HTTP and the mirrors still providing FTP at lagging (e.g. KDE, see [1]). A common case is to simply use Apache to serve the directories, but it will deliver a HTML view on the directory contents (using mod_autoindex [3]). In [2] Ludo wrote: So we need a way to list the latest releases somehow. If they publis= h JSON, XML, or some other structured info format, that=E2=80=99s fine = too. But HTTP alone is not good: we=E2=80=99d have to infer the information fr= om HTML pages, which sounds fragile. IMHO we can not expect project and mirror sites to provide these additional data. Most projects simply will not do since this would require the server to generate some data-files n the fly. OTOH, I assume the delivered directory index pages to be well-formed (X)HTML. Thus parsing the HTML should be quite simple: We only need to pattern-match "" tags, or =E2=80=93 if guile has some decent one =E2=80= =93 a=20 xml/html-parser use this to query the data. Only relative links without slash (except a trailing one) have to be handled. Links with a trailing slash can be assumed to be a directories. (Since auto-index only works if URL is pointing to a directory and the directory is marked by a training slash we can assume the generated links for directories will all have the trailing slash.) At least this would be a good start which could be refined if necessary. Please note tha I'm not suggesting to write a general-purpose parser, but aiming for auto-index html-pages only. Some things I already found out: * Directory-listings generated by mod_autoindex can be provided as a simple list by passing the query-parameter "F=3D0" in the URL [4]. There are other query parameters for sorting and pattern matching. * nginx's "ngx_http_autoindex_module" [6] seem to not use query parameters, but can be configured (on the server-side) to provide the content as XML or json. The "fancy_index" module [7] si documented to "Allow choosing to sort elements", but [7] does not state how and if "fancy" can be switched off. * Lighttp supports some of these options [5]. [1] http://lists.gnu.org/archive/html/guix-devel/2017-05/msg00237.html [2] http://lists.gnu.org/archive/html/guix-devel/2017-05/msg00292.html [3] https://httpd.apache.org/docs/2.4/mod/mod_autoindex.html [4] https://httpd.apache.org/docs/2.4/mod/mod_autoindex.html#query [5] https://redmine.lighttpd.net/projects/1/wiki/Docs_ModDirlisting#Table-sor= ting [6] http://nginx.org/en/docs/http/ngx_http_autoindex_module.html [7] https://www.nginx.com/resources/wiki/modules/fancy_index/ --=20 Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | --------------EFF2D8968916C5A88B89D48D Content-Type: application/pgp-keys; name="0xBF773B65.asc" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0xBF773B65.asc" -----BEGIN PGP PUBLIC KEY BLOCK----- mQINBFJQM3oBEACiUXmMppc3+A7JpF3lPz8O/mRhfz1U6F4EOTlacTrCcm7Xg5U2 JsPPNapVr5+vHnKBK+xbaX3s/A/G+SwrUZhi1X1HRnSNY1CqL8CT7rZmOtfjg4Ex nOuCjie2bKhb9JKmlJ7MrNimIbNQoX1mRcQ4VMEflhyWfaPGbT73siuMkbr020Ex hzW8T18JIC01SgWMULYMBXOBoGY21am/vaTFCK8bym1P4HVN8i64uOWL0agkAMHb ju6SZtG2fYJ68eS3P/97bXRg1pveEdpaFgaFZhquecw4WdedwLwt1xNcjAg/p6tN 73W3asEZTgMHa+iNzbJgcyhWpci09wQZfZ1uL0HdM+ohng38ccgu9hJx6YzCN7Fe 14JooKbPukG/WfClAgAzZSHRKpS4zGdGlg6D6EWayyWWoLR3KoMA4LIIlaQbqaOh fe85b4mNgB1hqd0uRTHOah/6T+FUoSQ1IAeKLIDqj6rW7X4ISRn1CXGSLGDn2QKq R3KtU3cLf8hAeDeO7Qe1jTvLrG1Mfca8lEmC7/yN1gI7L4/cs6lhmXUgMaevuxss BxO2kkh0OS8HVFf+QQ7LZ5vt91yQVT9HVvOuVob0YtG+3rvkpMaHQilKloNoEkmM iHpwypBaIfAC6NP9smgionvvmQ5RWSEaH5/pfSUAYbqzWbqDxtqEF/mPOwARAQAB tCtIYXJ0bXV0IEdvZWJlbCA8aC5nb2ViZWxAZ29lYmVsLWNvbnN1bHQuZGU+iQJA BBMBCgAqAhsDAh4BAheAAhkBBQsJCAcDBRUKCQgLBRYCAwEABQJX+5kVBQkJYJ16 AAoJEHt1KBG/dztlxhEP/Aml67QpOiUtNsDSaVoCdK6cG89UWLv4K8j7mz2U6Gjp NsFfFaWtBfb0EBbmi4fvqzQGi85PHRqnQ/nzlYwREmou3UPNeFKAiYFcYwcL4W9s YuiukJF1ZNXIVKapVp6JhVaqjQxM6GSnZof2raCo+JRPWwB0CbliExyEgPIUf5IR s+GhuDHbRBkh9GbUKfmw2NB91Zu6HSektPP83m52Od63AHxYPaXUVQ2/B5RCUcxy BEdoVnvwOnSuLTEUPyZGmnQYUrM4DXqaYGwGRVvyErijnzGt975zD8rY57kot55Q Tc+ZKHTlBBvgXJN3ZKWn6EAIh9Up0DR0Ke0HJcI86beUjZ5ck/VEaVzF321RzgUW 9CkLP69SiM3aIrhB4SleClWvkJa/+IMnh2v0Z6F/Hg1+kzr8t1qH9ZcsAdJ1w08h KL3HfWyMSr+ZHG0FfnZtEn2Nzjm0uOR+6RjQKMUtBqjPkv0nbYPhXGhFf8fPuDVn crTkJXbzZeOwymzGW667VDSBi+VaPw4MRE1vX91JnGmuOlfwLJalrqs5OFu4MoYh Pg7ZoB85Wirtsuwu1DWDb0I2TR7bm7ZCsWDNSfOS5JqRl/Ce/oe8T+e/ZhF0nrp1 KlP13hGZZqmYMPeoKLX7r3JZuUXGiPjqDpKoMTExzsJ/QSbwIqwrjxT8pgOgUgcO iQI+BBMBAgAoBQJSUDN6AhsDBQkFo5qABgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIX gAAKCRB7dSgRv3c7ZbfPEACRmUAnpD8NAA1z/BT1yPNqeX8dSHG4s9pbJQKmxLes 4VxI66LrrN9nktdHocreW8zKdQOAtMSIWtHSdINxrRuA4d3PoJ7hBloKRhRs1Dfg m7MqBTTOS8BpGXVu5XDT4QihanoI3/5pgYb06DKedjSnUbxLUEBirkqCfknOZWmx C6/TkAWWFSu3XIOwGxefG+85klibAW/3j9AB1UWQamM2dQ70FOCWurQ+uzaThjJ2 k+PBUWgHuujaPE5plArbLXd5JJVS1RihnsW0UzVm1FWADMDlr9tgG5hPiTPHPOa+ wba9BWih1BT0FBIEOuB2/yxTTFQ27QnpiOAXQjWubzPnjKqhs/GWvTJY3q9k3lst 5lVoa+xg/Avc4NKbwawCZ3bG8A0j3Wt44j/JErKCMCxIp4YYJtx8eb6Jud8lGr1g jTwdtmhd3mswfakmqcMTOout1cCiGlfDW3bXwd7s/9WZyyUoNsk4whiOEO55Diyd l+JVJB9pXOmE3X2sutxTj7TAoqb8m/y21XRDslzL2E6C82+UGvwN/tcC8B1k5y+w BTtULPyhMyk9jayS4Y8Shm9JA7sc/7YH63jgzBbjW1RjDLXJ1GiOoaO5wXZFrBzT WsmvP30qZ0+vjQnT2Ba/hz9QmS40j41P9KzUoThjbfpUFWt1pXR+LzsvLIc/XSV9 N4kCHAQQAQIABgUCU1wGGgAKCRAVIu2Z2EUnndSDEACuqXngMO4A4NlnkFjnTYfD 5Q9/pdqRmezISzwHKF3aoudGedtstZVE54Bs+9q84JeQDRt6HLSeB7Fg+0IWkALl mfqwvIP/1kL9nIwd6jJ/NsO2/ehYAbmKObIafSHM00hVSe7ETDpTUx+HOa19AkOL VxhxuQDgN5St/vImWQFJh2kK9B/RGAgxAudtWKJHGfSg85eg2gmyn2rTmdDJAErn 2UWy98jzFp1SzuyEjadNcMcwRCkEQhAlv2ZRqUJWzxC8IQ3nNj2ci0e+M4ge3BwC Onbt/F26i8iksn0X6tLB+PhRiyJ7Cdbts/QKVo4PHT2AFx2n2/jiIdgkQbX8SIU1 nlY507QV+mnhWC4ESSXuptlpd2jej0k3M4iLEV68pl1vwiAjgTHhNaLz3odQkdC5 NqMaFVYd3eSwLXV/gaYq53SL+W3FXVF7PI9hwqfggyHCyEATH3vf4l584Oj54Ci2 7isbz+7aqBgKMaRJmtvm0yH8zJh9yGjOQIjzJImGNjGcCUMLkTXDKF3OyJawCGn1 JSlDkuWoDOPaTYLjQ5vEEkpAl2pLq6gtSFP66nzObHthRpPAFTPEbqsuVx1iZFQC hFDQ9J9L2wWcoVAe9/0L5m4awWyjtSdyP+0wb+gIIvgAq1KwDsh2uzI+EZyKDoDi jAKPdBdRYWQMgv+7oZ9OzYheBBARCAAGBQJSn6+xAAoJELfvYUaKFOZUUMsA/2vu lFb7bJAWcZpILzNW8lu6gT4s+U3MG7hjVEureJuKAP4sqDZYRmtUuEfKjTsLBLmV LBMjYk/tTKksMiajHxiDKokCHAQTAQIABgUCVOuE+QAKCRDrmXakz6NJM5OPEACo EllO40HpT/H/szbLvZzFiY9BhH8T6dHYh0ICjRkziit/fs3mLAzrFcciSYU//Gwt U6TWV2v+lTqyCkdapEhTZ/S3cbhvwEFXEf4AlZiE2oIhEPGOecrjk0JNhHmpKi09 O4RKdCETAW7ejhuzksWxj3BP1iIOeuWcpIsOAlF2Qo3ejfE+LQGRy6nEnBFS2+WE 3jQ2jRv+I9VUX97oiTwKF3bpQVWAeXSsqE3bhVimthAa6SdxCs/V40us2MRCe+ae bLmsJBleh8wJlNUbirT04E976FZAQX07W+rstUpVgoBsTvYm4jA/yZDqnt3Ml2rG h+pmd0e4CUuiKGUtE1TyDZ+rH1GGQqt32SyMMnjs1cGI3l6JffOvCMzm+cc86/+S aOi4pQ4vs9MpOtJVN0FXep2JPe6DkwnavnpLO+5tEQkuoSSBBHmwBLk1HRZwHSDD EfnzOkP/djCcoY024cgvIIgtu4h5TsTXkb5vwH3t+6E8WaJK6Wx2XR2cLdF65SyS orqryO3jLPVMLAEBPaX1dOVNSsMjAWtjIHYXWrAVLQFpDh9Va+SVlDhNoKoIQoa4 Qg/RYdUB+KxamiavmLG4wyqv3v0Pi/lA3fQoXRnO6+HwdCNU6vPb/Kq5iguRS311 6zQ/qBUgILx5Ri/F3JKG4qTIfEj3lAcxdpPkO54BkYkCIgQQAQIADAUCVcxKmQWD B4YfgAAKCRAhEV35W7p8RHw3EACEjIKUek7Rxa1olNxURMwzSpUVGLzlMRz41N2g edD4J/Rms9QdcTMcnyWpj7kUOUHaanO46lr6NRNx0b/1HDTCPxjVT83QiqMY39OX CXesTL8Y9JVQbvEN3IwwemQo5YGOaf2oisL6CwBP+PmKTqqugClNibCdtmuQYTMO QMrgcba7VHDlhH1JiRqIPF6+CgnOFwSCF+cwJPK3zJhe7QYZ6FGXwLnTSJQw7Jxh 3lF50px1dXh/5iIMAg6oTEShuHbBV6R/5FIzovvrkOAHopidUsARoc3Ca7Ar1ZT0 De1RZ3SZ6sOczo9cKIvPRLGAMj3iCe2ovRVWABo+gMEGjX8YND0UcUywXn4CIp7k yAjXvqtkWyG0RU7yFmTLaPJVKGxAYGCBI4unI+4I+fT46gp9gIA/LJQZjpQ5x1Ji c3gtKySv7wN8jCJRCx8qeaBRcWONuZeTEZnfrO1MK9ftsaREW9wxXDtBKijjv7Z5 b0L32oS02nvTb8RjkbijZyZgn/YU3Jxyev0FTM3e//q8q4qelaHoNfbIaM8V0OUN W9m8zEwFczq9f10+IEXDoL5MU/lsYGphlPfAU/Is6nhWKBgcg6bIfW+cdZgTrjGF 4rGct5NGnzvHcmieIGr60uWwDRVa5wplz4GyrLcfJmCG1VnZf8CTOaFEeTL13ikA jboU2IkCQQQTAQIAKwIbAwUJBaOagAYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AF AlMgZ7wCGQEACgkQe3UoEb93O2WMxg//ahuuLNR3llEcRfx+z7NS5hWp+NmtXK/X xjThEJFn/H4Vm4DI1bH/NKGPfZIvFQsIPFm/kg77om4BNHcLXe9whYJGYEJaZS3g 30ORjNKMTkV4xupfaaXrPZebz4G3uk+H4UZq2FTjyFyaMRv1hrZEEe+x1LvMSQFu B9wwrO7jHzRgt3tkfpWHZDMDmThDkbTvDJHgBYo+xWoAAXYvN7BHb8lpzaPdzFkm tUbyvNCFozDyrhaY9l2xPCytTETlHjUsy/rZo5E2x+aNkZhUMyReElOD18S6Ct9Y QPdJlCthx3tiLzr91CXo7fEEIBuh7sXMLP18aNtpbaEo2hH9Go/buwaHyKt5twPb Tuh5lkfe7L06ifXeI7cHEPIALEDAbaetpKcmlqLOZM6T3g8O7c73CHaat4hJ9+OQ 7mnM2iVYlP2WxpYMYIZdXpM8661PS/6y0IgVxEN+v/TaOuE/C0Zqp0EzTvRANF+/ 5RVXrZQ5Br9StsIb5Gpyr+NQ85lfxCPkx+3LgKKlK9Lh+pKNWd5pocTmevgyetOq vLEyQUWh7eCHyta9KJiy+rbMIxnqbcMVX49TNWyllWPi8XR0OL1f8tzgbjQiLXLo vWvRHYMMVR4i/BOxjMEjF/l+C77vEzF5oTSQhe7+T34jRyG1WszI90rhH5gS6ucW 9XmI2JDfu/20MUhhcnRtdXQgR29lYmVsIDxoYXJ0bXV0LmdvZWJlbEBkaWdpdGFs Y291cmFnZS5kZT6JAj0EEwEKACcCGwMCHgECF4AFCwkIBwMFFQoJCAsFFgIDAQAF Alf7mRkFCQlgnXoACgkQe3UoEb93O2UDrA//V4A7Wiz+r+KJADLCSoi1LAxwqslY wRDYlQ91Z3qmmZcCtG3PyrTOE9KaPFMkdf1dd5gASCBXbP2n8lz3aTe5oR9BTcmW JWPj6GftzmUz3YGsA/QADyDEEl9kBfpsOfDIO/cgygUIn4TYAe+/1mmi2yW4QJ1q BCiFYsClTgLnjNkeu8A3vuEUWsh/FyTvszCJYjCeepbs3kuUoUqcHIqriPfTBszJ KCfF4wvx+dZu2wEQ8uEX5493VNSSNGp33Lrj7zl1CVW8+Yh1O5tJzFdVSy9aa1kz 2Des10n9prxT06IpfljcWJOOiCgsm/4mW8Lysy2M3nnXiIbW1BPrJh0VBx9p3giR yE83ofBOTGhnUl5gcJgLEE/+g0xL7dRzaDGTs0rQ23Cu6YCyvpn3j1iVmhT166Hb 0eALgK8ne1aJw6bOgSpI5qy1UBy/obpHhz6mgq76EZAfK4ub/aLwwjkdKYyIh6xd 3yRto0xa2USTp8/JVyq6X1ZLCtfB6Gp+yQIsFQGAHXbYX2A2RkMkUBXcMa5ig/EC UrYyRx8csOQ7330cLEOfnn8J0lC4dtcwGIm2zo4tZJy7fW6R2dw56PRGnG207B7Q 2Yty+P5uhB+9eNTtM1g8MbVfLn8JIm/mYOtYyo0hMJoNisZvxclZgwXXop6ROqAq NcI/myaCIB862lOJAhwEEwECAAYFAlTrhPkACgkQ65l2pM+jSTOGTBAAhQitQDJz +f3HSovpOmz4qXCLjiTLkQM1TLifRa0f5UgSNWomrpZ/KticGZ0PfRx+d5t8YDCB 8iQqrKAsY6LqMhXEfMJSWt4WndGiIqf51w3sPSxkqWjgTbwglNcj+7sB0Bu1CDfr yc+r5KBBj/kV4Jx707PLMYpqJmczsLN9TQsEd0Y21trnsjrZkFP6zenX7P/ZAscw 8gbBq+G/Ge6qVtyyrx9KzXBdr4UYwlXxZ83znclTl2KJjTFKjrptsXBGMC6OmHOU qCXcFYf7WYMRiOle8kN1C8cIR9ENNySA15a3ayUA/33o4JeVBEeYAbERAdEqqlIt 7xGVEgG65kvj2pnDNly2t6rodLooZgDZgMdav41Nx2MZurbQf2IiN3jLVE4JoGTj jYUWVaaIGdX8Xn0o92u9uWB1hOP/937ckQPygnk/ldBh/vy/ttYIe+NZN15ptSom TZxjkNEzUs+X8i0Ny7QxQNvghzJfaOcEpzy+dlxyuJsqWYrkU4iaEU3VSqSY8fcn JU/IcLVmxngF4LoDlYs24eZC4E6iyOtSHaiiCDW6Mq2K74c37m4dJlEMttGvcSJd 4Z5D/sAgS6it6BKw550y8Hpczd4jiy+wllOjR5Cs4+ppZd3C5hYWSPZlyJ54coJU a9q+3AvhxdEI/6gfiWTkEBmrvbpCSKAkj8iJAiIEEAECAAwFAlXMSpkFgweGH4AA CgkQIRFd+Vu6fETXsQ//VZCg1DIT9yHx2gC309Alp59cO9WNVPXRfHIpqCxFWYW3 KXVmB6aCWoiRwryQYl6LrGnmOIXjRQICDQkMwRQKaO1Y+C93HKvdqnY5mqa10hJw YcnBA11ShV/Ya4e8ZA/v5/IUDrt5UUwgUgjifnODKaM2RmuWwdEFzVcKov95H7RR Z+QESJURzJhO3C0PwFimzv3di1iKt8PV4yTYmERZpzQM0epDFCOkNMDXqGUNaXUM hllT3q+DGe9eWFjvtqB45Iqh4ImfQu17vqOW1VtXLUc73fyGkkbmKCrYJi29zQm4 wwRvbhJvjQPQctEeZOT2QyB4jqjfdjOhk7Z7G3tuDka++LJ1ybudysmamlIkooBR rSM5KxIesz8fwCTqEHBNQbawVE8zZqhu7j5suwm39QthqMHe8bujQNr2HO3zq2OL uaSoOruMGVeBa4Ax98UEah0AAtvibuMHFvXJJ3MV6W5iPW2MkByRvgpM4tL1Ez2v ezUVrjLjLxp2R4dZyGcgdtO3WnKuydFxzYUaUZuML9XoMxjOQ9l0UZS6tkJvSqgH wrgV7kv2qQ3OB9Euq7lC2brBh7khmQ+VJE2QOEfMPeJL0onvEq2mNF4N9o1lZiFZ pCpcmmp+K2tsMbRG32UfmQ5acJT8rlti1LSNFV7RIxLS9l2EWxTkVN0YWNeJvZKJ Aj4EEwECACgFAlMgZuACGwMFCQWjmoAGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheA AAoJEHt1KBG/dztlbEsP/38Y+5gWkPZZus7mhX8YjiACjUZeyA9d1YoSSlpbpBOY hac8tmuw+eYQKV4OIjSEzSF+G0dAAyXlCHVCWkOQkHPoRzSUa36Ejof6vfTAbed1 /+9qLL+hsBt/Wt71mGUH0aQdw7Jit5v7R5vzzJsDL0cKunmQetml32lXIQCdyKAL IZPj5Nhvqc0KHgmTFPJJ30ENLWyaZlKrKqPnogzZT0i3cE+QwcTdIl+3BWX6k1/7 3IK6V3iQm3lHOXhDIiKtZFRPAxphUT31pmDQ3jDb9kzyYMaveIlakWh7QtDP/01Z +gdzRN32kQ5mHIlHmwiWUWjCLmj/a8egINa/h+kz4MJUN2j+t0ALRpXvhd9y+OdV 1m/1QXwLI2d8nsB8ikBANvp+tw1GyfdFT+g4pI+RMyFEfvCbSCJ9yvKiWvSZdgzh /YlpIPuKHl+X8b01sJ+Zutx2zwcremJEUaJeRjXCEdT94/zElnAuRSbP04eE5Zga HibKv9x/LgeLjnx76iKw9vgnaFbPkL28e4sS7eXBTsE2DwT7FX176LlYKFKMvVbO GYR9x4TEFVX7/Jr/aOQXVhQc0UY/mo3xE5tMHDk9X2e+ka85PqgVr6B4Ik3hFjP4 bIBfk2sopg2BLucjtbg3y+92/AKvamt/CdXXdGRgpWk9dKoeWmOqtyNEnQyM1BUg tC1IYXJ0bXV0IEdvZWJlbCA8aC5nb2ViZWxAY3JhenktY29tcGlsZXJzLmNvbT6J Aj0EEwEKACcCGwMCHgECF4AFCwkIBwMFFQoJCAsFFgIDAQAFAlf7mRkFCQlgnXoA CgkQe3UoEb93O2Ua/Q/9EKcB8wuKtqpxLqOoIPdySnpNnTF6kfw6I+z3KWX1fbhO 7QhOGBSmCIQd0MXlVRZUNtyImveW6iiEBDnEHOHknuZ8xalXI/tbIwPY3bhuZVWl d4h4gmHMhDuc/nnNt4sgPoc+e+RjhGPQYQYbFPwzdH7cu2tEsW9iXKj6c3HngEFn wO2Y+3fkL9PMstBXPijHfmVXqVskAVjCh4x4HrArSjhTKvwGP/F2Qppv5rxBMKIk ACqMntEgIX9luz9TBqG+Cj++xlBcfatBwcpb4nKP54s6TrwysmhyD0qtjo50Dbg8 NS5zbmgz5PiF/xzFL7+LUQfRuK96+5tP3KLEr2LOMa8WtW8wDjhUWU+UQznk+y1Y 06XCRYcdQ+o72rRHQmt44fOPN9PBnf2wAFT6lmbOUT87TLvJoLa4tAlzGgDeA4Jy h/QmY5LWls7AM6ZXBFmQOXXvSHLfvB7L93g1Mgpl0LYqiqVZ79PPWQXanzw78qrE BZ+ftcUkkflbRaW0oqPFNl3rOPIZPZTEuM4NTIczpfQvTJeENeq/Y6hzcfJTOPcN UdHnT4oPlP3c4zH13rAlt53Po4yxf6xnFyQXf3jGSzwwUi/uRXPVoL/6VmYqQlfl EXbnrglZW2MCVLZ9gvu0XONa273UlwjLRytetGuX1KZw5/2cPHecZe67KQE4Re2J AiIEEAECAAwFAlXMSpkFgweGH4AACgkQIRFd+Vu6fEReyA//epitdszV5ZQuuL4z 8XPibWlTaVwLh2PvW0dEB6ID3qPdGINhfff9OT1rjAzSSzH++GAhDjwVmeI3fKBy 5ZNnOKSw+QmBZDOVcMEdf7EbeA9elm2IEzjEHO45c1ZU96jWwD4TX/32uA8CYZ1k aeqVl4Pnu34rQaCiSxdtCbC9AATLL3Q+3ezImwTXeicslnhT9HdwQKkiGfzc0MJM wlcPAc6QSD9sXGZNPFb1+aEHdk2xB381jGhPuE0nyKGfpJadlkW95yFt6hcRi25s DvypcDIRU8QLErH7Efi2kYtza3I0zmsWXarZ7hzDLY7FGpwYO8nvYFgL82AFM/3v TnWqv8nqM9RX5F6B9rYmM9ln+VmzUV8Ntr3nqXPV3oRJeeqydR5NI8sB1901d9PP WpL/QN+acmMh0zgapMUBzBETx3rhR+Ad1ynZgeMq25k5tC7+BK7sd/2rxie/u6vj 6CWYKHNjIRNHhYPspXcI1KlsqTYyzpcB7pQaz/CrXVGi7tpVBpOHdxGsbDttjahu tDPAvrSryBn4W0/PWn001fRTLkyRzRNCOvp7WqcKWmTUkfWf1/IojTGug0WN3Q2S /Il8OcHAb/yxNQILSrS+OaLnDFlKayV1lvaN8tweqdMudhHJnm5SP7NFRQvy2Z5g X/lSu/9rVAyHxAzPzT81/4kFdlqJAj4EEwECACgFAlRyUlsCGwMFCQWjmoAGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEHt1KBG/dztlqJkP/0zKrL9PbZ/OuSeF 2UiORhdr4sr2awJZNdQYIcXNLJbqa+78J1lL9YCYmoiR0msQ2RkQlP7J7XFJD6Z7 nHp3iKA1ZezKHJz9cESZClK6RD23HpJtoGR6farGZXDhb1emeCRgMSRVZColkAmv K/i8FNgP0Zgje8nF1KyZX/Yz2A76wVTI1EfsTs41x+XoT8EC9nPwBXEFmLrlPYUp VjffMAdrTC1nTE3uLUAPRgcPhN/h+w/xdQW32IGQU0YEon5qHldYmWpbetL3S4ef a5cCyJqYx/nKAQCpEqvrFDNbe4Bmn8RWEHNibza2HpXYPsqo2fdgXMzcNkJvjtU6 CSgRTOxVVuvGRZQZUpCdP8WjV0HohPXZ6Y8iOVlsk9vey11EAMElDLXKlQXucJ20 864GUZHIO1ENzAs5Kyv9fdAYc7F+XeHfG0D+OUnJ6E+1J6DVBpdSNq4/ILteO4Fu rjRK5OrpPLzoDvBgwcdyw5pzqAoLFC07YWBU9ci4SP0yAnYCNq3TWGMeOypfA9qH L/AIJvSdPhgIAH4q0b4VboxChHOhrJinxPWY7JL59HVJwdFwwzCxuo+RGn6fLcRa bci0MEuATvx4pND08mmZFCqNA0cCwRQRARTDGeYktGC8yzgH+w7diZNszL/BM0/D manBaawPK7m3D5wFrbA9UXy+nNkQtCpIYXJ0bXV0IEdvZWJlbCA8aGFydG11dEBn b2ViZWwtY29uc3VsdC5kZT6JAj0EEwEKACcCGwMCHgECF4AFCwkIBwMFFQoJCAsF FgIDAQAFAlf7mRkFCQlgnXoACgkQe3UoEb93O2W+og/9G8CCk9yAnxqn4UUaMOum aVL041k2YDEiJd/KgDTIEOrSnPCuJ/pCrI6aNiChY3jmjAqxittV8uhyDlvmBGu+ iwgdhHAyQTHQCaV17XqdvVjIG3dMt6IAm1+hZczl0H/OCBvW5MEVBD7wYCQ05Ssf FXvV8a2JUaD2iwnjRNZMd5OqpMNz2ws6H7zoy1RrzS0XQeBtLy851DJqphc2AjDH PI8wbZLcEXeiP+fSMMZgR0Eu+mLA5XCeNiJbVcjwnDc2CndLKErsFNhuWX2+HFZo ihdC93RbCVXWsFs1NCcD3pWR84tMZay87ONTyLr72CHDCFGhcVWdCcjVdRJiaSLr kLkB+UFeQ0KmlUOMRGTbdpCi6dJxbpy/gMxAEQRSFCZeR6yM4rl8AOqdkpbW/DOu sjIiVEJPE/3QLqx3K8Vr18f9zuIJf6q01ZEzCtD+jrj+HD4ciTnjrpqUdg6INu3u JQnYFTrUwI4nXYnzXgGFzgXjTPIKt4QEfE7D5/zFJLbqRsQCmJh5r14mShbuLhR1 XH7Mfgi+PNVg6Ko8j1ogBRZuLSUaGMw3rrweyulvWZYlpzt2WVmsVd1LIxs8yKUH WEDDyIBS+7RIlI3VnMAZ5pBLAqIJVFANUKVHgSU4HrjghTmm70R4wYRwjPBMvB/Y aSMX7Eq9X0Jc6RYOocDX+UeJAiIEEAECAAwFAlXMSpkFgweGH4AACgkQIRFd+Vu6 fEQ3oRAAgpy8MdVf3NDT/+ntWeOY3ltSk7xsBnILeudGoq2lF2z4N/JCJaqnl0i8 fh4L4bYOL0qgzVG2+dzkZ3PjcX2CzLHDhcqgjqIYqDbVQ1BOIv1vUdLFK5b2I9hC KH8KOp/MWqIwX/rj7NFK670TtiLvgiM/z+PUSJ9aiSMEIy35OgodbQ9RVjq9HZlE s7E6sHXiKqkDlagM0QBXg1cSD+192RKB5bZkOjxzkqWGJ3IyCYJFpD2VSIjEe4cJ upOIlW9GEYdZ4H849PYiFbQ+zTx+VilhP8qGmYjSMlcrVs36sQso1P4DW4h5H6AW NQnSJkXhkHwGt/yh0Rt6i1tgHyFVwjPIUYNpNG+G+YBNz7ClIcvivA3wqeQBpA8n 7V9bHSWxyJEGHA1Gpo+zzwnNU1aQVJP+JIRy7qo0hxS92qyS2jn62zx06gJ0XM1N uyq8aj9ZKWA4NeKhxbIkEoiYRwGg2arQOK5arjOVMbBTMb0HouV/aXbNbBqLQbzD JE0ZQFMAZEb9M8SAtqi9Bmqb/b0IZuu98q+EdDWZYmGPB9T+fdgB0smeDmTGZaKa bbg+RJ9RJCfQsG2HZbIT1Gbl1uv241N9QXZBHWPHp7j19LxHV7Qo3V713QFYsbiS DrOITZ0p3aezR1JLzQlkczGR2myaOSxrb5ucS8rdMdNLz+g1wa6JAj4EEwECACgF AlTrhl4CGwMFCQWjmoAGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEHt1KBG/ dztldoMP/0HrbHGRL7LT9qtbr1Oe8Jge5RZPQQvCFKzNUaiJqlci/0c6bJtEFwn9 53tCquvBQBc11y/vsk+0uSUaa3zLTOlGtHHaEkAgjv45QfaimXptTd3qPHkT63ZS NWMZ6C4RXsP0O8NhNw9aNfbVfk/jpJffiTJ6dnCsDVX2KAXHKxNfevY8LcBuJTSi 3hYScMOgqlGWZn+pwt7KrgpJ+cbsK49rbyGhnrePdCD/U11Cp4aOM41360Wc1EXJ 9yD8eTSqilgNh0+z6jFBFcVp030kjfI3nP76e5WAT5iF7j/iJF2hZWiJPN0gIYGv 2YRGy4XnsmgCXh82ynyrmSlKagUPvk0mhj4gWtMIQOpp/+hIPCHEOqSge0l6pvW2 rO716dJ4xZCqz7vdgM1xiaDeodoLEDRuNweuWcZ34c89lTm2m8/e+RN+kS2caD8o gg3scqFeLQtY9ZwHb7Vk967HxtY/GcQHInAE2/xpb9n3dgWeBaNdO6TcfDOSeVUN +hhkc9ylmwbgilDHg44k/MM3f2O0xTbUSUg6q+0iJ/vT4N+dH6+aGEYKGtN3ELAp q9K0y2rks9A6I967+QYJOsUVF3T+sDEiYh4ZlMeOgjdKw31Puv330q4C+ZMlar0i Zsfg/xetSLp65uWdoF/CqIjJwDF5P509eXTiT+8kv5U/da98jLvguQINBFJQM3oB EAC95Ts/roR7nRL6LyVXRMZfjExy6WHGowNEC8w13tI4gGHQfiab5uwgCmp4utf3 qNUpx8kTkRC48Zf3y98rY4N8kb1QDNiPsYO5SAeKly+a0eSvIheKcf42FdggSIsD ikOoFRFZNoaZf7T6j90HTil3S7t7DjNWIINzzeX3PiOwepbnIcegbuzQEe51ip+g 9TXTiNVGS+g6+jHv+oTVSQRrwxdxZtsdAqCB7oxZLHzsWbcKGTF59dzdfGQ8kDjA PPXhzUJRa4vYucRt5WjAFtjNG6M3/hcEwV6JiFpE9k5I96JlvGSHH2oLNkqUZEJe OaROuZyyit+nkTaDWwaL6impTMoSrLRU6SI402UTCORGfO4+4GF/4IQrLNi2c+F9 Fn7x+pWLfGL47yPUrZeBDbzbpEGUmYCKWmgIGXHl8ri2o/hWNXFVytwKE4fpaK3X 16900gcZW9c+3DR6rCPb3UcLHwvjYPskazRpzcNNPH4A6vX4WmbrNZK7T0M08icz MPcZyyrh4RxW5Bw2ltRm/XZxAgRfGk9JI334BDRGuSum38gBhmec4pAbivRGXSEN CoCz+eYXZ2Oz4oF2s0rL2MVQetvgdJEWMY8jO1AF+efkHg+6ZD4QZkaM3jWGc0Mb vXMng/N3tWH+3XZwZJC11ck8yiLrRodkdgZHJpGwSk79uwARAQABiQIlBBgBCgAP AhsMBQJX+5lPBQkJYJ3PAAoJEHt1KBG/dztlvv4P/0jNxvHfEu/WfodAEpo+nZhN PokGjVV7SWpLMGfxBakWv0kkZakb7t5YyWmo6/qajxKBmIYOdDAOqF6oHpCRjc8M sNY501VXcNlURqX2tnMmhOmBG2cTIPwTmQlZLSpfaNKuzNGshrdJ+N0eGnG5HPp8 xbQEjSd/vcxBjVh3nD+OpmkWDQR5EVSdT5YnQW3gUf2mUSLn17Hnel7ANfmhLSQC Ye7OOCEhny4zBnDCry6RM2Vooskr2fqmFGgvKVBOKoMyU+QHtvAEbTluqi4L2K0c Hnzjio3oAyrTLNIg9uqT2MTSgDmpHYVBFgLC1q/8vm8Y0+/CV69O+9rB3cm8oAVl OKLcQg+BdcnIwvU5ot7aZQ5u/75ALGyhMvOF/5Gf8KKVxlYg+nrsZN/8tNSqm489 dWwxfCqIrvxH/3hGjbkSt25bef0h7p7FIJe7h+cBWXTtm3Euh889nXccdfnqPUkY ardcAipeIQaggBZxiTH7cJYu1FaJlKWNt2mzVZRMiX79YCyDvAjSmkiB8AFMlspX Bo/d9Iy0ldRocCJzprSjz5XBFX6WxA0BmWWbdUade6CoBirib6fo8o+1TjZ86hv0 44pMWL9BTJp8KuJDpfHmiO8YQb6aZDOqcWfHyygBn/bYbi//GKa03j5Ij5INNJXZ MKO95dDUGyQf00JGQUCZ =7jty -----END PGP PUBLIC KEY BLOCK----- --------------EFF2D8968916C5A88B89D48D-- From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 22 04:57:34 2017 Received: (at 28159) by debbugs.gnu.org; 22 Aug 2017 08:57:34 +0000 Received: from localhost ([127.0.0.1]:48580 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dk509-0003x9-Um for submit@debbugs.gnu.org; Tue, 22 Aug 2017 04:57:34 -0400 Received: from eggs.gnu.org ([208.118.235.92]:59066) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dk508-0003wr-N6 for 28159@debbugs.gnu.org; Tue, 22 Aug 2017 04:57:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dk4zy-00063y-Oh for 28159@debbugs.gnu.org; Tue, 22 Aug 2017 04:57:27 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:54698) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dk4zy-00063I-Lt; Tue, 22 Aug 2017 04:57:22 -0400 Received: from [193.50.110.251] (port=36974 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dk4zy-0000FZ-0q; Tue, 22 Aug 2017 04:57:22 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Hartmut Goebel Subject: Re: bug#28159: Updater needs to support HTTP(S) servers References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 5 Fructidor an 225 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Tue, 22 Aug 2017 10:57:20 +0200 In-Reply-To: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> (Hartmut Goebel's message of "Sun, 20 Aug 2017 14:06:02 +0200") Message-ID: <87poboasjz.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28159 Cc: 28159@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hi Hartmut, Hartmut Goebel skribis: > our updater currently only supports FTP servers, More precisely, several updaters rely on FTP (gnu, kernel.org, kde, etc. see (guix gnu-maintenance)), but others rely on structured data retrieved over HTTP(S) (pypi, cran, elpa, etc.) > but more and more projects shutdown the FTP service and provide > HTTP(S) servers only (e.g the Linux kernel). For other projects, the > main distribution point has changed to HTTP and the mirrors still > providing FTP at lagging (e.g. KDE, see [1]). The FTP updater had the advantage of being simple and fairly generic, but here we=E2=80=99ll probably have to go for project specific methods. So I would suggest picking one updater, say kde, and implementing it using whatever metadata can be retrieved from kde.org. This should be simpler than trying to figure out a generic method that will work for every software project. HTH! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 23 06:20:09 2017 Received: (at 28159) by debbugs.gnu.org; 23 Aug 2017 10:20:09 +0000 Received: from localhost ([127.0.0.1]:50392 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dkSlc-0002ny-N8 for submit@debbugs.gnu.org; Wed, 23 Aug 2017 06:20:09 -0400 Received: from mail-out.m-online.net ([212.18.0.10]:40659) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dkSlZ-0002no-Oq for 28159@debbugs.gnu.org; Wed, 23 Aug 2017 06:20:06 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 3xck2D4xQrz1qsZC; Wed, 23 Aug 2017 12:20:04 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 3xck2D36Dnz3jgYF; Wed, 23 Aug 2017 12:20:04 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id LwCVYh1ebD3B; Wed, 23 Aug 2017 12:20:03 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-93-104-85-186.dynamic.mnet-online.de [93.104.85.186]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Wed, 23 Aug 2017 12:20:03 +0200 (CEST) Received: from [192.168.110.2] (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id BB371603E5; Wed, 23 Aug 2017 12:20:02 +0200 (CEST) From: Hartmut Goebel Subject: Re: bug#28159: Updater needs to support HTTP(S) servers To: =?UTF-8?Q?Ludovic_Court=c3=a8s?= References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> Organization: crazy-compilers.com Message-ID: <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> Date: Wed, 23 Aug 2017 12:20:02 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <87poboasjz.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 28159 Cc: 28159@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Am 22.08.2017 um 10:57 schrieb Ludovic Courtès: > So I would suggest picking one updater, say kde, and implementing it > using whatever metadata can be retrieved from kde.org. I'm not sure if I understood what you mean with "whatever metadata can be retrieved from kde.org". By change, download.kde.org indeed provides a "ls-lR" and "ls-lR.bz" file at the top-level. I was not aware of this up to just now. Using this might be an option (It is lagging a bit, though I think this is acceptable. From what I've ssen I guess it is generated each hour if some file changed.) So for kde we might find a simpler solution. But in the long-run IMHO we need a simple html parser. I'm not skilled enough in scheme/guile to write such a parser, sorry. -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 23 17:30:47 2017 Received: (at 28159) by debbugs.gnu.org; 23 Aug 2017 21:30:47 +0000 Received: from localhost ([127.0.0.1]:51801 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dkdEd-0005eR-31 for submit@debbugs.gnu.org; Wed, 23 Aug 2017 17:30:47 -0400 Received: from eggs.gnu.org ([208.118.235.92]:34863) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dkdEb-0005eE-9n for 28159@debbugs.gnu.org; Wed, 23 Aug 2017 17:30:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dkdEU-0003kv-HT for 28159@debbugs.gnu.org; Wed, 23 Aug 2017 17:30:40 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:57044) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dkdEU-0003kp-Dl; Wed, 23 Aug 2017 17:30:38 -0400 Received: from [2a01:e0a:1d:7270:6a6c:dc17:fc02:cfda] (port=51178 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dkdET-0007EC-SG; Wed, 23 Aug 2017 17:30:38 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Hartmut Goebel Subject: Re: bug#28159: Updater needs to support HTTP(S) servers References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 6 Fructidor an 225 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Wed, 23 Aug 2017 23:30:35 +0200 In-Reply-To: <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> (Hartmut Goebel's message of "Wed, 23 Aug 2017 12:20:02 +0200") Message-ID: <87ziaqj7k4.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28159 Cc: 28159@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hartmut Goebel skribis: > Am 22.08.2017 um 10:57 schrieb Ludovic Court=C3=A8s: >> So I would suggest picking one updater, say kde, and implementing it >> using whatever metadata can be retrieved from kde.org. > > I'm not sure if I understood what you mean with "whatever metadata can > be retrieved from kde.org". I mean using package metadata provided by kde.org (maybe they have a JSON representation of the package graph or something?), or the =E2=80=98ls= -lR=E2=80=99 files at worst. > By change, download.kde.org indeed provides a "ls-lR" and "ls-lR.bz" > file at the top-level. I was not aware of this up to just now. Using > this might be an option (It is lagging a bit, though I think this is > acceptable. From what I've ssen I guess it is generated each hour if > some file changed.) Sounds good. > So for kde we might find a simpler solution. But in the long-run IMHO we > need a simple html parser. In some cases yes, but maybe not in all cases. I also suspect that something that attempts to extract the latest release number from a home page may be brittle. > I'm not skilled enough in scheme/guile to write such a parser, sorry. This can be done along these lines: --8<---------------cut here---------------start------------->8--- scheme@(guile-user)> ,use(sxml simple) scheme@(guile-user)> ,use(web client) scheme@(guile-user)> ,use(sxml match) scheme@(guile-user)> (define page (xml->sxml (call-with-values=20 (lambda () (http-get "http://www.gnu.org/software/guix/guix.html" #:streaming= ? #t)) (lambda (response port) port)))) --8<---------------cut here---------------end--------------->8--- =E2=80=A6 where =E2=80=98page=E2=80=99 is the SXML representation of the we= b page. The difficulty is to browse this page (using =E2=80=98match=E2=80=99 or =E2=80= =98sxml-match=E2=80=99.) HTH, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 05:54:31 2017 Received: (at 28159) by debbugs.gnu.org; 26 Aug 2017 09:54:31 +0000 Received: from localhost ([127.0.0.1]:56007 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlXnT-0001d6-2v for submit@debbugs.gnu.org; Sat, 26 Aug 2017 05:54:31 -0400 Received: from eggs.gnu.org ([208.118.235.92]:45954) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlXnQ-0001cs-Tb for 28159@debbugs.gnu.org; Sat, 26 Aug 2017 05:54:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dlXnG-0001yF-VE for 28159@debbugs.gnu.org; Sat, 26 Aug 2017 05:54:23 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:60693) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dlXnG-0001yA-S2; Sat, 26 Aug 2017 05:54:18 -0400 Received: from x590d0e4d.dyn.telefonica.de ([89.13.14.77]:49840 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dlXnF-00022S-I8; Sat, 26 Aug 2017 05:54:18 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Hartmut Goebel Subject: Re: bug#28159: Updater needs to support HTTP(S) servers References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> Date: Sat, 26 Aug 2017 11:54:11 +0200 In-Reply-To: <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> (Hartmut Goebel's message of "Wed, 23 Aug 2017 12:20:02 +0200") Message-ID: <87r2vybqnw.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28159 Cc: 28159@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hello, I just learned that ftp://ftp.gnu.org will be retired on Nov. 1st, 2017, so we=E2=80=99ll have to implement a replacement for the =E2=80=98gnu=E2=80= =99 updater at least. At worst, we=E2=80=99ll parse HTML index files like the one at , but I=E2=80=99m trying to see if the FSF sysadmin could generate an =E2=80=98ls-lR=E2=80=99 file or similar. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 06:33:39 2017 Received: (at 28159) by debbugs.gnu.org; 26 Aug 2017 10:33:39 +0000 Received: from localhost ([127.0.0.1]:56071 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlYPL-0005FL-9b for submit@debbugs.gnu.org; Sat, 26 Aug 2017 06:33:39 -0400 Received: from mail-out.m-online.net ([212.18.0.10]:43881) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlYPK-0005FD-AY for 28159@debbugs.gnu.org; Sat, 26 Aug 2017 06:33:38 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 3xfZBS5Z26z1qrdc; Sat, 26 Aug 2017 12:33:36 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 3xfZBS40Cwz3jgYf; Sat, 26 Aug 2017 12:33:36 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id 7WIr-80G4Xne; Sat, 26 Aug 2017 12:33:35 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-93-104-85-186.dynamic.mnet-online.de [93.104.85.186]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Sat, 26 Aug 2017 12:33:35 +0200 (CEST) Received: from [192.168.110.2] (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id 4B0F260162; Sat, 26 Aug 2017 12:33:35 +0200 (CEST) Subject: Re: bug#28159: Updater needs to support HTTP(S) servers To: =?UTF-8?Q?Ludovic_Court=c3=a8s?= References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> <87r2vybqnw.fsf@gnu.org> From: Hartmut Goebel Organization: crazy-compilers.com Message-ID: <263fe0e0-b9f5-3377-30ad-2675698d41c8@crazy-compilers.com> Date: Sat, 26 Aug 2017 12:33:34 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <87r2vybqnw.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 28159 Cc: 28159@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Hi, > I just learned that ftp://ftp.gnu.org will be retired on Nov. 1st, 2017, > so we’ll have to implement a replacement for the ‘gnu’ updater at least. By change, also this server provides a `ls-lrRt.txt.gz` file. Unfurtunaly is as a slightly different (date-) format than the one at kde.org: kde: drwxr-xr-x   3 ftpadmin packager       6 2000-10-01 14:07 adm gnu: drwxr-xr-x   2 root root      4096 Aug  2  2003 third-party Also by chance ftp.gnu.org also provides a file `find.txt.gz`, listing all files, including the full path: ./video/Stephen_Fry-Happy_Birthday_GNU-nq_600px_425kbit.ogv ./old-gnu/g77/g77-0.5.21.tar.gz ./old-gnu/guile ./old-gnu/guile/guile-www-1.0.1.tar.gz ./old-gnu/guile/guile-1.3.2.tar.gz > At worst, we’ll parse HTML index files like the one at > , This is what Ihis bug is about :-) Please mind the query-parameters one can pass to apache: is much more terse. -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 03 17:40:33 2017 Received: (at 28159) by debbugs.gnu.org; 3 Sep 2017 21:40:33 +0000 Received: from localhost ([127.0.0.1]:46989 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1docd6-0008LF-PD for submit@debbugs.gnu.org; Sun, 03 Sep 2017 17:40:33 -0400 Received: from eggs.gnu.org ([208.118.235.92]:59061) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1docd5-0008L2-8Y for 28159@debbugs.gnu.org; Sun, 03 Sep 2017 17:40:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1doccv-0002JO-Iv for 28159@debbugs.gnu.org; Sun, 03 Sep 2017 17:40:26 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:60979) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1doccv-0002JK-Fw; Sun, 03 Sep 2017 17:40:21 -0400 Received: from [2a01:e0a:1d:7270:6a6c:dc17:fc02:cfda] (port=45840 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1doccu-0000QZ-U6; Sun, 03 Sep 2017 17:40:21 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Hartmut Goebel Subject: Re: bug#28159: Updater needs to support HTTP(S) servers References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> <87r2vybqnw.fsf@gnu.org> <263fe0e0-b9f5-3377-30ad-2675698d41c8@crazy-compilers.com> Date: Sun, 03 Sep 2017 23:40:18 +0200 In-Reply-To: <263fe0e0-b9f5-3377-30ad-2675698d41c8@crazy-compilers.com> (Hartmut Goebel's message of "Sat, 26 Aug 2017 12:33:34 +0200") Message-ID: <87pob71mwt.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28159 Cc: 28159@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hi Hartmut, Hartmut Goebel skribis: > Also by chance ftp.gnu.org also provides a file `find.txt.gz`, listing > all files, including the full path: > > ./video/Stephen_Fry-Happy_Birthday_GNU-nq_600px_425kbit.ogv > ./old-gnu/g77/g77-0.5.21.tar.gz > ./old-gnu/guile > ./old-gnu/guile/guile-www-1.0.1.tar.gz > ./old-gnu/guile/guile-1.3.2.tar.gz This one is nice and smaller than =E2=80=98ls-lR=E2=80=99. I reimplemented= the GNU updater in terms of this file, and kept the previous FTP-based updater around for GNU packages not hosted on ftp.gnu.org: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3D100b216d8a4218dae= c4a79024d62d54b52dc07be =E2=80=9Cguix refresh -t gnu=E2=80=9D is now much faster. The next step may be to have a more-or-less generic updater based on =E2=80=98ls-lR=E2=80=99 files. Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 08 04:30:56 2017 Received: (at 28159) by debbugs.gnu.org; 8 Sep 2017 08:30:56 +0000 Received: from localhost ([127.0.0.1]:55233 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dqEgi-0008LV-Eb for submit@debbugs.gnu.org; Fri, 08 Sep 2017 04:30:56 -0400 Received: from eggs.gnu.org ([208.118.235.92]:40520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dqEgg-0008LF-QI for 28159@debbugs.gnu.org; Fri, 08 Sep 2017 04:30:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dqEgW-0007EU-Qf for 28159@debbugs.gnu.org; Fri, 08 Sep 2017 04:30:49 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42963) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dqEgW-0007E6-LF; Fri, 08 Sep 2017 04:30:44 -0400 Received: from [193.50.110.231] (port=57858 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dqEgV-0002ly-Vc; Fri, 08 Sep 2017 04:30:44 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: sysadmin@gnu.org Subject: Re: bug#28159: Updater needs to support HTTP(S) servers References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> <87r2vybqnw.fsf@gnu.org> <263fe0e0-b9f5-3377-30ad-2675698d41c8@crazy-compilers.com> <87pob71mwt.fsf@gnu.org> Date: Fri, 08 Sep 2017 10:30:42 +0200 In-Reply-To: <87pob71mwt.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Sun, 03 Sep 2017 23:40:18 +0200") Message-ID: <87r2vhk2x9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28159 Cc: 28159@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hello sysadmins! How frequently is ftp.gnu.org/find.txt.gz updated? It seems to be less than once a day. Could we arrange to have it regenerated every time a new file is uploaded? I suppose uploads aren=E2=80=99t this frequent, but regenerating =E2=80=98f= ind.txt.gz=E2=80=99 right after an upload would ensure that it=E2=80=99s always current. Thanks in advance! Ludo=E2=80=99. PS: For context, see . PPS: Please reply to all. From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 14 13:04:18 2017 Received: (at 28159) by debbugs.gnu.org; 14 Sep 2017 17:04:18 +0000 Received: from localhost ([127.0.0.1]:41332 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dsXYo-0000d7-4R for submit@debbugs.gnu.org; Thu, 14 Sep 2017 13:04:18 -0400 Received: from rt2p.gnu.org ([74.94.156.212]:41502 helo=rt.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dsXLV-0006qo-Pf for 28159@debbugs.gnu.org; Thu, 14 Sep 2017 12:50:34 -0400 Received: from www-data by rt.gnu.org with local (Exim 4.86_2) (envelope-from ) id 1dsXLQ-0001z3-Aa for 28159@debbugs.gnu.org; Thu, 14 Sep 2017 12:50:28 -0400 Subject: [gnu.org #1238656] Re: bug#28159: Updater needs to support HTTP(S) servers From: "=?UTF-8?B?UnViw6luIFJvZHLDrWd1ZXogUMOpcmV6?= via RT" In-Reply-To: <87r2vhk2x9.fsf@gnu.org> References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> <87r2vybqnw.fsf@gnu.org> <263fe0e0-b9f5-3377-30ad-2675698d41c8@crazy-compilers.com> <87pob71mwt.fsf@gnu.org> <87r2vhk2x9.fsf@gnu.org> Message-ID: X-RT-Loop-Prevention: gnu.org X-RT-Ticket: gnu.org #1238656 X-Managed-BY: RT 4.2.13-5-gc649048 (http://www.bestpractical.com/rt/) X-RT-Originator: ruben@fsf.org Content-Type: text/plain; charset="utf-8" X-RT-Original-Encoding: utf-8 Precedence: bulk Date: Thu, 14 Sep 2017 12:50:28 -0400 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 1.2 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: On Fri Sep 08 04:31:05 2017, ludo@gnu.org wrote: > Hello sysadmins! Hi Ludo > How frequently is ftp.gnu.org/find.txt.gz updated? It seems to be > less > than once a day. [...] Content analysis details: (1.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: gnu.org] 1.2 MISSING_HEADERS Missing To: header -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain X-Debbugs-Envelope-To: 28159 X-Mailman-Approved-At: Thu, 14 Sep 2017 13:04:17 -0400 Cc: 28159@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: sysadmin@gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.2 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: On Fri Sep 08 04:31:05 2017, ludo@gnu.org wrote: > Hello sysadmins! Hi Ludo > How frequently is ftp.gnu.org/find.txt.gz updated? It seems to be > less > than once a day. [...] Content analysis details: (1.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: gnu.org] 1.2 MISSING_HEADERS Missing To: header -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain On Fri Sep 08 04:31:05 2017, ludo@gnu.org wrote: > Hello sysadmins! Hi Ludo > How frequently is ftp.gnu.org/find.txt.gz updated? It seems to be > less > than once a day. It is run by cron.daily > Could we arrange to have it regenerated every time a new file is > uploaded? > > I suppose uploads aren’t this frequent, but regenerating ‘find.txt.gz’ > right after an upload would ensure that it’s always current. I've modified the cron script to apply that change, now on the look for the next upload to see if it worked. Regards, -- Ruben Rodriguez | Senior Systems Administrator, Free Software Foundation GPG Key: 05EF 1D2F FE61 747D 1FC8 27C3 7FAC 7D26 472F 4409 https://fsf.org | https://gnu.org From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 15 03:49:58 2017 Received: (at 28159) by debbugs.gnu.org; 15 Sep 2017 07:49:58 +0000 Received: from localhost ([127.0.0.1]:41820 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dslNu-0006Y8-Lc for submit@debbugs.gnu.org; Fri, 15 Sep 2017 03:49:58 -0400 Received: from eggs.gnu.org ([208.118.235.92]:60686) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dslNr-0006Xt-FC for 28159@debbugs.gnu.org; Fri, 15 Sep 2017 03:49:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dslNi-00041H-BH for 28159@debbugs.gnu.org; Fri, 15 Sep 2017 03:49:50 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD, TVD_RCVD_SPACE_BRACKET,UNPARSEABLE_RELAY,URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42487) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dslNi-00041D-80 for 28159@debbugs.gnu.org; Fri, 15 Sep 2017 03:49:46 -0400 Received: from [193.50.110.249] (port=37282 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dslNh-0006Ab-Op for 28159@debbugs.gnu.org; Fri, 15 Sep 2017 03:49:46 -0400 Resent-To: 28159@debbugs.gnu.org Resent-From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Resent-Date: Fri, 15 Sep 2017 09:49:44 +0200 Resent-Message-ID: <87efr8765j.fsf@gnu.org> Received: from solo.fdn.fr ([unix socket]) by solo (Cyrus v2.4.16-Debian-2.4.16-4+deb7u2) with LMTPA; Thu, 14 Sep 2017 18:50:36 +0200 X-Sieve: CMU Sieve 2.4 Received: by solo.fdn.fr (Postfix) id 50DE5D0A88; Thu, 14 Sep 2017 18:50:36 +0200 (CEST) Received: from eggs.gnu.org (eggs.gnu.org [IPv6:2001:4830:134:3::10]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by solo.fdn.fr (Postfix) with ESMTPS id 27D0DD0A87 for ; Thu, 14 Sep 2017 18:50:36 +0200 (CEST) Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsXLS-0007Wn-Nf for ludovic.courtes@fdn.fr; Thu, 14 Sep 2017 12:50:34 -0400 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:52431) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsXLS-0007WX-KV for ludovic.courtes@fdn.fr; Thu, 14 Sep 2017 12:50:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57621) by fencepost.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dsXLS-0001gY-9C for ludo@gnu.org; Thu, 14 Sep 2017 12:50:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsXLR-0007VU-6Z for ludo@gnu.org; Thu, 14 Sep 2017 12:50:30 -0400 Received: from rt2p.gnu.org ([74.94.156.212]:52524 helo=rt.gnu.org) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dsXLR-0007V2-2x for ludo@gnu.org; Thu, 14 Sep 2017 12:50:29 -0400 Received: from www-data by rt.gnu.org with local (Exim 4.86_2) (envelope-from ) id 1dsXLQ-0001z7-Dc for ludo@gnu.org; Thu, 14 Sep 2017 12:50:28 -0400 Subject: [gnu.org #1238656] Re: bug#28159: Updater needs to support HTTP(S) servers From: "\=\?UTF-8\?B\?UnViw6luIFJvZHLDrWd1ZXogUMOpcmV6\?\= via RT" In-Reply-To: <87r2vhk2x9.fsf@gnu.org> References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> <87r2vybqnw.fsf@gnu.org> <263fe0e0-b9f5-3377-30ad-2675698d41c8@crazy-compilers.com> <87pob71mwt.fsf@gnu.org> <87r2vhk2x9.fsf@gnu.org> Message-ID: X-RT-Loop-Prevention: gnu.org X-RT-Ticket: gnu.org #1238656 X-Managed-BY: RT 4.2.13-5-gc649048 (http://www.bestpractical.com/rt/) X-RT-Originator: ruben@fsf.org To: ludo@gnu.org Content-Type: text/plain; charset="utf-8" X-RT-Original-Encoding: utf-8 Precedence: bulk Date: Thu, 14 Sep 2017 12:50:28 -0400 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28159 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: sysadmin@gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) On Fri Sep 08 04:31:05 2017, ludo@gnu.org wrote: > Hello sysadmins! Hi Ludo > How frequently is ftp.gnu.org/find.txt.gz updated? It seems to be > less > than once a day. It is run by cron.daily > Could we arrange to have it regenerated every time a new file is > uploaded? > > I suppose uploads aren’t this frequent, but regenerating ‘find.txt.gz’ > right after an upload would ensure that it’s always current. I've modified the cron script to apply that change, now on the look for the next upload to see if it worked. Regards, -- Ruben Rodriguez | Senior Systems Administrator, Free Software Foundation GPG Key: 05EF 1D2F FE61 747D 1FC8 27C3 7FAC 7D26 472F 4409 https://fsf.org | https://gnu.org From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 15 03:50:46 2017 Received: (at 28159) by debbugs.gnu.org; 15 Sep 2017 07:50:46 +0000 Received: from localhost ([127.0.0.1]:41824 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dslOf-0006ZY-VP for submit@debbugs.gnu.org; Fri, 15 Sep 2017 03:50:46 -0400 Received: from eggs.gnu.org ([208.118.235.92]:60871) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dslOe-0006ZN-I0 for 28159@debbugs.gnu.org; Fri, 15 Sep 2017 03:50:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dslOY-0004LV-O7 for 28159@debbugs.gnu.org; Fri, 15 Sep 2017 03:50:39 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42493) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dslOY-0004LN-7K; Fri, 15 Sep 2017 03:50:38 -0400 Received: from [193.50.110.249] (port=37284 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dslOX-0006E0-Kh; Fri, 15 Sep 2017 03:50:37 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: =?utf-8?Q?Rub=C3=A9n_Rodr=C3=ADguez_P=C3=A9rez_via_RT?= Subject: Re: [gnu.org #1238656] Re: bug#28159: Updater needs to support HTTP(S) servers References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> <87r2vybqnw.fsf@gnu.org> <263fe0e0-b9f5-3377-30ad-2675698d41c8@crazy-compilers.com> <87pob71mwt.fsf@gnu.org> <87r2vhk2x9.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 29 Fructidor an 225 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Fri, 15 Sep 2017 09:50:35 +0200 In-Reply-To: (=?utf-8?Q?=22Rub=C3=A9n_Rodr=C3=ADguez_P=C3=A9rez?= via RT"'s message of "Thu, 14 Sep 2017 12:50:28 -0400") Message-ID: <878thg7644.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28159 Cc: 28159@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hi Rub=C3=A9n, "Rub=C3=A9n Rodr=C3=ADguez P=C3=A9rez via RT" skribis: > On Fri Sep 08 04:31:05 2017, ludo@gnu.org wrote: [...] >> Could we arrange to have it regenerated every time a new file is >> uploaded? >>=20 >> I suppose uploads aren=E2=80=99t this frequent, but regenerating =E2=80= =98find.txt.gz=E2=80=99 >> right after an upload would ensure that it=E2=80=99s always current. > > I've modified the cron script to apply that change, now on the look for t= he next upload to see if it worked. Awesome, thanks! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 25 18:40:05 2017 Received: (at 28159) by debbugs.gnu.org; 25 Sep 2017 22:40:05 +0000 Received: from localhost ([127.0.0.1]:59110 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dwc2n-00007u-25 for submit@debbugs.gnu.org; Mon, 25 Sep 2017 18:40:05 -0400 Received: from eggs.gnu.org ([208.118.235.92]:45178) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dwc2k-00007L-LU for 28159@debbugs.gnu.org; Mon, 25 Sep 2017 18:40:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dwc2e-0005jo-IN for 28159@debbugs.gnu.org; Mon, 25 Sep 2017 18:39:57 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:48347) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dwc2e-0005jd-Ee; Mon, 25 Sep 2017 18:39:56 -0400 Received: from [2a01:e0a:1d:7270:6a6c:dc17:fc02:cfda] (port=39940 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dwc2e-00040O-1F; Mon, 25 Sep 2017 18:39:56 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Hartmut Goebel Subject: Re: bug#28159: Updater needs to support HTTP(S) servers References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> <87r2vybqnw.fsf@gnu.org> <263fe0e0-b9f5-3377-30ad-2675698d41c8@crazy-compilers.com> <87pob71mwt.fsf@gnu.org> Date: Tue, 26 Sep 2017 00:39:54 +0200 In-Reply-To: <87pob71mwt.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Sun, 03 Sep 2017 23:40:18 +0200") Message-ID: <87tvzqpflx.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28159 Cc: 28159@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > This one is nice and smaller than =E2=80=98ls-lR=E2=80=99. I reimplement= ed the GNU > updater in terms of this file, and kept the previous FTP-based updater > around for GNU packages not hosted on ftp.gnu.org: > > https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3D100b216d8a4218d= aec4a79024d62d54b52dc07be > > =E2=80=9Cguix refresh -t gnu=E2=80=9D is now much faster. Commit c1d8b3b3b5af8282328b87dd7a8d09357cbb0af7 rewrites the GNOME updater in terms of the =E2=80=98cache.json=E2=80=99 files that can be foun= d in each package directory at . Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Nov 10 17:38:29 2018 Received: (at 28159) by debbugs.gnu.org; 10 Nov 2018 22:38:29 +0000 Received: from localhost ([127.0.0.1]:45155 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gLbtd-0005ua-Bu for submit@debbugs.gnu.org; Sat, 10 Nov 2018 17:38:29 -0500 Received: from eggs.gnu.org ([208.118.235.92]:36163) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gLbtb-0005uN-MZ for 28159@debbugs.gnu.org; Sat, 10 Nov 2018 17:38:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gLbtS-0002up-MB for 28159@debbugs.gnu.org; Sat, 10 Nov 2018 17:38:20 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:37081) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gLbtS-0002uV-F2; Sat, 10 Nov 2018 17:38:18 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51598 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gLbtS-0006Hd-6Q; Sat, 10 Nov 2018 17:38:18 -0500 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Hartmut Goebel Subject: Re: bug#28159: Updater needs to support HTTP(S) servers References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> <87r2vybqnw.fsf@gnu.org> <263fe0e0-b9f5-3377-30ad-2675698d41c8@crazy-compilers.com> <87pob71mwt.fsf@gnu.org> <87tvzqpflx.fsf@gnu.org> Date: Sat, 10 Nov 2018 23:38:16 +0100 In-Reply-To: <87tvzqpflx.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Tue, 26 Sep 2017 00:39:54 +0200") Message-ID: <87efbsv9rb.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28159 Cc: 28159@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > >> This one is nice and smaller than =E2=80=98ls-lR=E2=80=99. I reimplemen= ted the GNU >> updater in terms of this file, and kept the previous FTP-based updater >> around for GNU packages not hosted on ftp.gnu.org: >> >> https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3D100b216d8a4218= daec4a79024d62d54b52dc07be >> >> =E2=80=9Cguix refresh -t gnu=E2=80=9D is now much faster. > > Commit c1d8b3b3b5af8282328b87dd7a8d09357cbb0af7 rewrites the GNOME > updater in terms of the =E2=80=98cache.json=E2=80=99 files that can be fo= und in each > package directory at . Commit 5230dce154a8861d806fcd667f2d424def571ed6 rewrites the kernel.org updater so that it=E2=80=99s based on an analysis of HTML directory listings such as . Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 10 13:26:03 2019 Received: (at 28159) by debbugs.gnu.org; 10 Sep 2019 17:26:04 +0000 Received: from localhost ([127.0.0.1]:41722 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i7jty-00044B-JC for submit@debbugs.gnu.org; Tue, 10 Sep 2019 13:26:03 -0400 Received: from mail-out.m-online.net ([212.18.0.10]:56260) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i7jtx-00043l-1N for 28159@debbugs.gnu.org; Tue, 10 Sep 2019 13:26:01 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 46SX4S2JfBz1rW5x for <28159@debbugs.gnu.org>; Tue, 10 Sep 2019 19:26:00 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 46SX4S0vY4z1qqkn for <28159@debbugs.gnu.org>; Tue, 10 Sep 2019 19:26:00 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id fj6zrXQDXwXR for <28159@debbugs.gnu.org>; Tue, 10 Sep 2019 19:25:58 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-61-50.dynamic.mnet-online.de [188.174.61.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS for <28159@debbugs.gnu.org>; Tue, 10 Sep 2019 19:25:58 +0200 (CEST) Received: from [192.168.110.2] (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id B96E56028B for <28159@debbugs.gnu.org>; Tue, 10 Sep 2019 19:28:06 +0200 (CEST) Subject: Re: bug#28159: Updater needs to support HTTP(S) servers References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> To: 28159@debbugs.gnu.org From: Hartmut Goebel Openpgp: preference=signencrypt Organization: crazy-compilers.com Message-ID: <901a1993-ed8e-4306-d3f2-f3dbcac30424@crazy-compilers.com> Date: Tue, 10 Sep 2019 19:25:58 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <87poboasjz.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 28159 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Am 22.08.17 um 10:57 schrieb Ludovic Courtès: > More precisely, several updaters rely on FTP (gnu, kernel.org, kde, > etc. see (guix gnu-maintenance)), but others rely on structured data > retrieved over HTTP(S) (pypi, cran, elpa, etc.) For the records: KDE no longer relies on FTP access. It now fetches the ls-lR.bz2 file list using HTTPS from download.kde.org, converts it into a list of file paths and caches the list. See http://debbugs.gnu.org/cgi/bugreport.cgi?bug=36919 commit 4eb69bf0d33810886ee118f38989cef696e4c868 -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 29 04:22:05 2020 Received: (at 28159) by debbugs.gnu.org; 29 Apr 2020 08:22:06 +0000 Received: from localhost ([127.0.0.1]:42324 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jThyn-0004WW-NR for submit@debbugs.gnu.org; Wed, 29 Apr 2020 04:22:05 -0400 Received: from relay8-d.mail.gandi.net ([217.70.183.201]:55899) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jThyk-0004VX-Kk for 28159@debbugs.gnu.org; Wed, 29 Apr 2020 04:22:04 -0400 Received: from webmail.gandi.net (webmail18.sd4.0x35.net [10.200.201.18]) (Authenticated sender: brice@waegenei.re) by relay8-d.mail.gandi.net (Postfix) with ESMTPA id 711601BF205; Wed, 29 Apr 2020 08:21:56 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Wed, 29 Apr 2020 08:21:56 +0000 From: Brice Waegeneire To: 28159@debbugs.gnu.org Subject: Closing bug #28159? Updater needs to support HTTP(S) servers Message-ID: <07e7317fe16fc58790cd99c0a712fcf5@waegenei.re> X-Sender: brice@waegenei.re User-Agent: Roundcube Webmail/1.3.8 X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 28159 Cc: h.goebel@crazy-compilers.com, ludo@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hello Guix, It looks like now most of the major updaters that relied on FTP (GNU, kernel.org, KDE and Gnbome) now support HTTP(S). I think we can close this bug. Ludovic Courtès wrote on Tue Aug 22 10:57:20+0200 2017: > More precisely, several updaters rely on FTP (gnu, kernel.org, kde, > etc. see (guix gnu-maintenance)), but others rely on structured data > retrieved over HTTP(S) (pypi, cran, elpa, etc.) Ludovic Courtès wrote on Sun Sep 03 23:40:18+0200 2017: > This one is nice and smaller than ‘ls-lR’. I reimplemented the GNU > updater in terms of this file, and kept the previous FTP-based updater > around for GNU packages not hosted on ftp.gnu.org: Ludovic Courtès wrote on Tue Sep 26 00:39:54+0200 2017: > Commit c1d8b3b3b5af8282328b87dd7a8d09357cbb0af7 rewrites the GNOME > updater in terms of the ‘cache.json’ files that can be found in each > package directory at . Ludovic Courtès wrote on Sat Nov 10 23:38:16+0100 2018: > Commit 5230dce154a8861d806fcd667f2d424def571ed6 rewrites the kernel.org > updater so that it’s based on an analysis of HTML directory listings > such as . Hartmut Goebel wrote on Tue Sep 10 19:25:58+0200 2019: > For the records: KDE no longer relies on FTP access. It now fetches the > ls-lR.bz2 file list using HTTPS from download.kde.org, converts it into > a list of file paths and caches the list. - Brice From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 30 17:15:22 2020 Received: (at 28159-done) by debbugs.gnu.org; 30 Apr 2020 21:15:22 +0000 Received: from localhost ([127.0.0.1]:47837 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jUGWf-000560-QL for submit@debbugs.gnu.org; Thu, 30 Apr 2020 17:15:22 -0400 Received: from eggs.gnu.org ([209.51.188.92]:39766) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jUGWS-00055U-2x for 28159-done@debbugs.gnu.org; Thu, 30 Apr 2020 17:15:20 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58547) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jUGWL-0007bL-7q; Thu, 30 Apr 2020 17:15:01 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=35284 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jUGWK-0003ws-OH; Thu, 30 Apr 2020 17:15:01 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Brice Waegeneire Subject: Re: Closing bug #28159? Updater needs to support HTTP(S) servers References: <07e7317fe16fc58790cd99c0a712fcf5@waegenei.re> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 12 =?utf-8?Q?Flor=C3=A9al?= an 228 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 30 Apr 2020 23:14:59 +0200 In-Reply-To: <07e7317fe16fc58790cd99c0a712fcf5@waegenei.re> (Brice Waegeneire's message of "Wed, 29 Apr 2020 08:21:56 +0000") Message-ID: <87lfmcoeuk.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 28159-done Cc: h.goebel@crazy-compilers.com, 28159-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Brice, Brice Waegeneire skribis: > It looks like now most of the major updaters that relied on FTP (GNU, > kernel.org, KDE and Gnbome) now support HTTP(S). I think we can close > this > bug. Yup. There=E2=80=99s still the =E2=80=98gnu-ftp=E2=80=99 and the =E2=80=98= xorg=E2=80=99 updaters which, according to =E2=80=98guix refresh --list-updaters=E2=80=99, account for 2.= 2% of the packages. We can change them later when it becomes necessary. Closing, thank you! Ludo=E2=80=99. From unknown Sat Aug 16 16:13:07 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 29 May 2020 11:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator