Subject: 26.1; url.el does not support socks5h:// proxy scheme

From: Łukasz Stelmach <[email-masked]>
Date: Mon, 30 Aug 2021 19:12:06 +0200

I've got HTTPS_PROXY environment variable set to
"socks5h://127.0.0.1:9050" to use tor. Unfortunatelly the url package
discards the sock5h:// scheme in url-scheme-register-proxy and sets
url-proxy-services to (("https" . "127.0.0.1:9050")) which makes the
rest of the url package think the proxy supports HTTP. When Emacs makes
HTTP requests tor responds 501 Not implemented. 

While it wouldn't be good for tor users if Emacs would ignore socks5h
proxies making direct connections, current behaviour isn't helpful
either as it requires certain lisp knowledge and some time to understand
the situation.

IMHO there are two possible solution which can actually go togeher.
First and quick is to report error if a proxy in a *_PROXY environment
variable contains a scheme (e.g. socks5h) that isn't supported instead
of discarding it, and thus treating it as http. Second is to fully
support SOCKS via socks.el (reporting unsupported schemes in proxy urls
will still be helpful).

In GNU Emacs 26.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.5)
 of 2021-01-31, modified by Debian built on x86-csail-01
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description:	Debian GNU/Linux 10 (buster)

Recent messages:
Result: "Using a proxy for https..."

Result: "Using a proxy for https..."

(("https" . "127.0.0.1:9050"))
Auto-saving...
Mark saved where search started
Making completion list... [2 times]
Quit [2 times]
Making completion list... [2 times]

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --enable-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --build
 x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd
 --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --with-x=yes
 --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs-9Yet8u/emacs-26.1+1=. -fstack-protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 THREADS LIBSYSTEMD LCMS2

Important settings:
  value of $LANG: pl_PL.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Emacs-Lisp

Minor modes in effect:
  bug-reference-prog-mode: t
  shell-dirtrack-mode: t
  diff-auto-refine-mode: t
  electric-pair-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/usr/share/emacs/site-lisp/elpa/debian-el-37/deb-view hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37/deb-view
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-el-autoloads hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37/debian-el-autoloads
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-el hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37/debian-el
/usr/share/emacs/site-lisp/elpa/debian-el-37/preseed hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37/preseed
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-el-pkg hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37/debian-el-pkg
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-bug hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37/debian-bug
/usr/share/emacs/site-lisp/elpa/debian-el-37/apt-sources hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37/apt-sources
/usr/share/emacs/site-lisp/elpa/debian-el-37/apt-utils hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37/apt-utils
/usr/share/emacs/site-lisp/elpa/debian-el-37/gnus-BTS hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37/gnus-BTS
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-autoloads hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37/debian-autoloads
/usr/share/emacs/site-lisp/elpa/editorconfig-0.7.14/editorconfig-pkg hides /usr/share/emacs/site-lisp/elpa-src/editorconfig-0.7.14/editorconfig-pkg
/usr/share/emacs/site-lisp/elpa/editorconfig-0.7.14/editorconfig-autoloads hides /usr/share/emacs/site-lisp/elpa-src/editorconfig-0.7.14/editorconfig-autoloads
/usr/share/emacs/site-lisp/elpa/editorconfig-0.7.14/editorconfig hides /usr/share/emacs/site-lisp/elpa-src/editorconfig-0.7.14/editorconfig
/usr/share/emacs/site-lisp/elpa/editorconfig-0.7.14/editorconfig-conf-mode hides /usr/share/emacs/site-lisp/elpa-src/editorconfig-0.7.14/editorconfig-conf-mode
/usr/share/emacs/site-lisp/elpa/editorconfig-0.7.14/editorconfig-fnmatch hides /usr/share/emacs/site-lisp/elpa-src/editorconfig-0.7.14/editorconfig-fnmatch
/usr/share/emacs/site-lisp/elpa/editorconfig-0.7.14/editorconfig-core-handle hides /usr/share/emacs/site-lisp/elpa-src/editorconfig-0.7.14/editorconfig-core-handle
/usr/share/emacs/site-lisp/elpa/editorconfig-0.7.14/editorconfig-core hides /usr/share/emacs/site-lisp/elpa-src/editorconfig-0.7.14/editorconfig-core
/usr/share/emacs/site-lisp/elpa/go-mode-1.5.0/go-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/go-mode-1.5.0/go-mode-autoloads
/usr/share/emacs/site-lisp/elpa/go-mode-1.5.0/go-mode hides /usr/share/emacs/site-lisp/elpa-src/go-mode-1.5.0/go-mode
/usr/share/emacs/site-lisp/elpa/go-mode-1.5.0/go-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/go-mode-1.5.0/go-mode-pkg
/usr/share/emacs/site-lisp/elpa/go-mode-1.5.0/go-guru hides /usr/share/emacs/site-lisp/elpa-src/go-mode-1.5.0/go-guru
/usr/share/emacs/site-lisp/elpa/go-mode-1.5.0/go-rename hides /usr/share/emacs/site-lisp/elpa-src/go-mode-1.5.0/go-rename
/usr/share/emacs/site-lisp/elpa/htmlize-1.54/htmlize hides /usr/share/emacs/site-lisp/elpa-src/htmlize-1.54/htmlize
/usr/share/emacs/site-lisp/elpa/htmlize-1.54/htmlize-pkg hides /usr/share/emacs/site-lisp/elpa-src/htmlize-1.54/htmlize-pkg
/usr/share/emacs/site-lisp/elpa/htmlize-1.54/htmlize-autoloads hides /usr/share/emacs/site-lisp/elpa-src/htmlize-1.54/htmlize-autoloads
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-fonts hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-fonts
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-reconcile hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-reconcile
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-flymake hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-flymake
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-occur hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-occur
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-sort hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-sort
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-pkg hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-pkg
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-test hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-test
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-regex hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-regex
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-fontify hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-fontify
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-navigate hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-navigate
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-check hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-check
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-post hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-post
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-xact hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-xact
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-report hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-report
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-mode hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-mode
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-texi hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-texi
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-autoloads hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-autoloads
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-complete hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-complete
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-state hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-state
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-context hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-context
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-init hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-init
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-commodities hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-commodities
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-schedule hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-schedule
/usr/share/emacs/site-lisp/elpa/ledger-3.1.2/ledger-exec hides /usr/share/emacs/site-lisp/elpa-src/ledger-3.1.2/ledger-exec
/usr/share/emacs/site-lisp/elpa/lua-mode-20151025/lua-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/lua-mode-20151025/lua-mode-pkg
/usr/share/emacs/site-lisp/elpa/lua-mode-20151025/lua-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/lua-mode-20151025/lua-mode-autoloads
/usr/share/emacs/site-lisp/elpa/lua-mode-20151025/init-tryout hides /usr/share/emacs/site-lisp/elpa-src/lua-mode-20151025/init-tryout
/usr/share/emacs/site-lisp/elpa/lua-mode-20151025/lua-mode hides /usr/share/emacs/site-lisp/elpa-src/lua-mode-20151025/lua-mode
/usr/share/emacs/site-lisp/elpa/yaml-mode-0.0.13/yaml-mode hides /usr/share/emacs/site-lisp/elpa-src/yaml-mode-0.0.13/yaml-mode
/usr/share/emacs/site-lisp/elpa/yaml-mode-0.0.13/yaml-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/yaml-mode-0.0.13/yaml-mode-pkg
/usr/share/emacs/site-lisp/elpa/yaml-mode-0.0.13/yaml-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/yaml-mode-0.0.13/yaml-mode-autoloads

Features:
(shadow emacsbug bug-reference url-about url-dav url-dired package-x
debug url-http url-gw url-cache url-auth conf-mode etags xref project
descr-text picture reporter debian-bug cl-print edebug mule-diag novice
yaml-mode warnings calc-map calc-stat calc-vec gnuplot-gui ox-org ox-odt
rng-loc rng-uri rng-parse rng-match rng-dt rng-util rng-pttrn nxml-parse
nxml-ns nxml-enc xmltok nxml-util ox-latex ox-icalendar ox-html table
ox-ascii ox-publish ox gnuplot info-look org-plot org-table mhtml-mode
css-mode eww mm-url url-queue url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf js imenu sgml-mode
flyspell ispell view holidays hol-loaddefs cal-julian lunar solar
cal-dst org-duration hashcash calccomp calc-bin calc-arith cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs tabify vc-git xgit-dvc xgit xgit-annotate rect dvc-annotate
xgit-log eieio-opt speedbar sb-image ezimage dframe help-fns radix-tree
two-column iso-transl doc-view jka-compr image-mode flow-fill sort
gnus-cite url-util shr-color color shr svg xml dom browse-url mm-archive
mail-extr gnus-bcklg tramp-cmds calc-aent calc-math calc-undo python
json map calc-yank calc-misc calc-mode calc-alg calc-ext calc-menu calc
calc-loaddefs calc-macs org-clock cal-iso cal-move xmtn-dvc dvc-status
xmtn-conflicts xmtn-ids xmtn-match xmtn-automate xmtn-compat xmtn-run
xmtn-basic-io xmtn-base cl org-w3m org-info org-habit org-agenda
org-gnus nnir org-bibtex bibtex org-bbdb files-x tramp-sh tramp-cache
tramp tramp-compat tramp-loaddefs trampver ucs-normalize shell misearch
multi-isearch tla smerge-mode dvc-state dvc-config dvc-diff dvc-fileinfo
diff tla-gnus dvc-gnus dvc-cmenu dvc-about dvc-version dvc-revlist
sh-script smie executable gnus-dup gnus-async cl-extra help-mode qp
gnus-ml disp-table cursor-sensor nnmaildir utf-7 epa-file network-stream
nsm starttls nnfolder spam spam-stat bbdb-com pcase crm gnus-uu yenc
gnus-delay gnus-draft gnus-agent gnus-srvr gnus-score score-mode
nnvirtual nntp gnus-cache gnus-msg gnus-art mm-uu mml2015 mm-view
mml-smime smime dig mailcap gnus-sum nndraft nnmh gnus-group gnus-undo
gnus-start gnus-cloud nnimap tls gnutls utf7 netrc nnoo parse-time
gnus-spec gnus-win nnmail gnus-int gnus-range mail-source message
sendmail rmc puny rfc822 mml mml-sec epa epg mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader gnus
nnheader gnus-util rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums
mail-utils mm-util mail-prsvr bzr-core tla-core pp tla-autoconf tla-defs
dvc-log vc vc-dispatcher dvc-unified dvc-tips dired-x ffap thingatpt
add-log diff-mode ediff-merg ediff-wind ediff-diff ediff-mult ediff-help
ediff-init ediff-util ediff xdarcs-core xgit-core xhg-core xmtn-minimal
hl-line dired dired-loaddefs bbdb derived bbdb-site timezone
polish-holidays diary-lib diary-loaddefs ob-ditaa edmacro kmacro
elec-pair dvc-autoloads dvc-core dvc-lisp dvc-buffers dvc-ui
dvc-register dvc-utils dvc-emacs ewoc dvc-defs dvc-site ledger-mode
ledger-check ledger-texi ledger-test ledger-sort ledger-report
ledger-reconcile ledger-occur ledger-fonts ledger-fontify ledger-state
ledger-complete ledger-schedule ledger-init ledger-xact ledger-post
ledger-exec ledger-navigate ledger-context ledger-commodities
org-element avl-tree generator org advice org-macro org-footnote
org-pcomplete org-list org-faces org-entities noutline outline
easy-mmode org-version ob-emacs-lisp ob ob-tangle org-src ob-ref ob-lob
ob-table ob-keys ob-exp ob-comint ob-core ob-eval org-compat org-macs
org-loaddefs format-spec find-func cal-menu calendar cal-loaddefs
pcomplete comint ansi-color ring esh-var esh-io esh-cmd esh-opt esh-ext
esh-proc esh-groups eshell esh-module esh-mode esh-arg esh-util cus-edit
cus-start cus-load wid-edit ledger-regex rx finder-inf info debian-el
go-mode-autoloads package easymenu epg-config url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache url-vars seq byte-opt gv bytecomp byte-compile cconv
cl-loaddefs cl-lib bbdb-loaddefs time-date mule-util tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 1097718 198779)
 (symbols 48 101148 3)
 (miscs 40 8429 15584)
 (strings 32 263362 36253)
 (string-bytes 1 7985155)
 (vectors 16 103854)
 (vector-slots 8 2681460 153680)
 (floats 8 1149 1951)
 (intervals 56 48508 2488)
 (buffers 992 163))

-- 
Miłego dnia,
Łukasz Stelmach