Subject: 27.0.50; Broken tests on macOS
From: Philipp <[email-masked]>
To: [email-masked]
Date: Sun, 04 Mar 2018 20:05:02 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

The following tests fail for me on macOS with a recent master build:

network-stream-tests:

Test echo-server-with-dns backtrace:
  make-network-process(:name "server" :server t :noquery t :family ipv
  make-server("p")
  (let* ((server (make-server (system-name))) (port (aref (process-con
  (closure (t) nil (let* ((server (make-server (system-name))) (port (
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name echo-server-with-dns :documentation n
  ert-run-or-rerun-test(#s(ert--stats :selector (not (or (tag :expensi
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/net/network-stream-t
  command-line()
  normal-top-level()
Test echo-server-with-dns condition:
    (error "p/0 nodename nor servname provided, or not known")

The macOS firewall prevents creating the server here.  Probably we
should just skip this test on macOS.

flymake-tests:

3 unexpected results:
   FAILED  included-c-header-files
   FAILED  warning-predicate-function-gcc
   FAILED  warning-predicate-rx-gcc

These tests fail because they seem to invoke GCC, and on macOS GCC is
actually Clang, so the output format is different from what's expected.
I'd suggest two fixes:

1. Instead of invoking GCC, save GCC's output once in a checked-in file.
   That makes the test more robust and reproducible because it no longer
   depends on the behavior of external programs.

2. Make Flymake work with the Clang output, i.e. remove the hard
   dependency on GCC.

In GNU Emacs 27.0.50 (build 57, x86_64-apple-darwin17.4.0, NS appkit-1561.20 Version 10.13.3 (Build 17D102))
 of 2018-03-02 built on p
Repository revision: e58f9a45d6a31be4ca379e10a866766e0993f49e
Windowing system distributor 'Apple', version 10.3.1561
System Description:  Mac OS X 10.13.3

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --with-modules --without-pop --with-mailutils
 --enable-gcc-warnings=yes --enable-checking
 --enable-check-lisp-object-type 'CFLAGS=-ggdb3 -O0''

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS
JSON

Important settings:
  value of $LANG: de_DE.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair time-date
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote kqueue cocoa ns
multi-tty make-network-process emacs)

Memory information:
((conses 16 204785 6390)
 (symbols 48 20096 1)
 (miscs 40 56 156)
 (strings 32 28897 1948)
 (string-bytes 1 771870)
 (vectors 16 35249)
 (vector-slots 8 722256 13436)
 (floats 8 52 64)
 (intervals 56 211 0)
 (buffers 992 11))