Subject: 29.0.50; todo-test-add-and-delete-file is a flaky test
From: Matt Armstrong <[email-masked]>
To: [email-masked]
Date: Wed, 12 Oct 2022 09:42:18 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

It seems that todo-test-add-and-delete-file is a flaky test.  It passes
most of the time but sometimes not.

Example error log:

Running 39 tests (2022-10-12 09:37:27-0700, selector `(not (or (tag :expensive-test) (tag :unstable) (tag :nativecomp)))')
Test todo-test-add-and-delete-file backtrace:
  signal(ert-test-failed (((should (equal todo-current-todo-file todo-
  ert-fail(((should (equal todo-current-todo-file todo-test-file-1)) :
  #f(compiled-function () #<bytecode -0x719631cd833b93d>)()
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name todo-test-add-and-delete-file :docume
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable) (tag :n
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable) (
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/calendar/todo-mode-t
  command-line()
  normal-top-level()
Test todo-test-add-and-delete-file condition:
    (ert-test-failed
     ((should
       (equal todo-current-todo-file todo-test-file-1))
      :form
      (equal nil "/home/matt/git/emacs-noverlay/test/lisp/calendar/todo-mode-resources/todo-test-1.todo")
      :value nil :explanation
      (different-types nil "/home/matt/git/emacs-noverlay/test/lisp/calendar/todo-mode-resources/todo-test-1.todo")))
   FAILED   1/39  todo-test-add-and-delete-file (0.033038 sec) at lisp/calendar/todo-mode-tests.el:823
Type C-x C-q to return to Todo mode.

Warning (todo): 
Type C-x C-q to return to Todo mode.

This also runs a file format check and signals an error if
the format has become invalid.  However, this check cannot
tell if the number of items or categories changed, which
could result in the file containing inconsistent information.
You can repair this inconsistency by invoking the command
`todo-repair-categories-sexp', but this will revert any
renumbering of the categories you have made, so you will
have to renumber them again (see `(todo-mode) Reordering
Categories').

Warning (todo): 
Type C-x C-q to return to Todo Archive mode.

This also runs a file format check and signals an error if
the format has become invalid.  However, this check cannot
tell if the number of items or categories changed, which
could result in the file containing inconsistent information.
You can repair this inconsistency by invoking the command
`todo-repair-categories-sexp', but this will revert any
renumbering of the categories you have made, so you will
have to renumber them again (see `(todo-mode) Reordering
Categories').

   passed   2/39  todo-test-current-file-in-edit-mode (0.270833 sec)
   passed   3/39  todo-test-done-items-separator01-bol (0.129566 sec)
   passed   4/39  todo-test-done-items-separator01-eol (0.145479 sec)
   passed   5/39  todo-test-done-items-separator02-bol (0.111079 sec)
   passed   6/39  todo-test-done-items-separator02-eol (0.112115 sec)
   passed   7/39  todo-test-done-items-separator03-bol (0.156204 sec)
   passed   8/39  todo-test-done-items-separator03-eol (0.154852 sec)
   passed   9/39  todo-test-done-items-separator04-bol (0.171032 sec)
   passed  10/39  todo-test-done-items-separator04-eol (0.193185 sec)
   passed  11/39  todo-test-done-items-separator05-bol (0.154420 sec)
   passed  12/39  todo-test-done-items-separator05-eol (0.152174 sec)
   passed  13/39  todo-test-done-items-separator06-bol (0.169586 sec)
   passed  14/39  todo-test-done-items-separator06-eol (0.126120 sec)
   passed  15/39  todo-test-done-items-separator07 (0.187008 sec)
   passed  16/39  todo-test-edit-item-date-month (0.192969 sec)
Warning (todo): 
Type C-x C-q to return to Todo mode.

This also runs a file format check and signals an error if
the format has become invalid.  However, this check cannot
tell if the number of items or categories changed, which
could result in the file containing inconsistent information.
You can repair this inconsistency by invoking the command
`todo-repair-categories-sexp', but this will revert any
renumbering of the categories you have made, so you will
have to renumber them again (see `(todo-mode) Reordering
Categories').

Warning (todo): 
Type C-x C-q to return to Todo Archive mode.

This also runs a file format check and signals an error if
the format has become invalid.  However, this check cannot
tell if the number of items or categories changed, which
could result in the file containing inconsistent information.
You can repair this inconsistency by invoking the command
`todo-repair-categories-sexp', but this will revert any
renumbering of the categories you have made, so you will
have to renumber them again (see `(todo-mode) Reordering
Categories').

   passed  17/39  todo-test-edit-quit (0.354015 sec)
   passed  18/39  todo-test-item-highlighting (0.030907 sec)
   passed  19/39  todo-test-move-item01 (0.179885 sec)
   passed  20/39  todo-test-move-item02 (0.135768 sec)
   passed  21/39  todo-test-move-item03 (0.081981 sec)
   passed  22/39  todo-test-move-item04 (0.087317 sec)
   passed  23/39  todo-test-move-item05 (0.135557 sec)
   passed  24/39  todo-test-multiline-item-indentation-1 (0.122302 sec)
Type C-x C-q to return to Todo mode.

   passed  25/39  todo-test-multiline-item-indentation-2 (0.166801 sec)
Warning (todo): 
Type C-x C-q to return to Todo mode.

This also runs a file format check and signals an error if
the format has become invalid.  However, this check cannot
tell if the number of items or categories changed, which
could result in the file containing inconsistent information.
You can repair this inconsistency by invoking the command
`todo-repair-categories-sexp', but this will revert any
renumbering of the categories you have made, so you will
have to renumber them again (see `(todo-mode) Reordering
Categories').

   passed  26/39  todo-test-multiline-item-indentation-3 (0.316444 sec)
   passed  27/39  todo-test-raise-lower-priority (0.122647 sec)
   passed  28/39  todo-test-revert-buffer01 (0.134979 sec)
   passed  29/39  todo-test-revert-buffer02 (0.120267 sec)
   passed  30/39  todo-test-todo-mark-unmark-category (0.087925 sec)
   passed  31/39  todo-test-todo-quit01 (0.576536 sec)
   passed  32/39  todo-test-todo-quit02 (0.160571 sec)
   passed  33/39  todo-test-toggle-item-header01 (0.027441 sec)
   passed  34/39  todo-test-toggle-item-header02 (0.124355 sec)
   passed  35/39  todo-test-toggle-item-header03 (0.153855 sec)
   passed  36/39  todo-test-toggle-item-header04 (0.203886 sec)
   passed  37/39  todo-test-toggle-item-header05 (0.199669 sec)
Items unarchived.
   passed  38/39  todo-test-toggle-item-header06 (0.391414 sec)
   passed  39/39  todo-test-toggle-item-header07 (0.098373 sec)

Ran 39 tests, 38 results as expected, 1 unexpected (2022-10-12 09:37:34-0700, 6.913405 sec)

1 unexpected results:
   FAILED  todo-test-add-and-delete-file

In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.34, cairo version 1.16.0) of 2022-10-05 built on naz
Repository revision: e2027d60ba6c9a78ef9539234969ef2a44bf84fe
Repository branch: my-build
System Description: Debian GNU/Linux bookworm/sid

Configured using:
 'configure --with-native-compilation --with-pgtk'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP
NOTIFY INOTIFY PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS XIM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  pyvenv-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  msb-mode: t
  display-time-mode: t
  global-tab-line-mode: t
  tab-line-mode: t
  shell-dirtrack-mode: t
  auto-insert-mode: t
  keyfreq-autosave-mode: t
  keyfreq-mode: t
  savehist-mode: t
  icomplete-vertical-mode: t
  icomplete-mode: t
  editorconfig-mode: t
  which-key-mode: t
  electric-pair-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tab-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t

Load-path shadows:
~/env/elisp/ol-notmuch hides /home/matt/.config/emacs/elpa/ol-notmuch-20220428.1337/ol-notmuch
/home/matt/.config/emacs/elpa/transient-20220918.2101/transient hides /home/matt/git/emacs-build/lisp/transient

Features:
(shadow emacsbug completion vc-annotate flyspell ispell perl-mode
flymake-ruby flymake-easy ruby-mode whitespace conf-mode
flymake-yamllint yaml-mode calc-alg calc-ext calc-menu calc
calc-loaddefs calc-macs emoji-labels emoji multisession sqlite js qp
pylint tramp tramp-loaddefs trampver tramp-integration tramp-compat
ls-lisp yasnippet highlight-indentation elpy elpy-rpc pyvenv eshell
esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg esh-module esh-groups
esh-util elpy-shell elpy-profile elpy-django s elpy-refactor ido
hideshow grep cus-edit cus-start cus-load sh-script executable textsec
uni-scripts idna-mapping ucs-normalize uni-confusable textsec-check
macros apropos emacs-news-mode rect cl-print shortdoc help-fns
radix-tree magit-patch magit-subtree magit-gitignore magit-ediff ediff
ediff-merg ediff-mult ediff-wind ediff-diff ediff-help ediff-init
ediff-util term/xterm xterm git-rebase view facemenu two-column sort
company-oddmuse company-keywords company-etags company-gtags
company-dabbrev-code company-dabbrev company-files company-clang
company-capf company-cmake company-semantic company-template
company-bbdb company mail-extr files-x mm-archive tabify man notmuch
notmuch-tree notmuch-jump notmuch-hello notmuch-show notmuch-print
notmuch-crypto notmuch-mua notmuch-message notmuch-draft
notmuch-maildir-fcc notmuch-address notmuch-company notmuch-parser
notmuch-wash coolj goto-addr icalendar diary-lib diary-loaddefs
notmuch-tag notmuch-lib notmuch-compat hl-line magit-extras face-remap
magit-submodule magit-obsolete magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log which-func edebug
magit-diff smerge-mode diff git-commit log-edit add-log magit-core
magit-autorevert autorevert magit-margin magit-transient magit-process
with-editor magit-mode transient magit-git magit-base crm dabbrev gud
dired-aux misearch multi-isearch pulse display-line-numbers
bug-reference protbuf msb time copyright mule-util markdown-mode color
vc-hg vc-git diff-mode vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs
log-view pcvs-util vc vc-dispatcher go-mode find-file etags fileloop
eglot array jsonrpc ert pp ewoc debug backtrace xref flymake-proc
flymake compile pcase imenu clang-format+ clang-format cap-words
superword subword my llvm-c-style google-c-style cc-mode cc-fonts
cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
editorconfig-core editorconfig-core-handle editorconfig-fnmatch
org-element avl-tree generator ol-w3m ol-rmail ol-mhe ol-irc ol-info
org-habit org-agenda org-refile ol-gnus nnselect gnus-art mm-uu mml2015
mm-view mml-smime smime gnutls dig gnus-sum gnus-group gnus-undo
gnus-start gnus-dbus dbus gnus-cloud nnimap nnmail mail-source utf7 nnoo
parse-time gnus-spec gnus-int gnus-range message sendmail yank-media
rfc822 mml mml-sec epa derived epg rfc6068 epg-config mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums
mailabbrev gmm-utils mailheader gnus-win ol-eww eww xdg url-queue shr
pixel-fill kinsoku url-file svg xml dom puny mm-url gnus nnheader
gnus-util text-property-search mail-utils range wid-edit mm-util
mail-prsvr ol-doi org-link-doi ol-docview doc-view filenotify jka-compr
image-mode exif dired dired-loaddefs ol-bibtex ol-bbdb tab-line server
web-mode disp-table nix-mode ffap thingatpt smie nix-repl nix-shell
nix-store magit-section dash compat-27 compat-26 nix-instantiate
nix-shebang nix-format nix dirtrack ob-shell shell ob-ruby ob-python
python compat compat-macs ob-dot org-protocol org ob ob-tangle ob-ref
ob-lob ob-table ob-exp org-macro org-footnote org-src ob-comint
org-pcomplete pcomplete comint ansi-osc ansi-color ring org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp
ob-core ob-eval org-table oc-basic bibtex iso8601 time-date org-keys oc
org-loaddefs find-func cal-menu calendar cal-loaddefs finder-inf
ol-notmuch ol org-compat org-macs format-spec skeleton autoinsert advice
keyfreq project edmacro kmacro savehist icomplete comp comp-cstr
warnings icons rx editorconfig which-key package browse-url url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source eieio eieio-core password-cache json subr-x map byte-opt
url-vars cl-extra help-mode cl-macs gv cl-seq elec-pair use-package
use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core cl-loaddefs
cl-lib bytecomp byte-compile cconv info bazel-autoloads
clang-format+-autoloads clang-format-autoloads cmake-mode-autoloads
d-mode-autoloads debbugs-autoloads editorconfig-autoloads
eglot-autoloads elpy-autoloads company-autoloads
exec-path-from-shell-autoloads flymake-ruby-autoloads
flymake-easy-autoloads flymake-yamllint-autoloads go-mode-autoloads
google-c-style-autoloads graphviz-dot-mode-autoloads
highlight-indentation-autoloads magit-autoloads git-commit-autoloads
markdown-mode-autoloads meson-mode-autoloads nix-mode-autoloads
magit-section-autoloads dash-autoloads nixpkgs-fmt-autoloads
ol-notmuch-autoloads notmuch-autoloads orderless-autoloads
org-drill-autoloads ox-hugo-autoloads persist-autoloads pylint-autoloads
pyvenv-autoloads s-autoloads shfmt-autoloads reformatter-autoloads
tomelr-autoloads transient-autoloads use-package-autoloads
bind-key-autoloads vertico-autoloads web-mode-autoloads
which-key-autoloads with-editor-autoloads compat-autoloads
yaml-mode-autoloads yasnippet-autoloads rmc iso-transl tooltip eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/pgtk-win pgtk-win term/common-win pgtk-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
faces cus-face macroexp files window text-properties overlay sha1 md5
base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 1552683 200072)
 (symbols 48 60189 405)
 (strings 32 316503 22389)
 (string-bytes 1 9599931)
 (vectors 16 169883)
 (vector-slots 8 3447448 320750)
 (floats 8 774 987)
 (intervals 56 68555 6744)
 (buffers 1000 107))

-- 
matt (sent from an Emacs running the feature/noverlay branch)