GNU bug report logs - #79082
30.1; [reproducibility] sysinfo call during profile dump

Previous Next

To reply to this bug, email your comments to 79082 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: bug-gnu-emacs <at> gnu.org
Cc: Liliana Marie Prikler <liliana.prikler <at> gmail.com>
Subject: 30.1; [reproducibility] sysinfo call during profile dump
Date: Wed, 23 Jul 2025 16:09:41 +0200

Hi emacs,

I recently worked a bit on reproducibility which is not guaranteed in
particular in the profile dump.  I found that undeterminism during the
profile dump comes from two sources : calls to clock_gettime and
sysinfo.

clock_gettime is not that hard to patch using faketime, but it's called
multiple times so it'll be harder to find where in the codebase.

We don't have dedicated tools to patch sysinfo without compiling an
additional file an using the same LD_PRELOAD trick as faketime, but
there's actually an OK solution on the lisp side, IMHO (this is what I
propose for guix's emacs <at> 30.1, but it'd be great to add that on the next
release: 

(add-after 'unpack 'avoid-sysinfo-call-at-build-time
 (lambda _
  ;; This is a useful trick for reproducibility: when we configured
  ;; with --disable-build-details, (system-name) is nil at build
  ;; time on the lisp side.
  ;; Find those places with strace -k -e sysinfo.
  (substitute* "lisp/jit-lock.el"
    (("\\(condition-case nil \\(load-average\\) \\(error\\)\\)"
      all)
     (format #f "(and (system-name) ~a)" all)))))

It's only a single line change (with maybe a comment addition), please
proceed without my feedback if I don't answer, it doesn't deserve a
copyright citation.

-- 
Best regards,
Nicolas Graves

Message #8 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: "Dr. Werner Fink" <werner <at> suse.de>
To: Nicolas Graves <ngraves <at> ngraves.fr>
Cc: 79082 <at> debbugs.gnu.org, Liliana Marie Prikler <liliana.prikler <at> gmail.com>
Subject: Re: [emacs-bug] bug#79082: 30.1; [reproducibility] sysinfo call
 during profile dump
Date: Mon, 28 Jul 2025 16:13:08 +0200

[Message part 1 (text/plain, inline)]

On 2025/07/23 16:09:41 +0200, Nicolas Graves wrote:
> 
> Hi emacs,
> 
> I recently worked a bit on reproducibility which is not guaranteed in
> particular in the profile dump.  I found that undeterminism during the
> profile dump comes from two sources : calls to clock_gettime and
> sysinfo.
> 
> clock_gettime is not that hard to patch using faketime, but it's called
> multiple times so it'll be harder to find where in the codebase.
> 
> We don't have dedicated tools to patch sysinfo without compiling an
> additional file an using the same LD_PRELOAD trick as faketime, but
> there's actually an OK solution on the lisp side, IMHO (this is what I
> propose for guix's emacs <at> 30.1, but it'd be great to add that on the next
> release: 
> 
> (add-after 'unpack 'avoid-sysinfo-call-at-build-time
>  (lambda _
>   ;; This is a useful trick for reproducibility: when we configured
>   ;; with --disable-build-details, (system-name) is nil at build
>   ;; time on the lisp side.
>   ;; Find those places with strace -k -e sysinfo.
>   (substitute* "lisp/jit-lock.el"
>     (("\\(condition-case nil \\(load-average\\) \\(error\\)\\)"
>       all)
>      (format #f "(and (system-name) ~a)" all)))))
> 
> It's only a single line change (with maybe a comment addition), please
> proceed without my feedback if I don't answer, it doesn't deserve a
> copyright citation.

After some debugging with a helper function I've identified two
lisp objects, a Lisp Float which are set by the garbage collector,
and a Lisp Cons set by the initial lisp timestamp at startup.
Both are changing at every dump therefore avoid them or use
a fixed value at dump time.

During debugging I stumbled over some inconsistencies in
pdumper.c which are already fixed in upstream master branch.

Signed-off-by: Werner Fink <werner <at> suse.de>
---
 src/alloc.c   |    4 ++--
 src/pdumper.c |   10 +++++-----
 src/timefns.c |   34 ++++++++++++++++++++++++++++++++++
 3 files changed, 41 insertions(+), 7 deletions(-)

--- src/alloc.c
+++ src/alloc.c	2025-07-10 10:54:56.217020919 +0000
@@ -6702,8 +6702,8 @@ garbage_collect (void)
   image_prune_animation_caches (false);
 #endif
 
-  /* Accumulate statistics.  */
-  if (FLOATP (Vgc_elapsed))
+  /* Accumulate statistics, but not during dump to get reproducible pdmp images.  */
+  if (FLOATP (Vgc_elapsed) && !will_dump_p ())
     {
       static struct timespec gc_elapsed;
       gc_elapsed = timespec_add (gc_elapsed,
--- src/pdumper.c
+++ src/pdumper.c	2025-07-09 07:26:00.889706813 +0000
@@ -2140,9 +2140,9 @@ dump_interval_node (struct dump_context
   if (node->parent)
     dump_field_fixup_later (ctx, &out, node, &node->parent);
   if (node->left)
-    dump_field_fixup_later (ctx, &out, node, &node->parent);
+    dump_field_fixup_later (ctx, &out, node, &node->left);
   if (node->right)
-    dump_field_fixup_later (ctx, &out, node, &node->parent);
+    dump_field_fixup_later (ctx, &out, node, &node->right);
   DUMP_FIELD_COPY (&out, node, begin);
   DUMP_FIELD_COPY (&out, node, end);
   DUMP_FIELD_COPY (&out, node, limit);
@@ -2213,9 +2213,9 @@ dump_finalizer (struct dump_context *ctx
   /* Do _not_ call dump_pseudovector_lisp_fields here: we dump the
      only Lisp field, finalizer->function, manually, so we can give it
      a low weight.  */
-  dump_field_lv (ctx, &out, finalizer, &finalizer->function, WEIGHT_NONE);
-  dump_field_finalizer_ref (ctx, &out, finalizer, &finalizer->prev);
-  dump_field_finalizer_ref (ctx, &out, finalizer, &finalizer->next);
+  dump_field_lv (ctx, out, finalizer, &finalizer->function, WEIGHT_NONE);
+  dump_field_finalizer_ref (ctx, out, finalizer, &finalizer->prev);
+  dump_field_finalizer_ref (ctx, out, finalizer, &finalizer->next);
   return finish_dump_pvec (ctx, &out->header);
 }
 
--- src/timefns.c
+++ src/timefns.c	2025-07-11 07:32:33.928031852 +0000
@@ -600,6 +600,40 @@ make_lisp_time (struct timespec t)
 Lisp_Object
 timespec_to_lisp (struct timespec t)
 {
+  if (will_dump_p())	/* Use provided epoch at dump to get reproducible pdmp images */
+    {
+      char *epoch;
+      epoch = secure_getenv("SOURCE_DATE_EPOCH");
+      if (epoch)
+        {
+          char *endptr;
+          const char env[] = "Environment variable SOURCE_DATE_EPOCH: ";
+          errno = 0;
+          t.tv_sec = strtoull(epoch, &endptr, 10);
+          if ((errno == ERANGE && (t.tv_sec == ULLONG_MAX || t.tv_sec == 0)) ||\
+              (errno != 0 && t.tv_sec == 0))
+            {
+              fprintf(stderr, "%s strtoull: %m\n", env);
+              exit(EXIT_FAILURE);
+            }
+          if (endptr == epoch)
+            {
+              fprintf(stderr, "%s No digits were found: %s\n", env, endptr);
+              exit(EXIT_FAILURE);
+            }
+          if (*endptr != '\0')
+            {
+              fprintf(stderr, "%s Trailing garbage: %s\n", env, endptr);
+              exit(EXIT_FAILURE);
+            }
+          if (t.tv_sec > ULONG_MAX)
+            {
+              fprintf(stderr, "%s value must be smaller than or equal to %lu but was found to be: %ld \n", env, ULONG_MAX, t.tv_sec);
+              exit(EXIT_FAILURE);
+            }
+          t.tv_nsec = 0ULL;
+        }
+    }
   return Fcons (timespec_ticks (t), timespec_hz);
 }
 

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Dr. Werner Fink" <werner <at> suse.de>
Cc: 79082 <at> debbugs.gnu.org, ngraves <at> ngraves.fr, liliana.prikler <at> gmail.com
Subject: Re: bug#79082: [emacs-bug] bug#79082: 30.1;
 [reproducibility] sysinfo call during profile dump
Date: Mon, 28 Jul 2025 17:58:15 +0300

> Cc: 79082 <at> debbugs.gnu.org, Liliana Marie Prikler <liliana.prikler <at> gmail.com>
> Date: Mon, 28 Jul 2025 16:13:08 +0200
> From: "Dr. Werner Fink" <werner <at> suse.de>
> 
> > Hi emacs,
> > 
> > I recently worked a bit on reproducibility which is not guaranteed in
> > particular in the profile dump.  I found that undeterminism during the
> > profile dump comes from two sources : calls to clock_gettime and
> > sysinfo.
> > 
> > clock_gettime is not that hard to patch using faketime, but it's called
> > multiple times so it'll be harder to find where in the codebase.
> > 
> > We don't have dedicated tools to patch sysinfo without compiling an
> > additional file an using the same LD_PRELOAD trick as faketime, but
> > there's actually an OK solution on the lisp side, IMHO (this is what I
> > propose for guix's emacs <at> 30.1, but it'd be great to add that on the next
> > release: 
> > 
> > (add-after 'unpack 'avoid-sysinfo-call-at-build-time
> >  (lambda _
> >   ;; This is a useful trick for reproducibility: when we configured
> >   ;; with --disable-build-details, (system-name) is nil at build
> >   ;; time on the lisp side.
> >   ;; Find those places with strace -k -e sysinfo.
> >   (substitute* "lisp/jit-lock.el"
> >     (("\\(condition-case nil \\(load-average\\) \\(error\\)\\)"
> >       all)
> >      (format #f "(and (system-name) ~a)" all)))))
> > 
> > It's only a single line change (with maybe a comment addition), please
> > proceed without my feedback if I don't answer, it doesn't deserve a
> > copyright citation.
> 
> After some debugging with a helper function I've identified two
> lisp objects, a Lisp Float which are set by the garbage collector,
> and a Lisp Cons set by the initial lisp timestamp at startup.
> Both are changing at every dump therefore avoid them or use
> a fixed value at dump time.
> 
> During debugging I stumbled over some inconsistencies in
> pdumper.c which are already fixed in upstream master branch.

Thanks.

However, in order for us to accept these patches, they must fulfill
the following conditions:

 . the special code which avoids causing the build to become not
   reproducible must be conditioned on an optional variable, or at
   least on some compile-time symbol (like, for example, something
   derived from --disable-build-details); and
 . the code changes should either be portable, or, if they are
   specific to GNU/Linux, they should be conditioned on an appropriate
   preprocessor macro to avoid compiling it on other platforms

> --- src/pdumper.c
> +++ src/pdumper.c	2025-07-09 07:26:00.889706813 +0000
> @@ -2140,9 +2140,9 @@ dump_interval_node (struct dump_context
>    if (node->parent)
>      dump_field_fixup_later (ctx, &out, node, &node->parent);
>    if (node->left)
> -    dump_field_fixup_later (ctx, &out, node, &node->parent);
> +    dump_field_fixup_later (ctx, &out, node, &node->left);
>    if (node->right)
> -    dump_field_fixup_later (ctx, &out, node, &node->parent);
> +    dump_field_fixup_later (ctx, &out, node, &node->right);

This code looks differently on master.

> @@ -2213,9 +2213,9 @@ dump_finalizer (struct dump_context *ctx
>    /* Do _not_ call dump_pseudovector_lisp_fields here: we dump the
>       only Lisp field, finalizer->function, manually, so we can give it
>       a low weight.  */
> -  dump_field_lv (ctx, &out, finalizer, &finalizer->function, WEIGHT_NONE);
> -  dump_field_finalizer_ref (ctx, &out, finalizer, &finalizer->prev);
> -  dump_field_finalizer_ref (ctx, &out, finalizer, &finalizer->next);
> +  dump_field_lv (ctx, out, finalizer, &finalizer->function, WEIGHT_NONE);
> +  dump_field_finalizer_ref (ctx, out, finalizer, &finalizer->prev);
> +  dump_field_finalizer_ref (ctx, out, finalizer, &finalizer->next);
>    return finish_dump_pvec (ctx, &out->header);

This is already in the code on master.

> --- src/timefns.c
> +++ src/timefns.c	2025-07-11 07:32:33.928031852 +0000
> @@ -600,6 +600,40 @@ make_lisp_time (struct timespec t)
>  Lisp_Object
>  timespec_to_lisp (struct timespec t)
>  {
> +  if (will_dump_p())	/* Use provided epoch at dump to get reproducible pdmp images */
> +    {
> +      char *epoch;
> +      epoch = secure_getenv("SOURCE_DATE_EPOCH");

secure_getenv is specific to glibc.  If we want to use it on other
platforms, we need to import the Gnulib secure_getenv module.

We should also make sure this change doesn't interfere with the build,
since we do have code in Emacs that manipulates time and is used
during the build.

Message #14 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: "Dr. Werner Fink" <werner <at> suse.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79082 <at> debbugs.gnu.org, ngraves <at> ngraves.fr, liliana.prikler <at> gmail.com
Subject: Re: bug#79082: [emacs-bug] bug#79082: 30.1; [reproducibility]
 sysinfo call during profile dump
Date: Tue, 29 Jul 2025 10:57:21 +0200

[Message part 1 (text/plain, inline)]

On 2025/07/28 17:58:15 +0300, Eli Zaretskii wrote:
> Thanks.
> 
> However, in order for us to accept these patches, they must fulfill
> the following conditions:
> 
>  . the special code which avoids causing the build to become not
>    reproducible must be conditioned on an optional variable, or at
>    least on some compile-time symbol (like, for example, something
>    derived from --disable-build-details); and
>  . the code changes should either be portable, or, if they are
>    specific to GNU/Linux, they should be conditioned on an appropriate
>    preprocessor macro to avoid compiling it on other platforms
> 
> > --- src/pdumper.c
> > +++ src/pdumper.c	2025-07-09 07:26:00.889706813 +0000
> > @@ -2140,9 +2140,9 @@ dump_interval_node (struct dump_context
> >    if (node->parent)
> >      dump_field_fixup_later (ctx, &out, node, &node->parent);
> >    if (node->left)
> > -    dump_field_fixup_later (ctx, &out, node, &node->parent);
> > +    dump_field_fixup_later (ctx, &out, node, &node->left);
> >    if (node->right)
> > -    dump_field_fixup_later (ctx, &out, node, &node->parent);
> > +    dump_field_fixup_later (ctx, &out, node, &node->right);
> 
> This code looks differently on master.
> 
> > @@ -2213,9 +2213,9 @@ dump_finalizer (struct dump_context *ctx
> >    /* Do _not_ call dump_pseudovector_lisp_fields here: we dump the
> >       only Lisp field, finalizer->function, manually, so we can give it
> >       a low weight.  */
> > -  dump_field_lv (ctx, &out, finalizer, &finalizer->function, WEIGHT_NONE);
> > -  dump_field_finalizer_ref (ctx, &out, finalizer, &finalizer->prev);
> > -  dump_field_finalizer_ref (ctx, &out, finalizer, &finalizer->next);
> > +  dump_field_lv (ctx, out, finalizer, &finalizer->function, WEIGHT_NONE);
> > +  dump_field_finalizer_ref (ctx, out, finalizer, &finalizer->prev);
> > +  dump_field_finalizer_ref (ctx, out, finalizer, &finalizer->next);
> >    return finish_dump_pvec (ctx, &out->header);
> 
> This is already in the code on master.

I know as this was derived from master for 30.1.  In the attachment I've
skipped that and moved from will_dump_p() to !build_details, switched from
secure_getnev() to getenv(), and add a comment about SOURCE_DATE_EPOCH.

> > --- src/timefns.c
> > +++ src/timefns.c	2025-07-11 07:32:33.928031852 +0000
> > @@ -600,6 +600,40 @@ make_lisp_time (struct timespec t)
> >  Lisp_Object
> >  timespec_to_lisp (struct timespec t)
> >  {
> > +  if (will_dump_p())	/* Use provided epoch at dump to get reproducible pdmp images */
> > +    {
> > +      char *epoch;
> > +      epoch = secure_getenv("SOURCE_DATE_EPOCH");
> 
> secure_getenv is specific to glibc.  If we want to use it on other
> platforms, we need to import the Gnulib secure_getenv module.
> 
> We should also make sure this change doesn't interfere with the build,
> since we do have code in Emacs that manipulates time and is used
> during the build.

That is the reason to use timespec_to_lisp() as these seems to be called
only once at start of the internal lisp engine. Without the attached patch
and the set SOURCE_DATE_EPOCH envvar there is always a changing timestamp

BUILD/emacs-30.1-build/emacs-30.1/src> od  -tx1 emacs.pdmp > x1
BUILD/emacs-30.1-build/emacs-30.1/src> diff -up x1 x2
--- x1  2025-07-29 08:38:00.204476936 +0000
+++ x2  2025-07-29 08:37:31.027684840 +0000
@@ -173419,7 +173419,7 @@
 15427560 00 50 00 03 00 00 00 40 30 2b 36 00 00 00 00 00
 15427600 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 15427620 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00
-15427640 fe 15 37 7a 29 b0 5a 61 02 28 6b ee 00 00 00 00
+15427640 1a 9b 1a d3 08 b0 5a 61 02 28 6b ee 00 00 00 00
 15427660 00 50 00 03 00 00 00 40 00 00 00 00 00 00 00 00
 15427700 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 *

In my spec file I use the line

 SOURCE_DATE_EPOCH="$(sed -n '/^----/n;s/ - .*$//;p;q' emacs.changes| date -u -f - +%%s)"
 export SOURCE_DATE_EPOCH

with the changelog which gets added to the spec file.

-- 
  "Having a smoking section in a restaurant is like having
          a peeing section in a swimming pool." -- Edward Burr

[emacs-30.1-pdumper.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Dr. Werner Fink" <werner <at> suse.de>
Cc: 79082 <at> debbugs.gnu.org, ngraves <at> ngraves.fr, liliana.prikler <at> gmail.com
Subject: Re: bug#79082: [emacs-bug] bug#79082: 30.1; [reproducibility]
 sysinfo call during profile dump
Date: Tue, 29 Jul 2025 14:26:43 +0300

> Date: Tue, 29 Jul 2025 10:57:21 +0200
> From: "Dr. Werner Fink" <werner <at> suse.de>
> Cc: ngraves <at> ngraves.fr, 79082 <at> debbugs.gnu.org, liliana.prikler <at> gmail.com
> 
> I know as this was derived from master for 30.1.  In the attachment I've
> skipped that and moved from will_dump_p() to !build_details, switched from
> secure_getnev() to getenv(), and add a comment about SOURCE_DATE_EPOCH.
> 
> > > --- src/timefns.c
> > > +++ src/timefns.c	2025-07-11 07:32:33.928031852 +0000
> > > @@ -600,6 +600,40 @@ make_lisp_time (struct timespec t)
> > >  Lisp_Object
> > >  timespec_to_lisp (struct timespec t)
> > >  {
> > > +  if (will_dump_p())	/* Use provided epoch at dump to get reproducible pdmp images */
> > > +    {
> > > +      char *epoch;
> > > +      epoch = secure_getenv("SOURCE_DATE_EPOCH");
> > 
> > secure_getenv is specific to glibc.  If we want to use it on other
> > platforms, we need to import the Gnulib secure_getenv module.
> > 
> > We should also make sure this change doesn't interfere with the build,
> > since we do have code in Emacs that manipulates time and is used
> > during the build.
> 
> That is the reason to use timespec_to_lisp() as these seems to be called
> only once at start of the internal lisp engine.

I'm not sure we can rely on that.  Some future change might call it
more than once, in which case it will return the same value and could
break something.

So I would prefer making such changes where the time value is used in
a way that it makes the build not reproducible, instead of changing a
low-level function that can be used in other context.

> In my spec file I use the line
> 
>  SOURCE_DATE_EPOCH="$(sed -n '/^----/n;s/ - .*$//;p;q' emacs.changes| date -u -f - +%%s)"
>  export SOURCE_DATE_EPOCH
> 
> with the changelog which gets added to the spec file.

Isn't this completely unportable?

Message #20 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: "Dr. Werner Fink" <werner <at> suse.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79082 <at> debbugs.gnu.org, ngraves <at> ngraves.fr, liliana.prikler <at> gmail.com
Subject: Re: bug#79082: [emacs-bug] bug#79082: 30.1; [reproducibility]
 sysinfo call during profile dump
Date: Tue, 29 Jul 2025 13:46:59 +0200

[Message part 1 (text/plain, inline)]

On 2025/07/29 14:26:43 +0300, Eli Zaretskii wrote:
> 
> I'm not sure we can rely on that.  Some future change might call it
> more than once, in which case it will return the same value and could
> break something.
> 
> So I would prefer making such changes where the time value is used in
> a way that it makes the build not reproducible, instead of changing a
> low-level function that can be used in other context.

Is there any way to remember which value will be stored in the final
(p)dump image?  Like set a mark/tag in the Lisp Cons or Lisp Float to
not to dump such values into the resulting (p)dump images?

> 
> > In my spec file I use the line
> > 
> >  SOURCE_DATE_EPOCH="$(sed -n '/^----/n;s/ - .*$//;p;q' emacs.changes| date -u -f - +%%s)"
> >  export SOURCE_DATE_EPOCH
> > 
> > with the changelog which gets added to the spec file.
> 
> Isn't this completely unportable?

Sorry but this is the defacto standard to get reproducible builds, see

  https://reproducible-builds.org/docs/source-date-epoch/

and any distribution depends on it. On how SOURCE_DATE_EPOCH is
determined is an other story.  I've choosen the latest changelog
entry as this is here the last change before a package gets
submitted to QA staging area.  Means no changes of the package
will results in a reproducible build on every rebuild.

Werner

-- 
  "Having a smoking section in a restaurant is like having
          a peeing section in a swimming pool." -- Edward Burr

[signature.asc (application/pgp-signature, inline)]

Message #23 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Dr. Werner Fink" <werner <at> suse.de>
Cc: 79082 <at> debbugs.gnu.org, ngraves <at> ngraves.fr, liliana.prikler <at> gmail.com
Subject: Re: bug#79082: [emacs-bug] bug#79082: 30.1; [reproducibility]
 sysinfo call during profile dump
Date: Wed, 30 Jul 2025 14:14:34 +0300

> Date: Tue, 29 Jul 2025 13:46:59 +0200
> From: "Dr. Werner Fink" <werner <at> suse.de>
> Cc: ngraves <at> ngraves.fr, 79082 <at> debbugs.gnu.org, liliana.prikler <at> gmail.com
> 
> On 2025/07/29 14:26:43 +0300, Eli Zaretskii wrote:
> > 
> > I'm not sure we can rely on that.  Some future change might call it
> > more than once, in which case it will return the same value and could
> > break something.
> > 
> > So I would prefer making such changes where the time value is used in
> > a way that it makes the build not reproducible, instead of changing a
> > low-level function that can be used in other context.
> 
> Is there any way to remember which value will be stored in the final
> (p)dump image?  Like set a mark/tag in the Lisp Cons or Lisp Float to
> not to dump such values into the resulting (p)dump images?

I thought you already knew that.  If not, how did you figure out that
timespec_to_lisp is the place where to make the change?

> > 
> > > In my spec file I use the line
> > > 
> > >  SOURCE_DATE_EPOCH="$(sed -n '/^----/n;s/ - .*$//;p;q' emacs.changes| date -u -f - +%%s)"
> > >  export SOURCE_DATE_EPOCH
> > > 
> > > with the changelog which gets added to the spec file.
> > 
> > Isn't this completely unportable?
> 
> Sorry but this is the defacto standard to get reproducible builds, see
> 
>   https://reproducible-builds.org/docs/source-date-epoch/
> 
> and any distribution depends on it. On how SOURCE_DATE_EPOCH is
> determined is an other story.  I've choosen the latest changelog
> entry as this is here the last change before a package gets
> submitted to QA staging area.  Means no changes of the package
> will results in a reproducible build on every rebuild.

Then I guess this will only work on Posix systems with Bash.  Too bad.

Message #26 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: "Dr. Werner Fink" <werner <at> suse.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79082 <at> debbugs.gnu.org, ngraves <at> ngraves.fr, liliana.prikler <at> gmail.com
Subject: Re: bug#79082: [emacs-bug] bug#79082: 30.1; [reproducibility]
 sysinfo call during profile dump
Date: Wed, 30 Jul 2025 14:23:06 +0200

[Message part 1 (text/plain, inline)]

On 2025/07/30 14:14:34 +0300, Eli Zaretskii wrote:
> > Date: Tue, 29 Jul 2025 13:46:59 +0200
> > From: "Dr. Werner Fink" <werner <at> suse.de>
> > Cc: ngraves <at> ngraves.fr, 79082 <at> debbugs.gnu.org, liliana.prikler <at> gmail.com
> > 
> > On 2025/07/29 14:26:43 +0300, Eli Zaretskii wrote:
> > > 
> > > I'm not sure we can rely on that.  Some future change might call it
> > > more than once, in which case it will return the same value and could
> > > break something.
> > > 
> > > So I would prefer making such changes where the time value is used in
> > > a way that it makes the build not reproducible, instead of changing a
> > > low-level function that can be used in other context.
> > 
> > Is there any way to remember which value will be stored in the final
> > (p)dump image?  Like set a mark/tag in the Lisp Cons or Lisp Float to
> > not to dump such values into the resulting (p)dump images?
> 
> I thought you already knew that.  If not, how did you figure out that
> timespec_to_lisp is the place where to make the change?

I've used a C helper function with

 static int sigsegv;
 //static const unsigned char bytarr[] = {0x68, 0x65, 0x6d, 0x0a, 0x62, 0x79, 0x20, 0x55, 0x54, 0x46, 0x2d, 0x38, 0x2e, 0x00};
 static const unsigned char bytarr[] = {0x61, 0x02, 0x28, 0x6b, 0xee};
 static const size_t bytlen = sizeof(bytarr)/sizeof(bytarr[0]);
 static size_t memmatch(const void* mem, size_t mem_len, const void* pattern,
     size_t pattern_length, size_t* pattern_index)
 {
     eassert(*pattern_index < pattern_length);
     size_t idx = 0; 
     register size_t index = *pattern_index;
     for (; idx < mem_len;) {
         if (((const unsigned char*)mem)[idx++] == ((const unsigned char*)pattern)[index]) {
             ++index;
             if (pattern_length == index) {
                 break; // ++idx;
             }
         } else if (index) {
             index = 0; // reset
         }
     }
     *pattern_index = index;
     return idx;
 }

in dump_write() to trigger SIGSEGV or run direct in gdb with break/watch points
and with the help of emacs-30.1/src/.gdbinit I've determined the Lisp Float and
the Lisp Cons which changes on every run of temacs at pdump.  The first as well
as the second bytarr is simply taken form the diffs of two od dump outputs. With
the first something like 'by UTF-8.' and the second a very large usigned integer
which had me reminded in gdb on seconds past epoch. The rest was a diligent but
routine piece of work.

Werner

-- 
  "Having a smoking section in a restaurant is like having
          a peeing section in a swimming pool." -- Edward Burr

[signature.asc (application/pgp-signature, inline)]

Message #29 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: "Dr. Werner Fink" <werner <at> suse.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79082 <at> debbugs.gnu.org, ngraves <at> ngraves.fr, liliana.prikler <at> gmail.com
Subject: Re: bug#79082: [emacs-bug] bug#79082: 30.1; [reproducibility]
 sysinfo call during profile dump
Date: Wed, 30 Jul 2025 14:32:39 +0200

[Message part 1 (text/plain, inline)]

On 2025/07/30 14:14:34 +0300, Eli Zaretskii wrote:
> > Sorry but this is the defacto standard to get reproducible builds, see
> > 
> >   https://reproducible-builds.org/docs/source-date-epoch/
> > 
> > and any distribution depends on it. On how SOURCE_DATE_EPOCH is
> > determined is an other story.  I've choosen the latest changelog
> > entry as this is here the last change before a package gets
> > submitted to QA staging area.  Means no changes of the package
> > will results in a reproducible build on every rebuild.
> 
> Then I guess this will only work on Posix systems with Bash.  Too bad.

You can use every shell like interpreter which is able to determine an
integer value and set an environment variable.

It would be nice if GNU Emacs upstream would fix the problem with the
two changing values of the gc statistics and lisp engine startup time.
If with my proposal or any other way I simply do not care but the
configure option --disable-build-details should do its job.

Werner
-- 
  "Having a smoking section in a restaurant is like having
          a peeing section in a swimming pool." -- Edward Burr

[signature.asc (application/pgp-signature, inline)]

Message #32 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Dr. Werner Fink" <werner <at> suse.de>
Cc: 79082 <at> debbugs.gnu.org, ngraves <at> ngraves.fr, liliana.prikler <at> gmail.com
Subject: Re: bug#79082: [emacs-bug] bug#79082: 30.1; [reproducibility]
 sysinfo call during profile dump
Date: Wed, 30 Jul 2025 15:45:36 +0300

> Date: Wed, 30 Jul 2025 14:23:06 +0200
> From: "Dr. Werner Fink" <werner <at> suse.de>
> Cc: ngraves <at> ngraves.fr, 79082 <at> debbugs.gnu.org, liliana.prikler <at> gmail.com
> 
> On 2025/07/30 14:14:34 +0300, Eli Zaretskii wrote:
> > > Date: Tue, 29 Jul 2025 13:46:59 +0200
> > > From: "Dr. Werner Fink" <werner <at> suse.de>
> > > Cc: ngraves <at> ngraves.fr, 79082 <at> debbugs.gnu.org, liliana.prikler <at> gmail.com
> > > 
> > > On 2025/07/29 14:26:43 +0300, Eli Zaretskii wrote:
> > > > 
> > > > I'm not sure we can rely on that.  Some future change might call it
> > > > more than once, in which case it will return the same value and could
> > > > break something.
> > > > 
> > > > So I would prefer making such changes where the time value is used in
> > > > a way that it makes the build not reproducible, instead of changing a
> > > > low-level function that can be used in other context.
> > > 
> > > Is there any way to remember which value will be stored in the final
> > > (p)dump image?  Like set a mark/tag in the Lisp Cons or Lisp Float to
> > > not to dump such values into the resulting (p)dump images?
> > 
> > I thought you already knew that.  If not, how did you figure out that
> > timespec_to_lisp is the place where to make the change?
> 
> I've used a C helper function with
> 
>  static int sigsegv;
>  //static const unsigned char bytarr[] = {0x68, 0x65, 0x6d, 0x0a, 0x62, 0x79, 0x20, 0x55, 0x54, 0x46, 0x2d, 0x38, 0x2e, 0x00};
>  static const unsigned char bytarr[] = {0x61, 0x02, 0x28, 0x6b, 0xee};
>  static const size_t bytlen = sizeof(bytarr)/sizeof(bytarr[0]);
>  static size_t memmatch(const void* mem, size_t mem_len, const void* pattern,
>      size_t pattern_length, size_t* pattern_index)
>  {
>      eassert(*pattern_index < pattern_length);
>      size_t idx = 0; 
>      register size_t index = *pattern_index;
>      for (; idx < mem_len;) {
>          if (((const unsigned char*)mem)[idx++] == ((const unsigned char*)pattern)[index]) {
>              ++index;
>              if (pattern_length == index) {
>                  break; // ++idx;
>              }
>          } else if (index) {
>              index = 0; // reset
>          }
>      }
>      *pattern_index = index;
>      return idx;
>  }
> 
> in dump_write() to trigger SIGSEGV or run direct in gdb with break/watch points
> and with the help of emacs-30.1/src/.gdbinit I've determined the Lisp Float and
> the Lisp Cons which changes on every run of temacs at pdump.  The first as well
> as the second bytarr is simply taken form the diffs of two od dump outputs. With
> the first something like 'by UTF-8.' and the second a very large usigned integer
> which had me reminded in gdb on seconds past epoch. The rest was a diligent but
> routine piece of work.

Thanks.  I'm not sure I understood your technique, but it's imperative
to know what Lisp values depend on build time, and then we could
discuss how best to avoid dumping them.

Message #35 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Dr. Werner Fink" <werner <at> suse.de>
Cc: 79082 <at> debbugs.gnu.org, ngraves <at> ngraves.fr, liliana.prikler <at> gmail.com
Subject: Re: bug#79082: [emacs-bug] bug#79082: 30.1; [reproducibility]
 sysinfo call during profile dump
Date: Wed, 30 Jul 2025 15:50:03 +0300

> Date: Wed, 30 Jul 2025 14:32:39 +0200
> From: "Dr. Werner Fink" <werner <at> suse.de>
> Cc: ngraves <at> ngraves.fr, 79082 <at> debbugs.gnu.org, liliana.prikler <at> gmail.com
> 
> It would be nice if GNU Emacs upstream would fix the problem with the
> two changing values of the gc statistics and lisp engine startup time.
> If with my proposal or any other way I simply do not care but the
> configure option --disable-build-details should do its job.

I understand, but I cannot in good faith uphold changes about whose
correctness I have doubts, as I explained in previous messages.  We
should not overwrite every time value by a single fixed value, because
this could easily break the build, if it doesn't already.

Message #38 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: "Dr. Werner Fink" <werner <at> suse.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79082 <at> debbugs.gnu.org, ngraves <at> ngraves.fr, liliana.prikler <at> gmail.com
Subject: Re: bug#79082: [emacs-bug] bug#79082: 30.1; [reproducibility]
 sysinfo call during profile dump
Date: Mon, 8 Sep 2025 15:50:45 +0200

[Message part 1 (text/plain, inline)]

On 2025/07/30 15:50:03 +0300, Eli Zaretskii wrote:
> > Date: Wed, 30 Jul 2025 14:32:39 +0200
> > From: "Dr. Werner Fink" <werner <at> suse.de>
> > Cc: ngraves <at> ngraves.fr, 79082 <at> debbugs.gnu.org, liliana.prikler <at> gmail.com
> > 
> > It would be nice if GNU Emacs upstream would fix the problem with the
> > two changing values of the gc statistics and lisp engine startup time.
> > If with my proposal or any other way I simply do not care but the
> > configure option --disable-build-details should do its job.
> 
> I understand, but I cannot in good faith uphold changes about whose
> correctness I have doubts, as I explained in previous messages.  We
> should not overwrite every time value by a single fixed value, because
> this could easily break the build, if it doesn't already.

Just to be noted: No bug reports so far. It simply works.

Werner

-- 
  "Having a smoking section in a restaurant is like having
          a peeing section in a swimming pool." -- Edward Burr

[signature.asc (application/pgp-signature, inline)]

Message #41 received at 79082 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Dr. Werner Fink" <werner <at> suse.de>
Cc: 79082 <at> debbugs.gnu.org, ngraves <at> ngraves.fr, liliana.prikler <at> gmail.com
Subject: Re: bug#79082: [emacs-bug] bug#79082: 30.1; [reproducibility]
 sysinfo call during profile dump
Date: Mon, 08 Sep 2025 18:21:59 +0300

> Date: Mon, 8 Sep 2025 15:50:45 +0200
> From: "Dr. Werner Fink" <werner <at> suse.de>
> Cc: ngraves <at> ngraves.fr, 79082 <at> debbugs.gnu.org, liliana.prikler <at> gmail.com
> 
> On 2025/07/30 15:50:03 +0300, Eli Zaretskii wrote:
> > > Date: Wed, 30 Jul 2025 14:32:39 +0200
> > > From: "Dr. Werner Fink" <werner <at> suse.de>
> > > Cc: ngraves <at> ngraves.fr, 79082 <at> debbugs.gnu.org, liliana.prikler <at> gmail.com
> > > 
> > > It would be nice if GNU Emacs upstream would fix the problem with the
> > > two changing values of the gc statistics and lisp engine startup time.
> > > If with my proposal or any other way I simply do not care but the
> > > configure option --disable-build-details should do its job.
> > 
> > I understand, but I cannot in good faith uphold changes about whose
> > correctness I have doubts, as I explained in previous messages.  We
> > should not overwrite every time value by a single fixed value, because
> > this could easily break the build, if it doesn't already.
> 
> Just to be noted: No bug reports so far. It simply works.

Thanks, noted.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.