Subject: bug#41571: 27.0.91; "(elisp) Interpolated Strings" is under "(elisp) Text"

From: "Basil L. Contovounesios" <[email-masked]>
Date: Sun, 31 May 2020 10:24:46 +0100

Eli Zaretskii <[email-masked]> writes:

>> From: "Basil L. Contovounesios" <[email-masked]>
>> Cc: [email-masked]
>> Date: Fri, 29 May 2020 19:35:31 +0100
>> 
>> How's the attached for emacs-27?
>
> Thanks, it's much better, but it still "needs work".

Thanks for the very helpful feedback.

> (Btw, why are you attachments appear before the text?  It makes
> responding harder, because I need to bring the citations to the
> front.)

Sorry, I got into the habit of doing that because I wasn't sure whether
attachments should go before or after the email signature.  I'm guessing
after?

>> +The functions described in this section accept a fixed set of
>> +specification characters.  The next section describes a function
>> +@code{format-spec} which accepts custom specification characters.
>
> This would benefit from making it clear what you mean by
> "specification characters".  An example would clarify that.
>
> (It is actually a general comment to your text: you frequently use
> terms that are left unexplained, which makes the reader stumble and
> try to understand what you mean.  Specific examples below.)

I agree that this sentence can be further clarified, but in general I
tried to reuse existing terminology and phrasing to the greatest extent
possible (maybe even too much).

E.g. the preceding paragraphs in "(elisp) Formatting Strings" define and
make extensive use of the terms "format string", "format specification",
"format specification character", as well as the shorter forms
"specification character" and "specification".

>> +It is, in some circumstances, useful to allow users to control how
>
> The beginning of this sentence is unnecessarily complex.  A much
> simpler variant would be
>
>   Sometimes it is useful to allow Lisp programs to control...
>
> Note that I replaced "users" with "Lisp  programs", since we are not
> talking about Emacs users here.

Done.

>> +A more convenient format string for such cases would be something like
>> +@code{"%f %l <%e>"}, where each specification character carries more
>> +semantic information and can easily be rearranged relative to other
>> +specification characters.  The function @code{format-spec} described
>> +in this section performs a similar function to @code{format}, except
>> +it operates on format control strings that comprise arbitrary
>> +specification characters.
>
> "comprise" => "include", or even just "use"

Done.

>> +@defun format-spec format specification &optional only-present
>> +This function returns a string equal to the format control string
>
> The "equal" here is confusing, because equality is not really
> important here, especially since the job of this function is to
> produce strings that are NOT equal to the original.

I agree; this is just a copy of the corresponding text from "(elisp)
Formatting Strings".

>> +@var{format}, replacing any format specifications it contains with
>> +values found in the alist @var{specification} (@pxref{Association
>> +Lists}).
>> +
>> +Each key in @var{specification} is a format specification character,
>> +and its associated value is the string to replace it with.  For
>> +example, an alist entry @code{(?a . "alpha")} means to replace any
>> +@samp{%a} specifications in @var{format} with @samp{alpha}.
>
> You say "key in SPECIFICATION", but SPECIFICATION is an alist, and a
> key in an alist has well-known meaning.  The "key" above should be
> "association".

"Each key in [the alist] is a [...] character" means each key in each
association in the alist is a character, so the wording is not wrong,
just unclear.

> And in general I'd rearrange the text to make the
> format of SPECIFICATION more explicit, something like:
>
>   @defun format-spec template spec-alist &optional only-present
>   This function returns a format string suitable for using in
>   @code{format} and similar functions.  The format string is produced
>   from @var{template} according to conversions specified in
>   @var{spec-alist}, which is an alist (@pxref{Association Lists}) of
>   the form @w{@code{(@var{letter} . @var{replacement})}}.  Each
>   specification @code{%@var{letter}} in @var{template} will be
>   replaced by @var{replacement} when producing the resulting format
>   string.

This wording is much clearer, but the description of the output is
wrong: 'format-spec' and 'format' both produce the same result - a
formatted string, not a format string.  'format-spec' is an alternative
to 'format', not a precursor.

>> +Some useful properties are gained as a result of @var{specification}
>> +being an alist.  The alist may contain more unique keys than there are
>> +unique specification characters in @var{format}; unused keys are
>> +simply ignored.  If the same key is contained more than once, the
>> +first one found is used.  If @var{format} contains the same format
>> +specification character more than once, then the same value found in
>> +@var{specification} is used as a basis for all of that character's
>> +substitutions.
>
> Here you use "key" without first explaining what it is.

I was relying on the preceding xref to the node on alists, which defines
the terms "alist", "key", and "associated value".

> Also, this paragraph describes several distinct features, so it is
> better to use an itemized list instead of just one sentence after
> another: it makes the description easier to grasp by dividing it into
> distinct smaller chunks.

Done, including mentioning that associations can appear in a different
order to their corresponding format specifications in the format string.

>> +The optional argument @var{only-present} indicates how to handle
>> +format specification characters in @var{format} that are not found in
>> +@var{specification}.  If it is @code{nil} or omitted, an error is
>> +emitted.
>
> Passive tense alert!  Suggest to rephrase
>
>   If it is @code{nil} or omitted, the function signals an error.

Fire extinguished.

>> +The syntax of format specifications accepted by @code{format-spec} is
>> +similar, but not identical, to that accepted by @code{format}.  In
>> +both cases, a format specification is a sequence of characters
>> +beginning with @samp{%} and ending with an alphabetic letter such as
>> +@samp{s}.  The only exception to this is the specification @samp{%%},
>> +which is replaced with a single @samp{%}.
>
> How is what's described in the last sentence "an exception"?  Format
> strings used by 'format' also behave like that, right?

It's an exception to "beginning with % and ending with a letter".

Would it be clearer if I said "the only specification that does not end
in a letter is %%, which is replaced with a single % in the output"?

>> +Unlike @code{format}, which assigns specific meanings to a fixed set
>> +of specification characters, @code{format-spec} accepts arbitrary
>> +specification characters and treats them all equally.  For example:
>> +
>> +@example
>> +(format-spec "su - %u %l"
>> +             `((?u . ,(user-login-name))
>> +               (?l . "ls")))
>> +     @result{} "su - foo ls"
>> +@end example
>
> This example stops short of explaining why this function is useful:
> making the replacements fixed strings, as in "ls", is not the reason.
> OTOH, the use of user-login-name is obfuscated by the backtick
> notation, which seems to say that some magic is needed here.
>
> I think the reason for having this function should be explained
> better, with more meaningful examples.

Hm, I'm not sure how to give a better existential justification; this
example just serves as a usage example.

The main use case for format-spec I've seen is where one part of the
program produces an alist with all the information that could ever be
needed, and another part of the program formats an often
user-customisable format string using this data.

An example of such a use case is in battery.el, where the alist produced
by battery-status-function is used to format battery-echo-area-format
and battery-mode-line-format (battery.el doesn't currently use
format-spec, but it could and my WIP patch for master changes that).

Would replicating such a use case make a better example?  E.g.:

  (setq my-site-info
        (list (cons ?s system-name)
              (cons ?t (symbol-name system-type))
              (cons ?c system-configuration)
              (cons ?v emacs-version)
              (cons ?e invocation-name)
              (cons ?p (number-to-string (emacs-pid)))
              (cons ?a user-mail-address)
              (cons ?n user-full-name)))

  (format-spec "%e %v (%c)" my-site-info)
    => "emacs 28.0.50 (x86_64-pc-linux-gnu)"

  (format-spec "%n <%a>" my-site-info)
    => "Emacs Developers <[email-masked]>"

>> +@item 0
>> +This flag causes any padding inserted by the width, if specified, to
>                                 ^^^^^^^^^^^^^^^^^^^^^
> Width cannot insert anything, so this should be reworded.  Same in a
> few other items.

Most of this phrasing is taken from "(elisp) Formatting Strings".

Is it clear enough to say "...causes any padding specified by the width
to..."?

>> +@item <
>> +This flag causes the substitution to be truncated to the given width,
>> +if specified, by removing characters from the left.
>
> "truncated ... by removing characters" is unnecessarily complicated.
> Why not say simply "truncated on the left"?

Done.

>> +@item >
>> +This flag causes the substitution to be truncated to the given width,
>> +if specified, by removing characters from the right.
>
> Same here.

Done.

>> +As is the case with @code{format}, a format specification can include
>> +a width, which is a decimal number that appears after any flags.  If a
>> +substitution contains fewer characters than its specified width, it is
>> +extended with padding, normally comprising spaces inserted on the^^^^^
>>  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> +left:
>>  ^^^^
>
> "it is padded on the left" is simpler and more clear.

Done.

>> +Here is a more complicated example that combines several
>> +aforementioned features:
>> +
>> +@example
>> +(format-spec "%<06e %<06b"
>> +             '((?b . "beta")
>> +               (?e . "epsilon")))
>> +     @result{} "psilon 00beta"
>> +@end example
>
> Can we make this example be less trivial?  This use case doesn't
> justify using format-spec at all.

It's hard coming up with a simple enough example that both justifies
format-spec and showcases modifier combinations.  Would replicating a
subset of the output of battery-status-function be any good?  E.g.:

  (setq my-battery-info
        (list (cons ?p "73")      ; Percentage
              (cons ?L "Battery") ; Status
              (cons ?t "2:23")    ; Remaining time
              (cons ?c "24330")   ; Capacity
              (cons ?r "10.6")))  ; Rate of discharge

  (format-spec "%>^-3L : %3p%% (%05t left)" my-battery-info)
    => "BAT :  73% (02:23 left)"

  (format-spec "%>^-3L : %3p%% (%05t left)"
               (cons (cons ?L "AC")
                     my-battery-info))
    => "AC  :  73% (02:23 left)"

> Even the subtle point of having the format specs in the order
> different from the alist is not evident unless you make a point of
> mentioning it (something that IMO should have been done earlier in the
> description).

Done.

How's the new attached version?

-- 
Basil

>From 096755e58259a0520e44787a00d891e3bf6d4048 Mon Sep 17 00:00:00 2001
From: "Basil L. Contovounesios" <[email-masked]>
Date: Thu, 28 May 2020 00:53:42 +0100
Subject: [PATCH] Improve format-spec documentation (bug#41571)

* doc/lispref/text.texi (Interpolated Strings): Move from here...
* doc/lispref/strings.texi (Custom Format Strings): ...to here,
renaming the node and clarifying the documentation.
(Formatting Strings): End node with sentence referring to the next
one.
* lisp/format-spec.el (format-spec): Clarify docstring.
---
 doc/lispref/strings.texi | 168 +++++++++++++++++++++++++++++++++++++++
 doc/lispref/text.texi    |  64 ---------------
 lisp/format-spec.el      |  49 +++++++-----
 3 files changed, 198 insertions(+), 83 deletions(-)

diff --git a/doc/lispref/strings.texi b/doc/lispref/strings.texi
index 70c3b3cf4b..9098774fe9 100644
--- a/doc/lispref/strings.texi
+++ b/doc/lispref/strings.texi
@@ -28,6 +28,7 @@ Strings and Characters
 * Text Comparison::           Comparing characters or strings.
 * String Conversion::         Converting to and from characters and strings.
 * Formatting Strings::        @code{format}: Emacs's analogue of @code{printf}.
+* Custom Format Strings::     Formatting custom @code{format} specifications.
 * Case Conversion::           Case conversion functions.
 * Case Tables::               Customizing case conversion.
 @end menu
@@ -1122,6 +1123,173 @@ Formatting Strings
 NaNs and can lose precision and type, and @samp{#x%x} and @samp{#o%o}
 can mishandle negative integers.  @xref{Input Functions}.
 
+The functions described in this section accept a fixed set of
+specification characters.  The next section describes a function
+@code{format-spec} which can accept custom specification characters,
+such as @samp{%a} or @samp{%z}.
+
+@node Custom Format Strings
+@section Custom Format Strings
+@cindex custom format string
+@cindex custom @samp{%}-sequence in format
+
+Sometimes it is useful to allow Lisp programs to control how certain
+text is generated via custom format control strings.  For example, a
+format string could control how to display someone's forename,
+surname, and email address.  Using the function @code{format}
+described in the previous section, the format string could be
+something like @w{@code{"%s %s <%s>"}}.  This approach quickly becomes
+impractical, however, as it can be unclear which specification
+character corresponds to which piece of information.
+
+A more convenient format string for such cases would be something like
+@w{@code{"%f %l <%e>"}}, where each specification character carries
+more semantic information and can easily be rearranged relative to
+other specification characters.  The function @code{format-spec}
+described in this section performs a similar function to
+@code{format}, except it operates on format control strings that use
+arbitrary specification characters.
+
+@defun format-spec template spec-alist &optional only-present
+This function returns a string produced from the format string
+@var{template} according to conversions specified in @var{spec-alist},
+which is an alist (@pxref{Association Lists}) of the form
+@w{@code{(@var{letter} . @var{replacement})}}.  Each specification
+@code{%@var{letter}} in @var{template} will be replaced by
+@var{replacement} when formatting the resulting string.
+
+The characters in @var{template}, other than the format
+specifications, are copied directly into the output, including their
+text properties, if any.  Any text properties of the format
+specifications are copied to their replacements.
+
+Using an alist to specify conversions gives rise to some useful
+properties:
+
+@itemize @bullet
+@item
+If @var{spec-alist} contains more unique @var{letter} keys than there
+are unique specification characters in @var{template}, the unused keys
+are simply ignored.
+@item
+If @var{spec-alist} contains more than one association with the same
+@var{letter}, the closest one to the start of the list is used.
+@item
+If @var{template} contains the same specification character more than
+once, then the same @var{replacement} found in @var{spec-alist} is
+used as a basis for all of that character's substitutions.
+@item
+The order of specifications in @var{template} need not correspond to
+the order of associations in @var{spec-alist}.
+@end itemize
+
+The optional argument @var{only-present} indicates how to handle
+specification characters in @var{template} that are not found in
+@var{spec-alist}.  If it is @code{nil} or omitted, the function
+signals an error.  Otherwise, those format specifications and any
+occurrences of @samp{%%} in @var{template} are left verbatim in the
+output, including their text properties, if any.
+@end defun
+
+The syntax of format specifications accepted by @code{format-spec} is
+similar, but not identical, to that accepted by @code{format}.  In
+both cases, a format specification is a sequence of characters
+beginning with @samp{%} and ending with an alphabetic letter such as
+@samp{s}.  The only specification that does not end in a letter is
+@samp{%%}, which is replaced with a single @samp{%} in the output.
+
+Unlike @code{format}, which assigns specific meanings to a fixed set
+of specification characters, @code{format-spec} accepts arbitrary
+specification characters and treats them all equally.  For example:
+
+@example
+@group
+(setq my-site-info
+      (list (cons ?s system-name)
+            (cons ?t (symbol-name system-type))
+            (cons ?c system-configuration)
+            (cons ?v emacs-version)
+            (cons ?e invocation-name)
+            (cons ?p (number-to-string (emacs-pid)))
+            (cons ?a user-mail-address)
+            (cons ?n user-full-name)))
+
+(format-spec "%e %v (%c)" my-site-info)
+     @result{} "emacs 27.1 (x86_64-pc-linux-gnu)"
+
+(format-spec "%n <%a>" my-site-info)
+     @result{} "Emacs Developers <[email-masked]>"
+@end group
+@end example
+
+A format specification can include any number of the following flag
+characters immediately after the @samp{%} to modify aspects of the
+substitution.
+
+@table @samp
+@item 0
+This flag causes any padding specified by the width to consist of
+@samp{0} characters instead of spaces.
+
+@item -
+This flag causes any padding specified by the width to be inserted on
+the right rather than the left.
+
+@item <
+This flag causes the substitution to be truncated on the left to the
+given width, if specified.
+
+@item >
+This flag causes the substitution to be truncated on the right to the
+given width, if specified.
+
+@item ^
+This flag converts the substituted text to upper case (@pxref{Case
+Conversion}).
+
+@item _
+This flag converts the substituted text to lower case (@pxref{Case
+Conversion}).
+@end table
+
+The result of using contradictory flags (for instance, both upper and
+lower case) is undefined.
+
+As is the case with @code{format}, a format specification can include
+a width, which is a decimal number that appears after any flags.  If a
+substitution contains fewer characters than its specified width, it is
+padded on the left:
+
+@example
+@group
+(format-spec "%8a is padded on the left with spaces"
+             '((?a . "alpha")))
+     @result{} "   alpha is padded on the left with spaces"
+@end group
+@end example
+
+Here is a more complicated example that combines several
+aforementioned features:
+
+@example
+@group
+(setq my-battery-info
+      (list (cons ?p "73")      ; Percentage
+            (cons ?L "Battery") ; Status
+            (cons ?t "2:23")    ; Remaining time
+            (cons ?c "24330")   ; Capacity
+            (cons ?r "10.6")))  ; Rate of discharge
+
+(format-spec "%>^-3L : %3p%% (%05t left)" my-battery-info)
+     @result{} "BAT :  73% (02:23 left)"
+
+(format-spec "%>^-3L : %3p%% (%05t left)"
+             (cons (cons ?L "AC")
+                   my-battery-info))
+     @result{} "AC  :  73% (02:23 left)"
+@end group
+@end example
+
 @node Case Conversion
 @section Case Conversion in Lisp
 @cindex upper case
diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi
index de436fa9e6..a14867e1d1 100644
--- a/doc/lispref/text.texi
+++ b/doc/lispref/text.texi
@@ -58,7 +58,6 @@ Text
                        of another buffer.
 * Decompression::    Dealing with compressed data.
 * Base 64::          Conversion to or from base 64 encoding.
-* Interpolated Strings:: Formatting Customizable Strings.
 * Checksum/Hash::    Computing cryptographic hashes.
 * GnuTLS Cryptography:: Cryptographic algorithms imported from GnuTLS.
 * Parsing HTML/XML:: Parsing HTML and XML.
@@ -4662,69 +4661,6 @@ Base 64
 is optional, and the URL variant of base 64 encoding is used.
 @end defun
 
-
-@node Interpolated Strings
-@section Formatting Customizable Strings
-
-It is, in some circumstances, useful to present users with a string to
-be customized that can then be expanded programmatically.  For
-instance, @code{erc-header-line-format} is @code{"%n on %t (%m,%l)
-%o"}, and each of those characters after the percent signs are
-expanded when the header line is computed.  To do this, the
-@code{format-spec} function is used:
-
-@defun format-spec format specification &optional only-present
-@var{format} is the format specification string as in the example
-above.  @var{specification} is an alist that has elements where the
-@code{car} is a character and the @code{cdr} is the substitution.
-
-If @var{only-present} is @code{nil}, errors will be signaled if a
-format character has been used that's not present in
-@var{specification}.  If it's non-@code{nil}, that format
-specification is left verbatim in the result.
-@end defun
-
-Here's a trivial example:
-
-@example
-(format-spec "su - %u %l"
-             `((?u . ,(user-login-name))
-               (?l . "ls")))
-     @result{} "su - foo ls"
-@end example
-
-In addition to allowing padding/limiting to a certain length, the
-following modifiers can be used:
-
-@table @asis
-@item @samp{0}
-Pad with zeros instead of the default spaces.
-
-@item @samp{-}
-Pad to the right.
-
-@item @samp{^}
-Use upper case.
-
-@item @samp{_}
-Use lower case.
-
-@item @samp{<}
-If the length needs to be limited, remove characters from the left.
-
-@item @samp{>}
-Same as previous, but remove characters from the right.
-@end table
-
-If contradictory modifiers are used (for instance, both upper and
-lower case), then what happens is undefined.
-
-As an example, @samp{"%<010b"} means ``insert the @samp{b} expansion,
-but pad with leading zeros if it's less than ten characters, and if
-it's more than ten characters, shorten by removing characters from the
-left.''
-
-
 @node Checksum/Hash
 @section Checksum/Hash
 @cindex MD5 checksum
diff --git a/lisp/format-spec.el b/lisp/format-spec.el
index f418cea425..9278bd74c4 100644
--- a/lisp/format-spec.el
+++ b/lisp/format-spec.el
@@ -29,35 +29,46 @@
 
 (defun format-spec (format specification &optional only-present)
   "Return a string based on FORMAT and SPECIFICATION.
-FORMAT is a string containing `format'-like specs like \"su - %u %k\",
-while SPECIFICATION is an alist mapping from format spec characters
-to values.
+FORMAT is a string containing `format'-like specs like \"su - %u %k\".
+SPECIFICATION is an alist mapping format specification characters
+to their substitutions.
 
 For instance:
 
   (format-spec \"su - %u %l\"
-               `((?u . ,(user-login-name))
+               \\=`((?u . ,(user-login-name))
                  (?l . \"ls\")))
 
-Each format spec can have modifiers, where \"%<010b\" means \"if
-the expansion is shorter than ten characters, zero-pad it, and if
-it's longer, chop off characters from the left side\".
+Each %-spec may contain optional flag and width modifiers, as
+follows:
 
-The following modifiers are allowed:
+  %<flags><width>character
 
-* 0: Use zero-padding.
-* -: Pad to the right.
-* ^: Upper-case the expansion.
-* _: Lower-case the expansion.
-* <: Limit the length by removing chars from the left.
-* >: Limit the length by removing chars from the right.
+The following flags are allowed:
 
-Any text properties on a %-spec itself are propagated to the text
-that it generates.
+* 0: Pad to the width, if given, with zeros instead of spaces.
+* -: Pad to the width, if given, on the right instead of the left.
+* <: Truncate to the width, if given, on the left.
+* >: Truncate to the width, if given, on the right.
+* ^: Convert to upper case.
+* _: Convert to lower case.
 
-If ONLY-PRESENT, format spec characters not present in
-SPECIFICATION are ignored, and the \"%\" characters are left
-where they are, including \"%%\" strings."
+The width modifier behaves like the corresponding one in `format'
+when applied to %s.
+
+For example, \"%<010b\" means \"substitute into the output the
+value associated with ?b in SPECIFICATION, either padding it with
+leading zeros or truncating leading characters until it's ten
+characters wide\".
+
+Any text properties of FORMAT are copied to the result, with any
+text properties of a %-spec itself copied to its substitution.
+
+ONLY-PRESENT indicates how to handle %-spec characters not
+present in SPECIFICATION.  If it is nil or omitted, emit an
+error; otherwise leave those %-specs and any occurrences of
+\"%%\" in FORMAT verbatim in the result, including their text
+properties, if any."
   (with-temp-buffer
     (insert format)
     (goto-char (point-min))
-- 
2.26.2