GNU bug report logs - #6330
Feature request: mktemp creates named pipes

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 6330 in the body.
You can then email your comments to 6330 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sebastien Andre <wayana <at> users.sourceforge.net>
To: bug-coreutils <at> gnu.org
Subject: Feature request: mktemp creates named pipes
Date: Wed, 2 Jun 2010 12:08:34 +0200

[Message part 1 (text/plain, inline)]

Hello guys!

When needing a temporary named pipe in shell scripts, I've often been
writing the following function:

mktempfifo() {
    local path=$(mktemp -t)
    rm "$path"
    mkfifo -m 0600 "$path"
    echo "$path"
}

I was wondering if anybody would be interested in having an option -p --pipe
(or -f --fifo since -p is deprecated)
to create temporary named pipes?

Example:
   $ file $(mktemp -tp)
   /tmp/tmp.24665457: fifo (named pipe)


PS: I can try to provide a patch if my bug is accepted

Thanks,
Sebastien

[Message part 2 (text/html, inline)]

Message #8 received at 6330 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: Sebastien Andre <wayana <at> users.sourceforge.net>
Cc: 6330 <at> debbugs.gnu.org
Subject: Re: bug#6330: Feature request: mktemp creates named pipes
Date: Wed, 2 Jun 2010 10:13:27 -0600

Sebastien Andre wrote:
> When needing a temporary named pipe in shell scripts, I've often been
> writing the following function:
> 
> mktempfifo() {
>     local path=$(mktemp -t)
>     rm "$path"
>     mkfifo -m 0600 "$path"
>     echo "$path"
> }

Ew...  That isn't safe.  There is a time gap between when you remove
the temporary file and create the pipe.  That isn't good.

> I was wondering if anybody would be interested in having an option -p --pipe
> (or -f --fifo since -p is deprecated)
> to create temporary named pipes?
> 
> Example:
>    $ file $(mktemp -tp)
>    /tmp/tmp.24665457: fifo (named pipe)

The traditional way to deal with the entire range of issues such as
this (creating files with different suffixes, whatever) is to have
mktemp create a directory and then create your special files within
the directory.  It is fully safe that way.  Because the directory is
uniquely named the file within can have a fixed name.  No race
condition exists.

This is off of the top of my head, untested, but you might try
something like this example.  This is a do-nothing but with enough to
hopefully show you the technique.

  #!/bin/sh
  unset tmpdir
  trap 'cd / ; rm -rf "$tmpdir"' EXIT
  tmpdir=$(mktemp -d) || exit 1
  tmppipe="$tmpdir/pipe"
  mknod "$tmppipe" p
  ls -log "$tmppipe"
  exit 0

Bob

Message #11 received at 6330 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Sebastien Andre <wayana <at> users.sourceforge.net>
Cc: 6330 <at> debbugs.gnu.org
Subject: Re: bug#6330: Feature request: mktemp creates named pipes
Date: Wed, 02 Jun 2010 10:14:41 -0600

[Message part 1 (text/plain, inline)]

On 06/02/2010 04:08 AM, Sebastien Andre wrote:
> Hello guys!
> 
> When needing a temporary named pipe in shell scripts, I've often been
> writing the following function:
> 
> mktempfifo() {
>     local path=$(mktemp -t)
>     rm "$path"
>     mkfifo -m 0600 "$path"
>     echo "$path"
> }

First off, thanks for the suggestion.

What's wrong with the example given in the manual (info coreutils
mktemp), as adjusted to your usage pattern:

mktempfifo() {
    local dir
    dir=$(mktemp -t -d) || return 1
    mkfifo -m 0600 "$dir/fifo" || return 1
    echo "$dir/fifo"
}

other than the fact that you have to remember to also remove the
temporary directory?  And if you need to create more than one secure
temporary object, it becomes a much easier paradigm to create a single
secure directory in which to place all the other objects, and use a
simple 'rm -rf "$tmpdir"' in your cleanup paths.  That's how coreutils
'make check' works - every test is run inside its own temporary
directory, and within the body of the test, there is no need to worry
about name collisions from external processes.

> 
> I was wondering if anybody would be interested in having an option -p --pipe
> (or -f --fifo since -p is deprecated)
> to create temporary named pipes?

You are correct that a short option -p cannot be used.  And I'm
reluctant to burn -f unless there is another implementation that already
does it.  But you have a point that adding a long option --fifo may make
sense.  However, I thought about that same point the last time I touched
mktemp(1), and did not implement it at that time because I felt that a
temporary directory was enough.  But I can be swayed if you can present
good arguments why the addition is better than using existing tools.

> 
> PS: I can try to provide a patch if my bug is accepted

Patches speak volumes, although by the time you add the code and the
documentation, your contribution would probably be non-trivial and
require copyright assignment to the FSF.  Let us know if you'd like to
start that process.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #14 received at 6330 <at> debbugs.gnu.org (full text, mbox):

From: Sebastien Andre <wayana <at> users.sourceforge.net>
To: Bob Proulx <bob <at> proulx.com>
Cc: 6330 <at> debbugs.gnu.org
Subject: Re: bug#6330: Feature request: mktemp creates named pipes
Date: Thu, 3 Jun 2010 12:04:34 +0200

[Message part 1 (text/plain, inline)]

On Wed, Jun 2, 2010 at 6:13 PM, Bob Proulx <bob <at> proulx.com> wrote:

> Sebastien Andre wrote:
> > When needing a temporary named pipe in shell scripts, I've often been
> > writing the following function:
> >
> > mktempfifo() {
> >     local path=$(mktemp -t)
> >     rm "$path"
> >     mkfifo -m 0600 "$path"
> >     echo "$path"
> > }
>
> Ew...  That isn't safe.  There is a time gap between when you remove
> the temporary file and create the pipe.  That isn't good.
>
> > I was wondering if anybody would be interested in having an option -p
> --pipe
> > (or -f --fifo since -p is deprecated)
> > to create temporary named pipes?
> >
> > Example:
> >    $ file $(mktemp -tp)
> >    /tmp/tmp.24665457: fifo (named pipe)
>
> The traditional way to deal with the entire range of issues such as
> this (creating files with different suffixes, whatever) is to have
> mktemp create a directory and then create your special files within
> the directory.  It is fully safe that way.  Because the directory is
> uniquely named the file within can have a fixed name.  No race
> condition exists.
>
> This is off of the top of my head, untested, but you might try
> something like this example.  This is a do-nothing but with enough to
> hopefully show you the technique.
>
>  #!/bin/sh
>  unset tmpdir
>  trap 'cd / ; rm -rf "$tmpdir"' EXIT
>  tmpdir=$(mktemp -d) || exit 1
>  tmppipe="$tmpdir/pipe"
>  mknod "$tmppipe" p
>  ls -log "$tmppipe"
>  exit 0
>
> Bob
>

Thank you Bob for the suggestion

I agree my example was bad, my goal was to show the hassle in creating a
temporary named fifo in a shell script.
The traditional way, with the directory safely created and removed on exit,
is better but still not as convenient as a "mktemp -f" would be.

Sebastien

[Message part 2 (text/html, inline)]

Message #17 received at 6330 <at> debbugs.gnu.org (full text, mbox):

From: Sebastien Andre <wayana <at> users.sourceforge.net>
To: Eric Blake <eblake <at> redhat.com>
Cc: 6330 <at> debbugs.gnu.org
Subject: Re: bug#6330: Feature request: mktemp creates named pipes
Date: Thu, 3 Jun 2010 21:09:53 +0200

[Message part 1 (text/plain, inline)]

On Wed, Jun 2, 2010 at 6:14 PM, Eric Blake <eblake <at> redhat.com> wrote:

> On 06/02/2010 04:08 AM, Sebastien Andre wrote:
> > Hello guys!
> >
> > When needing a temporary named pipe in shell scripts, I've often been
> > writing the following function:
> >
> > mktempfifo() {
> >     local path=$(mktemp -t)
> >     rm "$path"
> >     mkfifo -m 0600 "$path"
> >     echo "$path"
> > }
>
> First off, thanks for the suggestion.
>
> What's wrong with the example given in the manual (info coreutils
> mktemp), as adjusted to your usage pattern:
>
> mktempfifo() {
>    local dir
>    dir=$(mktemp -t -d) || return 1
>    mkfifo -m 0600 "$dir/fifo" || return 1
>    echo "$dir/fifo"
> }
>
> other than the fact that you have to remember to also remove the
> temporary directory?  And if you need to create more than one secure
> temporary object, it becomes a much easier paradigm to create a single
> secure directory in which to place all the other objects, and use a
> simple 'rm -rf "$tmpdir"' in your cleanup paths.  That's how coreutils
> 'make check' works - every test is run inside its own temporary
> directory, and within the body of the test, there is no need to worry
> about name collisions from external processes.
>
> >
> > I was wondering if anybody would be interested in having an option -p
> --pipe
> > (or -f --fifo since -p is deprecated)
> > to create temporary named pipes?
>
> You are correct that a short option -p cannot be used.  And I'm
> reluctant to burn -f unless there is another implementation that already
> does it.  But you have a point that adding a long option --fifo may make
> sense.  However, I thought about that same point the last time I touched
> mktemp(1), and did not implement it at that time because I felt that a
> temporary directory was enough.  But I can be swayed if you can present
> good arguments why the addition is better than using existing tools.
>
>
Thank you Eric for your answer,

I see two reasons why the addition of a --fifo option is better than using
existing tools:

    * Creating a temporary directory to finally create a pipe just because
it is safe this way is kind of a trick. For the clarity of scripts, it would
be better having mktemp to ensure the uniqueness of a fifo, even if it's
created in /

    * mktemp is not only a tool to create unique files, it's also a name
generator. The example given in the manual works for one or two fifos, but
if the number of fifos is unknown there is no choice but implementing
something to generate names, which is another potential source of bugs.


Below is an example of two functions using temp fifos, with and without the
-f option (code not tested):


# merge the output of every given command, line by line (fifos are not
removed for clarity)

merge_output_mktempfifo() {
    local fifos=()
    if [ $# -gt 0 ]; then
        for cmd in "$@"; do
            local fifo=$(mktemp -tf) || return 1
            ( $cmd > $fifo ) &
            fifos=( ${fifos[@]} $fifo )
        done
        paste "${fifos[@]}"
       wait
    fi
}

merge_output_traditional() {
    local tmpdir=$(mktemp -td) || return 1
    local i=0
    if [ $# -gt 0 ]; then
        for cmd in "$@"; do
            i=$(expr $i + 1)
            mkfifo -m 0600 "$tmpdir/fifo$i"
            ( $cmd > "$tmpdir/fifo$i" ) &
        done
        paste $tmpdir/fifo*
        wait
    fi
}

The merge_output_traditional() function is more difficult to understand than
merge_output_mktempfifo() because of the use
of $i to ensure unicity of fifo names. If for any reason $tmpdir must be
shared with other processes which use the exact same function it will
lead to a mess, that's why I prefer merge_output_mktempfifo().

I think the "traditional way" to deal with named fifos is good and efficient
but tricky. The -f option would be more straightforward.

>
> > PS: I can try to provide a patch if my bug is accepted
>
> Patches speak volumes, although by the time you add the code and the
> documentation, your contribution would probably be non-trivial and
> require copyright assignment to the FSF.  Let us know if you'd like to
> start that process.
>
>
Yes, I'm interested in starting that process.




>  --
> Eric Blake   eblake <at> redhat.com    +1-801-349-2682
> Libvirt virtualization library http://libvirt.org
>
>

[Message part 2 (text/html, inline)]

Message #20 received at 6330 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Sebastien Andre <wayana <at> users.sourceforge.net>
Cc: Eric Blake <eblake <at> redhat.com>, 6330 <at> debbugs.gnu.org
Subject: Re: bug#6330: Feature request: mktemp creates named pipes
Date: Sat, 05 Jun 2010 09:25:25 +0200

Sebastien Andre wrote:
...
> I see two reasons why the addition of a --fifo option is better than using
> existing tools:
>
>     * Creating a temporary directory to finally create a pipe just because
> it is safe this way is kind of a trick. For the clarity of scripts, it would
> be better having mktemp to ensure the uniqueness of a fifo, even if it's
> created in /

"because it is safe" is often a good reason to learn and use a new idiom.
When you first encounter such an idiom, it does indeed look like a "trick".

>     * mktemp is not only a tool to create unique files, it's also a name
> generator. The example given in the manual works for one or two fifos, but
> if the number of fifos is unknown there is no choice but implementing
> something to generate names, which is another potential source of bugs.

Any code addition is an opportunity to introduce bugs ;-)

What if someone wants a uniquely-named symlink?
New option?  ... or a shared memory object?  Add another?
Wouldn't it be better to use the tried and true (safe) idiom
that will work with all versions of mktemp?  Then
your script will work also on systems that use the
version of mktemp that predated the one in coreutils.

Message #23 received at 6330 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Sebastien Andre <wayana <at> users.sourceforge.net>
Cc: 6330 <at> debbugs.gnu.org
Subject: Re: bug#6330: Feature request: mktemp creates named pipes
Date: Mon, 07 Jun 2010 18:41:44 -0600

[Message part 1 (text/plain, inline)]

On 06/03/2010 01:09 PM, Sebastien Andre wrote:
> I see two reasons why the addition of a --fifo option is better than using
> existing tools:
> 
>     * Creating a temporary directory to finally create a pipe just because
> it is safe this way is kind of a trick. For the clarity of scripts, it would
> be better having mktemp to ensure the uniqueness of a fifo, even if it's
> created in /

Why do you need to create fifos directly in /?  Unless you can prove
that it is needed there, then what's wrong with creating it elsewhere?
Maybe the real feature we should be asking for is a way to ensure that
'mv /path1/to/fifo /path2/to/fifo' is atomic, and fails if
/path2/to/fifo is a dangling symlink rather than accidentally creating
the fifo at an unintended location?

> 
>     * mktemp is not only a tool to create unique files, it's also a name
> generator. The example given in the manual works for one or two fifos, but
> if the number of fifos is unknown there is no choice but implementing
> something to generate names, which is another potential source of bugs.

If you are worried about creating lots of fifos, and want to avoid
collisions, you can still use the existing semantics of unpatched mktemp
to get that: merely make a secure directory with mktemp -d, then use
mkfifo "$(mktemp -u -p "$dir")".  That is, once you have a safe
directory, then mktemp -u within that directory is no longer a security
hole, and serves as a great way to create unique names without also
creating files.

> of $i to ensure unicity of fifo names. If for any reason $tmpdir must be
> shared with other processes which use the exact same function it will
> lead to a mess, that's why I prefer merge_output_mktempfifo().

As long as the multiple processes are closely cooperating, and all use
the same secure temporary directory, then mktemp -u is an appropriate
way for each of those processes to name their fifo.

>> Patches speak volumes, although by the time you add the code and the
>> documentation, your contribution would probably be non-trivial and
>> require copyright assignment to the FSF.  Let us know if you'd like to
>> start that process.
>>
>>
> Yes, I'm interested in starting that process.

I'm sending you some details off-list.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #26 received at 6330 <at> debbugs.gnu.org (full text, mbox):

From: Sebastien Andre <wayana <at> users.sourceforge.net>
To: Jim Meyering <jim <at> meyering.net>
Cc: Eric Blake <eblake <at> redhat.com>, 6330 <at> debbugs.gnu.org
Subject: Re: bug#6330: Feature request: mktemp creates named pipes
Date: Tue, 8 Jun 2010 08:36:53 +0200

[Message part 1 (text/plain, inline)]

On Sat, Jun 5, 2010 at 9:25 AM, Jim Meyering <jim <at> meyering.net> wrote:

> Sebastien Andre wrote:
> ...
> > I see two reasons why the addition of a --fifo option is better than
> using
> > existing tools:
> >
> >     * Creating a temporary directory to finally create a pipe just
> because
> > it is safe this way is kind of a trick. For the clarity of scripts, it
> would
> > be better having mktemp to ensure the uniqueness of a fifo, even if it's
> > created in /
>
> "because it is safe" is often a good reason to learn and use a new idiom.
> When you first encounter such an idiom, it does indeed look like a "trick".
>
>
Well, after all an idiom might just be a "popular trick"



> >     * mktemp is not only a tool to create unique files, it's also a name
> > generator. The example given in the manual works for one or two fifos,
> but
> > if the number of fifos is unknown there is no choice but implementing
> > something to generate names, which is another potential source of bugs.
>
> Any code addition is an opportunity to introduce bugs ;-)
>
> What if someone wants a uniquely-named symlink?
> New option?  ... or a shared memory object?  Add another?
> Wouldn't it be better to use the tried and true (safe) idiom
> that will work with all versions of mktemp?  Then
> your script will work also on systems that use the
> version of mktemp that predated the one in coreutils.
>

Thank you for broadening the question.

You made me realize that what I really want is to use the tempnam(3)
function
in shell scripts. But having a tempnam command in a shell script would be
unsafe unless
something is created atomically when invoked. If mktemp(1) is not the right
place for this, how about adding a -t/--template option to the mkfifo(1) ?

Wanting to get a unique name in a given directory according to a template
doesn't
sound too specific to me, maybe some other people need it?

[Message part 2 (text/html, inline)]

Message #29 received at 6330 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Sebastien Andre <wayana <at> users.sourceforge.net>
Cc: Eric Blake <eblake <at> redhat.com>, 6330 <at> debbugs.gnu.org
Subject: Re: bug#6330: Feature request: mktemp creates named pipes
Date: Tue, 08 Jun 2010 11:07:24 +0200

Sebastien Andre wrote:

> On Sat, Jun 5, 2010 at 9:25 AM, Jim Meyering <jim <at> meyering.net> wrote:
...
>     "because it is safe" is often a good reason to learn and use a new idiom.
>     When you first encounter such an idiom, it does indeed look like a "trick".
>
> Well, after all an idiom might just be a "popular trick"

"Effective technique" is more accurate.
This is the "unix way".

>     >     * mktemp is not only a tool to create unique files, it's also a name
>     > generator. The example given in the manual works for one or two fifos,
>     but
>     > if the number of fifos is unknown there is no choice but implementing
>     > something to generate names, which is another potential source of bugs.
>
>     Any code addition is an opportunity to introduce bugs ;-)
>
>     What if someone wants a uniquely-named symlink?
>     New option?  ... or a shared memory object?  Add another?
>     Wouldn't it be better to use the tried and true (safe) idiom
>     that will work with all versions of mktemp?  Then
>     your script will work also on systems that use the
>     version of mktemp that predated the one in coreutils.
>
> Thank you for broadening the question.
>
> You made me realize that what I really want is to use the tempnam(3) function
> in shell scripts. But having a tempnam command in a shell script would be
> unsafe unless
> something is created atomically when invoked. If mktemp(1) is not the right
> place for this, how about adding a -t/--template option to the mkfifo(1) ?

And to mknod
and to ln?

> Wanting to get a unique name in a given directory according to a template
> doesn't sound too specific to me, maybe some other people need it?

If that given directory has only one writer, then using mktemp -u is fine.

Other people use mktemp -d, and then can create names however
they like (and safely) in the just-created directory.

The point that someone might want to create an object of a type other
than "fifo" (e.g., symlink, hard link, block dev., etc.) was to make
you think about adding a more generic option, but then you'd have to
consider how to specify the target of a symlink and major,minor numbers
for a block device.

I was hoping you would then conclude that this would lead to mktemp
subsuming most of the functionality of tools like ln, mkfifo and mknod,
and thus rethink your premise that it would be better to change mktemp
than to use it the way everyone else does.

Message #32 received at 6330 <at> debbugs.gnu.org (full text, mbox):

From: Sebastien Andre <wayana <at> users.sourceforge.net>
To: Eric Blake <eblake <at> redhat.com>
Cc: 6330 <at> debbugs.gnu.org
Subject: Re: bug#6330: Feature request: mktemp creates named pipes
Date: Mon, 14 Jun 2010 20:39:02 +0200

[Message part 1 (text/plain, inline)]

On Tue, Jun 8, 2010 at 2:41 AM, Eric Blake <eblake <at> redhat.com> wrote:

> On 06/03/2010 01:09 PM, Sebastien Andre wrote:
> > I see two reasons why the addition of a --fifo option is better than
> using
> > existing tools:
> >
> >     * Creating a temporary directory to finally create a pipe just
> because
> > it is safe this way is kind of a trick. For the clarity of scripts, it
> would
> > be better having mktemp to ensure the uniqueness of a fifo, even if it's
> > created in /
>
> Why do you need to create fifos directly in /?  Unless you can prove
> that it is needed there, then what's wrong with creating it elsewhere?
> Maybe the real feature we should be asking for is a way to ensure that
> 'mv /path1/to/fifo /path2/to/fifo' is atomic, and fails if
> /path2/to/fifo is a dangling symlink rather than accidentally creating
> the fifo at an unintended location?
>
> >
> >     * mktemp is not only a tool to create unique files, it's also a name
> > generator. The example given in the manual works for one or two fifos,
> but
> > if the number of fifos is unknown there is no choice but implementing
> > something to generate names, which is another potential source of bugs.
>
> If you are worried about creating lots of fifos, and want to avoid
> collisions, you can still use the existing semantics of unpatched mktemp
> to get that: merely make a secure directory with mktemp -d, then use
> mkfifo "$(mktemp -u -p "$dir")".  That is, once you have a safe
> directory, then mktemp -u within that directory is no longer a security
> hole, and serves as a great way to create unique names without also
> creating files.
>
>
I don't really need to create temp fifos in /, just wanted it to be as
convenient as
creating temp files. Even if the 'dir=$(mktemp -td)' followed by 'mkfifo
"$(mktemp -u -p "$dir")"'
is not as straightforward as I would expect, it fits my need ~


> of $i to ensure unicity of fifo names. If for any reason $tmpdir must be
> > shared with other processes which use the exact same function it will
> > lead to a mess, that's why I prefer merge_output_mktempfifo().
>
> As long as the multiple processes are closely cooperating, and all use
> the same secure temporary directory, then mktemp -u is an appropriate
> way for each of those processes to name their fifo.
>
> >> Patches speak volumes, although by the time you add the code and the
> >> documentation, your contribution would probably be non-trivial and
> >> require copyright assignment to the FSF.  Let us know if you'd like to
> >> start that process.
> >>
> >>
> > Yes, I'm interested in starting that process.
>
> I'm sending you some details off-list.
>
> Thanks

> --
> Eric Blake   eblake <at> redhat.com    +1-801-349-2682
> Libvirt virtualization library http://libvirt.org
>
>

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.