Re: "here strings" and tmpfiles

2019-04-12 Thread konsolebox
On Fri, Apr 12, 2019 at 10:32 PM Chet Ramey wrote: > > On 4/11/19 3:08 PM, konsolebox wrote: > > >>> It has slightly inconvenient semantics, in that you can't open it more > >>> than once, and if you can't do that, you can't convert it from read-write > >>> to readonly. > >> > >> > >> Perhaps it c

Re: "here strings" and tmpfiles

2019-04-12 Thread Chet Ramey
On 4/11/19 3:08 PM, konsolebox wrote: >>> It has slightly inconvenient semantics, in that you can't open it more >>> than once, and if you can't do that, you can't convert it from read-write >>> to readonly. >> >> >> Perhaps it can be reopened via /dev/fd. > > Also file sealing maybe. The way it

Re: "here strings" and tmpfiles

2019-04-11 Thread Chet Ramey
On 4/11/19 3:15 AM, Robert Elz wrote: > (Substitute cat if you're that kind of weirdo!). We're really going to throw down right here? -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@c

Re: "here strings" and tmpfiles

2019-04-11 Thread Chet Ramey
On 4/11/19 12:12 AM, Jason A. Donenfeld wrote: > I keep forgetting things. The other thing I wanted to bring up is that > I suspect bash's actual implementation of temporary files is > problematic and might have some of the classic /tmp and TOCTOU style > attacks. It's a peripheral issue, since t

Re: "here strings" and tmpfiles

2019-04-11 Thread Chet Ramey
On 4/11/19 12:02 AM, Jason A. Donenfeld wrote: > Hi Chet, Hi. > I hope that can shed light on the motivation a bit. Pass got hit by > this a bit ago: > https://git.zx2c4.com/password-store/commit/?id=367efa5846492e1b0898aad8a2c26ce94163ba24 I note that the pipe-for-small-enough-heredocs works fo

Re: "here strings" and tmpfiles

2019-04-11 Thread konsolebox
On Thu, Apr 11, 2019 at 5:31 AM konsolebox wrote: > > On Thu, Apr 11, 2019, 4:45 AM Chet Ramey wrote: >> >> On 4/10/19 4:33 PM, konsolebox wrote: >> > On Wed, Apr 10, 2019 at 11:15 PM Chet Ramey wrote: >> >> If we're going to go off into hypotheticals and speculation, it would be >> >> nice if m

Re: "here strings" and tmpfiles

2019-04-11 Thread Chet Ramey
On 4/10/19 7:18 PM, Daniel Kahn Gillmor wrote: > On Wed 2019-04-10 16:16:44 -0400, Chet Ramey wrote: >> Is it just that people have not realized all along that most shells, >> certainly all historical shells, that implement here documents use temp >> files to do it? It's really only the ash-based s

Re: "here strings" and tmpfiles

2019-04-11 Thread Daniel Kahn Gillmor
On Thu 2019-04-11 10:04:02 +0200, Andreas Schwab wrote: > On Apr 10 2019, Daniel Kahn Gillmor wrote: > >> data written to the local filesystem can be discovered by someone >> analyzing the disk controller data path, or by someone with access to >> the underlying storage medium. > > Do you have swa

Re: "here strings" and tmpfiles

2019-04-11 Thread konsolebox
On Thu, Apr 11, 2019, 10:42 PM Andreas Kusalananda Kähäri < andreas.kah...@abc.se> wrote: > On Thu, Apr 11, 2019 at 09:01:50PM +0800, konsolebox wrote: > > On Thu, Apr 11, 2019, 4:04 PM Andreas Schwab wrote: > > > > > On Apr 10 2019, Daniel Kahn Gillmor wrote: > > > > > > > data written to the l

Re: "here strings" and tmpfiles

2019-04-11 Thread Andreas Kusalananda Kähäri
On Thu, Apr 11, 2019 at 09:01:50PM +0800, konsolebox wrote: > On Thu, Apr 11, 2019, 4:04 PM Andreas Schwab wrote: > > > On Apr 10 2019, Daniel Kahn Gillmor wrote: > > > > > data written to the local filesystem can be discovered by someone > > > analyzing the disk controller data path, or by some

Re: "here strings" and tmpfiles

2019-04-11 Thread konsolebox
On Thu, Apr 11, 2019 at 9:06 PM Greg Wooledge wrote: > So... yes. Because everyone in 2019 has a laptop and therefore has swap > enabled because it's used for hibernation. Sure captain. It was a joke. -- konsolebox

Re: "here strings" and tmpfiles

2019-04-11 Thread Greg Wooledge
On Thu, Apr 11, 2019 at 09:01:50PM +0800, konsolebox wrote: > On Thu, Apr 11, 2019, 4:04 PM Andreas Schwab wrote: > > Do you have swap enabled? > > It's 2019. So... yes. Because everyone in 2019 has a laptop and therefore has swap enabled because it's used for hibernation. That was what you me

Re: "here strings" and tmpfiles

2019-04-11 Thread konsolebox
On Thu, Apr 11, 2019, 4:04 PM Andreas Schwab wrote: > On Apr 10 2019, Daniel Kahn Gillmor wrote: > > > data written to the local filesystem can be discovered by someone > > analyzing the disk controller data path, or by someone with access to > > the underlying storage medium. > > Do you have sw

Re: "here strings" and tmpfiles

2019-04-11 Thread Greg Wooledge
On Thu, Apr 11, 2019 at 06:02:41AM +0200, Jason A. Donenfeld wrote: > what about internally treating "x < x"? Are these somehow not quite equivalent because x is in a subshell > in one but not the other, or something like that? cmd <<< string opens a temporary file for writing, dumps the string

Re: "here strings" and tmpfiles

2019-04-11 Thread Andreas Schwab
On Apr 10 2019, Daniel Kahn Gillmor wrote: > data written to the local filesystem can be discovered by someone > analyzing the disk controller data path, or by someone with access to > the underlying storage medium. Do you have swap enabled? Andreas. -- Andreas Schwab, SUSE Labs, sch...@suse.

Re: "here strings" and tmpfiles

2019-04-11 Thread Robert Elz
Date:Thu, 11 Apr 2019 06:02:41 +0200 From:"Jason A. Donenfeld" Message-ID: | Now, it might be the case that bash really isn't the | right tool for that kind of thing, and I shouldn't use bash for tasks | with security requirements as such. But I sort of love

Re: "here strings" and tmpfiles

2019-04-10 Thread Jason A. Donenfeld
I keep forgetting things. The other thing I wanted to bring up is that I suspect bash's actual implementation of temporary files is problematic and might have some of the classic /tmp and TOCTOU style attacks. The current implementation is three-fold via ifdefs: char * sh_mktmpname (nameroot, flag

Re: "here strings" and tmpfiles

2019-04-10 Thread Jason A. Donenfeld
On Thu, Apr 11, 2019 at 6:02 AM Jason A. Donenfeld wrote: > curious about is - what about internally treating "x <

Re: "here strings" and tmpfiles

2019-04-10 Thread Jason A. Donenfeld
Hi Chet, On Wed, Apr 10, 2019 at 3:07 PM Chet Ramey wrote: > This is unnecessary hyperbole. The existing file-based mechanism works > just fine. We're talking about what's essentially an optimization. > [...] > This doesn't make any sense. > [...] > There isn't an "insecure path." I'm a bit late

Re: "here strings" and tmpfiles

2019-04-10 Thread Daniel Kahn Gillmor
On Wed 2019-04-10 16:16:44 -0400, Chet Ramey wrote: > Is it just that people have not realized all along that most shells, > certainly all historical shells, that implement here documents use temp > files to do it? It's really only the ash-based shells (not an insignificant > portion of the shells

Re: "here strings" and tmpfiles

2019-04-10 Thread konsolebox
On Thu, Apr 11, 2019, 4:45 AM Chet Ramey wrote: > On 4/10/19 4:33 PM, konsolebox wrote: > > On Wed, Apr 10, 2019 at 11:15 PM Chet Ramey wrote: > >> If we're going to go off into hypotheticals and speculation, it would be > >> nice if memfd_create were available universally. > > > > I see many pa

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/10/19 4:33 PM, konsolebox wrote: > On Wed, Apr 10, 2019 at 11:15 PM Chet Ramey wrote: >> If we're going to go off into hypotheticals and speculation, it would be >> nice if memfd_create were available universally. > > I see many parts in lib/* that adapts to available system features > like

Re: "here strings" and tmpfiles

2019-04-10 Thread Peter & Kelly Passchier
On 10/4/2019 09:04, Greg Wooledge wrote: > On Wed, Apr 10, 2019 at 11:59:19AM -0400, Daniel Kahn Gillmor wrote: >> If we look at the problem from the perspective of the risk of >> herestring/heredoc content leaking to non-ephemeral storage, > > The content is already in the damned SHELL SCRIPT. >

Re: "here strings" and tmpfiles

2019-04-10 Thread konsolebox
On Wed, Apr 10, 2019 at 11:15 PM Chet Ramey wrote: > If we're going to go off into hypotheticals and speculation, it would be > nice if memfd_create were available universally. I see many parts in lib/* that adapts to available system features like mmap and MAP_ANONYMOUS. I don't see why memfd_c

Re: "here strings" and tmpfiles

2019-04-10 Thread konsolebox
On Tue, Apr 9, 2019 at 10:01 PM Jason A. Donenfeld wrote: > A real solution for this issue involves getting rid of the temporary file > all together. Since we're talking about a bash string, it's already in > memory. Why not just fork() if the write() will block? A simple way would be > to always

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/10/19 11:59 AM, Daniel Kahn Gillmor wrote: > On Wed 2019-04-10 09:07:27 -0400, Chet Ramey wrote: > I think we all agree that avoiding the filesystem where possible is > likely to be an optimization and improvement (it means the heredoc will > work in some circumstances where it didn't work be

Re: "here strings" and tmpfiles

2019-04-10 Thread Greg Wooledge
On Wed, Apr 10, 2019 at 11:59:19AM -0400, Daniel Kahn Gillmor wrote: > If we look at the problem from the perspective of the risk of > herestring/heredoc content leaking to non-ephemeral storage, The content is already in the damned SHELL SCRIPT. How much more "non-ephemeral" can it get?

Re: "here strings" and tmpfiles

2019-04-10 Thread Daniel Kahn Gillmor
On Wed 2019-04-10 09:07:27 -0400, Chet Ramey wrote: > On 4/9/19 2:56 AM, Jason A. Donenfeld wrote: >> Since originally raising this issue with dkg (leading to this email >> thread), I've only followed along from a bit of a distance. But it does >> look like there's been some good progress: there's

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/9/19 12:19 AM, Robert Elz wrote: > Date:Mon, 08 Apr 2019 17:04:41 -0700 > From:L A Walsh > Message-ID: <5cabe199.9030...@tlinx.org> > > | On 4/8/2019 7:10 AM, Chet Ramey wrote: > > | > Pipes are objectively not the same as files. They > | > > | > 1. Do n

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/9/19 7:48 PM, L A Walsh wrote: > > I am aware of that, however, if a pipe implementation > *stops* on reaching a full condition from some 'tmp-storage-space' > and awaits for space to become available, a similar dynamic would > apply. That's all. Is there a pipe implementation ou

Re: "here strings" and tmpfiles

2019-04-10 Thread konsolebox
On Wed, Apr 10, 2019, 1:09 AM Eli Schwartz wrote: > That being said, it seems like a rather odd place to configure and use a > heavyweight shell merely to allow third parties to include > downstream-specific bashisms. I think there is a great deal of wisdom in > the fact that the referenced issue

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/9/19 10:07 AM, konsolebox wrote: > Perhaps bash can also look at /dev/shm. It's a common tmpfs, but I > haven't checked if it's standard and what utility mounts it. I don't > really use it. Another non-portable feature. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/9/19 2:56 AM, Jason A. Donenfeld wrote: > Since originally raising this issue with dkg (leading to this email > thread), I've only followed along from a bit of a distance. But it does > look like there's been some good progress: there's now a commit that > fills the pipe up to the OS's maximum

Re: "here strings" and tmpfiles

2019-04-10 Thread Chet Ramey
On 4/9/19 1:03 AM, pepa65 wrote: > I think Linda's main drive is to seek improvement in how bash works. Now > that lack of memory is in no way a constraint for the vast majority of > situations where bash is commonly used, it would be great if that memory > could be used instead of writing to a fi

Re: "here strings" and tmpfiles

2019-04-10 Thread Greg Wooledge
On Tue, Apr 09, 2019 at 04:48:30PM -0700, L A Walsh wrote: > But the implementation of process substitution in bash > isn't implemented that way in the currently released version. It > uses a tmp file on a disk of fixed size to store *all* of the output > of the 'writer' before the reader is c

Re: "here strings" and tmpfiles

2019-04-10 Thread Greg Wooledge
On Tue, Apr 09, 2019 at 04:48:30PM -0700, L A Walsh wrote: > > cp <(process) /tmp/foo > --- > *red face* I'd never tried to copy something that > looked like input redirection. My apologies on my misconception. One of the archetypal examples that we give when explaining process substitut

Re: "here strings" and tmpfiles

2019-04-09 Thread L A Walsh
On 4/8/2019 9:19 PM, Robert Elz wrote: > > | Optionally, I would accept that > | an implementation would support forward seeking as some equivalent > | to having read the bytes. > > I suppose one could make pipes do that, but no implementation I have > ever seen does, so I don't think yo

Re: "here strings" and tmpfiles

2019-04-09 Thread Eli Schwartz
On 4/9/19 10:25 AM, konsolebox wrote: > On Mon, Apr 8, 2019 at 10:39 PM Greg Wooledge wrote: >> That's incorrect in this context. We're talking about boot scripts here, >> not interactive user shells. In boot scripts, on every operating system >> I've ever used, the shell being used is either PO

Re: "here strings" and tmpfiles

2019-04-09 Thread konsolebox
On Tue, Apr 9, 2019 at 11:28 PM Chet Ramey wrote: > > On 4/9/19 11:25 AM, konsolebox wrote: > > On Tue, Apr 9, 2019 at 10:28 PM Chet Ramey wrote: > >> > >> On 4/9/19 10:10 AM, konsolebox wrote: > >>> On Wed, Mar 20, 2019 at 8:19 PM Greg Wooledge wrote: > > Just like that one time L. Wa

Re: "here strings" and tmpfiles

2019-04-09 Thread Chet Ramey
On 4/9/19 11:25 AM, konsolebox wrote: > On Tue, Apr 9, 2019 at 10:28 PM Chet Ramey wrote: >> >> On 4/9/19 10:10 AM, konsolebox wrote: >>> On Wed, Mar 20, 2019 at 8:19 PM Greg Wooledge wrote: Just like that one time L. Walsh tried to write a bash boot script that used <() to populat

Re: "here strings" and tmpfiles

2019-04-09 Thread konsolebox
On Tue, Apr 9, 2019 at 10:28 PM Chet Ramey wrote: > > On 4/9/19 10:10 AM, konsolebox wrote: > > On Wed, Mar 20, 2019 at 8:19 PM Greg Wooledge wrote: > >> > >> Just like that one time L. Walsh tried to write a bash boot script that > >> used <() to populate an array, and it failed because she was

Re: "here strings" and tmpfiles

2019-04-09 Thread Chet Ramey
On 4/9/19 10:10 AM, konsolebox wrote: > On Wed, Mar 20, 2019 at 8:19 PM Greg Wooledge wrote: >> >> Just like that one time L. Walsh tried to write a bash boot script that >> used <() to populate an array, and it failed because she was running >> it too early in the boot sequence, and /dev/fd/ wasn

Re: "here strings" and tmpfiles

2019-04-09 Thread konsolebox
On Mon, Apr 8, 2019 at 10:39 PM Greg Wooledge wrote: > That's incorrect in this context. We're talking about boot scripts here, > not interactive user shells. In boot scripts, on every operating system > I've ever used, the shell being used is either POSIX sh or Bourne sh. > > Everyone who write

Re: "here strings" and tmpfiles

2019-04-09 Thread Greg Wooledge
On Tue, Apr 09, 2019 at 10:10:44PM +0800, konsolebox wrote: > @Chet, Isn't bash supposed to use named pipes alternatively, and > dynamically? Or does it just decide what to use based on the current > system? The second thing. On platform X, bash uses named pipes. On platform Y, bash uses /dev/f

Re: "here strings" and tmpfiles

2019-04-09 Thread konsolebox
On Wed, Mar 20, 2019 at 8:19 PM Greg Wooledge wrote: > > Just like that one time L. Walsh tried to write a bash boot script that > used <() to populate an array, and it failed because she was running > it too early in the boot sequence, and /dev/fd/ wasn't available yet. @Chet, Isn't bash suppose

Re: "here strings" and tmpfiles

2019-04-09 Thread konsolebox
On Wed, Mar 20, 2019 at 9:05 AM Robert Elz wrote: > Note: I am not suggesting bash should change - using files for here docs > is the way they were originally implemented (in the Bourne sh) (though it > had bugs, which could leave the files lying around in some cases). > > However, using files for

Re: "here strings" and tmpfiles

2019-04-09 Thread Jason A. Donenfeld
Since originally raising this issue with dkg (leading to this email thread), I've only followed along from a bit of a distance. But it does look like there's been some good progress: there's now a commit that fills the pipe up to the OS's maximum pipe size, and then falls back to the old (buggy, vu

Re: "here strings" and tmpfiles

2019-04-09 Thread Chet Ramey
On 4/9/19 8:36 AM, Greg Wooledge wrote: > Bash always forks for $() as far as I'm aware, which is why bash 3.1 > introduced printf -v var. It's not, but it was a nice side effect. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hipp

Re: "here strings" and tmpfiles

2019-04-09 Thread Greg Wooledge
On Tue, Apr 09, 2019 at 02:32:38PM +0700, Robert Elz wrote: > The idea is basically just to do > > var=$( cmd ) > > right? But without a fork. That's something that can be done today, > no new syntax needed (bash might even do it sometimes, I don't know, the > FreeBSD shell does.) wool

Re: "here strings" and tmpfiles

2019-04-09 Thread Robert Elz
Date:Mon, 8 Apr 2019 23:36:39 -0700 From:pepa65 Message-ID: | When in the past I proposed this syntax: | cmd >>>var | the idea was to commit the output of a command into memory (in the form | of a variable), without requiring a pipe or file. In general

Re: "here strings" and tmpfiles

2019-04-08 Thread pepa65
On 8/4/2019 22:53, Robert Elz wrote: > [Aside: when the destination is a builtin, another strategy is to > simply write the here doc into mem, and have the builtin read directly > from the mem buffer - no actual I/O of any kind involved at all.] When in the past I proposed this syntax: cmd >>>va

Re: "here strings" and tmpfiles

2019-04-08 Thread Robert Elz
Date:Mon, 8 Apr 2019 22:03:25 -0700 From:pepa65 Message-ID: | What is nice about here docs/strings is that there are no subshells | involved. When they use files, that's correct, but when a pipe is used, unless the data size is both known in advance, and limited

Re: "here strings" and tmpfiles

2019-04-08 Thread pepa65
On 8/4/2019 21:19, Robert Elz wrote: > In general here docs (and here strings) are overused - it is always > possible to simply write a pipe instead What is nice about here docs/strings is that there are no subshells involved. I think Linda's main drive is to seek improvement in how bash works. N

Re: "here strings" and tmpfiles

2019-04-08 Thread Robert Elz
Date:Mon, 08 Apr 2019 17:04:41 -0700 From:L A Walsh Message-ID: <5cabe199.9030...@tlinx.org> | On 4/8/2019 7:10 AM, Chet Ramey wrote: | > Pipes are objectively not the same as files. They | > | > 1. Do not have file semantics. For instance, they are not seeka

Re: "here strings" and tmpfiles

2019-04-08 Thread L A Walsh
On 4/8/2019 7:10 AM, Chet Ramey wrote: > On 4/7/19 4:21 PM, L A Walsh wrote: > >> On 3/22/2019 6:49 AM, Chet Ramey wrote: >> >>> Yes, that's how bash chooses to implement it. There are a few portable >>> ways >>> to turn a string into a file descriptor, and a temp file is one of them (a

Re: "here strings" and tmpfiles

2019-04-08 Thread Greg Wooledge
On Mon, Apr 08, 2019 at 10:53:46AM -0400, Chet Ramey wrote: > On 4/8/19 10:36 AM, Greg Wooledge wrote: > > > That's incorrect in this context. We're talking about boot scripts here, > > not interactive user shells. In boot scripts, on every operating system > > I've ever used, the shell being us

Re: "here strings" and tmpfiles

2019-04-08 Thread Chet Ramey
On 4/8/19 10:36 AM, Greg Wooledge wrote: > That's incorrect in this context. We're talking about boot scripts here, > not interactive user shells. In boot scripts, on every operating system > I've ever used, the shell being used is either POSIX sh or Bourne sh. This is clearly wrong in general,

Re: "here strings" and tmpfiles

2019-04-08 Thread Greg Wooledge
On Sun, Apr 07, 2019 at 01:06:21PM -0700, L A Walsh wrote: > On 3/20/2019 5:19 AM, Greg Wooledge wrote: > > Just like that one time L. Walsh tried to write a bash boot script that > > used <() to populate an array, and it failed because she was running > > it too early in the boot sequence, and /de

Re: "here strings" and tmpfiles

2019-04-08 Thread Chet Ramey
On 4/7/19 4:21 PM, L A Walsh wrote: > On 3/22/2019 6:49 AM, Chet Ramey wrote: >> Yes, that's how bash chooses to implement it. There are a few portable >> ways >> to turn a string into a file descriptor, and a temp file is one of them (a >> child process using a pipe is another, but pipes have othe

Re: "here strings" and tmpfiles

2019-04-08 Thread Chet Ramey
On 4/7/19 4:06 PM, L A Walsh wrote: >> Just like that one time L. Walsh tried to write a bash boot script that >> used <() to populate an array, and it failed because she was running >> it too early in the boot sequence, and /dev/fd/ wasn't available yet >> > > --- > /dev/fd was available,

Re: "here strings" and tmpfiles

2019-04-07 Thread L A Walsh
On 3/22/2019 6:49 AM, Chet Ramey wrote: > Yes, that's how bash chooses to implement it. There are a few portable > ways > to turn a string into a file descriptor, and a temp file is one of them (a > child process using a pipe is another, but pipes have other issues). > Such as? That are more co

Re: "here strings" and tmpfiles

2019-04-07 Thread L A Walsh
On 3/20/2019 5:19 AM, Greg Wooledge wrote: > On Wed, Mar 20, 2019 at 07:49:34AM +0700, Robert Elz wrote: > >> However, using files for here docs makes here docs unusable in a shell >> running in single user mode with no writable filesystems (whatever is >> mounted is read only, until after file

Re: "here strings" and tmpfiles

2019-03-24 Thread konsolebox
On Tue, Mar 19, 2019, 9:36 PM Greg Wooledge wrote: > On Tue, Mar 19, 2019 at 09:20:33AM -0400, Daniel Kahn Gillmor wrote: > > On Tue 2019-03-19 08:25:50 -0400, Greg Wooledge wrote: > > > On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: > > >> strace -o tmp/bash.herestring.

Re: "here strings" and tmpfiles

2019-03-22 Thread Greg Wooledge
On Fri, Mar 22, 2019 at 10:28:52AM -0400, Chet Ramey wrote: > On 3/20/19 8:19 AM, Greg Wooledge wrote: > > > Even if Chet changed how here docs work in bash 5.1, nobody would > > be safe to use those features in their "I'm feeding a password with > > a here string" scripts for at least 20 years, b

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/20/19 8:19 AM, Greg Wooledge wrote: > Even if Chet changed how here docs work in bash 5.1, nobody would > be safe to use those features in their "I'm feeding a password with > a here string" scripts for at least 20 years, because there will > still be people running older versions of bash for

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/19/19 9:07 AM, Daniel Kahn Gillmor wrote: > Thanks for the feedback, Eduardo-- > > On Mon 2019-03-18 17:40:17 -0700, Eduardo A. Bustamante López wrote: >> I don't think the implementation details of herestrings are documented >> anywhere, >> and I'm not too sure if they should (i.e. IMO if y

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/20/19 7:36 AM, Daniel Kahn Gillmor wrote: > On Tue 2019-03-19 09:31:55 -0400, Greg Wooledge wrote: >> There are scripts that *rely* on the seekability of the temporary files >> created by here-documents and here-strings. "Improving" the "situation" >> would break backward compatibility. > >

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/19/19 8:49 PM, Robert Elz wrote: > Date:Tue, 19 Mar 2019 08:25:50 -0400 > From:Greg Wooledge > Message-ID: <20190319122550.khv5jp66iobjo...@eeg.ccf.org> > > | Yes, just like here documents do. And have always done, in all shells. > > That's not correct. The

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/18/19 8:40 PM, Eduardo A. Bustamante López wrote: > Having said that, have you tried process substitution as an option? > > You should be able to do something like: > > > mycommand < <(printf %s 'super secret') > > > That will: > > - not write the 'super secret' string to the file-sys

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/19/19 8:25 AM, Greg Wooledge wrote: > On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: >> strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"' >> It turns out that this creates a temporary file, actually touching the >> underlying filesystem: > > Yes,

Re: "here strings" and tmpfiles

2019-03-22 Thread Chet Ramey
On 3/18/19 5:18 PM, Daniel Kahn Gillmor wrote: > hi bash developers-- > > I ran the following command to get a sense of how bash deals with here > strings under the hood: > > strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"' > > (i'm testing with bash 5.0-2 on debian tes

Re: "here strings" and tmpfiles

2019-03-20 Thread Greg Wooledge
On Wed, Mar 20, 2019 at 07:49:34AM +0700, Robert Elz wrote: > However, using files for here docs makes here docs unusable in a shell > running in single user mode with no writable filesystems (whatever is > mounted is read only, until after file system checks are finished). Meanwhile, proposals ba

Re: "here strings" and tmpfiles

2019-03-20 Thread Greg Wooledge
On Wed, Mar 20, 2019 at 07:36:41AM -0400, Daniel Kahn Gillmor wrote: > How do you replicate 3<<<"$secret" with a pipeline? Not strictly a pipeline, but: 3< <(printf %s "$secret") This is actually preferred in many cases, because it doesn't add a newline. <<< always adds a newline to the result,

Re: "here strings" and tmpfiles

2019-03-20 Thread Daniel Kahn Gillmor
On Tue 2019-03-19 09:31:55 -0400, Greg Wooledge wrote: > There are scripts that *rely* on the seekability of the temporary files > created by here-documents and here-strings. "Improving" the "situation" > would break backward compatibility. i hope you noticed that of my suggested improvements, on

Re: "here strings" and tmpfiles

2019-03-19 Thread Robert Elz
Date:Tue, 19 Mar 2019 08:25:50 -0400 From:Greg Wooledge Message-ID: <20190319122550.khv5jp66iobjo...@eeg.ccf.org> | Yes, just like here documents do. And have always done, in all shells. That's not correct. There are shells that don't use files for here docs.

Re: "here strings" and tmpfiles

2019-03-19 Thread Daniel Kahn Gillmor
On Tue 2019-03-19 08:25:50 -0400, Greg Wooledge wrote: > On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: >> strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"' >> It turns out that this creates a temporary file, actually touching the >> underlying filesys

Re: "here strings" and tmpfiles

2019-03-19 Thread Greg Wooledge
On Tue, Mar 19, 2019 at 09:20:33AM -0400, Daniel Kahn Gillmor wrote: > On Tue 2019-03-19 08:25:50 -0400, Greg Wooledge wrote: > > On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: > >> strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"' > >> It turns out th

Re: "here strings" and tmpfiles

2019-03-19 Thread Daniel Kahn Gillmor
Thanks for the feedback, Eduardo-- On Mon 2019-03-18 17:40:17 -0700, Eduardo A. Bustamante López wrote: > I don't think the implementation details of herestrings are documented > anywhere, > and I'm not too sure if they should (i.e. IMO if you need that degree of > control > over the implementat

Re: "here strings" and tmpfiles

2019-03-19 Thread Daniel Kahn Gillmor
On Mon 2019-03-18 17:18:10 -0400, Daniel Kahn Gillmor wrote: > A few possible options for trying to improve the situation: > > a) use socketpair(2) or pipe(2) instead of making a tmpfile. this has > the potential downside that the semantics of access to the remaining > file descriptor wou

Re: "here strings" and tmpfiles

2019-03-19 Thread Greg Wooledge
On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: > strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"' > It turns out that this creates a temporary file, actually touching the > underlying filesystem: Yes, just like here documents do. And have always don

Re: "here strings" and tmpfiles

2019-03-18 Thread Eduardo A . Bustamante López
On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote: > hi bash developers-- (...) > a) use socketpair(2) or pipe(2) instead of making a tmpfile. this has > the potential downside that the semantics of access to the remaining > file descriptor would be subtly different from

"here strings" and tmpfiles

2019-03-18 Thread Daniel Kahn Gillmor
hi bash developers-- I ran the following command to get a sense of how bash deals with here strings under the hood: strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"' (i'm testing with bash 5.0-2 on debian testing/unstable). It turns out that this creates a temporary fil