bug#69535: Problem with copying an EXTREMELY large file - cmp finds a mismatch
I don't know whether the problem I've found is with cp or with cmp, so I don't know whether to address this report to coreutils or diffutils. If you think I've guessed wrong, please tell me so. I am trying to make a backup copy of a very large (40 Gigabyte) data file - yes, I have plenty of disk space! :) It's a binary file, 200 byte fixed length records to be precise, not a text file. I have downloaded, compiled and used the latest versions of cp and cmp and the problem persists. My system is a 16-core AMD Ryzen desktop running Linux Mint 21.3. The steps to reproduce the problem are simple, provided you have the data file! I have a folder called original in the data directory. From a terminal prompt, I run cp data.dat original this apparently completes correctly - at least, no error messages are seen I then run cmp -l data.dat original/data.dat and I get something around 100 bytes of differences. On the basis of three attempted copy and comparison pairs, the addresses of these differences vary, but they're always a single block of contiguous locations, and always towards the end of the file (the last time, they were in the 35,000,000,000s). I have run a fsck on the drive (a 14 TB Seagate connected to one of the motherboard SATA ports) and no problems were found. Any advice, please? I'm close to the limits of my debugging knowledge. Please note that I have absolutely zero knowledge of the C language or its derivatives. I'm a (retired) scientist turned database programmer, I know Pascal, FORTRAN and SQL, and that's about it. Thanks, Brian.
bug#69535: Problem with copying an EXTREMELY large file - cmp finds a mismatch
On 3/4/24 03:10, Paul Eggert wrote: Try running 'strace -o tr cp data.dat original' and then look at the file 'tr' (which could be quite large). Look for the syscalls near the start, and near the end, of the bulk copy. Quite possibly it's a bug in your Linux drivers or your firmware or hardware. For example, if you're using ZFS, see: https://github.com/openzfs/zfs/issues/15526 The strace output might help figure this out. My drives are formatted using ext4. The command above did indeed produce a large output file, almost 40 Megabytes of it, but deleting every line that started with read(3, or write(4, (there were over 300,000 pairs) got the file down to a far more manageable 7 KB. At first glance, it doesn't make much sense to me, but I will try going through it line-by-line tomorrow (it's silly o'clock at the moment) and see whether anything jumps out at me. Thanks for the help.
bug#69535: update
Sorry for the delay in updating this problem - I've been doing some testing! The first thing I did was wrote a quick and dirty Pascal program to do a byte-by-byte comparison of the data files, just in case it was cmp that was causing the problem, not cp. The results were the same using my program. OK, so cmp is innocent. I then wrote a program which generated 40 GB of more or less random data - 200,000,000 fixed length records of 200 bytes each - and sent it to a friend with the request that he do the test I did, i.e. generate the file, copy it, and compare the copies. He's also using Mint 21.3, as I am, but with a different type of drive. The files compared correctly on his system. OK, so the next thing was I tried using different drives. I have a 2 GB SSD in this system, so I tried that. No problem, the files compared correctly. I got the same result using my RAID 5 backup device, which contains 5 of the same type of drive as was producing the error when connected direct to the SATA port, but obviously the backup device's software has an effect there. So, the bottom line is that I'm chasing some kind of goofy bug which likely involves the drivers for my hardware - a 14 TB Seagate connected to the mobo SATA port - and cp is also innocent. Please consider this bug report to be closed. I'm not sure if/how I can do that via e-mail.
Re: date +%s ignores TZ
Jan Engelhardt wrote: > I wanted to get the number of seconds since the start of the day. > > echo $[`date +%s` % 86400]; How about: echo $[$(date +%s) - $(date -d '' +%s)] Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Feature Request - prepend to file
Eric Blake wrote: > machines. Finally, what syntax did you have in mind? It is probably > possible to write a shell script that has that exact same syntax, but uses > existing commands, to achieve the task without needing to add a new coreutil. I agree that for light duty a few simple shell commands are probably best. However, there is something to be said for writing a complete solution once and then reusing it, rather than reinventing something every time it comes up. I'm thinking specifically about all the random trivialities and corner cases: - securely choosing a unique filename for the temporary file - preventing a possible race when replacing the original - coping with running out of space or being interrupted - ensuring that temporary files are always cleaned up I think these are all examples of things that traditionally the coreutils have had to worry about above and beyond the basic functionality of a command, so it's not totally unreasonable to make such a request. The portability argument of course still holds, and if that's a concern it's much better to do it with the shell fragment -- but for some people portability is not an issue. In addition, there's the possibility of doing this in-place if there's not enough free space and/or if the size of the header fits into memory, however that is dangerous since you suffer data loss if interrupted. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Makefile.am issue (and patch)
I'm having a small issue with make install of coreutils. I've included
below a patch that solves my problem.
I am using coreutils-6.12, and building it under cygwin, and I'm
cross-compiling it to a PPC-440.
Here's my build steps:
./configure --host=powerpc-440-linux-gnu --prefix="`pwd`/__images"
make
make install
configure and make work fine. But make install fails with:
./ginstall: ./ginstall: cannot execute binary file
To fix this, I'm using the patch below. Note that the change in my
patch applies to code from a very recent checkin:
http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=9d595099372e2eabbecce2f303058743e93f0748
Perhaps my issue is only related to cross-compiling? Or is there a
better fix I should use?
Here's my patch:
diff -urN coreutils-6.12/src/Makefile.am coreutils-6.12.new/src/Makefile.am
--- coreutils-6.12/src/Makefile.am2008-05-31 14:15:33.0 -0400
+++ coreutils-6.12.new/src/Makefile.am2008-07-03 12:46:25.260074200
-0400
@@ -434,7 +434,7 @@
install-exec-am:
@case '$(program_transform_name):$(EXEEXT)' in\
's,x,x,:') cu=cu-;; *) cu= ;; esac;\
-$(MAKE) $(AM_MAKEFLAGS) binPROGRAMS_INSTALL=./ginstall\
+$(MAKE) $(AM_MAKEFLAGS) \
$${cu}install-binPROGRAMS
@$(NORMAL_INSTALL)
$(MAKE) $(AM_MAKEFLAGS) install-exec-hook
--
Brian Silverman
Concept X, LLC
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: md5sum results with V5.2.1 and V5.97 differ
Aparna Rallabandi wrote: > $ md5sum --string "654321" > c33367701511b4f6020ec61ded352059 "654321" This is the md5 of the six characters "654321". > $ echo "654321" |md5sum --text > 4a62cf6ee3f8d889e65af1cc271f20fa - This is the md5 of the seven characters "654321\n". Use echo -n if you don't want the newline included. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: cut - lack of --merge-delimiters option
Jan Skowron wrote: > coreutils program "cut" could use a "merge delimiters" option. > > Common use case: ls -l | cut ... > One needs to print 7-th column of ls -l to see all times of > modifications. But there is no constant number of delimiters between If you are only interested in times, why not ask the tool for just that information, e.g. stat -c %y * > merge delimiters option would help a lot in such cases. Without it > users are forced to use "gawk" instead of "cut". And what's wrong with that? Or stated differently, awk is a standard and portable tool, whereas "new version of cut with new option that isn't yet present on many systems" isn't. Based on traffic on this list it seems there is still a large number of people using coreutils 5.x which is more than 2 years old at best. Any new option that gets added thus has this waiting period of a number of years before it can be reasonably assumed to be in common use. And even then, people that want to write portable scripts still can't use the option, because it won't be in other 'cut' implementations, such as Darwin's or Sun's or the BSD's, etc. In short: adding options to standard commands when a portable solution already exists does not make a lot of sense, and the coreutils maintainers are very resistant to doing it as a result. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: weird bug in tr (translate or delete character)
Arch Drone wrote: > [EMAIL PROTECTED]:~/tmp$ ls > a > [EMAIL PROTECTED]:~/tmp$ echo "abcDEF.<>" | tr -d [:alpha:] > bcDEF.<> > [EMAIL PROTECTED]:~/tmp$ mv a b; ls > b > [EMAIL PROTECTED]:~/tmp$ echo "abcDEF.<>" | tr -d [:alpha:] > .<> This is not a bug. Without quoting, [:alpha:] is interpreted by the shell as a glob that matches the filename 'a', so you're really running "tr -d a" in the first example. You need to quote the argument if you don't want it to spuriously match a filename: tr -d '[:alpha:]' You commonly see this also with find, where you have to quote globs that you want to survive past the shell without interpolation and be passed on to the command: find . -name '*.ext' Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
su vulnerability on coreutils 6.9 (64-bit Linux)
I have built the coreutils 6.9 package (the latest) on a 64-bit x86 Linux system (Linux 2.6). If as myself (not root) I type: % su I become root. No password asked! Note: This software resides in AFS space. If I build it locally, the problem does not occur. BTW, I've also built coreutils 6.9 on the following platforms (all out of AFS space): Solaris 9 Solaris 10 (SPARC and x86) Mac OS X AIX 5.3 Linux 2.4 Linux 2.6 (32-bit) None of them have this problem. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Possible bug with grep/sed/tail?
Pádraig Brady wrote:
> It's probably useful to add this functionality to all coreutil filters.
> I still need to be convinced that glibc is not the correct place for this.
> If it was there then any users of stdio could be controlled.
It has been said before in previous threads but I want to re-state that
this is possible in a very simple and general way with the LD_PRELOAD facility:
$ echo '__attribute__((constructor)) void f() { setvbuf (stdout, NULL, _IOLBF,
0); }' | \
gcc -include stdio.h -x c - -fPIC -shared -o linebuf.so
$ ( while true; do echo "foo"; sleep 1; done ) | LD_PRELOAD=./linebuf.so grep
foo | cat
foo
foo
foo
... # immediate output
Brian
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Possible bug with grep/sed/tail?
Jim Meyering wrote:
> local setvbuf='__attribute__ ((constructor)) void
> f () { setvbuf (stdout, NULL, _IOLBF, 0); }'
> echo "$setvbuf" | gcc -s -include stdio.h -x c - -fPIC -shared \
> -o "$line_buffer_so"
Note that I named the function 'f()' for brevity in the example, but for
a general solution I worry that it should have a name that cannot clash
with any other global symbol in the program or any of its libraries.
One way of achieving this is to put it in an anonymous namespace, which
obliges the compiler to give it a guaranteed-unique name:
local setvbuf='namespace { __attribute__ ((constructor)) void
f () { setvbuf (stdout, NULL, _IOLBF, 0); } }'
echo "$setvbuf" | gcc -s -include stdio.h -x c++ - -fPIC -shared \
-o "$line_buffer_so"
> # test with this:
> # (while :; do printf 'x\ny\n'; sleep 1; done) |line-buffer grep x
Is that really a valid test? Doesn't the stdout of grep need to be a
pipe?
> > Of course this solution depends on gcc and LD_PRELOAD etc.
>
> The mixed blessing of shared libraries...
Well, __attribute__((constructor)) is gcc-specific but it was another
shorthand used for brevity to squeeze things into one line. The ctor
could however just as well be implemented in plain ISO C++. Naturally
the specific set of required options to produce a shared library does
tend to be compiler specific though.
I'd be more concerned about the portability of LD_PRELOAD, which tends
to be a feature of ELF systems. Cygwin does provide an emulation of
LD_PRELOAD however, so the example at least does work there as well,
modulo s/-fPIC// and s/.so/.dll/.
Brian
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: du --files-from feature request
Aaron Peterson wrote: > I spend many hours trying to figure this out / and searching for solution. > It turns out that the \0 delimited list requirement was very painful. > I wish I could cat the output of find into du and get the total It's meant to be used like: $ find . -type f -print0 | du --files0-from=- Note also that xargs takes -0 for the same reason: $ find . -type f -print0 | xargs -0 command Using tr is not the intended use pattern, as it completely misses the point. \0 is the only character that cannot be part of a filename (other than '/'), so the only reliable way to deal with arbitrary filenames without complicated quoting is to \0-delimit them. Using tr to convert newlines to \0 fails if you have filenames with newlines in them, not to mention that it's pointless overhead. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
New Feature Desired for tail
The new feature is demonstrated by a wrapper script around tail which gives me the ability to use tail to drive arbitrary alerts like this (only the core concept lines are shown): # put it into the background tail -n 0 -f error_file > working_file & #wait for some lines to arrive while ! test -s working_file do echo nothing yet... Going to sleep for:timeout:$timeout: >&2 sleep $timeout done #got some lines so cat them cat working_file This allows me to watch a file for maybe 10 second intervals, and grab all lines that arrived during that time interval. If nothing arrived, then it keeps on waiting. This effectively allows me to drive shell scripts with stdin model, but only on when new lines arrive. It is used like this: ./recent_line_tail error_log | alarm_handler_script This functionality could be put into tail with a single new option. Proposed Usage (a for alert, or maybe n for new?): tail -a 23 error_log I propose this would check for output each 23 seconds, and if it finds any it will cat it and stop. If there are no lines then it would wait another 23 seconds. McQueen ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
RE: New Feature Desired for tail
I'm glad to hear about that program but it does not meet the need I am describing. Even tail will work on several files, but that's not what I am interested in. The most important featur is responding when new lines are added to a log file. Simply watching it not enough. When new lines are added to the log file, my alert must be triggered, and I want those new lines to be sent into my alert via the stdio model. The current tail does send things on, but it goes on forever: tail -f files | my_alert Tail will keep on running and pumping lines into my_alert. If tail were to find a few new lines and send them off and die, then my_alert is a very simple script to write - this is what I want. I want to know if ANY lines came in within the last x seconds. The example above is to check for errors every 2 seconds and trigger my alarm if any have occurred. But that may be more than is needed, maybe once per day is enough for some circumstances. If I have to write this into the alert, then its much more complicated then it needs to be, but if this timer is put into tail then its all simple Unix filter IO. Imagine there are a series of 30 minute tests you want to run on a server. Then you set up this tail on the log file so after each test case, the log file lines are sent off to make a report for each test. Also if the program is signaled, then it stops immediately, sending all lines to stdout and terminates immediately. So when a test finishes early, instead of waiting for the full 30 minute time interval, its Ctrl-C and the captured lines are sent to stdout and captured in the test report. -Original Message- From: j...@gmail.com [mailto:j...@gmail.com] On Behalf Of James Youngman Sent: Thursday, May 07, 2009 3:59 AM To: Brian McQueen Cc: bug-coreutils@gnu.org Subject: Re: New Feature Desired for tail On Wed, May 6, 2009 at 11:15 PM, Brian McQueen wrote: > The new feature is demonstrated by a wrapper script around tail which > gives me the ability to use tail to drive arbitrary alerts like this > (only the core concept lines are shown): > > > > # put it into the background > > tail -n 0 -f error_file > working_file & > > > > #wait for some lines to arrive > > while ! test -s working_file > > do > > > > echo nothing yet... Going to sleep for:timeout:$timeout: >&2 > > sleep $timeout > > > > done > > > > #got some lines so cat them > > cat working_file > > > > This allows me to watch a file for maybe 10 second intervals, and grab > all lines that arrived during that time interval. If nothing arrived, > then it keeps on waiting. This effectively allows me to drive shell > scripts with stdin model, but only on when new lines arrive. > > > > It is used like this: > > > > ./recent_line_tail error_log | alarm_handler_script > > > > This functionality could be put into tail with a single new option. > Proposed Usage (a for alert, or maybe n for new?): > > > > tail -a 23 error_log > > > > I propose this would check for output each 23 seconds, and if it finds > any it will cat it and stop. If there are no lines then it would wait > another 23 seconds. Why not use something designed for the purpose of watching log files, for example http://www.fourmilab.ch/webtools/logtail/ but ... ? James. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: DD converts LF -> CR / LF
Eric Blake wrote: > Predefining O_BINARY as the default input_flags and output_flags is a > stopgap measure. While it is fine for other programs, such as od, to Doesn't that overly complicate things? Seems to me that whenever you use dd you are interested in copying fixed record length data. It just doesn't seem very likely that you would actually ever want dd to operate in text mode. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Maybe a bug
nosair wrote: > scp -r [EMAIL PROTECTED]:~/* ~/ECEJUNE2006 | tee transfer_june_2006.txt > > I get the prompt for password. After that, the downloading of files and > folders begins as normal. However, the file transfer information does not show > on screen, and transfer_june_2006.txt remains empty during the downloading. This is not a bug. scp will not print any progress output if the standard output is not a tty. Even if it did, you would probably not like the result since it prints a line to the screen every second, prefixed with a \r so that each successive line overwrites the next. If you redirected that to a file you would get a bunch of nonsense when you tried to view it later. You can actually do this with the 'script' utility, which simulates a tty to the command with a pseudoterminal: script transfer_june_2006.txt scp -r [EMAIL PROTECTED]:~/* ~/whatever However, when you view the .txt file with 'less' it will likely be fairly unintelligible, certainly not a plain text log. Though if you 'cat' it to a terminal you will recreate the scp output in fast-forward mode. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64 tool?
Simon Josefsson wrote: > > As for whether choosing on "base64 abc" whether abc is the filename > > containing text to be encoded, or the actual text to be encoded, I would > > lean towards filenames. > > Me too. The tool started out as a debugging tool for me, but it is > not the typical use. And 'base64 -s foo' is simple for short tests. > Or should it be 'base64 -e foo'? To match 'grep -e ...' or 'perl -e > ...'. 'foo' is a string here, not a filename. IMHO it should not accept input strings as a command line argument at all. If you want to accept short strings you can always do 'echo foo | base64 -' or similar. In your examples of perl/sed/grep those all take program/script code (or in the case of grep, a regular expression, a type of program) as parameters, which is not the same as the input stream to be processed. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: "line" utility
Michael Tewner wrote: > Is there a reason that there isn't a tool that prints one or more lines > from within a file? > > For years, everyone has been doing this with ` head | tail ` > > ...or is there a program that does this already? seq 1 50 | sed -n 3,25p sed 1 50 | sed -n 11,+10p # gnu sed only Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: coreutils version number policy
Heiner Steven wrote: > I have two different systems (Debina and SuSE), both with ^^^ I think that's they key right there. Distros like Debian tend to add lots of local patches to make things run more smoothly for them. In the case of Debian: <http://ftp.debian.org/debian/pool/main/c/coreutils/coreutils_5.2.1-2.diff.gz>. As you can see, they change a ton of things. Debian is probably one of the farthest from "upstream" distros that exists. If you just look at the actual GNU coreutils upstream tarball, you should see a unique set of features per version as you expect. As to "how to write a script" question, the 'Autoconf' way is to test for the functionality rather than checking version numbers. But more importantly, if you write a script that depends on a certain feature from a certain release of coreutils, chances are that script will be highly unportable and will fail to work on many systems. If you want to write portable scripts you have to stick to a "lowest common denominator" approach, which you can read more about in the autoconf book. If you don't care about portability then you don't have to bother checking for anything because you will be responsible for knowing that the tools support whatever feature you want to use. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: test binary
Kenneth J Vojtech wrote: > In zLinux for o/s390 (SuSe v 8 64 bit) is a binary 'test' that we use to > test for the existence of files. One of our tests uses a wildcard to check You are probably using the bash builtin command 'test' here and not the binary /bin/test that is a part of coreutils. Use the command "type test" to see if this is the case. That does not change the nature of your problem, though. > for any *.hdr files in a certain directory. When there is more than one > *.hdr file in this directory, the test fails with an exit 2, so we are not > processing any files. Is there an option that will allow the test command > to end with exit 0 if multiple files are found? Or is this a bug? This is how 'test -e' is designed to work, that is, to take a single filename to test for existence. When you use a wildcard, the shell first expands that into multiple arguments before invoking the command, which has no idea that a wildcard was ever involved. All it knows is that where it expected a single thing it got more than one, e.g. "test -e 123.hdr 456.hdr", which is not valid syntax. I suggest that you use something like the following to see if one or more files matching a wildcard are present: test x`find . -maxdepth 1 -name \*.hdr -printf yes -quit" = xyes This invokes find to search the directory '.' for files matching *.hdr. Because we want find to do the searching and not the shell, the * wildcard must be quoted so that the shell doesn't expand it as above. -maxdepth 1 causes find not to recurse into subdirectories. When it finds the first file matching that pattern it will print "yes" and exit without further work. If no such file is found it will print nothing, which is why both arguments to '=' are prefixed with 'x'. Otherwise, test would be called as "test = yes" on a non-match which is invalid syntax -- the left hand side argument is missing. You could skip the -printf and test the output against the empty string, but then if one of the found files had a space in its name you would end up invoking test with invalid syntax. You could play quoting games to get around that but I find using -printf to be simpler. YMMV. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
renameutils 0.6.0
(cc: to bug-coreutils in case this is a coreutils problem)
We have talked about this before, but I have never been able to get imv
to work without patching it to change the order of the arguments passed
to "mv". icp has the same issue. qmv works fine.
Based on your last email to me about this (September, sorry!), I tried
this at the command line:
-
> unalias mv
-
The order of the arguments passed my imv doesn't work:
> strace imv setup-gentoo-chroot
execve("/usr/bin/mv", ["mv", "setup-gentoo-chroot", "--",
"setup-gentoo-chroot.sh"], [/* 68 vars */]) = 0
uname({sys="Linux", node="syrinx", ...}) = 0
mv: when moving multiple files, last argument must be a directory
Try `mv --help' for more information.
-
Adding a mv-only option doesn't help:
> imv -v setup-gentoo-chroot
> setup-gentoo-chroot.sh
mv: when moving multiple files, last argument must be a directory
Try `mv --help' for more information.
> imv --verbose setup-gentoo-chroot
> setup-gentoo-chroot.sh
mv: when moving multiple files, last argument must be a directory
Try `mv --help' for more information.
-
Testing at the command line, without imv:
> mv setup-gentoo-chroot -- setup-gentoo-chroot.sh
mv: when moving multiple files, last argument must be a directory
Try `mv --help' for more information.
Adding a leading "\" to the mv command, or specifying the path doesn't
help:
> \mv setup-gentoo-chroot -- setup-gentoo-chroot.sh
mv: when moving multiple files, last argument must be a directory
Try `mv --help' for more information.
> /usr/bin/mv setup-gentoo-chroot -- setup-gentoo-chroot.sh
/usr/bin/mv: when moving multiple files, last argument must be a
directory
Try `/usr/bin/mv --help' for more information.
> mv -v setup-gentoo-chroot.sh -- setup-gentoo-chroot
mv: when moving multiple files, last argument must be a directory
Try `mv --help' for more information.
-
This order works:
> mv -- setup-gentoo-chroot.sh setup-gentoo-chroot
as does this (which I think was your intent):
> mv -v -- setup-gentoo-chroot setup-gentoo-chroot.sh
`setup-gentoo-chroot' -> `setup-gentoo-chroot.sh'
-
The man page for mv simply says that "--" should "terminate the option
list". Hmmm. I guess it expects "--" to occur as the last NON-FILE
argument? If so, puttting it in between the file arguments is the source
of the problem. Do you have a reference that says something like
mv name1 -- name2
is legal syntax? If not, I see no particular benefit to passing the
arguments in this order. Granted, perhaps it should be accepted by mv
and cp, but what advantage is there in ordering the arguments like this?
I would humbly suggest swapping the order of the arguments passed to
mv and cp via execve to be:
#1 all arguments passed to imv/icp on the command line (not --help,etc)
#2 --
#3 file argument #1
#4 file argument #2
Maybe the version of coreutils (mv,cp) you're using is different
than mine? I am runing Gentoo 2005.1 on ~amd64.
> epm -q coreutils
coreutils-5.2.1-r6
> epm -q renameutils
renameutils-0.6.0
--
Brian Hall
Linux Consultant
http://pcisys.net/~brihall
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: (Query): Changing the Functionality of rm Command
"Chawla, Sumit" wrote: > I need to change the functionality of one of the existing binaries of > Linux. For example that of command 'rm'. Please guide how to proceed in that > case. I need to perform some pre-deletion and post deletion task . For that > I need to either design a hookup function or / to replace the existing rm > binary with some new binary having the same name. Can u please guide in this > regard. If you want to patch 'rm' then patch 'rm'. I don't really understand the question. If you want to know where to get the source code or how to build it, then ask that. (But this is documented elsewhere, so you should read the INSTALL and README files first.) But it seems like you may be under the impression that patching 'rm' will have a system-wide effect. It won't. It will only affect commands or processes that use /bin/rm to delete files. But rm is just a wrapper around system calls (unlink and rmdir) and most applications that delete files use these syscalls directly. In other words, what you are asking sounds somewhat misguided, so you might consider detailing what you are actually trying to achieve, rather that the particular method you are trying to use. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: chown - Operation not permitted
Bob Proulx wrote:
> Please see this frequently given answer in the reference below.
>
> http://www.gnu.org/software/coreutils/faq/
>
> Look for "Why can only root chown files?"
Incidently, there is an incomplete sentence in that section of the FAQ
("Such as creating...").
BrianIndex: coreutils-faq.texinfo
===
RCS file: /webcvs/coreutils/coreutils/faq/coreutils-faq.texinfo,v
retrieving revision 1.16
diff -u -r1.16 coreutils-faq.texinfo
--- coreutils-faq.texinfo 13 Aug 2005 04:44:50 - 1.16
+++ coreutils-faq.texinfo 1 Sep 2005 16:07:07 -
@@ -351,7 +351,7 @@
@item
A user can create files that they cannot remove. With the old semantics
it was possible for normal users to create situations that only the
-superuser could fix. Such as creating a non-writable directory of files
+superuser could fix, such as creating a non-writable directory of files
and then giving the ownership away. Since they no longer own the files
they can't change them, nor can they remove them. This is commonly a
problem when untaring files that other people created with restrictive
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Suggestion for new coreutil: "every" prints every M out of N lines
Wayne Hayes wrote: > SYNOPSIS: > every [-M[,N]] [OPTIONS] [list-of-files] > > DESCRIPTION > Print N lines out of every M lines in a file. If no files are > listed, use the standard input. N and M both default to 1, so that > every(1) without options acts like cat(1). Every(1) can also work > on bytes or blocks instead of lines. Can't you pretty much achieve the same thing with: awk '(FNR-1) % M < N' file Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Incorrect lexographical sort in sort (coreutils)5.2.1
Murray Stokely wrote: > The current version of GNU sort on my Linux box incorrectly > sorts the following simple file, while an older version of GNU > sort on MacOS X 10.4 works fine. > > $ sort --version > sort (coreutils) 5.2.1 Version 5.2.1 is more than 18 months old (released 2004-03-12) and there have been a great number of changes since then. You might consider upgrading to the current stable version (5.92). But that is not relevant to the behavior below: >$ sort scrambled > version0/data/r=2/1/42 12.3 > version0/data/r=2/1/45 3.2 > version0/data/r=2 146839.1 > version0/data/r=2/1/51 10.9 > version0/data/r=2/1/52 6.6 > version0/data/r=2/1/54 7.5 This is working correctly as designed, though probably not how you expected. <http://www.gnu.org/software/coreutils/faq/coreutils-faq.html#Sort-does-not-sort-in-normal-order_0021>. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Feature request: ignore non-existant files with diff -r
"Bryce Nesbitt (mailing list account)" wrote: > This is a request for a command line switch to "diff -r", The diff command is not a part of coreutils: http://www.gnu.org/software/diffutils/diffutils.html ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: ls -L should say just what file doesn't exist
Dan Jacobson wrote: > lrwxrwxrwx 1 2 2005-11-18 07:08 xx -> yy > grep: xx: No such file or directory > > User thinks: the problem is really Grep isn't part of coreutils, so this is off-topic for this list. You may want to make your case on the grep mailing list: <http://www.gnu.org/software/grep/devel.html>. But as I see it the whole point of symlinks is that the program that is accessing the link sees it just like a normal file (unless it specifically checks) and the operating system takes care of resolving the link. So from grep's point of view 'xx' really doesn't exist, and it shouldn't have to know or care about the details of how symlinks works. In a way the error is correctly telling you that there is a problem with 'xx', namely that the link points to something that doesn't exist. Also, someone suggested something similar in another thread and one of the responses made the point that in order to implement this, the program would have to potentially traverse a number of links (e.g. xx -> yy -> zz) and in doing so it would need to be careful not to get stuck in a loop (xx -> yy -> xx) and other such issues. So implementing this may not be trivial. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: logwatch bug - script zz-disk_space
[EMAIL PROTECTED] wrote: > OS: RHEL 4 Update 1 > File: /etc/log.d/scripts/services/zz-disk_space > change line 20: "df -h | grep '^/dev/'" > to: "df -h" You need to report this to your OS vendor, in this case Red Hat. This mailing list has nothing to do with that script or any other distro-specific file. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Apparently irrational behaviour in sort
The Wanderer wrote: > I had considered that, but did not have a clear idea of what to add or > where to add it... and, now that I've both thought about that further > and looked more closely, the copy of the source I have (from Debian's > 5.93 source package) does not appear to contain the file from which > sort.1 is autogenerated, at least not in any obvious location. The man page is generated from the --help output, I believe. > (On completely another subject, and quite offtopic, do you happen to > know where one would go to complain about problems arising from mailing > lists which do not set the Reply-To header to the list address?) That is an often-discussed topic but the basic summary is that many people (including a number of open source projects and the sites that host them) feel that this is a feature not a bug. See also <http://www.unicom.com/pw/reply-to-harmful.html>. I wouldn't expect it to be changed any time soon. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Apparently irrational behaviour in sort
The Wanderer wrote: > Instead, I'll just ask how > exactly I'm supposed to make sure that one and *only* one copy of any > messages I receive from the list gets sorted into the folder in which I > keep list messages... while simultaneously making sure that any messages > which happen to have been replies to list messages but which are not > sent through the list do *not* wind up in that folder? Filter on the List-Id: header present in every list message. Regardless of reply-to issues I find this to be much more reliable than trying to match addresses in all the various To/CC/BCC/whatever fields. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: (no subject)
"Lakkimsetti, Chalapathi G" wrote: >I want to work on mysql in linux. i have gentoo linux in my system. After > i loaded mysql5.0.17, if i want to install the mysql it is giving error "NO > file/directory found" eventhough it is having that file. I don't think anyone on this list has any idea of what you're talking about. This list is about coreutils, and has nothing to do with mysql, gentoo, or general linux questions. You need to post your question to a mysql forum, and include *exactly* what commands you are typing, the *exact* output, what you expected to happen, etc. Your message does not include nearly enough information. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Sort bug
"linux-os (Dick Johnson)" wrote: > Clearly it's not sorted. > Since "_" (0x5f) is lower in value than 'a' (0x61), clearly > the sorted list should have all the names beginning with > the underscore first! Only if you're using the C locale. Read the FAQ. http://www.gnu.org/software/coreutils/faq/coreutils-faq.html#Sort-does-not-sort-in-normal-order_0021 Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug#340236: coreutils: cp -p to nfs destination failsto preserve timestamps
The Wanderer wrote: > The strace output does not refer to utimes by name. The relevant EPERM > is returned by something referred to as SYS_271 - presumably the label > simply means "strace could not identify the name of this syscall" - and > all five of the arguments are given only as hex values. Since utimes > takes only two arguments, I don't think that's what this is. Any idea > how to figure out what call is being made? It looks like on x86 linux, 271 is in fact utimes(), according to <http://www.lxhp.in-berlin.de/lhpsysc0.html>. The fact that strace shows 5 arguments is probably because it cannot identify it (and thus does not know the nature of its arguments), not because it actually has 5. It is a bit odd that your strace does not identify this -- are you sure you're using a strace that was compiled against the proper kernel headers? It sounds like you're using an old strace with a newer kernel. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: something iteresting
Bob Proulx wrote: > You probably wanted the [EMAIL PROTECTED] mailing list for asking > for gcc help. That sound probably be gcc-help (at) gcc.gnu.org Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
[ program
Coreutils: After compiling the coreutils, I looked in the coreutils-5.93/src directory, and noticed and executable named "[" doing an ls -l \[ gave me... -rwxr-xr-x 1 bbeck bbeck 73120 2006-01-02 16:03 [ when I tried running the file it output... ./[: missing `]' on a whim I ran the program with "]" as an argument. It returned nothing. I was wondering if that [ program is supposed to be there, or if it's a typo. If I run info [ I get the info page for test, so I wasn't certain if they were related. Thanks, Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Localization based problem with sort
Dirk Stoecker wrote: > So please tell me a way to use the sort program in contexts which allow no > environment variable settings. I will be happy to accept it. You can use "env LC_COLLATE=C sort -opts" as the sort command. Or 'sh -c "LC_COLLATE=C sort -opts"'. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: dircolors aliases
Nathan Fearnley wrote: > I've looked through the documentation, I've looked through configuration > files, I've even taken a glance through the source, I can't figure out where > dircolors decides to define the aliases. Any clue? Is it hard coded in the > source, or is it part of a configuration file? You must be using a patched coreutils because none of that alias junk is in the official tarballs. Check the patches that your distribution adds. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Filename pattern in grep --include=....
Com MN PG P E B Consultant 3 wrote: > >From the grep man-page: > >--include=PATTERN > Recurse in directories only searching file matching > PATTERN. > > What type of PATTERN can be used here (i.e. glob, regex, extended regex, > etc.)? grep is not a part of coreutils, so you're asking in the wrong list. But if it were me, I would try something like: $ find D -name no\* -o -name uh\* -print0 | xargs -0 grep abc The 'find' command is designed specifically for this purpose and has many more features and knobs for recursion, compared to trying to make grep do this job. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Slash appears twice using ls -ldF
I'm not certain if this is a bug, but if I do a ls -ldF I get back something like this... drwxr-xr-x3 4096 Dec 13 2004 // I don't think there ought to be two slashes after the directory name. If I am in error I apologize about wasting your time. Version Information: ls (coreutils) 5.2.1 (5.2.1-2ubuntu2) Thanks, Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: make error
Paul Eggert wrote: > > LINK.EXE /subsystem:console /DLL /nologo /base:"0x4ad0" /NOENTRY > > /IMPLIB:ic > > udt.lib /out:icudt34.dll stubdata.o > > LINK: extra operand `/nologo' > > Try `LINK --help' for more information. > > make[1]: *** [icudt34.dll] Error 1 > > make[1]: Leaving directory `/cygdrive/c/icu/source/stubdata' > > I don't see the connection between this and coreutils; perhaps you > should be asking on a Cygwin mailing list rather than on > bug-coreutils? It seems like he's using some kind of Makefile that expects to call LINK.EXE (the Microsoft linker) to do linking, but instead it is trying to invoke /bin/link from coreutils. In that case it's a problem with his Makefile and/or his PATH and/or his environment, but has nothing to do with Cygwin or coreutils. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Possible bug w/sort: As of coreutils-5.9x, sort loses command-linecolumn offset (e.g. +1.0n)
"John P. Eisenmenger" wrote: > So is this a bug, or do I just need to learn the new way of specifying > column offsets? It's not a bug. See the NEWS file. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: error in date command
Adam Miller wrote: >According to the man page, /bin/date +%x should report the > date in the following format: mm/dd/yy No it doesn't, it says it reports the date in the format appropriate to the current locale: %x locale's date representation (e.g., 12/31/99) Note that "e.g." means it's an example of one possible format. >However, when run in bash the command above reports the date in the > following format: mm/dd/ Compare the output of: LANG=C /bin/date +%x to LANG=en_US /bin/date +%x >I have been calling a bash script in through cron, which reports the > date as it should. So why would running the command in bash be different? You most likely have something in your rc/profile scripts which only runs for interactive login shells that sets the locale (LC_* and/or LANG environment variables.) When run from the cron job they are not set and the default "C" locale is used. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: patch: contrib/compare_tests
Andrew Pinski wrote: > > I was lazy today and decided to use compare_tests. Guess what, it doesn't > > work on recent coreutils/sort (i.e. the one on FC5). > > > > >From the texinfo doc: > > > > On older systems, `sort' supports an obsolete origin-zero syntax > > `+POS1 [-POS2]' for specifying sort keys. This obsolete behavior can > > be enabled or disabled with the `_POSIX2_VERSION' environment variable > > (*note Standards conformance::), but portable scripts should avoid > > commands whose behavior depends on this variable. For example, use > > `sort ./+2' or `sort -k 3' rather than the ambiguous `sort +2'. > > This is the same problem as tail and head. Someone should tell Coreutils > that again they broking stuff that should work no matter what environment > variable is set. > > Also Redhat (and all other distros) should think about bycotting GNU > Coreutils until they fix this bug. Coreutils is just implementing the decisions of the Austin Common Standards Revision Group. If someone posted here that e.g. g++ rejected code that was not valid C++0x the response would be exactly the same, "fix your code" not "g++ is buggy for not accepting this broken code and should be boycotted." Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: `configure' and `make/bison' build issues
vy wrote: > roamantic:~/continuing/_study/coreutils/lib vy$ bison -y getdate.y > getdate.y:215: unrecognized: %parse_param > getdate.y:215:Skipping to next % > getdate.y:216: unrecognized: %lex_param > getdate.y:216:Skipping to next % This could mean the version of bison that you have installed is not sufficient. It is odd that you are running this by hand though, shouldn't this be the result of 'make' output? You should never have to invoke tools directly like that, AFAIK. > roamantic:~/continuing/_study/coreutils vy$ ./configure > --prefix=/Users/vy/local --exec-prefix=/Users/vy/local/ppc > ./configure: line 9: ===: command not found > ./configure: line 12: syntax error near unexpected token `>>' > ./configure: line 12: `>>>>>>> 1.277' This indicates that there was a conflict when you tried to update the file "configure" from cvs, and cvs indicated the conflicting hunks with those characters (which cause it to no longer be a valid shell script.) This would happen for example if you modified the file locally and then tried to patch/"cvs up" it and didn't notice the conflicts. I suggest that you start over with a fresh checkout of the source, or at the very least, do a "cvs up -C configure" (which will replace your copy of configure with the current one from the repository.) There may be other files with conflicts though. In general whenever you do a "cvs up" or apply a patch you have to look at the output of the command for files with conflicts and manually fix them by hand before they will ever build. Also it is strange that there would be conflicts on a file like configure because that is a generated file that should never be hand edited, so I don't see how you would get conflicts. Consult the CVS manual for information on conflicts, there is a whole section about this (10.3 in my copy.) You should have gotten a warning from cvs that there were conflicts (and/or seen "C" as the status character next to the filename.) Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: `configure' and `make/bison' build issues
Eric Blake wrote: > Actually, it is very common to get conflicts in generated files, when the > version of the generator is different between the person who checked into > cvs vs. the person doing the build (for example autoconf 2.59 vs 2.59c). > That's why so many projects try to avoid generated files in CVS, and > instead have a huge bootstrap script that requires a user to install > prerequisite generator tools. Certainly differences in autotools versions will cause conflicts in generated files. But I guess the point I was trying to convey was that in such cases where generated files like 'configure' are stored in CVS, in almost all casual usage patterns (i.e. checking out the source just to build the package or perhaps make minor changes) the checked out copy will be used without needing to be rebuilt. It is only the developer who actually modifies a configure.ac or similar input file that would ever set the stage for a conflict in a generated file, and this type of change is comparitively rare -- certainly not something that I would expect from the OP who stated that he has yet to successfully build the package at all. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: comm command bug while using with Tilda delimited files
Anand Ramamoorthy wrote: > This is the first time I'm having issues with "comm" command on an HP-UX Unix > box (Model: 9000/800, Release: B11.11) > > I'm trying to compare two files that are "~" delimited. > > File 1 > > 1~10 > 5~30 > 55~40 > 6~20 > > > File 2 > > 1~20 > 5~35 > 55~40 > 7~27 'comm' requires that the input files be sorted. These are not sorted, because "~" comes after "5" in the ascii order. You can test this for yourself with something like: $ echo -e "5~30\n55~40" | sort 55~40 5~30 It works when you use "," because that character comes before the digits in the ascii order. Run your input files through 'sort' before passing them to 'comm'. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: "ls -h -l" returns 255GB, while "ls -l" returns 272 gigabytes
"Jameson C. Burt" wrote: > Upon rebooting, the resulting filesize listed with "ls" is inconsistent, >ls -h -l partition.img > -rw-r--r-- 1 root root 255G 2006-10-24 23:10 partition.img >ls -l partition.img > -rw-r--r-- 1 root root 272751509504 2006-10-24 23:10 partition.img The only thing inconsistent here is your understanding of the units, I'm afraid. HD manufacturers use the definition of GB = 10**9, because it makes their drives look bigger. Everyone else in the computer industry uses the definition of GB = 1024**3 (= 2**30), which is what actually makes sense since the disk is allocated in sectors of 512 bytes each, and most operations on computers occur in units of base-2 sizes. So that is what you get when you use ls -h, as that is the widely used custom when working with sizes. Technically, the power-of-2 based units should be referred to as Gibibyte, Mebibyte, Kibibyte, et al. and the power-of-10 based units referred to as Gigabyte, Megabyte, Kilobyte. But that is not common usage anywhere in the industry except for perhaps the HD manufacturers since it gives them a small advantage. So when you buy a "250GB" hard drive you are getting 250 billion bytes, which is actually a factor of (1000/1024)**3 smaller in GB. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: mv command fails on files >4GB
Paul Eggert wrote: > "Rich Morgan" <[EMAIL PROTECTED]> writes: > > > I'm trying to move large files from one filesystem to another and the mv > > command fails with the message "File size limit exceeded". The partial > > destination file is not removed and is 4,294,967,295 bytes in size. > > Most likely that's not a bug in mv; it's a problem with your user > limits. If you're running Bash, type "ulimit -a". mv is subject to > these limits, just like any other program is. It could also be trying to move the file from e.g. ext2/3 to a brain-dead filesystem (e.g. FAT32) which has a 4GB limit. That's also not something that mv can deal with, it's a limitation of the filesystem that cannot be avoided. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Feature-Request: random sort
Matthias Schniedermeyer wrote: > As far as i can see it the opposite of sort is missing. > Randomize/shuffling the incoming data lines, so you can e.x. use it for > a playlist. > > find . -type f | sort --shuffle | xargs <...> You must not be using the current version. This was added in 6.1: `-R' `--random-sort' Sort by hashing the input keys and then sorting the hash values. Choose the hash function at random, ensuring that it is free of collisions so that differing keys have differing hash values. This is like a random permutation of the inputs (*note shuf invocation::), except that keys with the same value sort together. If multiple random sort fields are specified, the same random hash function is used for all fields. To use different random hash functions for different fields, you can invoke `sort' more than once. The choice of hash function is affected by the `--random-source' option. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Replacing text?
Solarius wrote:
> Is there any way to replace string with another, for example from
> standard input to standard output via commandline with GNU toolbox?
This is possible with sed, awk, perl, and probably a zillion other
tools, depending on exactly what you want to do. But none of these are
part of GNU coreutils, they are their own projects. However, any system
that has coreutils installed will likely have at least one of them as
well.
sed -e 's,search,replace,g'
perl -pe 's,search,replace,g'
awk '{ gsub(/search/, "replace"); print }'
> I need to "parse" text from a file or input, and replace a string and
> put conclusion to output.
That's way too generic of a description to really be able to offer any
concrete advice.
> I tried to find a way, but I didn't:(
http://www.oreilly.com/catalog/shellsrptg/
http://www.oreilly.com/catalog/unixnut4/
Brian
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [coreutils] "cut" from last column
Philip Ganchev wrote: > functionality very often when working with tabular data. Currently, > the expression to achieve this is clumsy: > > cut -f 3,5-$(echo $(head -1 myfile | wc -w) - 3 | bc) myfile You're right, that's very clumsy, especially the "head -1" part which is problematic. (You should use the unambiguous "head -n 1" instead.) > The new syntax could use +1 to specify the last column, +2 to specify > the second-last, etc. This syntax is the same as for the "tail" > command. Then you can achieve the same as the above expression by > writing: > > cut -f 3,5-+3 myfile How about: perl -F'\t' -nae 'print join "\t", @F[2,4..$#F-3]' myfile Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [coreutils] "cut" from last column
Philip Ganchev wrote: > > > cut -f 3,5-+3 myfile > > > > How about: > > > > perl -F'\t' -nae 'print join "\t", @F[2,4..$#F-3]' myfile > > Even more verbose and complicated (clumsy). Your original cut example requires forking four external binaries and one shell builtin, and involves at least two subshells. To me, that is a lot more complicated (in terms of both syscalls and points of failure) than just running a single perl (or awk) command with no subshells. > Plus, it depends on > having Perl, which is not a coreutil as far as I know. You're asking for a feature that currently does not exist in 'cut' to be added. Do you realize how long it will take before you could count on being able to use this feature in a portable script? Even if the feature was added today I bet that you would find many systems in three years from now that are still running an out of date coreutils that would not support that feature. Just look at all the people that post to this list that are still using coreutils 5.2x or even its predecessors, textutils/fileutils/sh-utils. And on systems that use the vendor 'cut' instead of GNU 'cut', the feature will essentially never be available. On the other hand, a unix-like system that is not an embedded device that does not have perl (or awk) installed is pretty rare. So if the question is availability or portability of a particular feature, to me it looks like there is no contest. > going to use Perl, why do we need "cut" at all? It is just sensical > that if a command allows you to cut from the beginning, it will let > you cut from the end. I agree that it sounds like a logical idea, but the problem comes when you try to add features to programs like 'cut' that have existed for probably decades and that are standardized by IEEE/POSIX, and have multiple implementations (i.e. not everybody uses GNU cut.) Any features you add become GNU extensions, and can't be used in anything but private scripts that you don't ever intend to distribute to anyone since there's a very low chance they will work on other people's systems. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: sort does not use tab as delimiter
The Wanderer wrote: > Actually, I just tested - entered a tab in this mail, copied it with > Ctrl-C, attempted to paste it into an xterm with Ctrl-V. No result. You need to read the bash manpage. Search for "tab-insert" in the readline section. It has nothing to do with copy and paste, it is a way to tell readline to take the next keystroke literally, and it works in any readline app (e.g. gdb.) Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: rmdir problem in Molinux
"M. David Frost" wrote: > rmdir filename (error message - directory not empty) > > rmdir -rf filename (error message - invalid option -- r) > > When I type: > > dir filename > > the directory is empty. It may not be empty. ls doesn't show everything by default. Files that begin with "." for example are not shown. You need to use "ls -a" (or -A if you don't want to see . and ..) or similar to see them. [ Also, that you are using "dir" and not the usual "ls" suggests some background in DOS/Windows. I suggest you don't let this confuse you, as many concepts are similar but completely different in the details, such as this case of ls not listing dot-files. So throw out the old knowledge, and instead just sit down with "info coreutils" and browse. ] > Does the Molinux distribution not support -rf? And what is the solution? -r and -f are options for rm, not rmdir. They are different commands. Note that you can always run "command --help" or "info command" or "man command" if you want to see what options it supports. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: rm doesnt' delete
Lise Slama wrote: > I have some trouble with "rm". > > When I do > > rm --recursive --verbose a.out > > or more generally when I use "rm -R", it only deletes the corresponding > files in the current directory. > Do you know if there is some thing to configure or some way to correct that? That is the expected behavior. "--recursive" does not work the way you think it does. It does not mean to recursively look in every directory for the given filename. It means that if a directory name is given as one of the arguments to "rm", to remove it and all its subdirectories. If you want to remove all instances of a file with a specified name in a directory tree the best method is to use "find". If you are only interested in deleting files then you can use find's -delete option. find . -name a.out -delete You can use wildcards with find, but remember that they should be quoted, as you want "find" to do the wildcard expansion, not the shell: find . -name 'foo*.bar' -delete If you wanted to do something other than delete them, then you can use the "find | xargs" idiom. For details and examples run "info find" or "info xargs" and browse. These commands are extremely flexible and the way to go for doing most tasks involving recursively searching for all files of a given specification. Pretty much the only time that the "-r" flag to rm is useful is when you want to remove a directory and all its contents, as in "rm -rf dir". Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: basename sucks
dusan halicky wrote: > This is the exact case why to do it, look at \"my\" version: > > $ cat filename | basename Arg! Why why why do people do this nonsense? There is *NO* reason to use cat here. "basename http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: cp should look for destination file operand BEFORE wildcardexpansion
Anshul Nigham wrote: > I don't think this could be intended or expected behavior in any > scenario. I suggest that the cp behavior be modified to look for a > destination file operand before expanding wildcards, in which case a > command like "cp [options] wildcard-expr" would still exit with an > error of missing destination file operand, even if the wildcard-expr > expands to two or more files. You have a misconception as to how globbing works. It is not done by 'cp' at all, it is entirely the domain of the shell. By the time 'cp' looks at its arguments, they are already expanded and 'cp' has no way to differentiate whether the user specified "*.txt" or "a.txt b.txt". You can trivially verify this yourself with a 5 line C program that just prints the contents of 'argv[]'. This is by design. The original intent of unix was along the lines of "a lot of small programs, each that do one thing really well." In that vein, the shell does all globbing (and variable expansion/substitution, quoting, etc.) for all program invocations, rather than having to add glob expansion code to every little individual program that might ever need to process more than one file. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: dirname - bug or feature?
Fill Manuel wrote: There's no need to reinvent this wheel. > [EMAIL PROTECTED]:~> dirname ../testfile > .. > [EMAIL PROTECTED]:~> dirname-ng.pl ../testfile > /home > [EMAIL PROTECTED]:~> $ dirname `readlink -m ../testfile` /home > [EMAIL PROTECTED]:~> dirname ../../../././testfile > ../../.././. > [EMAIL PROTECTED]:~> dirname-ng.pl ../../../././testfile > / > [EMAIL PROTECTED]:~> $ dirname `readlink -m ../../../././testfile` / > [EMAIL PROTECTED]:~> dirname ../testfile /tmp/testfile2 > dirname: too many arguments > Try `dirname --help' for more information. > [EMAIL PROTECTED]:~> dirname-ng.pl ../testfile /tmp/testfile2 > /home > /tmp > [EMAIL PROTECTED]:~> $ for F in ../testfile /tmp/testfile2; do dirname `readlink -m $F`; done /home /tmp We already did this thread just a few weeks ago. <http://lists.gnu.org/archive/html/bug-coreutils/2007-02/msg00019.html> > [EMAIL PROTECTED]:~> find /tmp/test/* | dirname | sort | uniq > dirname: too few arguments > Try `dirname --help' for more information. > [EMAIL PROTECTED]:~> find /tmp/test/* | dirname-ng.pl - | sort | uniq > /tmp/test > /tmp/test/a > /tmp/test/b > /tmp/test/c > /tmp/test/d Now why in the world would you go to all that trouble? You're listing every single file and directory in the tree, only to then go and strip the filename component entirely, resulting in massive duplication that you then have to remove. What's wrong with just telling find to list directories: find /tmp/test -type d Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: chown - not like other Unixes.
Sam G wrote: > I was wondering why the gnu chown doesn't have an option to behave > like other unixes, where the 'owner' of a file can chown. FAQ: http://www.gnu.org/software/coreutils/faq/#Why-can-only-root-chown-files_003f ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
basename -bash generates an error
Hi there, I occasionally use basename $0 in scripts that I sometime source in my interactive shell. This construct seems to be broken now. Old behavior: red:~$ basename $0 -bash New behavior: orange:~/lib/sh$ basename $0 basename: invalid option -- b Try `basename --help' for more information. Looks like a misbehavior in option parsing. Randomly, Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: basename -bash generates an error
Thanks guys, you learn something new every day. Cheers, Brian Eric Blake <[EMAIL PROTECTED]> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > According to Brian Matheson on 4/18/2007 8:34 PM: > > Old behavior: > > red:~$ basename $0 > > -bash > > Broken. > > > > > New behavior: > > orange:~/lib/sh$ basename $0 > > basename: invalid option -- b > > Try `basename --help' for more information. > > Correct behavior, but undesired results. > > So what you really wanted to do: > > basename -- "$0" > > This correctly handles the case when $0 contains shell metacharacters, as > well as when it starts with -. > > > > > Looks like a misbehavior in option parsing. > > No, just a misunderstanding on your part. POSIX requires the current > behavior. > > - -- > Don't work too hard, make some time for fun as well! > > Eric Blake [EMAIL PROTECTED] > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.5 (Cygwin) > Comment: Public key at home.comcast.net/~ericblake/eblake.gpg > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFGJtzR84KuGfSFAYARAi9SAKC1lELw/+J6mLYBQ8hyIRcOAwWn9ACg1apm > cs2wFv7zfGaZDV95JD6nxYc= > =79ow > -END PGP SIGNATURE- ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: sort by line length
Jim Meyering wrote:
> You can do this in perl (or awk, ruby, etc) as a one-liner,
> so it may not be worth adding to a C application:
>
> echo 1 938 four aa a | fmt -1 \
> | perl -ne 'push @line, $_;sub END{print sort {length $a<=>length $b}
> @line}'
> 1
> aa
> 938
> four
> a
Not to derail on perl golfing but how about just:
$ perl -e 'print sort {length($a)<=>length($b)} <>' file
Brian
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: comm separator document tab
Dan Jacobson wrote: > comm documents don't mention that columns are made via tabs, and it > one is not happy with that what one can do about that (nothing, even > if one's data has tabs in it and one wants to use a different separator.) What do you mean? It says it right there: With no options, `comm' produces three-column output. Column one contains lines unique to FILE1, column two contains lines unique to FILE2, and column three contains lines common to both files. Columns are separated by a single TAB character. Of course this is the texinfo documentation, viewable with "info coreutils 'comm invocation'". If you're referring to the man page, that is just an autogenerated summary which is the same as the "comm --help" information, which is meant as a brief summary, not full documentation. So, in other words, use the info documentation. If you don't like the 'info' program there are numerous alternatives, the easiest of which is HTML: <http://www.gnu.org/software/coreutils/manual/html_node/comm-invocation.html>. The ability to specify something other than TAB would probably be a useful feature to have, though. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Potential Bug Report
[EMAIL PROTECTED] wrote: >I would like to report what I think is a bug: why the following linux > "date" command fails to run within a shell script. But, in runs fine on > the command line. I'm running 2.6.21-1.3194.fc7 on a IBM Thinkpad. I've > also tested on an IBM eServer running 2.6.9-55.ELsmp with the same > results. From the shell, it fails with an "extra operand" error. It seems > to misinterpreting the date string and/or formatting string. It's good that you let us know what kernel versions you're running, but that's sort of irrelevant. It would have been better to tell us what version of coreutils you're using, but in this case the problem isn't related to coreutils either. > TDATE="2007-06-01 01:01:01" > echo $TDATE > # > echo "Test#1" > date +%s -d $TDATE > #OR > echo "Test#2" > date -d $TDATE +'%s' > #OR > echo "Test#3" > EDATE=`date -d $TDATE +'%s'` > echo $EDATE > #OR > echo "Test#4" > EDATE=`date +'%s' -d $TDATE` > echo $EDATE This is not a date bug; it's not a shell bug either, it's a misunderstanding of how shell quoting works. The thing after -d must be specified to 'date' as a single argument. When you enter the command at the prompt, you accomplish this by surrounding it in quotes. However, in your script, the quoting is absent and so you end up passing each word of the date string as a separate argument, which is invalid input to date, which complains. Everywhere that you have $TDATE you need to use "$TDATE". Note that just using quotes when you assign to TDATE is not sufficient, because the shell re-parses the command when you invoke date and unless you use quotes there, it will not know that the multiple words contained in the variable should be kept together as one argument. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: aux.* as filename
Paul Eggert wrote: > Hmm, how is it that they are reserved for use by coreutils ports to > Windows, and yet Microsoft's POSIX tools can create files named "aux" > and the like? Is there something that the coreutils porters should > know? Microsoft says "programs control the policy for creating files > in Windows", which implies that coreutils should be able to Do The > Right Thing here. See: <http://support.microsoft.com/kb/120716>. I think this has to do with whether you are using the Win32 API or the Native API, i.e. the reserved-ness is at the Win32 layer, but if you use lower level calls you can create them. Cygwin does use the NT API for some but not all calls, and in fact there was even a release of Cygwin several years ago that accidently let users create files with these reserved words. However, all hell broke out on the mailing list because people were able to create these files that were absolutely un-removable by all normal means, because there are very few tools that don't use the Win32 layer, including the standard Microsoft ones e.g. Windows Explorer. It caused a lot of user confusion, and subsequent releases of Cygwin changed internally to disallow this again, so I think it would be considered a bug to go back to this behavior. (The Native layer is officially undocumented except for some common entry points such as NtCreateFile(), but there is enough unofficial reverse engineering such as <http://undocumented.ntinternals.net/> so that it is relatively well understood. As a very rough analogy to POSIX systems, using it might be likened to making direct int 0x80 syscalls on a POSIX system rather than going through the libc... i.e. somewhat ugly.) Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Can't set the timezone for "date".
Alan Mackenzie wrote: > I think it would be helpful if the manual said something like this. Even > coreutils.info just says (in a fairly buried place) "Normally, `date' > operates in the time zone indicated by `TZ', or the system default if > `TZ' is not set", without giving any indication of exactly how $TZ > indicates. I found this unhelpful and frustrating. The coreutils info page can't just say "run tzconfig" because that only applies to glibc systems, whereas coreutils is used on many other platforms as well -- and they all have different ways of setting the system time zone. Welcome to portable software, where documenting anything system-specific in a generic way is difficult if not impossible, unless you want to resort to a long list of "On Linux, do this; On Solaris, do this; On IRIX, do this; On HP-UX, do this; On Darwin, do this; On FreeBSD, do this; On Windows, do this." Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Command touch and irregular file names
- Tong - wrote: > In my script I have > > touch -r "$file1" "$file2" > > the file1/2 can be anything file, ../path/file, /root/file, etc. > > The problem is when file1/2 are irregular file names. E.g., -test.file1/2. > > I.e., anyway to make the following touch command works? > > $ touch -r "-test.file1" "-test.file2" > touch: invalid date format `est.file2' You can do: $ touch -r ./-test.file1 -- -test.file2 If you want to automate this from the script I suppose you could try something like: touch -r "$(readlink -f -- "$file1")" -- "$file2" That is, express file1 with a full pathname, and -- should work for $file2 since it is a non-option argument. > I think if the touch command uses the standard gnu getopt lib, then > the above code should work, shouldn't it? I don't think it should. It would require much more context for a generic options parser to speculatively treat certain groupings of options as filenames, test for existance of those files, and change behavior dynamically. That kind of thing would be rife with corner cases, unexpected outcomes (e.g. -file names a filename that's not supposed to exist), maintenance nightmares, etc. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: bug in /bin/test ?
Emanuele Tomasi wrote: > $> /usr/bin/test -n $casa && echo yes || echo no > yes > > $> /usr/bin/test $casa && echo yes || echo no > no > > $> /usr/bin/test -z $casa && echo yes || echo no > yes I think you may not be understanding the fact that the shell expands $casa into the empty string before invoking test, effectively swallowing it, thus the above are equivalent to: $ /usr/bin/test -n; echo $? 0 $ /usr/bin/test; echo $? 1 $ /usr/bin/test -z; echo $? 0 That is, the test program does not see any STRING argument at all. If you want to test an empty string you have to explicitly quote it in the shell so that an empty argument is actually passed to the program being invoked. Then you get the expected behavior: $ /usr/bin/test -n "$casa"; echo $? 1 $ /usr/bin/test "$casa"; echo $? 1 $ /usr/bin/test -z "$casa"; echo $? 0 You might have also seen the idiom 'test "x$foo" = x' which is another way of dealing with this problem, as well as several others such the case when $foo is "=" or a value that starts with a dash. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: what will happen to the --reply option?
alessandro salvatori wrote: > On top of that the only way for me to get the desired behaviour of skipping > existing files seems to be: See the previous thread from earlier this month: <http://lists.gnu.org/archive/html/bug-coreutils/2007-08/msg0.html> Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: chmod problem
John Gatewood Ham wrote: > I was under the impression 0700 was the octal code and would thus > result in 0700 permissions, not leaving 2700 permissions. How > can I use the octcal code to get 0700 permissions for a directory > with 2700 permissions? I tried 00700 but that didn't matter. I know > I can do 'chmod g-s testme' but since I can set the setgid flag > with octal codes I would like to be able to unset the setgid flag > with octal codes as well. I was able to achieve this in 2 steps > with 'chmod 0 testme' followed by 'chmod 0700 testme'. Can it be > done in 1 step using an octal number? Read these threads for the past history on this behavior: http://lists.gnu.org/archive/html/bug-coreutils/2006-07/msg00125.html http://lists.gnu.org/archive/html/bug-coreutils/2007-05/msg00132.html Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: feature request: zero-terminated lines for comm
James Youngman wrote: > While that is true, those utilities already coped with the space > character (for example, with "find -print" and "xargs -L"). xargs cannot cope with filenames containing spaces without -0/-print0. $ echo -e "one two\nthree four" | xargs showargs argv[0] = 'showargs' argv[1] = 'one' argv[2] = 'two' argv[3] = 'three' argv[4] = 'four' I don't see how the xargs -L option has anything to do with the situation. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: feature request: zero-terminated lines for comm
James Youngman wrote:
> I beg your pardon. Of course I meant -I.
>
> $ printf "onetwo\nthree four\n" | xargs -I {} showargs {}
> argv[0] = /home/youngman/bin/showargs
> argv[1] = onetwo
> argv[0] = /home/youngman/bin/showargs
> argv[1] = three four
I see. But, using -I forces one invocation of the command for every
argument which is a huge efficiency hit. You might as well stop using
xargs at that point and just do it with the shell as "| (while read F;
do command "$F"; done)" because you've lost the entire advantage of
using xargs. So I still maintain that xargs without -print0/-0 is
deficient.
Brian
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bugreport for ChMod
Michael Gorbovitski wrote: > Maybe we could add several examples, to either the help or the man page? > > And no, neither the man page nor the info page is much better. What are you talking about? The info page has an entire section on this: $ info coreutils "Symbolic modes" -o - --subnodes 2>/dev/null | wc 28815289654 That's a fifteen-hundred word explanation on the syntax and meaning behind symbolic modes, and there are 19 examples by my count. --help output and man pages are supposed to be short and concise. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: linecut development
Steven Schubiger wrote: > I'm currently struggling how to make pipe support work correctly, because > one needs to know beforehand how many lines of input are available (to > turn relative range line positions into their absolute counterparts), > which implies reading through the entire STDIN stream. Once this is done, > I see no way to re-read the same stream again. > > These are the solutions (that come to mind): > > a) allocate memory for each line and buffer accordingly > b) disallow inverted (relative to EOF) line positions > c) a sophisticated algorithm > d) ... > > I haven't decided yet how to proceed and would like to collect some opinions. GNU policy is that you can't have limits such as requiring the whole input to fit in memory, so there's really no choice but to copy stdin to a temporary file while you count lines. As an optimization I suppose you could spool the input into a memory buffer and only switch to filesystem-backed storage once the size exceeds some threshold. Not being able to support EOF-relative ranges seems like it would be a major loss, and isn't the reason for writing this (as opposed to just using e.g. sed) to be able to have a robust and featureful implementation? So if you ask me, there's no way to avoid having to do this dirtywork. I think this is part of the more general pattern that lots of people suggest adding their own pet utilities to coreutils but very few end up making it in, because the project sets quite a high standard: no size/space limitations, full texinfo and --help/man documentation, testcases, implementing features not found in the existing unix toolset, etc. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: coreutils: Where is the repository of the manual pages?
Jari Aalto wrote: > Could you point me to the manual (info?) sources and document > which explains the build procedure for *.N files. The .x files are just skeletons. The meat of the man pages are generated automatically from the --help output of each binary with the help2man script. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Bug in cat
"Cutler, David" wrote:
> When I downloaded Cygwin, I specified the use of DOS line terminators.
>
> I found a bug in what I suspect is cat when used with a simple bash shell
> script.
Since you're using Cygwin, the Cygwin mailing list is a better place to
post this because most coreutils developers don't use Cygwin.
This is not technically a bug however, because cat always works in
binary mode by design regardless of mount mode. This is required by
POSIX I believe. Eric has mentioned in the past adding a
Cygwin-specific --text option to cat but any time you start to maintain
special patches it makes maintainer burden go up. Even then, you would
have to give the option explicitly.
And besides, there are much better ways to deal with this:
> And you execute the command:
>for I in `cat list` ; do echo \"${i}\" ; done
Inefficient. Instead you can simply:
while read i; do echo \"${i}\"; done http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [cp v5.97] --noreply erroneously depreciated
Chris Velevitch wrote: > I thought rsync is meant for copying files between machines? In what There's no requirement that rsync can only be used between two different machines. > It did have it, > until it got depreciated. No, it didn't have such an option. Go read those old threads. --reply=no would *only* have worked in those situations where mv would have prompted the user. That means when -i was also specified and when stdout was a terminal. But for example if -i wasn't given or mv was used in a batch/cron situation with stdout redirected it would *not* have prompted, and thus the file would be overwritten despite the user giving --reply=no. This was the source of much confusion and the reason why it the option was deprecated, because it did work in the manner that people expected. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Canceling interactive mode - cp
August Zajonc wrote: > I'm used to be able to do something like this > > alias cp=cp -i I think the common opinion around here is that -i aliases like that are evil and wrong. They encourage bad behavior like training your muscle memory to always reply "y" without thinking (thus defeating the whole purpose of the prompt), and can get you in trouble when you use a standard system that has no such crutches. > then when I need to overwrite a file on the destination of a copy, I > usually do -bf > > This makes a backup but avoids the prompt, perfect for bash scripts and > similiar. > > Recently I've noticed that I can't cancel -i. There seems to be NO > command to reverse the -i. -f is not supposed to cancel -i, by design. > What's the suggested workaround in these situations. This seems like it > would be a pretty common approach, and given the somewhat more esoteric > options, it's odd that one can't reverse or cancel the effect of -i. The behavior is required by the standards: <http://lists.gnu.org/archive/html/bug-coreutils/2007-03/msg00093.html>. The suggested approach is to invoke the non-aliased version of the command: <http://lists.gnu.org/archive/html/bug-coreutils/2007-03/msg00099.html> (or better, not have any troublesome aliases in the first place, but obviously people are free to do whatever works best for them.) Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: argve0, psfool
"Daniel C. Bastos" wrote:
> I always miss these two programs on every system I meet. argv0 is very
> handy when dealing with programs that care about argv[0] and psfool is
> essential when giving out passwords through the command line. I figure
> these two should be in coreutils.
perl -e 'exec { "real" } "fake", "arg1", "arg2"'
As an observer of this list I have noted that coreutils maintainers are
highly reticent to add new commands to coreutils that are trivially
implemented with existing standard commands.
Brian
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
coreutils 6.9 "cp -p" fails on leopard (erroneous error message)
I'm running coreutils 6.9 on leopard 10.5.1. % cp file1 file2 works correctly. However: % cp -p file1 file2 cp: 'file1' no such file or directory What's really strange is that the cp ***does*** correctly occur (with permissions being preserved). Why the error message? If I compile the same code on tiger and run it on leopard I don't get that error message (and the copy is done correctly). ??? --Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
coreutils 6.9 "cp -p" failure on Linux 2.6
I ran into the same cp -p timestamp problem reported earlier (from another site): http://lists.gnu.org/archive/html/bug-coreutils/2007-11/msg00047.html i.e., if one does a cp -p on Linux 2.6, timestamps are not preserved when the copy is done ***into*** an NFS mounted file system. Someone suggested recompiling coreutils with HAVE_BUGGY_NFS_TIME_STAMPS defined. I did recompile with that define and it did fix the problem. Just another data point. --Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: coreutils 6.9 "cp -p" fails on leopard (erroneous error message)
Hi, Jim. coreutils-6.9.92 configured normally has the same problem. After configuring with --disable-acl, though, it works (no error message with cp -p)! We (UNC-Chapel Hill) run coreutils (actually, all of our shared applications software) over AFS so disabling acl's is probably not something we should do, though. ??? --Brian On Jan 17, 2008, at 3:49 AM, Jim Meyering wrote: Brian Biswas <[EMAIL PROTECTED]> wrote: I'm running coreutils 6.9 on leopard 10.5.1. % cp file1 file2 works correctly. However: % cp -p file1 file2 cp: 'file1' no such file or directory What's really strange is that the cp ***does*** correctly occur (with permissions being preserved). Why the error message? If I compile the same code on tiger and run it on leopard I don't get that error message (and the copy is done correctly). Thanks for the report. Would you please see if you can reproduce that with the latest test release? ftp://alpha.gnu.org/gnu/coreutils/coreutils-6.9.92.tar.gz ftp://alpha.gnu.org/gnu/coreutils/coreutils-6.9.92.tar.lzma If so, please try again, but this time start by running ./configure --disable-acl ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Building portions of coreutils
NightStrike wrote: > Thanks :) I agree that mingw is a very hard platform to work with. > My intent is to just get enough programs working to be able to > configure and build the toolchain (binutils, gcc, and gdb). I don't see how you have any hope of doing that. The whole reason for the creation of MSYS was to do exactly what you just said. If it were possible to build coreutils (and bash, etc) without an emulation layer like MSYS then it would never have been created. Brian ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
troubles chrooting oidentd
Hello all, I'm in the process of creating a chroot jail for oidentd on my Gentoo box, but I'm running into some problems. oidentd runs find just by itself, but it refuses to run in the chrooted environment. Here's what I've done: File Structure: /path_to_chroot_home/ bin dev etc lib tmp usr sbin var run The chrooted etc contains: group passwd shadow ld.so.cache (regenerated in the environment) /usr/sbin contains the oidentd executable. dev contains: crw-rw-rw- 1 root root 1, 3 Sep 2 15:00 null crw-rw-rw- 1 root root 1, 5 Sep 2 18:05 zero lib contains all the library files which were outputted by 'ldd /usr/sbin/oidentd' and 'lsof | grep oidentd', while oidentd was running of course. I realize I don't have syslogging setup, but that doesn't matter, because I'm running oidentd with the -S and -q flags, which suppress logging. I'll turn them on later... oidentd DOES run fine in the chrooted environment without any flags. But of course it runs as root, which is completely undesirable. As soon as I add -u1002 -g1002, nothing happens. No output, no errors, just nothing. a 'ps -ef | grep oidentd' shows nothing. My chrooted /etc/passwd and /etc/group contain the appropriate user and group, so there should be no problem. What am I missing? Do I have to regenerate the passwd database somehow? I know I'm missing something small and stupid, but all my googling says that it should be up and running. Any help would be appreciated. Thanks! -- Brian Kreulen "We shall not cease from exploration And the end of all our exploring Will be to arrive where we started And know the place for the first time" --T.S. Eliot ___ Bug-coreutils mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/bug-coreutils
bug#20575: possible bug with false?
'false --help' and 'false --version' print nothing and return an error. I honestly don't know if it's working as intended. If not, the man page needs to be updated.
bug#26766: Bug found while using 'tail' in a Docker container
The program 'tail' just reported a bug, and requested that I send in a bug report. This is the line: tail: unrecognized file system type 0x794c7630 for 'stderr.log'. please report this to bug-coreutils@gnu.org. reverting to polling I am using a bash shell inside a Docker container (Ubunut 14.04 as the client OS). The command I executed is: tail -f stderr.log I am happy to provide followup information or testing. Thanks, Brian Drawert
bug#33433: Bug in directory listing display
Hello, On a recent install of Ubuntu I discovered a new bug in 'ls' which I understand you guys maintain. Items with spaces are incorrectly listed surrounded by single quotes. This is problematic for a number of reasons. One of which is that files or directories that contain a mix of quotes in their titles are now displayed incorrectly. This bug also breaks existing scripts all over the place. Please resolve this bug on the next release. Brian Hartvigsen
bug#36383: date command processes timezone differently when doing math
When doing a math operation to a date command it appear to process the timezone differently. The system is Ubuntu 18.04.2 LTS. Versions: coreutils/bionic,now 8.28-1ubuntu1 amd64 [installed] #echo $datNow 2019-06-25 15:21:34 #date -d "$datNow " "+%Y-%m-%d %H:%M:%S" --debug date: parsed date part: (Y-M-D) 2019-06-25 date: parsed time part: 15:21:34 date: input timezone: system default date: using specified time as starting value: '15:21:34' date: starting date/time: '(Y-M-D) 2019-06-25 15:21:34' date: '(Y-M-D) 2019-06-25 15:21:34' = 1561501294 epoch-seconds date: timezone: system default date: final: 1561501294.0 (epoch-seconds) date: final: (Y-M-D) 2019-06-25 22:21:34 (UTC) date: final: (Y-M-D) 2019-06-25 15:21:34 (UTC-07) 2019-06-25 15:21:34 #date -d "$datNow + 1 minute" "+%Y-%m-%d %H:%M:%S" --debug date: parsed date part: (Y-M-D) 2019-06-25 date: parsed time part: 15:21:34 UTC+01 date: parsed relative part: +1 minutes date: input timezone: parsed date/time string (+01) date: using specified time as starting value: '15:21:34' date: starting date/time: '(Y-M-D) 2019-06-25 15:21:34 TZ=+01' date: '(Y-M-D) 2019-06-25 15:21:34 TZ=+01' = 1561472494 epoch-seconds date: after time adjustment (+0 hours, +1 minutes, +0 seconds, +0 ns), date: new time = 1561472554 epoch-seconds date: timezone: system default date: final: 1561472554.0 (epoch-seconds) date: final: (Y-M-D) 2019-06-25 14:22:34 (UTC) date: final: (Y-M-D) 2019-06-25 07:22:34 (UTC-07)
bug#50972: src/ls.c fails to build when __GNUC_PREREQ is not defined (e.g., OpenBSD)
Hello -- I am the GNU coreutils package maintainer for OpenBSD. While updating the OpenBSD package to 9.0, the build errored out while compiling src/ls.c with the following error message: src/ls.c:2276:6: error: function-like macro '__GNUC_PREREQ' is not defined I noticed that there are some #ifdefs defining and using the macro in the lib/ directory but clearly the definition is not reaching into src/. OpenBSD has a __GNUC_PREREQ__ macro defined in but not the __GNUC_PREREQ macro. If the preprocessor checks and associated pragmas are removed from src/ls.c then it builds OK. OpenBSD package builders use clang 11.1.0 and clang does not trigger the bogus warnings that these __GNUC_PREREQ-wrapped pragmas are defending against. Obviously that is not a universal fix, just a data point. Once this fix is done, the rest of coreutils-9.0 builds and works OK. There are no test failures for coreutils-9.0 on x86_64-unknown-openbsd7.0 Thanks. ~Brian
bug#50972: src/ls.c fails to build when __GNUC_PREREQ is not defined (e.g., OpenBSD)
Hi Paul -- On 10/02/2021 10:07 PM, Paul Eggert wrote: The first patch listed above should fix the problem you mentioned; you might want to look at the other patches as well. Much appreciated. Can confirm the tip of the tree builds OOTB now. By the way, how do you deal with all those false positives from clang? Do you suppress them with CFLAGS? I'm referring to the many warnings like "comparison is always true due to limited range of data type" and a few "integer overflow in expression" warnings. With so many false alarms I wouldn't be surprised if people stopped paying attention to compiler warnings even if they're valid. OpenBSD 7.0 (which is ready to go, just hasn't had an official announcement) ships with clang 11.1.0, whereas OpenBSD 6.9 ships with clang 10.0.1. I did not try to build coreutils-9.0 on 6.9 but on 7.0 there are virtually no warnings during the build (even before these fixes). I can send you a build log offlist if you'd like to see what the 7.0 build looks like. ~Brian
bug#50972: src/ls.c fails to build when __GNUC_PREREQ is not defined (e.g., OpenBSD)
On 10/03/2021 01:42 PM, Paul Eggert wrote: On 10/2/21 8:14 PM, Brian Callahan wrote: I can send you a build log offlist if you'd like to see what the 7.0 build looks like. It sounds like the more-recent clang has fixed most of the false alarms. It'd probably be a more-efficient use of our time for us to wait until 7.0 is out, so that I can then run 7.0 in a VM myself (as debugging via email is a pain) and look into the remaining diagnostics. If recent experience is any guide they'll be false alarms too, so this isn't urgent. Sounds good. Thanks again for the quick fix. ~Brian
bug#73865: cp 9.1+ fails to overwrite existing files even with -f under Cygwin
Hi folks,
Versions of coreutils cp from 9.1-9.5 have been failing with error message:
"cp: cannot create regular file '/tmp/b/c': File exists"
to overwrite existing files even with -f under Cygwin.
I have been trying to run down how and where cp is missing a check that the file
exists by tracing and with gdb without anything obvious grabbing my attention
and raising suspicions.
Any advice for changing my approach, or where specifically to concentrate, could
be helpful in progressing a new release.
A couple of Cygwin specific patches have been applied over many releases with
rebases to root inode handling and allowing case insensitivity that do not
appear to have had any other effect on the flow or operations.
--
Take care. Thanks, Brian Inglis Calgary, Alberta, Canada
La perfection est atteinte Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut
-- Antoine de Saint-Exupéry#!/bin/sh
# cp-9.1-fails-file-exists.sh
#set -vx
days=7
ver=9.1
ls_opt=-glo
rc=0
tmp=${TMPDIR:-${TMP:_${TEMP:-/tmp}}}
bin=coreutils-$ver-1.$(arch)/inst/usr/bin
for d in $tmp $bin
do
[ ! -d $d/ ] && echo "${0##*/}: $d: not found" >&2 && rc=2
done
[ 0 -eq $rc ] || exit $rc
export PATH="$PWD/$bin:$PATH"
for u in cp echo ls mkdir rm tee touch
do
if [ ! -s $bin/$u ] || [ ! -x $bin/$u ]
then
echo "${0##*/}: $u: not found" >&2 && rc=2
fi
done
[ 0 -eq $rc ] || exit $rc
for dir in a b
do
rm -fr $tmp/$dir/*
mkdir -p $tmp/$dir/
for f in c # dd eee f.fff gg.ggg hhh.hhh iii k
do
echo $f > $tmp/$dir/$f
done
days=$(($days+7))
done
for f in 0 y
do
echo $f > $tmp/a/$f
done
for f in 1 z
do
echo $f > $tmp/b/$f
done
ls $ls_opt $tmp/a/ $tmp/b/
for cp in $bin/cp /bin/cp
do
for f in $tmp/a/*
do
for p in "echo run:" command strace
do
[ "strace" = "$p" ] && o=" -o cp-${f##*/}.strace" || o=''
$p$o $cp -fv$tmp/a/*$tmp/b/
[ 0 -eq $? ] && [ "command" = "$p" ] && break
done
done
ls $ls_opt $tmp/b/
done
/tmp/a/:
total 3
-rw-r--r-- 1 2 Oct 18 10:26 0
-rw-r--r-- 1 2 Oct 18 10:26 c
-rw-r--r-- 1 2 Oct 18 10:26 y
/tmp/b/:
total 3
-rw-r--r-- 1 2 Oct 18 10:26 1
-rw-r--r-- 1 2 Oct 18 10:26 c
-rw-r--r-- 1 2 Oct 18 10:26 z
run: coreutils-9.1-1.x86_64/inst/usr/bin/cp -fv /tmp/a/0 /tmp/a/c /tmp/a/y
/tmp/b/
'/tmp/a/0' -> '/tmp/b/0'
'/tmp/a/c' -> '/tmp/b/c'
coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create regular file '/tmp/b/c':
File exists
'/tmp/a/y' -> '/tmp/b/y'
'/tmp/a/0' -> '/tmp/b/0'
/usr/src/coreutils/coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create
regular file '/tmp/b/0': File exists
'/tmp/a/c' -> '/tmp/b/c'
/usr/src/coreutils/coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create
regular file '/tmp/b/c': File exists
'/tmp/a/y' -> '/tmp/b/y'
/usr/src/coreutils/coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create
regular file '/tmp/b/y': File exists
run: coreutils-9.1-1.x86_64/inst/usr/bin/cp -fv /tmp/a/0 /tmp/a/c /tmp/a/y
/tmp/b/
'/tmp/a/0' -> '/tmp/b/0'
coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create regular file '/tmp/b/0':
File exists
'/tmp/a/c' -> '/tmp/b/c'
coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create regular file '/tmp/b/c':
File exists
'/tmp/a/y' -> '/tmp/b/y'
coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create regular file '/tmp/b/y':
File exists
'/tmp/a/0' -> '/tmp/b/0'
/usr/src/coreutils/coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create
regular file '/tmp/b/0': File exists
'/tmp/a/c' -> '/tmp/b/c'
/usr/src/coreutils/coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create
regular file '/tmp/b/c': File exists
'/tmp/a/y' -> '/tmp/b/y'
/usr/src/coreutils/coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create
regular file '/tmp/b/y': File exists
run: coreutils-9.1-1.x86_64/inst/usr/bin/cp -fv /tmp/a/0 /tmp/a/c /tmp/a/y
/tmp/b/
'/tmp/a/0' -> '/tmp/b/0'
coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create regular file '/tmp/b/0':
File exists
'/tmp/a/c' -> '/tmp/b/c'
coreutils-9.1-1.x86_64/inst/usr/bin/cp: cannot create regular file '/tmp/b/c':
File exists
'/tmp/a/y' -&g
bug#73865: cp 9.1+ fails to overwrite existing files even with -f under Cygwin
On 2024-10-18 11:32, Paul Eggert wrote: On 2024-10-18 09:47, Brian Inglis wrote: Hi folks, Versions of coreutils cp from 9.1-9.5 have been failing with error message: "cp: cannot create regular file '/tmp/b/c': File exists" to overwrite existing files even with -f under Cygwin. What does strace say? Thanks Paul, That made me think to redo and reread the straces and I noticed that the basename was being accessed in the builddir, as patches were now accessing the basename not the realpath, so mea culpa! That required a bunch of rebasing patches, and that now appears to work. However, I now appear to have the issue that the Cygwin emulated "root" is not being detected and handled properly, so during tests, chgrp/chown/chmod --preserve-root is being ignored, also presumably rm!, tests are taking forever, and all my account and CI system mount points are being trashed by the tests, and build/gt-preserve-root.sh.XKPb/out is over 1GB! I take it that should go into a different bug report, as I need to understand whether any access to "root" is just warned and skipped, or the command is terminated with prejudice. The info docs section "Treating ‘/’ specially" appears to be ambiguous about the ch??? commands, whereas the man pages say "fail to operate recursively on ’/’". Cygwin would probably want to default these commands to --preserve-root like rm, and also require -f --force to override, but ch??? redefine -f to --silent --quiet! -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut -- Antoine de Saint-Exupéry
bug#74094: chgrp/chmod/chown fails to recognize "root" and respect --preserve-root under Cygwin
Hi folks, I now appear to have the issue that the Cygwin emulated "root" is not being detected and handled properly, so during tests, chgrp/chown/chmod --preserve-root is being ignored, also presumably rm!, tests are taking forever, and all my account and CI system mount points are being trashed by the tests, and build/gt-preserve-root.sh.XKPb/out is over 1GB! I need to understand whether any access to "root" is just warned and skipped, or the command is terminated with prejudice, as I would expect. The info docs section "Treating ‘/’ specially" appears to be ambiguous about the ch??? commands, except that they default to --no-preserve-root whereas the man pages say "fail to operate recursively on ’/’", as I would expect. I know that I will have to handle the "root" issue, but advice on how the commands are conceptually expected to deal with "root" and --preserve-root would be useful, and any gotchas on other systems, before I dig into then hack at the operations. Cygwin would probably want to default these ch??? commands to --preserve-root like rm, and also require -f, --force --no-preserve-root to override, but ch??? redefine -f to --silent, --quiet, so would have to mandate the long --force option with --preserve-root, and also change the docs! -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut -- Antoine de Saint-Exupéry
chown clears suid bit!
Hi, I am pre-apologizing as I'm about to sound like yet another ingrateful user. :) sorry! I just spent litterally all day tracking down a bizzarre inexplicable brokenness trying to get a new box up & running. Scripts that have worked for years on many OS's (sco osr, sco unixware, solaris, linux, freebsd, hpux...) suddenly didn't work on a new SuSE 10 box. (namely, the setperms script that ships with filePro (commrcial database and 4gl) on several unices and hasn't changed significantly since '92, and no changes since '00) Skipping all the blind alleys I chased down, the end result was that chown now clears the suid bit from files it acts upon. As the man page doesn't mention this, and it's certainly new behaviour, and certainly deviant from every other unix, I submit that it should not do this. Thanks Brian K. White -- [EMAIL PROTECTED] -- http://www.aljex.com/bkw/ +[>+++[>+>+++<<-]<-]>>+.>.+.+++.-.[>+<---]>++. filePro BBxLinux SCO FreeBSD#callahans Satriani Filk! ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: chown clears suid bit!
Well that does explain why it didn't matter whether I used a shells built-in
chown like ksh93, or /bin/chown it always acted the same.
Your explanations do make sense but it sure was a surprise is all. I wasn't
bs'ing when I said the same script has been unchanged for years, and worked
on several platforms, including linux, for years.
Perhaps the fact that it's always root that runs this, and previously (or
still on other platforms) root was treated specially?
Maybe it's still supposed to do that and there is a kernel bug now that
decides even root shouldn't be allowed to chown a file without clearing any
suid?
Also there's the fact that 99% of the time the chown commands in this
particular script end up being no-ops because the script is intended to
correct errors, and normally all the files would already be what the script
wants, and maybe previously in the case of a no-op it really did (what I
think) the more sensible thing, nothing.
Anyways. As long as it's documented and can be counted on not to change (any
more), then it's fine...
Thanks much
Brian K. White -- [EMAIL PROTECTED] -- http://www.aljex.com/bkw/
+[>+++[>+>+++<<-]<-]>>+.>.+.+++.-.[>+<---]>++.
filePro BBxLinux SCO FreeBSD#callahans Satriani Filk!
- Original Message -
From: "Paul Eggert" <[EMAIL PROTECTED]>
To: "Brian K. White" <[EMAIL PROTECTED]>
Cc:
Sent: Thursday, April 13, 2006 2:11 AM
Subject: Re: chown clears suid bit!
"Brian K. White" <[EMAIL PROTECTED]> writes:
chown now clears the suid bit from files it acts upon.
This tends to be a property of the kernel, not of the chown
executable. You can verify this by running "strace chown user file"
on your GNU/Linux host. In the example below, the command "chown
eggert at" executed the chown32 system call as requested, but the
kernel decided to clear the setuid bit. Hence the bug (if there is
one) is in the kernel, not in coreutils.
penguin:/tmp# pwd
/tmp
penguin:/tmp# cp -p /usr/bin/at .
penguin:/tmp# ls -l at
-rwsr-xr-x 1 root root 34488 Jan 18 2002 at
penguin:/tmp# strace -o tr chown eggert at
penguin:/tmp# tail tr
munmap(0x40018000, 1130)= 0
close(3)= 0
lstat64("at", {st_mode=S_IFREG|S_ISUID|0755, st_size=34488, ...}) = 0
open(".", O_RDONLY|O_NONBLOCK|O_NOCTTY|O_LARGEFILE|O_DIRECTORY) = 3
fchdir(3) = 0
chown32("at", 1000, -1) = 0
fchdir(3) = 0
close(3)= 0
close(1)= 0
exit_group(0) = ?
penguin:/tmp# ls -l at
-rwxr-xr-x 1 eggert root 34488 Jan 18 2002 at
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: chown clears suid bit!
- Original Message -
From: "Bob Proulx" <[EMAIL PROTECTED]>
To: "Brian K. White" <[EMAIL PROTECTED]>
Cc:
Sent: Thursday, April 13, 2006 11:42 AM
Subject: Re: chown clears suid bit!
Brian K. White wrote:
Your explanations do make sense but it sure was a surprise is all. I
wasn't
bs'ing when I said the same script has been unchanged for years, and
worked
on several platforms, including linux, for years.
I think if you took your new system running the new linux 2.6 kernel
and booted an older linux 2.4 kernel on it you would find that the
behavior follows the kernel version. Your previous systems were
probably linux 2.4 kernels. This is probably the first time your
script has run on the new linux 2.6 kernel.
Also there's the fact that 99% of the time the chown commands in this
particular script end up being no-ops because the script is intended to
correct errors, and normally all the files would already be what the
script
wants, and maybe previously in the case of a no-op it really did (what I
think) the more sensible thing, nothing.
I use the following technique in my shell scripts that do similar
things. Hope you find it interesting.
mode=$(ls -dlnL "$i" | awk '{print$1}')
case $mode in
?r??r??r??*) : file permissions $mode is okay ;;
*) chmod a+r "$i" ;;
esac
ownergroup=$(ls -dlnL "$i" | awk '{print$3$4}')
case $ownergroup in
00) : file ownership $ownergroup is okay ;;
*) chown root:root "$i" ;;
esac
I don't change anything unless it needs to be changed. This way the
file ctime is not continuously being updated. Admins looking at what
changed today on the system won't find things that did not actually
change today. Although the matching technique is not completely
portable, some systems will print different information there, it
seems to be good enough in practice.
Also the use of intermediate variables here provide convenient tracing
with 'sh -x script' so that the -x tracing output shows me what is
happening relatively easily when developing and debugging.
Bob
Thats a great idea and I think I will incorporate it. Thanks.
There are some cases where it'll be hard to do though. This script for
example does a couple sweeping chown -R / chmod -R at the top of a tree of
around 30,000 files.
Replacing the -R with any kind of loop might make it take too long to run.
There are some other large sweeping passes that are a little more selective
that are based on find |xargs
I'll try it though, maybe it'll be fast as long as I avoid and screen output
and make a mychmod() function that does a "while read a b c d e f ... do ...
done", and I can pipe ls -lR into that so I only run ls once, and the while
loop gets both perms and filename in one shot and can simply compare values
with no need to execute any processes inside the loop 99% of the time.
That should be good enough. The same new way of ls | mychown() will work
the same way to replace both the chown -R and the find |xargs chown too.
It'll be harder to deal with bad filenames (spaces & other speacial
characters) than with -R of course but not terribly I think.
Food for thought, thanks.
Brian K. White -- [EMAIL PROTECTED] -- http://www.aljex.com/bkw/
+[>+++[>+>+++<<-]<-]>>+.>.+.+++.-.[>+<---]>++.
filePro BBxLinux SCO FreeBSD#callahans Satriani Filk!
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: chown, cpio: proposed change for userspec handling of USER:
Jim Meyering wrote:
Hello,
While writing a few tests for userspec (below), I was surprised to
re-learn that chown USER_NAME: has a special meaning. It is a
shorthand for chown USER_NAME:+$(id -g USER_NAME) ...
I had expected it to be equivalent to this:
chown USER_NAME ...
Since the above behavior is not specified by POSIX, and
is IMHO, counter-intuitive, I propose to change it. However,
it is documented both in coreutils and in cpio's manuals.
Here's the patch, followed by the coreutils part,
but I suppose I can't really apply them as-is.
Rather, I may make "chown USER_NAME: ..." warn-for-now yet
continue to work and "chown NUMERIC: ..." work like "chown NUMERIC ...".
Then, in say two years, I'd make the actual change.
Sergio, would you accept a similar preparatory patch for cpio?
Opinions?
BTW, just realized that changing userspec semantics like this
would require a NEWS update, too... But we're not there yet.
Also, regardless, I now expect to adjust this new test
module to test the current behavior.
From b86a5bd9960c33b753f7963ceb1502a34255f3f4 Mon Sep 17 00:00:00 2001
From: Jim Meyering
Date: Wed, 2 Dec 2009 20:01:36 +0100
Subject: [PATCH 1/2] userspec: do not treat USER: specially
* lib/userspec.c: Adjust semantics not to treat USER: specially.
GNU chown has had a "feature" whereby specifying "USER:" would make
chown change owner to the non-numeric "USER", and at the same time
change the group to the login group of USER. By a similar token,
since it's not generally possible to map a numeric USER to a user
name, chown would reject a numeric "USER:". Now, "USER:" is
treated the same as "USER", for both numeric and non-numeric USER.
---
ChangeLog | 11 +++
lib/userspec.c | 21 +
2 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 6eec830..c4098c0 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2009-12-02 Jim Meyering
+
+ userspec: do not treat USER: specially
+ * lib/userspec.c: Adjust semantics not to treat USER: specially.
+ GNU chown has had a "feature" whereby specifying "USER:" would make
+ chown change owner to the non-numeric "USER", and at the same time
+ change the group to the login group of USER. By a similar token,
+ since it's not generally possible to map a numeric USER to a user
+ name, chown would reject a numeric "USER:". Now, "USER:" is
+ treated the same as "USER", for both numeric and non-numeric USER.
+
2009-12-01 Jim Meyering
fts: fts_open: do not let an empty string cause immediate failure
diff --git a/lib/userspec.c b/lib/userspec.c
index 0388cb1..e12b567 100644
--- a/lib/userspec.c
+++ b/lib/userspec.c
@@ -105,7 +105,6 @@ parse_with_separator (char const *spec, char const
*separator,
{
static const char *E_invalid_user = N_("invalid user");
static const char *E_invalid_group = N_("invalid group");
- static const char *E_bad_spec = N_("invalid spec");
const char *error_msg;
struct passwd *pwd;
@@ -158,22 +157,12 @@ parse_with_separator (char const *spec, char const
*separator,
pwd = (*u == '+' ? NULL : getpwnam (u));
if (pwd == NULL)
{
- bool use_login_group = (separator != NULL && g == NULL);
- if (use_login_group)
-{
- /* If there is no group,
- then there may not be a trailing ":", either. */
- error_msg = E_bad_spec;
-}
+ unsigned long int tmp;
+ if (xstrtoul (u, NULL, 10, &tmp, "") == LONGINT_OK
+ && tmp <= MAXUID && (uid_t) tmp != (uid_t) -1)
+unum = tmp;
else
-{
- unsigned long int tmp;
- if (xstrtoul (u, NULL, 10, &tmp, "") == LONGINT_OK
- && tmp <= MAXUID && (uid_t) tmp != (uid_t) -1)
-unum = tmp;
- else
-error_msg = E_invalid_user;
-}
+error_msg = E_invalid_user;
}
else
{
--
1.6.6.rc0.324.gb5bf2
From 33cf9804ac5593a267d9c76482d5a5c85b01db84 Mon Sep 17 00:00:00 2001
From: Jim Meyering
Date: Sat, 28 Nov 2009 11:51:08 +0100
Subject: [PATCH 2/2] userspec-tests: test the userspec module
* tests/test-userspec.c: New file.
* modules/userspec-tests: Likewise.
---
ChangeLog |4 ++
modules/userspec-tests | 10
tests/test-userspec.c | 139
3 files changed, 153 insertions(+), 0 deletions(-)
create mode 100644 modules/userspec-tests
create mode 100644 tests/test-userspec.c
diff --git a/ChangeLog b/ChangeLog
index c4098c0..20720d0 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -34,6 +34,10 @@
2009-11-28 Jim Meyering
+ userspec: add unit tests
+ * tests/test-userspec.c: New file.
+ * modules/userspec-tests: Likewise.
+
userspec: depend on the inttostr module, too
* modules/us
bug#44704: uniq: replace repeated lines with a message about how many repeated lines
It would be a useful enhancement to uniq to replace all lines considered non-uniq (i.e. those that would be removed from the output) with a message about how many times the previous line was repeated. I.e. $ cat < signature.asc Description: This is a digitally signed message part
