Just ran into some weirdness that seems a bit "off" to me. Perhaps there
is a logical explanation, related to timing and the timing is causing the
weirdness, but I thought I'd run it by the list and see what you guys
thought.
I started with 2 files about 250MB in length (random, compressed data)
and wanted to create a 256MB file. I thought to concatenate the
256MB files with cat and use dd to read a fixed amount to an output
file:
cat testfile-250M testfile.gz | dd of=250MB bs=1M count=256
Is something "wrong" with the above"?...because it didn't work.
What I got (tried it multiple times and got different results
each time, comments are my "commentary, I combined the two
"dd output lines" in this email to save vertical space):
# started with parens around cat
test> (cat testfile-250M testfile.gz )|dd of=256MB bs=1M count=256
49+207 records in / 49+207 records out
65523712 bytes (66 MB) copied, 0.590972 s, 111 MB/s
#huh? is output really only 66MB?
test> ll 256MB
-rw-rw---- 1 65523712 Jan 18 15:33 256MB
#yep... remove parens; shouldn't be needed anyway
test> cat testfile-250M testfile.gz |dd of=256MB bs=1M count=256
65+191 records in / 65+191 records out
81899520 bytes (82 MB) copied, 0.741014 s, 111 MB/s
# !! better, but still bad; make sure output is gone -
# shouldn't make difference, but...
test> rm 256MB
test> cat testfile-250M testfile.gz |dd of=256MB bs=1M count=256
94+162 records in / 94+162 records out
105992192 bytes (106 MB) copied, 0.264301 s, 401 MB/s
# obviously that didn't do it...
test> cat testfile-250M testfile.gz |dd of=256MB bs=1M count=256
71+185 records in / 71+185 records out
89989120 bytes (90 MB) copied, 0.801108 s, 112 MB/s
# it's not even the same "random" amount each time
test> cat testfile-250M testfile.gz |dd of=256MB bs=1M count=256
50+206 records in / 50+206 records out
68370432 bytes (68 MB) copied, 0.614398 s, 111 MB/s
# I went off and created a 1000M file using cat:
test> lh testfile-1000M
-rw-rw---- 1 1000M Jan 18 15:28 testfile-1000M
# continuing w/cat
test> cat testfile-1000M |dd of=256MB bs=1M count=256
107+149 records in / 107+149 records out
124542976 bytes (125 MB) copied, 1.15536 s, 108 MB/s
# grr...what the heck, its fast...
test> cat testfile-1000M |dd of=256MB bs=1M count=256
70+186 records in / 70+186 records out
87945216 bytes (88 MB) copied, 0.783247 s, 112 MB/s
# >weird<; alright, forget cat
test> dd if=testfile-1000M of=256MB bs=1M count=256
256+0 records in / 256+0 records out
268435456 bytes (268 MB) copied, 0.781711 s, 343 MB/s
# Yay! It says it worked; did it?
test> lh 256MB
-rw-rw---- 1 256M Jan 18 15:36 256MB
------------------
So just to be sure -- this is "expected" behavior, right?
I'm rationalizing that 'dd' stops copying the first time it tried to
read and found the pipe either 1) less than full or 2) it got a
zero-length read cause 'cat' was keeping up due to it's relatively
small read/write defaults?
Rationalized or not -- sure isn't what I "expected" (in
terms of what I really wanted it to do!).
Machine has 4G of memory & 64-bit kernel if that
makes a difference -- though both cat and dd can run at
the same time as there are 4 cpu's available.
Does that qualify for weirdness or what?
Linda
_______________________________________________
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
