Hello,
Originally my previous post had an explanation of why I really did want
the pipes and redirects. I dropped it as I didn't want to be
long-winded. Short version: large intermediate data files can be avoided
by using pipes. Likewise, I frequently use multiple steps of filtering
Ok, so why not do the piping/redirect/ect etc without "large
intermediate file" in a controlled environment instead of one that is
nearly impossible to get the date you want?
So instead of
yabba.txt | foo.pl > yab.txt
do
foo.pl -in yabba.txt -out yab.txt
and in your script:
open my $in_fh, $arg{'-in'} or die "Input file failed: $!";
open my $out_fh, $arg{'-out'} or die "Output file failed: $!";
while(<$in_fh>) {
print {$out_fh} $_;
}
close $in_fh;
close $out_fh;
_log({
'in' => $arg{'-in'},
'out' => $arg{'-out'},
'env' => \%ENV,
});
That doesn't use *any* "large intermediate data files" and is easy on
memory, and you have every piece of data you could ever want to know
about where it came from, how it was modified (IE think process flag to
tell it to modify it somehow anf log that as well), where it went, when
it all happened, any problems, etc etc
Also you could drop it in 100% of servers that had perl and expect it to
run the same.
Heck you could evn use pipes if you really wanted to make it
unneccessarily complex.
if the intermediates aren't useful. Its just not practical with the data
I process to save every intermediary file. Many of the files are one to
several Gb in size. Pipes are nice things in my book :-)
Who ever mentioned using files? I certainly didn't. Its "just not
practical" to write a shell that gives the info you want and hope that
anyplace your script is run has that same shell :)
Feel free to do it that way if you want but you're causing yourself a
lot of extra headache for no good reason.
Basically my point is this:
There are much easier, portable, reliable ways to modify X with procee
Y and put it in Z and track every bit but shoot yourself in the head if
you like :)
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>
