On 26/01/2013 05:58, Rajeev Prasad wrote:
*From:* "rob.di...@gmx.com" <rob.di...@gmx.com>
Rajeev Prasad <rp.ne...@yahoo.com <mailto:rp.ne...@yahoo.com>> wrote:
i have a lot of data coming/pouring in from this:
my ($rout, $pid) = $ssh-> pipe_out($cmd);
while (my $line = <$rout> ) {
print filehandle $line;
}
I want to stop writing after certain size is written (say 1gb).
so i can try this: (it is working). But I am worried I am doing too
many stat (and i am imagining it would be very expensive operation).
there are about 500,000 lines coming in for 500mb data file, so below
is doing stat 500 thousand times!!!
my ($rout, $pid) = $ssh-> pipe_out($cmd);
while (my $line = <$rout> ) {
my
($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime,$blksize,$blocks)=
stat(filehandle);
if ($size < $huge_limit ){
print $tmp_fh $line;
} else {
seek($tmp_fh, 0, 0);
print $tmp_fh "file size limit reached\n";
kill TERM => $pid;
last;
}
} ###end while
so i am 'thinking' to try this: storing output in local temp array and
checking when it reaches 10mb ($huge_limit) then save it to file and
increase counter, when it happens total of $huge_limit_counter times
(50 for 500mb limit). I close the channel and proceed. Here I am
thinking, checking size of array 500,000 times (a 500k line array) is
somehow 'more efficient' then checking file-size using stat.
my $counter=1;my @tmp_arr;
my ($rout, $pid) = $ssh-> pipe_out($cmd);
while (my $line = <$rout> ) {
push(@tmp_arr,$line);
if (size(\@tmp_arr) > $huge_limit){
$counter++;
if ($counter > = $huge_limit_counter){
kill TERM => $pid;
last;
} else {
print $tmp_fh @tmp_arr;
undef(@tmp_arr);
}
}
}
if(tell($rout) != -1){close $rout;}
Please advice any suggestion for increasing efficiency of this code.
thank you.
Rajeev
You should use the `tell` operator on the output file handle to discover
how much data has been printed.
HTH,
Rob
oops ! sorry. this is not working.
Please reply to the Perl Beginners group and not directly to me.
Please also put your response /after/ the message you are quoting, as it
is the standard for this group.
What doesn't work exactly? You can see how much data has been printed to
your output file using `tell`. The code snippet below shows an example.
Rob
my ($rout, $pid) = $ssh->pipe_out($cmd);
while (my $line = <$rout>) {
if (tell $tmp_fh > $huge_limit) {
print $tmp_fh "\nFile size limit reached\n";
kill 'TERM', $pid;
last;
}
print $tmp_fh $line;
}
close $rout;
--
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/
