Re: Improving performance when working with large text files

2004-03-14 Thread James Edward Gray II
On Mar 14, 2004, at 2:25 AM, R. Joseph Newton wrote: I will have to give some thought to the implications of this for programming practice. If you find yourself playing with slurping (I'm a fan, as you can probably tell), treat yourself to IO:All from the CPAN, while you're at it. That modules

Re: Improving performance when working with large text files

2004-03-14 Thread R. Joseph Newton
James Edward Gray II wrote: > On Mar 13, 2004, at 1:09 PM, R. Joseph Newton wrote: > > > "West, William M" wrote: > > > > > >> by slurping the WHOLE file into memory at once (do you have room to > >> but a 45 > >> megabyte file into memory?) the speed of the processing went up- > > > > Highly unl

Re: Improving performance when working with large text files

2004-03-13 Thread James Edward Gray II
On Mar 13, 2004, at 1:09 PM, R. Joseph Newton wrote: "West, William M" wrote: by slurping the WHOLE file into memory at once (do you have room to but a 45 megabyte file into memory?) the speed of the processing went up- Highly unlikely. There is no real speed advantage with Perl in slurping

Re: Improving performance when working with large text files

2004-03-13 Thread R. Joseph Newton
"West, William M" wrote: > by slurping the WHOLE file into memory at once (do you have room to but a 45 > megabyte file into memory?) the speed of the processing went up- Highly unlikely. There is no real speed advantage with Perl in slurping a file. The only good reason I can think of is if

RE: Improving performance when working with large text files

2004-03-12 Thread West, William M
> >Price, Jason wrote: >> I'm trying to optimize a script used for processing large text log >> files (around 45MB). I think I've got all the processing fairly well >> optimized, but I'm wondering if there's anything I can do to speed up >> the initial loading of the file. oh the pain and suf

RE: Improving performance when working with large text files

2004-03-12 Thread Bob Showalter
Price, Jason wrote: > I'm trying to optimize a script used for processing large text log > files (around 45MB). I think I've got all the processing fairly well > optimized, but I'm wondering if there's anything I can do to speed up > the initial loading of the file. > > Currently, I'm performing

Re: Improving performance when working with large text files

2004-03-12 Thread Shiping Wang
At 07:55 PM 3/11/2004 -0600, James Edward Gray II wrote: On Mar 11, 2004, at 10:41 AM, Price, Jason wrote: I'm trying to optimize a script used for processing large text log files (around 45MB). I think I've got all the processing fairly well optimized, but I'm wondering if there's anything I can

Re: Improving performance when working with large text files

2004-03-11 Thread James Edward Gray II
On Mar 11, 2004, at 10:41 AM, Price, Jason wrote: I'm trying to optimize a script used for processing large text log files (around 45MB). I think I've got all the processing fairly well optimized, but I'm wondering if there's anything I can do to speed up the initial loading of the file. Curre

Improving performance when working with large text files

2004-03-11 Thread Price, Jason
I'm trying to optimize a script used for processing large text log files (around 45MB). I think I've got all the processing fairly well optimized, but I'm wondering if there's anything I can do to speed up the initial loading of the file. Currently, I'm performing operations on the file one line