Below is one possible optimization: the elimination of regular expressions.
################
## begin code ##
################
use strict;
use warnings;
use Benchmark;
# populate record list
my @records;
foreach my $record (<DATA>) {
chomp ($record);
push @records, $record;
}
# benchmark
timethese (1000000, {
do_regex => \&do_regex,
do_index => \&do_index
});
exit;
# find records enclosed in quotes (regex way)
sub do_regex {
my $record;
foreach my $record (@records) {
if ($record =~ /^"/ && $record =~ /"$/ && $record =~ tr/"// == 2) {
# handle enclosed record
}
}
}
# find records enclosed in quotes (substr/index)
sub do_index {
foreach my $record (@records) {
if (substr ($record, 0, 1) eq '"' && index ($record, '"', 1) == length
($record) - 1) {
# handle enclosed record
}
}
}
__DATA__
"Field1","Field2","Field3","Field4","Field5"
"Field1,Field2,Field3,Field4,Field5"
"Field1","Field2","Field3","Field4","Field5"
"Field1,Field2,Field3,Field4,Field5"
"Field1","Field2","Field3","Field4","Field5"
"Field1,Field2,Field3,Field4,Field5"
"Field1","Field2","Field3","Field4","Field5"
"Field1,Field2,Field3,Field4,Field5"
"Field1","Field2","Field3","Field4","Field5"
"Field1,Field2,Field3,Field4,Field5"
##############
## end code ##
##############
Benchmark results (P4/1.3GHz)
-----------------------------
Benchmark: timing 1000000 iterations of do_index, do_regex...
do_index: 21 wallclock secs (19.89 usr + 0.06 sys = 19.95 CPU) @ 50130.34/s (
n=1000000)
do_regex: 33 wallclock secs (30.01 usr + 0.05 sys = 30.06 CPU) @ 33263.48/s (
n=1000000)
I hope this helps.
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of
Michael D Schleif
Sent: Monday, May 23, 2005 1:43 PM
To: perl-win32-users mailing list
Subject: CSV: help optimize ???
I have an ongoing stream of CSV files of varying lengths (some quite
long), and of various CSV formats. I am using Text::CSV_XS to parse the
records; which works except in that case where the incoming file
surrounds each RECORD with double-quotes.
I find this to be aberrant behaviour. How can I tell Text::CSV_XS to
ignore double-quotes around records?
I have the following code that normalizes each line prior to parsing
with Text::CSV_XS:
if (
/^"/
&& /"$/
&& tr/"// == 2
){
s!(^"|"$)!!g;
}
Obviously, in the majority of cases this should be a no-op, since
double-quotes surround fields, NOT records.
How can I optimize this sub-routine for minimal processing overhead in
those cases where it does not apply?
--
Best Regards,
mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know. The more I know, the more I know I don't know . . .
--
_______________________________________________
Perl-Win32-Users mailing list
[email protected]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
