On Dec 23, 2006, at 9:25 PM, Chris Charley wrote:
----- Original Message ----- From: "Kirk Wythers" <[EMAIL PROTECTED]>
Newsgroups: perl.beginners
To: "John W. Krahn" <[EMAIL PROTECTED]>
Cc: "Perl Beginners" <beginners@perl.org>
Sent: Saturday, December 23, 2006 1:32 PM
Subject: Re: run script on multiple files
Thanks or the reply John. I have a couple of questions inline.
On Dec 22, 2006, at 10:53 PM, John W. Krahn wrote:
#! /usr/bin/perl -w
use strict;
use Date::Calc qw(Day_of_Year);
use DBI;
#MICIS climate data munger. Required input argument is the file to
process.
#Use > to redirect output to new file.
#Set the item delimiter to tabs instead of the default commas
and the line
The Output Field Separator ($,) has the default value of undef.
I guess I'm too new at this. I don't understand your point.
Above, you said 'Set the item delimiter to tabs instead of the
default commas'
and John is saying that the default of $, is undef, not a comma.
[snip]
#Part 1. Loop through the 11 header lines to identify the
station id.
#The 7th line contains the station ID, and has the format of
#STATION: SOME_STATION, STATE (Station ID: ######)
for(my $i=1;$i<=6;$i++) {
Your comment says eleven lines but your code says six?
A mistake on my part not updating the comments. The earlier file
format had 11 lines.
You need to correct this in John's code where it says:
if (1..11)
should then be
if (1..6)
[snip]
You are using the <> operator to read from the file(s) so this
*will* read all
the lines from all the files listed on the command line. The
only problem is
that you will not distinguish the headers from the second and
subsequent files
listed on the command line.
That will not do. I need to start fresh on each file. Just as if
I ran the program as:
./program.pl file1
./program.pl file2
./program.pl file3
ect....
And thats what John's solution does. In his code:
# At eof close the input filehandle to reset $.
if ( eof ) {
close ARGV;
next;
}
That 'resets' $. to 1 (beginning line of the next file) and the
'next' keyword
instructs your program to 'goto' the while (<>) statement above
thus not processing
any of the statements below it at the end of each file.
[snip]
This may work better for you:
#!/usr/bin/perl -w
use strict;
use Date::Calc qw(Day_of_Year);
use DBI;
my $dbh = DBI->connect( 'DBI:Pg:dbname=met_data;host=localhost',
'pguser',
'pguser' )
or die "Couldn't connect to PostgreSQL: $DBI::errstr
($DBI::err) \n";
my $sth = $dbh->prepare( 'INSERT INTO weather (station_id,
year, month, day,
doy, date, precip, tmin, tmax, snowfall, snowdepth, tmean) VALUES
(?,?,?,?,?,?,?,?,?,?,?,?)' );
my $station_id = '';
while ( <> ) {
# Part 1. Loop through the 11 header lines to identify the
station id.
# The station ID has the format of:
# STATION: SOME_STATION, STATE (Station ID: ######)
if ( 1 .. 11 ) {
$station_id = $1 if /\(Station ID:\s*(\S+)\)/;
Thi regular expression wants to match:
\( - a literal left parenthesis
Station ID: - then this text
\s* - 0 or more spaces
(\S+) - 1 or more non-space characters (enclosed in capturing
parentheses
whose value will be held in $1)
\) - a final literal right parenthesis
It seems that this is more flexable. ie not dependent upon a
certine number of header lines. Can you translate the f /\(Station
ID:\s*(\S+) \)/; part though?
next;
}
# At eof close the input filehandle to reset $.
if ( eof ) {
close ARGV;
next;
}
I think this is suppose to allow the script to jump to the next
file. Right?
Right
However, this script also reads the first file into the database,
then stops.
Don't know why - maybe someone else could say.
I think I see what is happening. John's script was crashing at the
end of the first file with an error that I saw earlier when writing
my script. There is a footer at the end of each file and as soon as
the script hits the footer junk, it gives:
Use of uninitialized value in concatenation (.) or string at ./
micis_final.pl line 38, <> line 37993.
Use of uninitialized value in concatenation (.) or string at ./
micis_final.pl line 38, <> line 37993.
In my script I solved this problem by telling the program to exit
when there is nothing in $year (essentially the first empty line
which always comes after the last line of data and before the
footer). Since Johns script was giving the same error, I added my two
bit solution. My guess is that I am telling the program to exit
before the the file handle closes and $ is reset.
I below is john's code with my addition. Any ideas how to get out of
each file when a blank line is hit, and still close the file handle
re-set $?
#!/usr/bin/perl -w
use strict;
use Date::Calc qw(Day_of_Year);
use DBI;
my $dbh = DBI->connect( 'DBI:Pg:dbname=met_data;host=localhost',
'pguser', 'pguser' )
or die "Couldn't connect to PostgreSQL: $DBI::errstr ($DBI::err)
\n";
my $sth = $dbh->prepare( 'INSERT INTO weather (station_id, year,
month, day, doy, date, precip, tmin, tmax, snowfall, snowdepth,
tmean) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)' );
my $station_id = '';
while ( <> ) {
#Part 1. Loop through the header lines to identify the station id.
#The station ID has the format of:
#STATION: SOME_STATION, STATE (Station ID: ######)
if ( 1 .. 11 ) {
$station_id = $1 if /\(Station ID:\s*(\S+)\)/;
next;
}
#At eof close the input filehandle to reset $.
if ( eof ) {
close ARGV;
next;
}
#Part 2. Loop through the records and prepare SQL statement.
my ( $year, $month, $day, $precip, $tmin, $tmax, $snowfall,
$snowdepth, $tmean, $obstime, $datasource ) = split;
#Stop reading data at the end of the file, when $year is empty.
This
#gets you out of the datafile before the program chokes on the
footer.
exit unless $year;
#Initialize and concatenate date as YYYMMDD.
my $date = $year . $month . $day;
#Initialize and calculate day of the year (doy)
my $doy = Day_of_Year( $year, $month, $day );
#Switch T (trace) to 0.01 and M (missing) to -999
$precip = 0.01 if $precip eq 'T';
for ( $precip, $tmin, $tmax, $snowfall, $snowdepth, $tmean ) {
$_ = -999 if $_ eq 'M';
}
$sth->execute( $station_id, $year, $month, $day, $doy, $date,
$precip, $tmin, $tmax, $snowfall, $snowdepth, $tmean );
#print join( "\t", $station_id, $year, $month, $day, $doy,
$date, $precip, $tmin, $tmax, $snowfall, $snowdepth, $tmean ), "\n";
}
#$sth->finish();
# Disconntect from database
$dbh->disconnect();
__END__
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>
