On Dec 22, 2006, at 8:33 PM, Chad Perrin wrote:
On Fri, Dec 22, 2006 at 08:04:39PM -0600, Kirk Wythers wrote:
I have written a short perl script that munges climate data and then
loads it into a postgres database. It works fine on one file at a
time... syntax is ./program.pl filename
I would like to run it in a directory with multiple files. I have
tried syntax ./program.pl file1 file2, but only the first file gets
processed. Can anyone help me figure out how to run this script on a
directory full of files that all need to be processed?
Yes, some of us probably can help. We'll probably need to see what
you're trying so far to be able to give the most helpful responses
possible, however, for solving your problem. Right off the top of my
head, without any other information, I'm just inclined to say "Try
using
'while (<>)' to access file contents." That may not suit your
needs at
all, though, since I don't know exactly how you need your file
access to
fit into the program.
Thanks for the reply Chad. Here is my script (I'm not sure if I
should be modifying the script itself, or piping something on the CL:
#! /usr/bin/perl -w
use strict;
use Date::Calc qw(Day_of_Year);
use DBI;
#MICIS climate data munger. Required input argument is the file to
process.
#Use > to redirect output to new file.
#Set the item delimiter to tabs instead of the default commas and the
line
#delimiter to a newline character
$, = "\t";
$\ = "\n";
#Instantiate the global station ID variable
my $station_id = "";
#Initialize I/O variables
my ($year,$month,$day,$doy,$date,$precip,$tmin,$tmax,$snowfall,
$snowdepth,$tmean,$obstime,$datasource);
#Part 1. Loop through the 11 header lines to identify the station id.
#The 7th line contains the station ID, and has the format of
#STATION: SOME_STATION, STATE (Station ID: ######)
for(my $i=1;$i<=6;$i++) {
my $header = <>;
#Remove the newline character
chomp $header;
if ($i == 2) {
#Split the line into an 3-item array based on the 2 colons.
my @line = split(":", $header);
#Extract everything after the 2nd colon.
$station_id = $line[2];
#Remove leading white spaces.
$station_id =~ s/^\s+//;
#Remove ending bracket.
$station_id =~ s/\)//;
}
}
#Connect to postgreql
my $dbh = DBI->connect( "DBI:Pg:dbname=met_data;host=localhost",
"pguser", "pguser" )
or die "Couldn't connect to PostgreSQL: $DBI::errstr ($DBI::err)\n";
#Part 2. Loop through the records and prepare SQL statement.
while (my $line=<>) {
chomp $line;
#Split the line on white spaces.
($year,$month,$day,$precip,$tmin,$tmax,$snowfall,$snowdepth,$tmean,
$obstime,$datasource) = split(/\s+/, $line);
#Stop reading data at the end of the file, when $year is empty. This
#gets you out of the datafile before the program chokes on the
footer.
exit unless $year;
# Initialize and concatenate date as YYYMMDD.
$date = $year . $month . $day;
# Initialize and calculate day of the year (doy)
$doy = Day_of_Year($year, $month, $day);
#Switch T (trace) to 0.01 and M (missing) to -999
if ($precip eq "T") { $precip = 0.01; }
elsif ($precip eq "M") {$precip = -999; }
if ($tmin eq "M") { $tmin = -999; }
if ($tmax eq "M") { $tmax = -999; }
if ($snowfall eq "M") { $snowfall = -999; }
if ($snowdepth eq "M") { $snowdepth = -999 }
if ($tmean eq "M") { $tmean = -999 }
my $sth = $dbh->prepare("INSERT INTO weather (station_id, year,
month, day, doy, date, precip, tmin, tmax, snowfall, snowdepth,
tmean) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)");
$sth->execute($station_id, $year, $month, $day, $doy, $date, $precip,
$tmin, $tmax, $snowfall, $snowdepth, $tmean);
#print $station_id, $year, $month, $day, $doy, $date, $precip, $tmin,
$tmax, $snowfall, $snowdepth, $tmean;
}
#$sth->finish();
#Disconntect from database
$dbh->disconnect();
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>
