Re: opening a list of files to process error

John W. Krahn Tue, 03 May 2005 18:00:10 -0700

S E wrote:

Hello,


Hello,

I've been using the following snippet of code to create a list of files
(received from external clients) to be processed, interrogate the files
for the client's id, confirm the id against the database and then rename
the files with using the client id in a unix environment.

open(FILE_LIST,"ls filename* | grep -v txt|" ) || die "ERROR: Cannot create list of 
files\n" ;

while ($cfile = <FILE_LIST>) {
  chomp $cfile;
  if ( -z $cfile ) {
     rename $cfile, "$cfile.empty.txt";
     print "$cfile is empty\n";
     print "$cfile has been renamed to $cfile.empty.txt\n\n";
  }
  else {
     print "$cfile has data\n";
     my $time = strftime '%m%d%y%H%M%S', localtime;
     open ($char_file, '<', $cfile) or die "Can't open `$cfile': $!\n";
     my $charcount;
     my $char_line=<$char_file>;
     chomp;
     $charcount=substr($char_line,8,1);
     if ( $charcount eq / ) {
        chomp($csv_client_id = `cut -c1-6 $cfile |sort -u`);
        my $sql_command = " SELECT DISTINCT CLIENT_ID ".
                          "   FROM CLIENT_SERVICER ".
                          "  WHERE CSV_CLIENT_ID = \'$csv_client_id\'".
                          "    AND TERMDATE IS NULL ".
                          "    AND TYPE_CODE = 'PAY' ";
        print "Executing <$sql_command>\n"  ;
        ($client_id) = $conn->selectrow_array($sql_command) ;
            if ( ! defined $client_id) {
                print "CLIENT_ID not found on P_DB for $cfile\n\n" ;
            }
            else {
                sleep 1;
                mv "/tmp/$cfile", "/tmp/$cfile.$time.$client_id.txt" or die "can't 
mv file: $!";
                print "$cfile has been renamed to 
/tmp/$cfile.$time.$client_id.txt\n";
            }


Its probably not the best written.  But, it was working well and I thought
I understood what it was doing through each step as a beginner.  Until, I
started receiving this error message:

'sort: missing NEWLINE added at end of input file STDIN'

Plus, the client_id variable was becoming:
'^Z
12345'

Instead of:
'12345'

I found some files from a particular client had this ^Z end of file
character in the file.  I found that if I ran the dos2unix command
manually on these files and the reran my job and I didn't receive the error.

I found this seems to perform the dos2unix command:

s/\r*$//;

which appears to work well when I pass it the file(s) with the ^Z at the
end of the file.

So, my question(s) is this:

Am I doing something wrong when I open my list of files to process?

Or

Should I be including the (s/\r*$//;) before I open my list of files to process? If so, what would be your advice on the best way to attack that?

It looks like part of the problem is using external programs to do what you could do better in perl. For example (untested):


opendir FILE_LIST, '.' or die "Cannot open the current directory: $!";

while ( my $cfile = readdir FILE_LIST ) {

    next unless $cfile =~ /^filename/;
    next if $cfile =~ /txt/;

    open my $char_file, '<', $cfile or die "Can't open `$cfile': $!\n";

    my $size = read $char_file, my $char_line, 9;

    if ( $size == 0 ) {
        close $char_file;
        rename $cfile, "$cfile.empty.txt";
        print "$cfile is empty\n",
              "$cfile has been renamed to $cfile.empty.txt\n\n";
        next;
        };
    print "$cfile has data\n";

    next unless substr( $char_line, 8, 1 ) eq '/';

    my $time = strftime '%m%d%y%H%M%S', localtime;
    # remove non-digit characters for id
    ( my $csv_client_id = $char_line ) =~ tr/0-9//cd;

    my $sql_command = " SELECT DISTINCT CLIENT_ID ".
                      "   FROM CLIENT_SERVICER ".
                      "  WHERE CSV_CLIENT_ID = \'$csv_client_id\'".
                      "    AND TERMDATE IS NULL ".
                      "    AND TYPE_CODE = 'PAY' ";
    print "Executing <$sql_command>\n";

( my $client_id ) = $conn->selectrow_array( $sql_command ); if ( ! defined $client_id ) { print "CLIENT_ID not found on P_DB for $cfile\n\n" ; } else { sleep 1; rename "/tmp/$cfile", "/tmp/$cfile.$time.$client_id.txt" or die "can't mv file: $!"; print "$cfile has been renamed to /tmp/$cfile.$time.$client_id.txt\n"; } }

John
--
use Perl;
program
fulfillment

--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>

Re: opening a list of files to process error

Reply via email to