handling large data files

[Jerry Preston]

Hi!,

I am look for a better way and a faster way to deal with a 4 - 8 meg data
file.   This file has been saved as an .cvs file for excel to read in.

All I am interested in is the first  three cells of ',' delimited data.

Die,Row 0, Column 11
Test Result,1
 Score,1
 PMark Score,0
k Score,0
Score,0

Defects,0

Mark Measurements,276
Measurement
0,0,8.030399,740.998413,21.542923,16.721996,817.562500,22.048611,881.062500,
29.847174,11.604215,17.210899,685.522644,16.721996,0,0
Measurement
1,1,12.346605,804.399353,25.516476,8.607447,817.562500,8.607447,881.062500,2
6.055706,28.836847,20.028336,748.923584,9.931009,0,0

  open( FI, $file_path )   || die "unable to open $file_path $!\n";
  @file_data = <FI>;
  close FI;

  LINE: foreach $_ ( @file_data ) {
    if( /^Die,Row/ ) {
      ($row, $col) = /(\d+)/g;
    }
    if( /^PMark Measurements/ ) {
      ($cnt) = /(\d+)/g;
      if( $cnt > $max ) {
        $max = $cnt;
      }
      if( $cnt > 0 ) {
        $row_col[ $jp++ ] = "$row,$col";
        while( <FI> ) {
          if( /^Die,Row/ ) {
            ($row, $col) = /(\d+)/g;
            $row_col[ $jp++ ] = "$row,$col";
            next LINE;
          }
          $Z=0;
          if( /^Measurement/ ) {
            (@data) = split( /\,/ );
            if( $data[ 2 ] > 0 ) {
              ($meas) = ($data[0]) =~ /(\d+)/g;
              $data{ "$row,$col" }{ $meas } = $data[2];
              $data{ "$row,$col" }{ $meas }{ $data[1] } = $data[2];
            }
          }
        }
      }
    }
  }
@file_data = ();