Memory Problems

[Greg Robertson]

I am trying to parse a large CSV (5MB) file using some code I found here:
http://www.macresearch.org/cocoa-scientists-part-xxvi-parsing-csv-data

but I am getting what I think is an out of memory error part way through, the 
error is:

DataImport(1324,0xa09f3500) malloc: *** mmap(size=16777216) failed (error 
code=12)
*** error: can't allocate region

I'm fairly certain I do not have a memory leak it is just that the NSString is 
very large. Is there anyway to increase the memory allocated or clean up the 
code to reduce its memory usage?

Thanks

Greg


Here is the code:

#import <Cocoa/Cocoa.h>
#import <Foundation/Foundation.h>

@interface NSString (ParsingExtensions)

-(NSArray *)csvRows;

@end

---------

#import "NSStringCategory.h"

@implementation NSString (ParsingExtensions)

-(NSArray *)csvRows
{
        NSMutableArray *rows = [NSMutableArray array];
        
    // Get newline character set
    NSMutableCharacterSet *newlineCharacterSet = (id)[NSMutableCharacterSet 
whitespaceAndNewlineCharacterSet];
    [newlineCharacterSet formIntersectionWithCharacterSet:[[NSCharacterSet 
whitespaceCharacterSet] invertedSet]];
        
    // Characters that are important to the parser
    NSMutableCharacterSet *importantCharactersSet = (id)[NSMutableCharacterSet 
characterSetWithCharactersInString:@",\""];
    [importantCharactersSet formUnionWithCharacterSet:newlineCharacterSet];
        
    // Create scanner, and scan string
    NSScanner *scanner = [NSScanner scannerWithString:self];
    [scanner setCharactersToBeSkipped:nil];
    while ( ![scanner isAtEnd] ) {        
        BOOL insideQuotes = NO;
        BOOL finishedRow = NO;
        NSMutableArray *columns = [NSMutableArray arrayWithCapacity:12];
        NSMutableString *currentColumn = [NSMutableString string];
        while ( !finishedRow ) {
            NSString *tempString;
            if ( [scanner scanUpToCharactersFromSet:importantCharactersSet 
intoString:&tempString] ) {
                [currentColumn appendString:tempString];
            }
                        
            if ( [scanner isAtEnd] ) {
                if ( ![currentColumn isEqualToString:@""] ) [columns 
addObject:currentColumn];
                finishedRow = YES;
            }
            else if ( [scanner scanCharactersFromSet:newlineCharacterSet 
intoString:&tempString] ) {
                if ( insideQuotes ) {
                    // Add line break to column text
                    [currentColumn appendString:tempString];
                }
                else {
                    // End of row
                    if ( ![currentColumn isEqualToString:@""] ) [columns 
addObject:currentColumn];
                    finishedRow = YES;
                }
            }
            else if ( [scanner scanString:@"\"" intoString:NULL] ) {
                if ( insideQuotes && [scanner scanString:@"\"" intoString:NULL] 
) {
                    // Replace double quotes with a single quote in the column 
string.
                    [currentColumn appendString:@"\""]; 
                }
                else {
                    // Start or end of a quoted string.
                    insideQuotes = !insideQuotes;
                }
            }
            else if ( [scanner scanString:@"," intoString:NULL] ) {  
                if ( insideQuotes ) {
                    [currentColumn appendString:@","];
                }
                else {
                    // This is a column separating comma
                    [columns addObject:currentColumn];
                    currentColumn = [NSMutableString string];
                    [scanner scanCharactersFromSet:[NSCharacterSet 
whitespaceCharacterSet] intoString:NULL];
                }
            }
        }
        if ( [columns count] > 0 ) [rows addObject:columns];
    }
        
    return rows;
}
@end

_______________________________________________

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com