Lionel MARTIN wrote: > >> Lionel MARTIN wrote: >> >>>> - Don't load large amounts of data into scalars. >>> Fine. Now I know why. But sometimes, you don't have the choice. >> >> I'd like to know what situations you encounter where you are forced to >> load >> large amounts of data into scalars. I can't think of any.
I keyed in on the phrase "don't have the choice". I didn't say it would be easier, just possible. Anytime you're working with data that could be very large, you have to treat it differently: > I don't have any clear situations right here in mind, but we could > imagine many: > -for example, a bulletin board system, where you are retrieving posted > message from a DB. Each message could weigh several dozens of kilo. > (especially if you store HTML formatting in the DB) Pull the records out a set at a time and send them to the browser a set at a time. > -another example hat comes to my mind is a project (implemented in PHP) > where I had to work. Part of the process was retrieving cached HTML > template pages from the server and do BB tags parsing before serving the > client. Of course, you would tell me that I could retrieve the data > chunk by chunk, but this is not as obvious, as some BB tags could spread > over several lines. So, this would need to devise an algorithm to be > sure we are not cutting in the middle of a tag. In this kind of > situation and if the file to be retrieved don't grow too large, I would > prefer to retrieve the file all at once, do the processing, serve it to > the client, and then, undef the buffer. Far easier than doing chunk by > chunk. This kind of thing happens all the time. Think of SAX XML parsers or mod_perl filters. It's not terribly difficult to parse something in chunks like that. I wasn't saying that it wouldn't be easier to have everything in memory. Heck I'd love it if I never had to read a file in line by line anymore and could just slurp them all into arrays. But knowing when to put all of something into memory and when not to is part of being a programmer. -- Michael Peters Developer Plus Three, LP
