I have a curiosity question about this type of thing. Is there a way to adjust things so that posting data to a url is not blocking? If so, you could take advantage of a webservers multithreading (or several webservers) to hand off "jobs" and then get the returned result.
Otherwise it should be possible using a server local to the machine to write temp files to a directory, then "load" the url with a GET parameter that points to the file in question with a callback message. Have an lc server (or php or whatever) script handle each, return the data, at which point your app can compile the results, sort, and be done with it. For small datasets this might not be worthwhile, but if there is enough volume, I think the returns would be worthwhile. Ok, just tried this. With 500k lines of data, split into 10k line chunks it's possible to do it in around 2.3 secs on my machine. A hotter machine with more cores is probably faster than that. Laying off really small chunks doesn't pay, too many threads on the server side, 10k lines per seems to be a good value. 15 to 20k per is very slightly faster. Of course code would have to be tweaked to manage the very last group that might not have the full number of 20k lines. If you want to mess with something like this, the stack is here: https://dl.dropbox.com/u/11957935/messingWithNumbers.livecode And the lc server script is as follows: <?lc put 10 into tChunkSize -- 10 lines per chunk to process put URL ("binfile:" & $_GET["tfile"]) into tData if tData is not empty then -- make sure the file is not empty set the itemdelimiter to tab put the number of lines in tData into tCount -- number of lines sent. Should always be a factor of 10 -- process using the sort and grab method. repeat tCount/tChunkSize times put line 1 to tChunksize of tData into tTemp delete line 1 to tChunksize of tData sort tTemp numeric ascending by item 2 of each put line 1 of tTemp & "<br>" & (the last line of tTemp) & "<br>" after tNewData end repeat put tNewData else put "Problem finding data?" end if ?> Of course everything is set up to match my server set up. Tried to just post the scripts but they ended up too large. Twice. _______________________________________________ use-livecode mailing list [email protected] Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
