I made a quick test stack to try out a few ides:
http://dl.dropbox.com/u/144280/Divide%20List%20Tests.livecode
It generates 100,000 random integer pairs into one field, then has four
buttons to do the sample division you gave to the two items in each line.
The first is a straight-up "repeat with a = 1 to the number of lines" making
sure to copy the data from the field into a variable, and write the new data
into a variable before saving it to the output field. (Because working
directly with fields is slower than pure variables.)
The second cuts the input list into ten 10,000 line chunks, and runs through
each of those in turn, tacking it all onto one output variable as it goes.
The third does the same thing, but cuts the input into a hundred 1,000 line
chunks.
And the fourth brings it down to a mere 100 lines per chunk. (But a thousand
chunks.)
Times were as follows on my not-so-hot machine:
Generate Randoms: 2 seconds
Divide all 100,000: 77 seconds
Divide in groups of 10,000: 10 seconds
Divide in groups of 1,000: 7 seconds
Divide in groups of 100: 7 seconds
Those totals are for doing ALL 100,000 regardless of how they are broken
down. I get the same output for all of them.
So you can see that the slowdown is probably in accessing "line 84,932" etc
of the input string. Whereas limiting the line numbers to less than 10,000
makes a HUGE difference in speed. And even a bit better with a line limit of
1,000. Any further savings of using tiny 100-line chunks is lost by having
to cut out 1,000 different chunks.
But this should show where the slowdown is, and offer a way to work around
it.
When the cookie is too big, break off what you can chew.
~ Chris Innanen
~ Nonsanity
On Thu, Mar 10, 2011 at 1:51 PM, FlexibleLearning <
ad...@flexiblelearning.com> wrote:
> Problem:
> I have a long list of several thousand lines.
> Each line contains two comma-separated numbers.
> I want to divide the first item of each line by one divisor, and divide the
> second item of each line by a different divisor.
> The list order must stay the same.
>
> Example:
> Using 2 and 5 as divisors...
> 10,10
> 12,15
> 8,12
> would become
> 5,2
> 6,3
> 4,2.4
>
> Options:
> Using "repeat with n=1 to num of lines" takes far too long.
> Using "repeat for each line L" either attempts to modify read-only data, or
> is only 25% faster using a dumping variable.
> Using split/combine will mess up the ordering (numeric array keys are not
> sorted numerically with combine).
>
> Any other ideas?
>
> Hugh Senior
> FLCo
>
>
>
>
>
> _______________________________________________
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
