Pete, I meant to ask, how does your array solution work? Where does the comparison take place? I've long used arrays for storing data but not much beyond that.
Thanks & Regards, Scott Rossi Creative Director Tactile Media, UX Design Recently, Pete wrote: > Thanks for the report back on the speed Alex. I guess its academic if the > speed is down to 100msecs but I'm wondering if a binary search technique > would be better or worse (assuming the lists were sorted of course). > > How did you create the two lists for your test? I'd like to try the binary > search but stuck with an easy way to generate two large files like that! > > Pete > Molly's Revenge <http://www.mollysrevenge.com> > > > > > On Thu, Oct 6, 2011 at 1:17 PM, Alex Tweedly <a...@tweedly.net> wrote: > >> Much faster. >> >> I tried the original script (with typo fixed) on 7000 lines of varying >> length between 100 and 300 chars - took about 2 minutes to run. The array >> version (again with typo fixed) took around 100 msec. >> >> -- Alex. >> >> On 06/10/2011 20:16, Scott Rossi wrote: >> >>> FWIW, I tried a quick test of Matthias's script using two fields with 5000 >>> lines of 256 chars each. I tried using "i is not among the lines of" and >>> "i >>> is not in" with identical results. Processing time was 1 min 6 secs in >>> both >>> cases (Mac Intel Core2 Duo). Perhaps the array option posted is faster. >>> >>> Regards, >>> >>> Scott Rossi >>> Creative Director >>> Tactile Media, UX Design >>> >>> >>> >>> Recently, Michael Kann wrote: >>> >>> Matthias, >>>> >>>> Your script should take a few seconds at most. There must be something >>>> else >>>> going on to slow you down. If you want to post the script itself and a >>>> few >>>> lines of data perhaps someone can figure it out. >>>> >>>> Mike >>>> >>>> --- On Wed, 10/5/11, Matthias >>>> Rebbe<matthias_livecode_**150...@m-r-d.de<matthias_livecode_150...@m-r-d.de >>>> >> >>>> wrote: >>>> >>>> From: Matthias >>>> Rebbe<matthias_livecode_**150...@m-r-d.de<matthias_livecode_150...@m-r-d.de>>>> > >>>>> >>>> Subject: how to compare 2 very large textfiles >>>> To: "How to use >>>> LiveCode"<use-livecode@lists.**runrev.com<use-livecode@lists.runrev.com> >>>>> >>>> Date: Wednesday, October 5, 2011, 5:00 PM >>>> >>>> >>>> Hi, >>>> >>>> i need to compare two very large text files with about 5000 - 7000 lines >>>> each >>>> with a lines size of up to 256 chars. >>>> >>>> I need to find out if there are lines missing in either file a or file b. >>>> >>>> What is the best way to do this with good speed? >>>> >>>> I tried to check each line in file a and if the line is in file b. >>>> And after that, i check for each line in file b and try to find out >>>> if the line is in file a. >>>> >>>> With large files it takes about 10 to 15 minutes to do the complete >>>> check. >>>> >>>> My script looks like this >>>> >>>> repeat for each line i in tTextA >>>> if i is not among the lines of tTextB then put i&return after tMissingInB >>>> end repeat >>>> >>>> repeat for each line i in tTextB >>>> if i is not among the lines of tTextA then put i&retrurn after >>>> tMissingInA >>>> end repeat >>>> >>>> Is there a better (faster) way? >>>> >>>> Regards, >>>> >>>> Matthias >>>> >>> >>> >>> ______________________________**_________________ >>> use-livecode mailing list >>> use-livecode@lists.runrev.com >>> Please visit this url to subscribe, unsubscribe and manage your >>> subscription preferences: >>> http://lists.runrev.com/**mailman/listinfo/use-livecode<http://lists.runrev. >>> com/mailman/listinfo/use-livecode> >>> >>> >> >> ______________________________**_________________ >> use-livecode mailing list >> use-livecode@lists.runrev.com >> Please visit this url to subscribe, unsubscribe and manage your >> subscription preferences: >> http://lists.runrev.com/**mailman/listinfo/use-livecode<http://lists.runrev.c >> om/mailman/listinfo/use-livecode> >> >> > _______________________________________________ > use-livecode mailing list > use-livecode@lists.runrev.com > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode _______________________________________________ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
