Ooh, that's very likely to be a good tip.
(I think the database stuff is a red herring, by the way: the bulk of the
delay is in the processing stage, which is just reading text files in and out.)
Thanks everyone for their input. I'll report back with what I can find (unless
I can persuade IT to let me use the 32 bit drivers!)
Ben
On 30/01/2020 22:41, Ralph DiMola via use-livecode wrote:
I found this as well. Another thing, it's faster to truncate the string and
search from the beginning than using a "start at" on the entire string when
searching for all occurrences of a string . This was counter intuitive to me
until Mark explained that skipping chars requires more work because
repetitive skipping of Unicode chars is slower than many "memcpy"s on very
long strings.
Ralph DiMola
IT Director
Evergreen Information Services
rdim...@evergreeninfo.net
-----Original Message-----
From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On Behalf
Of Neville via use-livecode
Sent: Thursday, January 30, 2020 4:49 PM
To: use-livecode@lists.runrev.com
Cc: Neville
Subject: Re: OMG text processing performance 6.7 - 9.5
Are you perchance using lineOffset searches? I have found that lineOffset
performance on utf8 text degrades exponentially with the length of the file,
presumably as it searches for line breaks. Use offset instead which remains
fast (and much faster still if you can search on the raw text before
textencoding, then utf8 encode the found chunks)
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
