On 08/03/2022 07:27, Neville Smythe via use-livecode wrote:
I believe there were two problems: a) the original search search algorithm was
too slow, and b) when implemented on certain devices the app went into an
unusable state. Both problems were caused by the wordlist being stored being
too large (275K words for a file size of 2.5 MB). The second issue we are
pretty sure was Virtual Memory swap space thrashing.
I guess I'm not convinced about VM space thrashing being the problem,
certainly not due to the wordlist. It's 2.5 Mb - i.e. 0.02% of the real
RAM in a Pixel 5. There may be something else in the app making the
total VM space very large - but the wordlist wouldn't do it on its own.
An alternative is to use an sqlite database, always an option which should be
considered when handling large data. This completely removes the issue of
having the wordlist in memory, and at the same time provides an extremely fast
search engine, so was worth exploring as a solution to both problems. Storing
the words in a db single table with an index on initial letter and word length,
or as lots of tables, one for each initial and length (!), both return a result
for a search for a word in a small fraction of a millisecond, so definitely
this would be a viable solution which would handle all letter distributions.
Sounds cool. I'd like to see an example of how this would be created and
used (I'm very much a SQL novice).
There is however a downside: both methods produce a db file size of 7.1 MB for
the SOWPODS wordlist, which rather bloats the app footprint, even when you
discount the text file version of the wordlist which no longer needs to be
stored.
You could do the opposite. Store the wordlist (700Kb compressed), and
decompress/populate into the SQLite database on initial run.
But now, speaking of databases, I have a question. I have an update to my
nsScriptDatabase stack which I want to upload to the Sample Stacks. Because it
really should be compiled to a standalone, I really need to upload a pair of
stacks, a launch stack and one to hold data which can one modified by the user.
But it would seem a sample stack must be a single item. What to do?
Yep -this is a big drawback of the Sample Stacks system.
Maybe you could make a single app, which contains the two desired stacks
as custom props.
(Sometimes it's easier to type code than to describe it :-)
In your preparation, you'd do:
put URL ("binfile:launchstack.livecode") into tmp
set the cLaunch of me to compress(tmp)
put URL ("binfile:userdatastack.livecode") into tmp
set the cUser of me to compress(tmp)
And then when the user downloads your sample stack, it does the inverse
to create the two local stacks, with an info box telling them about it.
Alex.
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
