Monte Goulding wrote:
> I agree it makes total sense to use the same format when
> reading/writing to disk. As all the data is loaded into memory anyway
> I wonder how much extra time it would take to turn it into a pointer
> that was just as fast to access as a regular array.... perhaps that
> could be done at first access?
ArrayEncode's format isn't optimized for searching, simply a linear list
of name/value pairs - no btree, no hashes, none of the goodies that make
them so fast in RAM.
Interestingly, some benchmarks I did some time ago suggest that the
format for custom properties is at least somewhat hashed, so that
accessing a single property tSomeKey in a property set uProp like this:
get the uProp[tSomeKey] of this stack
...is much faster than obtaining the same value if you'd stored the
array as a single property like this:
put the uProp of this stack into tArray
get tArray[tSomeKey]
In fact, the overhead of arrayDecode is so significant, and the
optmization of custom property access so nice, that in many cases you
can access a series of custom properties faster than you can access a
series of array elements from a variable if you must first obtain the
array from any other persistent store (either property or arrayEncoded
file).
> Either way my guess is you are having to do many iterations to see
> a measurable difference unless you are dealing with an amount of
> data that would be better in a database... so no matter what the
> engine needs to do the reason why we can't deal with multi-
> dimensional custom properties is because of the syntax ambiguity.
If we were as casual about all optimization opportunities, we might
parse text runs in fields rather than in htmlText. ;)
Not all data stores need be relational, and it's possible to craft
persistent stores in LC that outperform SQLite if your needs are modest
and performance is important for that particular use case.
So respectfully, the parser and the storage format are two separate
challenges, and both would need to be addressed to make good use of
nested arrays as persistent stores.
> The simplest resolution to the ambiguity is to only allow array
> syntax on sets so the set name must be the first key. If you need
> to access one of the regular custom properties there is already a
> mechanism in place for that by using empty as the set name.
>
> get the customKeys[""]["dgData"]["1"]
>
> get the customKeys["dgProps"]["row template"]
I'd vote for that.
And with that, how about a btree-based storage format so we could do
that from disk as well. :)
--
Richard Gaskin
Fourth World
LiveCode training and consulting: http://www.fourthworld.com
Webzine for LiveCode developers: http://www.LiveCodeJournal.com
Follow me on Twitter: http://twitter.com/FourthWorldSys
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
