Knapp Martin wrote:
> Thanks for the in-depth reply. Are there known issues with saving LC
> stacks to Dropbox? I was thinking of Roland's issue with corruption
> and Phil's suggested fix of watching the file with the tilde character
> before allowing a quit to complete. I have an app where users create
> documents (stacks) and saving them to Dropbox is a built-in option.
My hunch (read, "I'm likely wrong, wait for something more definitive
from someone on the core team or Brian Milby, since he reads the engine
source <g>) is that LC files are less prone to a wide range of
disk-related issues by virtue of being written in a single pass.
HyperCard, FileMaker, SQLite, and others are read and written in pages,
which works more reliably in some (SQLite) than others (no one misses
HC's 504 errors <g>). Moving stuff in and out of portions of a file
that may not change much otherwise isn't necessarily brittle in itself
(SQLite as a reputation for being enormously robust), but we can
envision how larger SQLite files may confuse a syncing system that can
only move chunks of data over a network so quickly.
Since LC files are, near as I recall, written fresh from beginning to
end with each save command, the result is usually a cleaner file than
with some paging systems.
And it's also faster to write everything in one pass than writing the
same amount of data by moving back and forth throughout a file, so my
hunch would be that LC stack files would likely be less prone to error
when synced over Dropbox than SQLite files.
But by no means should this be taken as gospel, and certainly not as any
indication that we should drop SQLite and use stack files instead.
Different formats for different purposes.
And as Mark Wieder noted, some of the biggest challenges with syncing
occur with multiple users accessing a given resource simultaneously.
Those are very real challenges, which may be why so many developers over
the years keep writing their own sync mechanisms, with no single
solution having emerged as the one-size-fits-all answer.
Consider this scenario with stack files, for example:
Mark writes a stack, then I open it, then you open it. You and I are
both making changes, and I save mine a few seconds before you save
yours. In that scenario, what's on disk?
When I get back to work on the file tomorrow, I'll likely be
disappointed to find that everything I'd done is now gone. I can see
only your changes, since yours were the last saved and when you saved
you hadn't yet updated to have a copy with my changes. So after Dropbox
faithfully performs everything we ask it to do, my changes are gone forever.
Write-locking can help with things like that, and that's a subject in
itself. At what level of granularity should we lock? And before
editing begins do we first compare what's in memory with what's in the
data store to ensure what we're about to edit is the most recent copy?
How do we handle conflicts, when one inevitably occurs?
So many ways to handle sync....
> I have one customer who stores their stacks on a network server and
> a few times they've ended up with a corrupted stack.
Corruption is very interesting to me. Lost data is usually explainable
through simple multiuser workflow weaknesses, such as inadequate locking
mechanisms. But corruption in LC means something interrupted the write,
or overwrote portions before the write was completed. And since LC
stacks files are written in a single pass, whatever caused it to corrupt
was very efficient at it, as it had little time to do it in.
I can conceive of a possible scenario in which neither Dropbox nor LC
has a bug per se at all, but which may account for corruption:
1. Your app issues a save command on a large stack.
2. Dropbox begins syncing the file to the server.
3. Your friend has the same file open on his machine.
4. On his machine, Dropbox notices the server has a newer version, and
prepares to update his copy.
5. He starts a save in his stack.
6. Dropbox has written a portion of the stack from the server to his
disk, but stopped when it noticed the file was being updated.
7. After your friend's save is done, the corrupted stack is then copied
to the server, where it later propagates to your machine.
RESULT: everyone now has the same file, and it's broken.
I haven't studied the file system triggers and server monitoring that
trigger reads and writes in Dropbox. And if it's proprietary, chances
are no one outside that company has either.
There are many ways that imagined scenario might for all I know be
accounted for in the way Dropbox is written. But there may also be
other scenarios that can produce the same corrupted result that I
haven't thought of.
--
Richard Gaskin
Fourth World Systems
Software Design and Development for the Desktop, Mobile, and the Web
____________________________________________________________________
ambassa...@fourthworld.com http://www.FourthWorld.com
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
