Very exciting! Are you thinking about sync between multiple nodes or just 2?
On Thu, Jun 20, 2013 at 12:02 AM, Yuri Z <vega...@gmail.com> wrote:
> Sounds fantastic, especially when it comes from you Joseph.
> Just a side note regarding SVN-Git issue - it is possible to combine both
> by using git-svn - it works fine for me.
>
>
> On Wed, Jun 19, 2013 at 9:22 PM, Joseph Gentle <jose...@gmail.com> wrote:
>
> > I've given half a dozen talks about ShareJS over the last 3 years, and
> > almost every time I give a talk, someone asks me whether you can use
> > ShareJS in a peer-to-peer way instead of just through a single server.
> >
> > "You say it works like subversion. Can it work like Git?"
> > "Can you have a document shared between multiple servers?"
> >
> > Sigh no. ShareJS & Wave's algorithms were invented in 1995. Back then,
> > it was news when someone put up a new website.
> >
> > "What about Wave Federation?" Appropriately, it works like IRC, but
> > using XML. Its complicated, its vulnerable to netsplits and its buggy.
> > I guess its like IRC except it doesn't actually work.
> >
> > So lets fix that! Lets modernize wave and make it federate properly.
> > On the way we have a great opportunity to make it simpler and cleaner.
> >
> >
> > To start, I want to build a generic P2P OT container. This is a simple
> > wrapper that contains a set of OT documents and defines a network
> > protocol for keeping them in sync. The container needs to be able to
> > talk to another instance of itself running somewhere else and
> > syncronize documents between the two instances.
> >
> > Thats all I want this container to do - it should be as lightweight as
> > possible, so we can port it to lots of different languages and
> > environments. I want that code running in websites, in giant server
> > farms, in vim, and everywhere in between. It won't have any database
> > code, network code, users or a user interface (though it'll need APIs
> > to support all of that stuff). At its core it just does OT + protocol
> > work to syncronize documents.
> >
> > What are the documents? Well, like ShareJS, I'd like to support
> > multiple different kinds of data. We'll need to be able to support
> > wave's conversation model, but I'd also like to support arbitrary
> > JSON. Doing OT over arbitrary JSON structures would allow other
> > applications to be built on top of wave, using wave as a data platform
> > ("Glorious messaging bus in the sky"). It'd also be super useful for
> > gadgets and user data.
> >
> > There's three models I can imagine for what wavelets could look like:
> >
> > Option 1: All documents in the container have a unique name and a
> > type. This is how ShareJS works. We could have a JSON type and a
> > wavelet type. This is simple, but not particularly extensible (it
> > makes it hard to embed JSON inside a conversation, and vice versa).
> >
> > Option 2: At the root of every document is a JSON object. Leaves in
> > the JSON structure can be subdocuments, which could be rich text for
> > blips, or any other type we think of down the road.
> >
> > Option 3: We make another layer, which can contain a set of documents.
> > So, a wavelet could contain a JSON document describing the
> > conversation structure, some rich text documents for blips and another
> > JSON document containing gadget data or something. Access control
> > rules are at the container level. This is (sort of) how wavelets work
> > today.
> >
> > The OT itself I imagine building along the lines of Torben Weis's P2P
> > OT theory that he made in Lightwave:
> > https://code.google.com/p/lightwave/ . Briefly, every operation gets a
> > hash (like git). We add tombstones to wave's OT type and remove
> > invertability, so the transform function supports TP2. We also add a
> > prune function (inverse transform) which allows the history list to be
> > reordered (so you don't have to transform out on every site). The hard
> > part is figuring out which operations to sync, and which operations
> > need to be reordered. I'd like to go over the details with Michael
> > MacFadden and anyone else who's interested - there may well be a
> > better system that we should use instead. If there is, I'd like to
> > know about it now.
> >
> >
> > Once thats built, we can start integrating it into WIAB. The simplest
> > way to do the client-server protocol and federation will be to simply
> > reuse the container's protocol (obviously wrapped for access control).
> > We could also strip it down for pure client-server interaction if we
> > want, to make it less chatty. (If we decide thats worthwhile.)
> >
> > I'm also thinking about full end-to-end encryption. Especially in the
> > wake of the PRISM stuff, I'd quite like to make something secure.
> > Snowden: "Encryption works. Properly implemented strong crypto systems
> > are one of the few things that you can rely on. Unfortunately,
> > endpoint security is so terrifically weak that NSA can frequently find
> > ways around it." --
> >
> >
> http://www.guardian.co.uk/world/2013/jun/17/edward-snowden-nsa-files-whistleblower
> > .
> >
> >
> > All of this should happen in the experiments branch (with a mirror on
> > github).
> >
> > The design decisions that we make here will be really hard to change
> > later, so I'd like to get this right. I'd like as much feedback as
> > possible. But please restrain yourself from complaining that its too
> > much work. You're not the boss of me :D
> >
> > Also, I expect the core OT piece to be no longer than a few thousand
> > lines. We can definitely pull that off - its just figuring out what
> > those lines are thats the tricky part.
> >
> > Cheers
> > Joseph
> >
>
