I'd encourage you to fire up a wiki page and start the discussion here. I suspect due to te nature of the topic, participants will quickly be self selecting.
Upayavira On Wed, Jun 19, 2013, at 10:11 PM, Joseph Gentle wrote: > On Wed, Jun 19, 2013 at 1:26 PM, Michael MacFadden > <michael.macfad...@gmail.com> wrote: > >>To start, I want to build a generic P2P OT container. This is a simple > >>wrapper that contains a set of OT documents and defines a network > >>protocol for keeping them in sync. The container needs to be able to > >>talk to another instance of itself running somewhere else and > >>syncronize documents between the two instances. > >> > >>Thats all I want this container to do - it should be as lightweight as > >>possible, so we can port it to lots of different languages and > >>environments. I want that code running in websites, in giant server > >>farms, in vim, and everywhere in between. It won't have any database > >>code, network code, users or a user interface (though it'll need APIs > >>to support all of that stuff). At its core it just does OT + protocol > >>work to syncronize documents. > > > > I strongly suggest we separate the OT Algorithms, the application level > > protocol, and the network transport layer. > > The network transport layer should definitely be separated out. I'd > also like to separate out the OT functions themselves (a la > share/ottypes). > > I'm imagining coupling the concurrency control system and the > application level protocol. This coupling might not be needed - I > don't know enough yet to be able to tell. I think we should focus on > figuring out what OT system(s) to use first. > > >>The OT itself I imagine building along the lines of Torben Weis's P2P > >>OT theory that he made in Lightwave: > >>https://code.google.com/p/lightwave/ . Briefly, every operation gets a > >>hash (like git). We add tombstones to wave's OT type and remove > >>invertability, so the transform function supports TP2. We also add a > >>prune function (inverse transform) which allows the history list to be > >>reordered (so you don't have to transform out on every site). The hard > >>part is figuring out which operations to sync, and which operations > >>need to be reordered. I'd like to go over the details with Michael > >>MacFadden and anyone else who's interested - there may well be a > >>better system that we should use instead. If there is, I'd like to > >>know about it now. > > > > I think this is another interesting area. One thing to consider is that I > > am not sure if the linear model that wave used is even the right option. > > If we start manipulating things like JSON Objects, Java Object, XML, or > > nested documents, a hierarchical path mechanism may be best. > > Yes. For ShareJS's JSON OT type, operations are defined by a path (a > list) and an operation at that path. I'm not convinced by this design > anymore - we should definitely discuss it. > > > My other > > instinct here is, we should make sure that we base the OT on something > > that has been proven to be correct. There has been some work to evaluate > > OT systems and prove that they are correct and solve the proper OT Puzzles > > and support the required properties. > > > > Two other things that we need to consider. Beyond TP1 and TP2, there are > > also IP1, IP2, and IP3 that you need to think about if you are going to > > support group undo, which in my opinion wave needs to support. If you > > can't undo things in collaboration, then you have problems. > > Using tombstones, I don't know how you can implement IP3. I've a mind > to abandon formal invertibility, and simply rely on undo stacks for > user level actions. But if we can find an architecture which meets our > needs and can support invertibility, then that would be even better. > > > Basically, I am not confident that we know that wave's OT or what is in > > lightwave is really what we want. I am not saying that they are not, I am > > just saying that I don't think we really have stated what we NEED from the > > OT and then proved that a particular approach salsifies those needs. > > Simply having a demo of something that works in a toy environment is > > likely to have holes that we don't see until much, much later in the > > development cycle. > > > > My recommendation here would be for us to form a small committee to do a > > literature review on the topic and to foster some technical discussion. > > The result should be something in the wiki that lays out a plan that the > > community can comment on. > > I completely agree. I don't know enough about what our other choices > are. You're much better read than I am and you're better connected to > the academic OT community. Would you be interested in organising / > chairing this committee? > > >>I'm also thinking about full end-to-end encryption. Especially in the > >>wake of the PRISM stuff, I'd quite like to make something secure. > >>Snowden: "Encryption works. Properly implemented strong crypto systems > >>are one of the few things that you can rely on. Unfortunately, > >>endpoint security is so terrifically weak that NSA can frequently find > >>ways around it." -- > >>http://www.guardian.co.uk/world/2013/jun/17/edward-snowden-nsa-files-whist > >>leblower > >>. > > I like this idea. There is a paper from France that talks about nested > > hashing, digital signing, etc that proves the authenticity of operations > > that might also be worth looking at. I will try to dig up the reference. > > Great! > > -Joseph
