Glad to participate as requested, though this is primarily the developers' discussion. John B
On Thu, Jun 20, 2013 at 4:31 AM, Upayavira <u...@odoko.co.uk> wrote: > Perfect. > > On Thu, Jun 20, 2013, at 03:57 AM, Michael MacFadden wrote: > > Upayavira, > > > > > > We will put a wiki page together. We will also have the discussion here. > > That said, I would like to select at least three people who sign up to > > shepherd the discussion. Everyone is welcome and the committee members > > won't have any specific authority. I am just looking for people to form > > a > > working group and commit to making sure we can define and meet some > > objectives. > > > > ~Michael > > > > On 6/19/13 6:25 PM, "Upayavira" <u...@odoko.co.uk> wrote: > > > > >I'd encourage you to fire up a wiki page and start the discussion here. > > >I suspect due to te nature of the topic, participants will quickly be > > >self selecting. > > > > > >Upayavira > > > > > >On Wed, Jun 19, 2013, at 10:11 PM, Joseph Gentle wrote: > > >> On Wed, Jun 19, 2013 at 1:26 PM, Michael MacFadden > > >> <michael.macfad...@gmail.com> wrote: > > >> >>To start, I want to build a generic P2P OT container. This is a > simple > > >> >>wrapper that contains a set of OT documents and defines a network > > >> >>protocol for keeping them in sync. The container needs to be able to > > >> >>talk to another instance of itself running somewhere else and > > >> >>syncronize documents between the two instances. > > >> >> > > >> >>Thats all I want this container to do - it should be as lightweight > as > > >> >>possible, so we can port it to lots of different languages and > > >> >>environments. I want that code running in websites, in giant server > > >> >>farms, in vim, and everywhere in between. It won't have any database > > >> >>code, network code, users or a user interface (though it'll need > APIs > > >> >>to support all of that stuff). At its core it just does OT + > protocol > > >> >>work to syncronize documents. > > >> > > > >> > I strongly suggest we separate the OT Algorithms, the application > > >>level > > >> > protocol, and the network transport layer. > > >> > > >> The network transport layer should definitely be separated out. I'd > > >> also like to separate out the OT functions themselves (a la > > >> share/ottypes). > > >> > > >> I'm imagining coupling the concurrency control system and the > > >> application level protocol. This coupling might not be needed - I > > >> don't know enough yet to be able to tell. I think we should focus on > > >> figuring out what OT system(s) to use first. > > >> > > >> >>The OT itself I imagine building along the lines of Torben Weis's > P2P > > >> >>OT theory that he made in Lightwave: > > >> >>https://code.google.com/p/lightwave/ . Briefly, every operation > gets a > > >> >>hash (like git). We add tombstones to wave's OT type and remove > > >> >>invertability, so the transform function supports TP2. We also add a > > >> >>prune function (inverse transform) which allows the history list to > be > > >> >>reordered (so you don't have to transform out on every site). The > hard > > >> >>part is figuring out which operations to sync, and which operations > > >> >>need to be reordered. I'd like to go over the details with Michael > > >> >>MacFadden and anyone else who's interested - there may well be a > > >> >>better system that we should use instead. If there is, I'd like to > > >> >>know about it now. > > >> > > > >> > I think this is another interesting area. One thing to consider is > > >>that I > > >> > am not sure if the linear model that wave used is even the right > > >>option. > > >> > If we start manipulating things like JSON Objects, Java Object, XML, > > >>or > > >> > nested documents, a hierarchical path mechanism may be best. > > >> > > >> Yes. For ShareJS's JSON OT type, operations are defined by a path (a > > >> list) and an operation at that path. I'm not convinced by this design > > >> anymore - we should definitely discuss it. > > >> > > >> > My other > > >> > instinct here is, we should make sure that we base the OT on > something > > >> > that has been proven to be correct. There has been some work to > > >>evaluate > > >> > OT systems and prove that they are correct and solve the proper OT > > >>Puzzles > > >> > and support the required properties. > > >> > > > >> > Two other things that we need to consider. Beyond TP1 and TP2, > there > > >>are > > >> > also IP1, IP2, and IP3 that you need to think about if you are going > > >>to > > >> > support group undo, which in my opinion wave needs to support. If > you > > >> > can't undo things in collaboration, then you have problems. > > >> > > >> Using tombstones, I don't know how you can implement IP3. I've a mind > > >> to abandon formal invertibility, and simply rely on undo stacks for > > >> user level actions. But if we can find an architecture which meets our > > >> needs and can support invertibility, then that would be even better. > > >> > > >> > Basically, I am not confident that we know that wave's OT or what is > > >>in > > >> > lightwave is really what we want. I am not saying that they are > not, > > >>I am > > >> > just saying that I don't think we really have stated what we NEED > > >>from the > > >> > OT and then proved that a particular approach salsifies those needs. > > >> > Simply having a demo of something that works in a toy environment is > > >> > likely to have holes that we don't see until much, much later in the > > >> > development cycle. > > >> > > > >> > My recommendation here would be for us to form a small committee to > > >>do a > > >> > literature review on the topic and to foster some technical > > >>discussion. > > >> > The result should be something in the wiki that lays out a plan that > > >>the > > >> > community can comment on. > > >> > > >> I completely agree. I don't know enough about what our other choices > > >> are. You're much better read than I am and you're better connected to > > >> the academic OT community. Would you be interested in organising / > > >> chairing this committee? > > >> > > >> >>I'm also thinking about full end-to-end encryption. Especially in > the > > >> >>wake of the PRISM stuff, I'd quite like to make something secure. > > >> >>Snowden: "Encryption works. Properly implemented strong crypto > systems > > >> >>are one of the few things that you can rely on. Unfortunately, > > >> >>endpoint security is so terrifically weak that NSA can frequently > find > > >> >>ways around it." -- > > >> > > >>>> > http://www.guardian.co.uk/world/2013/jun/17/edward-snowden-nsa-files-wh > > >>>>ist > > >> >>leblower > > >> >>. > > >> > I like this idea. There is a paper from France that talks about > > >>nested > > >> > hashing, digital signing, etc that proves the authenticity of > > >>operations > > >> > that might also be worth looking at. I will try to dig up the > > >>reference. > > >> > > >> Great! > > >> > > >> -Joseph > > > > >
