tldr; I need some volunteers to collaboratively edit a document together, so we can systematically evaluate algorithmic performance.
So recently Michael linked me to a paper[1] which evaluates a bunch of different concurrency algorithms on speed & memory usage. They got a bunch of students to collaboratively edit two documents and used the operations generated in their benchmarks. The paper has some glaring omissions[2], and the data they gathered isn't publicly available. Of course, I also want to test Torben's algorithm to see how well it performs with realistic usage. So I'd like to reproduce their experiment. To do this I need a few volunteers to collaboratively edit some documents. We should construct realistic editing scenarios. The paper did two things: - Transcribe an episode of big bang theory - Write a report I'm open to suggestions on what we should do - we could also try collaborative creative writing, writing notes on a youtube video, or something. It doesn't really matter so long as the activity is focused, realistic (no keyboard mashing) and involves collaboration. (Sequential editing scenarios aren't interesting) To do this, I'll set up a special instance of ShareJS with ~1s of artificially induced latency and extra logging for the experiment. I want to run this experiment either late next week or on the weekend. The more experimental runs the better - although I suspect most of what we learn will be from the first couple logs. I will publish the raw data from the logs and send out a followup email. The experiment will be anonymous, but don't say anything you wouldn't want publicly known. How does that sound? Who's willing to help out? -J [1] http://hal.archives-ouvertes.fr/docs/00/62/95/03/PDF/doce63-ahmednacer.pdf [2] Criticisms: - Operations only insert or remove a single character, which means that a copy+paste that one of the users did resulted in 5000 operations, each of which needed to be transformed individually. - Their text editor didn't batch changes - which is really stupid and unrealistic. - The students were all working locally (on a LAN), so there would have been fewer concurrent actions than we should realistically expect.
