Agreed, setting the WaveletDeltaRecord object as the top level document sounds sensible, I'll have a go at serializing it once I've built the rest of the model around it and created a workaround for the '.' issue that I'm seeing from attempting to save down the wave domain. I'll have to replace it with another char that can't be used inside domain names.
I have some concerns about serializing the WaveletDeltaRecord object as it has a fairly extensive network of dependencies, including proto classes for the delta ops, it may be easier to replicate the FileDeltaStore rather than base the mongo serialization on the memory implementation. But I'll have a go at the WaveletDeltaRecord first, it would be interesting to know why the file implementation is so different to the memory version, it may give us some guidance on the best route to follow. Additionally does anyone know if there are known issues for either of the memory or file delta stores? There are some issues on JIRA but only one that seems to directly name the file store as buggy. Any thoughts or advice from anyone here would be appreciated. Regards Ben On Thu, Jan 2, 2014 at 5:26 PM, Pablo Ojanguren <pablo...@gmail.com> wrote: > The document size limit only matters in a document-per-wavelet data model > approach. As long as updates (1) in existing documents are expensive > operations in mongoDB a more granular approach would be suitable. > > I suggest a first simple data model with a document-per-delta approach, > storing basically the data of a WaveletDeltaRecord object. It seems that > the major effort here is to serialize properly objects from the > WaveletOperation class heriarchy. > > Of course we need to upgrade mongoDB version to ensure durability (2) > > (1) See document growth operational factor > http://docs.mongodb.org/manual/core/data-model-operations/ > (2) See Write concerns http://docs.mongodb.org/manual/core/write-concern/ > > > > > > 2014/1/2 Ali Lown <a...@lown.me.uk> > > > I have seen the deltas files regularly hit >10MB. > > The largest root conversation wavelet I have is 13300kB. (I think this > > is only several hundred blips) > > > > My next question is why you are trying to store the serialized deltas > > for a whole wavelet in a single field in mongo? > > > > On 2 January 2014 16:45, Ben Hegarty <heg...@gmail.com> wrote: > > > Does anyone know if the deltas or index files are likely to ever hit > > 16MB, > > > as that is the maximum document size inside mongodb and if we are then > > the > > > persistence will need to be pushed into the mongodb GridFS? > > > > > > > > > On Thu, Jan 2, 2014 at 9:58 AM, Frank R. <renfeng...@gmail.com> wrote: > > > > > >> This feature is on my wish list for Wave. It's a good news. Thank you~ > > >> > > >> > > >> On Thu, Jan 2, 2014 at 5:24 PM, Ben Hegarty <heg...@gmail.com> wrote: > > >> > > >> > Hi Pablo, > > >> > Ironically I started work on this feature over the weekend as it is > > next > > >> on > > >> > my list of must haves for my own personal project which also uses > > >> MongoDB. > > >> > I've had to upgrade the MongoDB driver and add in the morphia plugin > > for > > >> > modelling the delta store which hopefully I'll be able to continue > > >> working > > >> > on over the weekend. My time is limited and so if you think you'll > be > > >> able > > >> > to finish before me let me know and I'll try and help with any > > knowledge > > >> > I've gained around mongoDB. It looks like getting the delta store > > >> finished > > >> > is going to be fiddly work from what I've seen so far, I started by > > >> > modelling the store on the memory version, but the first problem > I've > > >> found > > >> > is that the '.' in mongoDB is a reserved character so I'm going to > > have > > >> to > > >> > start modelling the store using a mongoDB specific implementation. > > >> > > > >> > If it helps I've uploaded my current work > > >> > here< > > >> > > > >> > > > https://drive.google.com/file/d/0B5FF_Ld8SzsNQ2JONHdGU2xJUG8/edit?usp=sharing > > >> > >which > > >> > just keep in mind is very much just the beginning. > > >> > > > >> > HTH > > >> > Ben > > >> > > > >> > > > >> > On Thu, Jan 2, 2014 at 8:41 AM, Pablo Ojanguren <pablo...@gmail.com > > > > >> > wrote: > > >> > > > >> > > Hi all, > > >> > > > > >> > > I am a developer working for an EU research project. There, we are > > >> going > > >> > to > > >> > > use Wave (via Kune project) so we would like to help fixing bugs > and > > >> > > improving the software along wiht the community. Currently I have > > some > > >> > > available time for this for the next 10 months. > > >> > > > > >> > > After check out the issues, first thing I've paid attention is the > > >> > Delta's > > >> > > storage. I have reviewed server / persistence code and I think I > can > > >> > start > > >> > > working in a mongoDB implementation. > > >> > > > > >> > > But maybe there is somebody else working on this. I would like to > > join > > >> > them > > >> > > and to avoid duplicated efforts. Or if there is another > outstanding > > >> > topics > > >> > > to suggest it would be great to know too. > > >> > > > > >> > > Thank you and Happy New Year. > > >> > > > > >> > > Pablo > > >> > > > > >> > > > >> > > > >> > > > >> > -- > > >> > Mobile Phone: +447767 322 122 > > >> > Home Phone: +4420 7737 6530 > > >> > > > >> > > > > > > > > > > > > -- > > > Regards > > > Ben > > > -- Regards Ben
