Thanks for all the responses. Originally I was leaning toward the batch transformation approach thinking that this would help keep map-reduce jobs as simple as possible since they wouldn't have to concern themselves with multiple versions of documents. Perhaps I'm just so used to schema migrations in RDBMS that this is just how I think. The lazy object upgrading technique is appealing. I have a small project coming up where I can test that out to see how it goes.
Thanks again. On Wed, Sep 19, 2012 at 6:52 PM, Todd Cullen <t...@thoughtleadr.com> wrote: > ThoughtLeadr uses Bip's approach. Its essential lazy schema migration. > > > On Wed, Sep 19, 2012 at 3:03 PM, Reid Draper <reiddra...@gmail.com> wrote: > >> I have also used Bip's method and found it successful. You do need to be >> careful to make sure >> all reads of your data go through this upgrade-path, but that is easy >> once you realize it :). >> >> >> Reid >> >> On Sep 19, 2012, at 5:44 PM, Bip Thelin <b...@kivra.com> wrote: >> >> We use a technique that works well for our use case: >> We store everything as JSON and use the erlang-client, for every object >> we store we add an opaque schema_version tag to the JSON and then in our >> model encode/decode routine we parse this schema_version and if the version >> is not the latest according to the model we recurse and upgrade the payload >> all the way up to the current version and then return the new object to the >> client and spawn an async process that saves the new object back. >> >> So for instance an object is at schema_version 1 when it get's read. The >> model is now on version 5 there will be five functions, i.e: >> upgrade_version(2, Payload) -> >> do_something(); >> upgrade_version(3, Payload) -> >> etc... >> >> every function knows how to upgrade itself from it's former version so >> fast forwarding through version is easy and every object will get upgraded >> eventually (if the get read). >> >> -- >> Bip Thelin >> >> KIVRA | Lugnets Allé 1 | 120 33 Stockholm >> Tel 08-533 335 37 | Mob 0735-18 18 90 >> www.kivra.com >> >> >> >> >> On 19 sep 2012, at 00:55, Allen Johnson <akjohnso...@gmail.com> wrote: >> >> Hey everyone, >> >> I'm beginning to experiment with Riak and I'm trying to better >> understand how to model my data. One question I have at the moment is >> how to evolve my data with a data store such as Riak? I know that >> it's schema-less and that I can add new fields as needed but I'm >> thinking more about the existing documents. >> >> For example, say hypothetically, that I have a fairly successful >> riak-based app. As the application code continues to evolve over >> several versions I begin to find a "better" way to model my data. By >> this time I have already stored many, many documents. What is the >> appropriate path here? Do I version my documents with metadata and >> rely on my application code to continue to deal with old-style >> documents -- or do I perform some sort of bulk transformation on these >> existing documents? >> >> Thanks, >> Allen >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > > > -- > Cheers, > > Todd Cullen | Founder & CEO of ThoughtLeadr > 177 Post Street, #550 SF 94108 > e. t...@thoughtleadr.com p. 415-254-0787 > twitter: @shakakai <http://www.twitter.com/shakakai> skype: todd.cullen > > Fast Company covers > ThoughtLeadr<http://www.fastcocreate.com/1680828/imgur-blurs-line-between-memes-and-ads-with-lipton-briskscumbag-steve-hybrid> > > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
