2012/2/23 Will Moss <wm...@bu.mp>: > Hey Dominik, > > To answer the question in your subject line, no. To quote one of the guys I > work with, "You're the bitch of data." That said, I'm extremely happy we did > it. Riak does what it says it's going to do when it says it will and doesn't > seem to really ever go down. I can't say the same for Mongo. > > We did this for some of the data we have in Mongo (and are planning to move > the rest in the near future) and it certainly wasn't painless. What we did > that worked reasonably well was: > 1. Change all our mongo query patterns so that we were effectively using it > a key-value store (i.e. no secondary indexes or other complex queries). > 2. Write code that read an existing record from Mongo and wrote it to Riak. > The actual schema stayed about the same, but we decided to go with protocol > buffers in Riak since they are much more efficient at representing data. We > mapped a single collection to a single bucket in Riak. > 3. Change all the queries to Riak instead of Mongo. > 4. Then, when we were ready to actually kick off the migration, we migrated > uses as the logged in. So, if they had not been migrated, we delayed their > login slightly while we copied the necessary data out of mongo and into > Riak. > > > Will
What Will said is a nice practice in general for migrating between data storage systems. My general recommendation is: - Dark launch new code paths that concurrently writes data to both systems - Optionally dark launch reads against the new subsystem as well but only log for comparison reasons. No presentation changes. - Validate data in new subsystem - When you're happy that things are looking good, you come to a bit of a fork: - migrate the user wholesale on login or - run in an incremental fashion on reading a subset of data (old user messages are copied into Riak only when accessed). The later requires a few more iterations and there's a probably a slight performance hit from talking to multiple datastores. However the alternative is to take a big maintenance window and risk a big migration that may or may not work. By going incremental with dark launches you make it mostly transparent to the user and can work out any latent bugs. I've done this personally at a few places. The last time was migrating blobs from Voldemort into S3 over the period of a month using a similar approach. We'll probably do something similar at the new gig when we migrate into Riak from MySQL. - John http://about.me/lusis _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
