I'm having to go through the same thing. My plan was to have a modified date on all the data. Then transfer over all data up to the current date and keep track of the timestamp. Do this while the site is live. Then put the site in read only mode then do the same transfer starting at that timestamp up until the latest and that will get all the extra difference, but it'll be a lot smaller. You also have to account for data that was modified that might already by in AWS from the first transfer and just update it instead of inserting it.
I'm also planning on pruning as much data as I can before hand. I haven't done any of this, but that's what my plan is right now. Let me know if you come up with something different. Hope that helps. David On Sep 1, 5:22 am, Millisecond <[email protected]> wrote: > Won't rehash the pricing discussions, but because of the pricing > changes and the way it's been handled we're 90% sure we're going to > move to AWS. > > Anyways... now I'm tasked with figuring out how to get our almost TB > of data over there either into SimpleDB or into an RDS instance, not > sure yet. > > Have other people done the move, how did it go? Did you pull from > AWS / push from GAE? Take an intermediate backup and then load from > some other mechanism (thinking S3) into SDB / RDS? > > Our app is adding over 1 GB / day with 20MM reads and 20MM writes, and > we'd rather not take it offline for too long... Thinking of some > crazy scheme based on descending keys to move data over from key A > "downwards", shut off app, move over everything from key A "upwards" > as we're mostly only writing new data. Maybe special-case a few > object classes and use timestamps to detect deltas. Realize it's > pretty app-specific, but wondering if other people have tackled that > same problem and what their experiences were. > > Thanks for any and all help, > -Casey -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
