Hi Daniel, I strongly suspect you're going to need a different solution to transfer that much data out in a timely manner. The best solution depends on your write rates and update patterns
Robert On Sep 1, 2011 1:15 PM, "Daniel" <[email protected]> wrote: > > We will be facing the same problem as we leave app engine. I believe > I'm going to attempt to use the bulk downloader to get the data to a > local computer. I am slightly worried about the cost of transferring > 11TB. I believe that the process make take a couple of days, so I plan > on moving inbound records to the new database before I start the d/l, > that way I can have reduced functionality over the data transfer > period but no data will be lost. > > -Daniel > > On Sep 1, 8:22 am, Millisecond <[email protected]> wrote: > > Won't rehash the pricing discussions, but because of the pricing > > changes and the way it's been handled we're 90% sure we're going to > > move to AWS. > > > > Anyways... now I'm tasked with figuring out how to get our almost TB > > of data over there either into SimpleDB or into an RDS instance, not > > sure yet. > > > > Have other people done the move, how did it go? Did you pull from > > AWS / push from GAE? Take an intermediate backup and then load from > > some other mechanism (thinking S3) into SDB / RDS? > > > > Our app is adding over 1 GB / day with 20MM reads and 20MM writes, and > > we'd rather not take it offline for too long... Thinking of some > > crazy scheme based on descending keys to move data over from key A > > "downwards", shut off app, move over everything from key A "upwards" > > as we're mostly only writing new data. Maybe special-case a few > > object classes and use timestamps to detect deltas. Realize it's > > pretty app-specific, but wondering if other people have tackled that > > same problem and what their experiences were. > > > > Thanks for any and all help, > > -Casey > > -- > You received this message because you are subscribed to the Google Groups "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to [email protected]. > For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
