Alan, In general, canaried rollouts are used whenever possible. Most of the time, maintenance happens without any downtime or impact to the user. App Engine is actually quite sophisticated in terms of instance migration and some of the limits of App Engine exist because of our scalable, highly available infrastructure: one of the reasons we require developers to break long running jobs into small tasks is because we build for processes that can be short lived and migrated around data centers as needed.
Scheduled maintenance occurs because there are some maintenance operations which either cannot be done in a phased manner or would simply cost users too much to maintaining full operability while it takes place - there's a balance here we maintain. Each of these maintenances is carefully considered and must meet the criteria of having the long-term payoff of less scheduled downtimes. There are always, always risks with maintenances, and it'd be naive to assume that phased rollouts aren't already being used - they are, but some upgrades just can't happen this way. On Wed, Feb 17, 2010 at 8:41 PM, Alan Xing <[email protected]> wrote: > GAE is still having serious post-maintenance hiccup. My cron jobs and > deferred tasks essentially stopped executing since 6:20pm, the time > maintenance was supposed to be done. Plus, we can not do deployment at this > moment. These kind of maintenance nightmare should not happen again. > > This should be a big lesson for GAE. People including me have good trust on > Google and GAE team. You guys should do the best to live up to people's > expectation. Otherwise, you will lose in the cloud computing game. > > One experience that I learned when I worked for Salesforce.com may help the > GAE team next time. In Salesforce.com, a major product release or scheduled > maintenance usually happen only to a portion of their data center, so that: > 1, a majority of the customer base won't get affected if something bad > happen. 2, most of the time, traffic can get redirected to the old portion > of data center if the upgrading portion has problem. > > Alan > -- > http://twitter.com/alanxing > http://twitter.com/xinghailiang > http://facebook.com/alanxing > http://www.linkedin.com/in/alanxing > http://twitter.com/snsanalytics > http://facebook.com/snsanalytics > http://www.snsanalytics.com provides targeted social marketing solutions > for everyone! > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
