This looks really great, thanks for sharing. Only thing I see that I'd be concerned about are reliability on larger databases.
You have a step where it's downloading the image from S3 onto the Heroku node. If you have a database that's more than a GB, I wonder if you could run into any problems there. Also, I'd previously used a solution where I use pg_dump for my backups. Unfortunately, I ran into a problem there where my database (about 1GB) was taking longer than the Heroku database query execution limit, and failing. Normal commands to increase the query execution timeout don't work for pg_dump since it runs in a separate environment. I conferred with Heroku's customer service about this, but there didn't seem to be a rectify this problem. The way I ended up doing my backups is to create a cron job that deletes my old bundle, and creates a new bundle. Bundles contain within them a pg_dump, but run in a special environment I guess that isn't subject to the limits. I then have the cron job push that to Amazon. Unfortunately, restores are much more of a pain, and I would have to wipe the Heroku database, download the bundle locally, pg_restore it locally, and then push it up with taps. On May 10, 8:50 pm, Carl Anderson <[email protected]> wrote: > Excellent to hear. > > Carl > > > > On Mon, May 10, 2010 at 5:48 PM, Jason <[email protected]> wrote: > > I sure have. I've run it to transfer between environments and as a > > cron. It will be running nightly for me from now on. > > > Jason > > > On May 10, 7:43 pm, Carl Anderson <[email protected]> wrote: > > > That looks pretty helpful. Have you tested it with any production > > datasets > > > yet? > > > > Carl > > > > On Mon, May 10, 2010 at 5:17 PM, Jason <[email protected]> wrote: > > > > Just wanted to let you guys/girls know that I've created a project up > > > > on github with backup and restore tasks using pg_dump. I've wanted > > > > this for a long time, and I figured some of you might too. I've seen > > > > a lot of backup solutions (which I've borrowed heavily from), but very > > > > few restores! I could just imagine scrambling to restore the > > > > database, having not tested it before. > > > > > Anyway, hope some people get some use out of this. > > > > >http://github.com/jpearl/heroku_backups > > > > > From the readme: > > > > > These rake tasks are basically a cleaned up and consolidated version > > > > of everything good I could find related to backup of heroku > > > > databases. It essentially takes advantage of pg_dump and pg_restore > > > > capailities of postgres. > > > > > You can use it to run a nightly backup cron, or to transfer between > > > > environments. > > > > > heroku rake db:backup --app from_application > > > > heroku rake db:restore -app to_application > > > > > Notes: > > > > * You'll need to place your backup bucket name where it indicates in > > > > database.rake > > > > * You will need aws_s3 and yaml_db, since we're dumping both a > > > > database agnostic yaml file and a pg_dump compressed > > > > backup > > > > > Things that could be done: > > > > * Most of you may not care about the yaml_db portion. I like it > > > > because I use mysql locally, even though I'm running on postgres at > > > > Heroku. You could easily rip this part out. > > > > > Hope this helps some folks. > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Heroku" group. > > > > To post to this group, send email to [email protected]. > > > > To unsubscribe from this group, send email to > > > > [email protected]<heroku%[email protected]> > > <heroku%[email protected]<heroku%[email protected]> > > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/heroku?hl=en. > > > > -- > > > You received this message because you are subscribed to the Google Groups > > "Heroku" group. > > > To post to this group, send email to [email protected]. > > > To unsubscribe from this group, send email to > > [email protected]<heroku%[email protected]> > > . > > > For more options, visit this group athttp:// > > groups.google.com/group/heroku?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Heroku" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<heroku%[email protected]> > > . > > For more options, visit this group at > >http://groups.google.com/group/heroku?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Heroku" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group > athttp://groups.google.com/group/heroku?hl=en. -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
