I'm going to be adding a number of discrete, but enormous (maybe many gigs each), datasets to my Heroku app's database. In many ways, I'm in a similar situation faced by Tobin in another current post, but with a different question: http://groups.google.com/group/heroku/browse_thread/thread/141c3ef84b22fc18
Right now I still haven't merged the datasets into my database yet. What's the best way for me to approach this? The lack of ability to push individual tables with taps suggests to me I'm going to want to do this probably as a one shot deal, rather than doing each dataset sequentially and testing that one before proceeding to the next. I'm thinking about doing a db:pull to get the current state of my database, and then shutting down my application in maintenance mode, running a local merge of the datasets (maybe taking days I'm guessing just to process the enormous things), doing some exhaustive local testing on the result, and then doing a push back to Heroku (maybe taking days again), before reactivating my app. Because of their massive size, it seems like after I've done one, doing any further db:pulls is going to be basically impossible. Just the idea of possibly having made a mistake in merging the datasets that I don't catch until after it's been pushed to the site gives me the shivers. Overall, I wonder if there could be a better way that I'm overlooking. One possible alternative I thought of is would it be possible to do something involving creating a local bundle from my database using YamlDB? But then I'm not sure how to get the bundle back onto the server and then to restore from it? The documentation on Heroku doesn't seem to really talk about that possibility. Also, in my case this data is integral to the application, so I'm not going to be able to split it up into a separate Heroku application like in Tobin's case. Is there going to be any practical way for me to be pulling just the non-dataset data from the server in order to use on a development machine? Does anyone have any ideas on how they would approach this problem? If so, I'd be filled with gratitude. Mike -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
