Those are really good ideas. Would there be any way you can think of to push data back up to the server?
On Mar 12, 9:43 am, Daniele <[email protected]> wrote: > For the import part if it has not to be "atomic" you could create > another app and use it to feed via a web service the data on the > production database while the main app is live. > > To download only the main dataset create a boundle, animate it on a > new application, delete the huge data with a migration or with a sql > query in the console and the pull the database locally. > > By the way I'm investigating about how to work with big db too. Coming > from a standard server world I love Heroku but some steps could be a > bit less flexible. Nothing is perfect :) > > On 12 Mar, 07:29, Mike <[email protected]> wrote: > > > I'm going to be adding a number of discrete, but enormous (maybe many > > gigs each), datasets to my Heroku app's database. In many ways, I'm > > in a similar situation faced by Tobin in another current post, but > > with a different > > question:http://groups.google.com/group/heroku/browse_thread/thread/141c3ef84b... > > > Right now I still haven't merged the datasets into my database yet. > > What's the best way for me to approach this? > > > The lack of ability to push individual tables with taps suggests to me > > I'm going to want to do this probably as a one shot deal, rather than > > doing each dataset sequentially and testing that one before proceeding > > to the next. I'm thinking about doing a db:pull to get the current > > state of my database, and then shutting down my application in > > maintenance mode, running a local merge of the datasets (maybe taking > > days I'm guessing just to process the enormous things), doing some > > exhaustive local testing on the result, and then doing a push back to > > Heroku (maybe taking days again), before reactivating my app. Because > > of their massive size, it seems like after I've done one, doing any > > further db:pulls is going to be basically impossible. Just the idea > > of possibly having made a mistake in merging the datasets that I don't > > catch until after it's been pushed to the site gives me the shivers. > > Overall, I wonder if there could be a better way that I'm overlooking. > > > One possible alternative I thought of is would it be possible to do > > something involving creating a local bundle from my database using > > YamlDB? But then I'm not sure how to get the bundle back onto the > > server and then to restore from it? The documentation on Heroku > > doesn't seem to really talk about that possibility. > > > Also, in my case this data is integral to the application, so I'm not > > going to be able to split it up into a separate Heroku application > > like in Tobin's case. Is there going to be any practical way for me > > to be pulling just the non-dataset data from the server in order to > > use on a development machine? > > > Does anyone have any ideas on how they would approach this problem? > > If so, I'd be filled with gratitude. > > > Mike > > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
