If you guys haven't seen Johan's community update, here's a link to sign up for early access to the migration tool:
https://docs.google.com/a/google.com/spreadsheet/viewform?authkey=CLXR0LMN&formkey=dERMcDZuMnlycHoyZDd4Vy1PNXlhWlE6MQ#gid=0 -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Fri, Aug 5, 2011 at 12:30 PM, Robert Kluin <[email protected]>wrote: > On Fri, Aug 5, 2011 at 07:17, Joops <[email protected]> wrote: > > Hi all, > > > > Just to through my migration hat in the ring. > > > > I migrated my (non-production app) from MS to HR and it worked fine. > > (or at least it seems to have worked fine) > > I had used the Key ID's instead of Keys to handle references between > > entities. > > > > Am i correct in saying that the only benefit of using key's over id's > > is that you get the nice syntactic sugar way of fetching things? > > Keys can also encode the namespace and any partent-child relationship > data; if those two things are not a concern then there is effectively > no difference. > > Storing the ids directly is also not-bad because you will not > accidentally dereference it leading to lots of inadvertent datastore > calls. Your code will be nice and explicit. > > > Robert > > > > > > > > > Thanks very much, > > > > J. > > > > On Aug 5, 5:45 am, Robert Kluin <[email protected]> wrote: > >> So here are my thoughts regarding this: if I have to map over all of > >> my data re-writing it to prepare it for the tool to convert, why not > >> just migrate to the other app while I'm doing that? Otherwise I'm > >> going to pay to read and write it to fix it, then pay again to read > >> it, transfer it, and rewrite it. This adds up fast if you've got much > >> data. > >> > >> Aside from the cost factor, the migration to HR is an opportunity to > >> "fix" or improve your schemas. Things like long field names, unused > >> indexes, entity groups, etc.... All of those items that require > >> rewriting all of your data to adjust -- the migration is a great time > >> to do it. On a migration I've recently been working on, I estimate I > >> can reduce our storage use by about 50% with very minor changes (field > >> names and indexes!). > >> > >> I've already migrated a couple apps, each time I've setup a job to map > >> over my data, maybe do some conversions, put it in some suitable > >> format (like csv or json) then send batches to the new app. On the > >> new app side I write a simple 'receiver' handler that does any needed > >> conversions and then writes the entities (in batches or one-by-one > >> transactionally, depending on the case). > >> > >> Writing the sender and receiver handlers has usually been pretty quick > >> for me, even with very complex data. The other advantage is that you > >> may not need to move all of your data. Some items like stats that are > >> computed from your original data, may be able to be recomputed by the > >> exact same code as it comes in. > >> > >> Would love to hear other people's thoughts on this. > >> > >> Robert > >> > >> > >> > >> On Thu, Aug 4, 2011 at 21:28, Greg Darke <[email protected]> wrote: > >> > Yes, this is the correctwayto perform this operation. I would > >> > suggest adding a new property to the model to store the keys and > >> > update your code to use this new property. This will allow you to > >> > continue using your application while you are performing your schema > >> > migration. > >> > >> > For example: > >> > >> > # Old model definition > >> > class MyModel(db.Model): > >> > links = db.StringListProperty() > >> > >> > # New model definition > >> > class MyModel(db.Model): > >> > links = db.StringListProperty() > >> > link_keys = db.ListProperty(db.Key) # You may also want to set > >> > indexed = False > >> > >> > def apply_schema_change(self): > >> > if not links: > >> > # There is nothing to update > >> > return > >> > self.link_keys = [db.Key(str_key) for str_key in self.links] > >> > self.links = [] # We don't use this anymore, so delete the data > >> > >> > Then you would call the apply_schema_change method after reading the > >> > model in from datastore, ensuring that the data is now in the new > >> > format. > >> > >> > Then I would use the mapper framework to apply the new schema to all > >> > of the entities in your datastore. > >> > >> > On 5 August 2011 10:53, Tim Hoffman <[email protected]> wrote: > >> >> Hi John > >> >> I have a db.ListProperty(db.String) or db.StringListProperty and I > have > >> >> stored keys in it, which of course will be in the str(key) form. > >> >> The conversion tools doesn't convert this sort of thing mainly > because > >> >> unless there is an additional mechanism for > >> >> describing what this field holds the conversion tools can't know. > >> >> So you would need to something like > >> >> Convert all the strings back to keys > >> >> keys = [db.Key(i) for i in obj.some_list_property] > >> >> then stick them in some property that is defined as > db.ListProperty(db.Key) > >> >> Now the trick will be working out the order of events and possibly > >> >> performing multiple stages > >> >> so you can get you data converted and make the sites work before and > after > >> >> transfer/conversion > >> >> Haven't really worked out the order of events I have to go through. > >> >> Rgds > >> >> Tim > >> > >> >> -- > >> >> You received this message because you are subscribed to the Google > Groups > >> >> "Google App Engine" group. > >> >> To view this discussion on the web visit > >> >>https://groups.google.com/d/msg/google-appengine/-/8_fgPFgpABAJ. > >> >> To post to this group, send email to > [email protected]. > >> >> To unsubscribe from this group, send email to > >> >> [email protected]. > >> >> For more options, visit this group at > >> >>http://groups.google.com/group/google-appengine?hl=en. > >> > >> > -- > >> > Greg Darke > >> > >> > -- > >> > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > >> > To post to this group, send email to > [email protected]. > >> > To unsubscribe from this group, send email to > [email protected]. > >> > For more options, visit this group athttp:// > groups.google.com/group/google-appengine?hl=en.- Hide quoted text - > >> > >> - Show quoted text - > > > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > [email protected]. > > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
