How many entities were there when the batch put failed? Was it the size of the entities or the number of entities that caused the batch put to fail?
Jeff On Sat, Oct 30, 2010 at 8:39 AM, Stephen <[email protected]> wrote: > > > On Oct 29, 6:24 pm, Joshua Smith <[email protected]> wrote: > > I'm running into a too-large exception when I bulk put a bunch of > entities. So obviously, I need to break up my puts into batches. I want to > do something like this pseudo code: > > > > size = 0 > > for o in objects: > > if size + o.size() > 1MB: > > db.put(list) > > size = 0 > > list = [] > > list.append(o) > > > > Any idea what I could use for the "o.size()" method? I could crawl > through all the fields and build up an estimate, but it seems likely to me > that there is a way to get the API-size of an entity more elegantly. > > > How about something like: > > > from google.appengine.api import datastore > from google.appengine.runtime import apiproxy_errors > > def put_all(entities, **kw): > try: > return datastore.Put(entities, **kw) > except apiproxy_errors.RequestTooLargeError: > n = len(entities) / 2 > a, b = entities[:n], entities[n:] > return put_all(a, **kw).extend(put_all(b, **kw)) > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Jeff -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
