On Fri, Sep 13, 2013 at 9:28 AM, Jason Collins <[email protected]> wrote:
> We have 50,000,000+ search documents that we need to change (i.e., delete > the old version, create a new version). These are sharded across many > individual indexes. > > I think the per-second rate limit for interacting with the search API is > 50/s. We need to leave some room for live traffic, so if we clamp our > re-indexing to 25/s, that means it will take 23 days to process all these > documents. > > Does anyone have other ideas? > Do you have a Silver or above Google Support package? If you do, you can try calling up Google to discuss a quota increase. See https://developers.google.com/appengine/docs/python/search/#Python_Search_API_quotas_and_pricing where it says: *Customers with Silver, Gold, or Platinum support who need to exceed these limits should contact their support representative to discuss their needs.* Other than that, there's not too many other options. Perhaps the next-best option would be to open up a Lucene ( http://lucene.apache.org/core/ ) instance and temporarily let your customers search with it, while Search API rebuilds the index. ----------------- -Vinny P Technology & Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
