@Robert Thanks. I understand what you say. I will try to boil down my set of problems into the ones that would affect the majority of my users. Thank you much for the awesome suggestion about Bloom Filters. I did not know about it and would not have, had you not pointed me to it. It seems like just the solution that would help me do fast searches over my ids.
I am not worried about false positives (if they happen in small percentages) and I will not be deleting any of the ids inserted. So this seems to fit right into my use case. And Billions of entities! Wow, I'm yet to go above a few million, it helps to have got suggestions from you :) @Greg Ah! Arrays, I think you just solved my problem of persisting all the many entities. So here's what I'm planning to do based on suggestions from all of you : 1. Persist the ids in an array 2. Have a Bloom Filter for searches My app can tolerate single digit false positive percentages in searches so I would not really need to unpack the arrays saved until the false positives increase by a huge margin (I can go for sharding and have one bloom filter for each shard in the future). At times I would need to analyze the ids, and I think I can live with having to unpack the id array. Hoping this approach is scalable and solves my problem. I'm thankful to all of you for taking time out to help me. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
