Saving 100 entities in a single request, especially if a TON of indexes will be created is very expensive and will cause a lot of CPU to be consumed. There probably is a design that will allow you to accomplish what you need to do without this limitation. Can you post your use case?
On Thu, Aug 12, 2010 at 11:02 PM, SC2PlayerFinder <[email protected] > wrote: > > Used appstat and didn't notice anything out of the ordinary. Still > thinking that the main issue here is exploding index and I'm not sure > how to get around that other than to cut search features from the > site, but that would more or less make the app not very useful. > > On Aug 12, 10:19 pm, SC2PlayerFinder <[email protected]> > wrote: > > Thanks for the response. I wanted to get back to you sooner, but I > > have been scrambling to optimize the application. I'm making some > > progress, however, it is still a relative CPU hog due to an exploding > > index and the frequent updates the application needs to make in order > > to keep the data current. 71000 CPU to save 100 records seems rather > > expensive?. > > > > My next step is to run appstat on it and see what is going on. > > > > Current version of the app is running a sc2pf6.appstat.com. The older > > version is running at sc2pf2.appspot.com. > > > > On Aug 9, 3:51 pm, "Ikai L (Google)" <[email protected]> wrote: > > > > > > > > > I've got a few questions for you: > > > > > 1. What is your application ID? > > > 2. What are you doing in the tasks? > > > 3. Are you using transactions or entity groups? > > > 4. Are you writing to the same datastore entities? > > > > > My guess is that you are running into write contention in the > datastore. > > > It'd be very helpful to understand what it is you're doing. > > > > > App Engine will scale up the number of instances if you are returning > web > > > requests in less than 800ms (thought 400ms is optimal), but if you are > > > writing to a small number of entities over and over it won't be able to > > > resolve that bottleneck. > > > > > On Mon, Aug 9, 2010 at 12:28 PM, SC2PlayerFinder > > > <[email protected]>wrote: > > > > > > Further explanation. I had a large increase in users which caused a > > > > chain reaction. Tasks that used to take about 10-20 seconds started > > > > taking longer and using more CPU time. Interestingly, these tasks are > > > > doing the exact same thing, but under a higher load, they take longer > > > > and use more CPU time. I can understand the process taking longer > > > > under load, but why would it use a significant amount more of CPU > > > > time? I will admit that my code is not the most optimal, but then > > > > again isn't the main focus of GAE built in scalability and speed? In > > > > short, I needed on the fly scalability and speed with an influx of > new > > > > users and I don't feel like I got that with GAE? Worse, there was no > > > > way for me to contact someone who could help me out. > > > > > > I'm left with the feeling that GAE does not do a good job of scaling > > > > up and it is kind of slow as well. Creating and updating records > > > > seems painfully expensive, time out due to new instance creation > > > > happens way to often. Now this is fine for a free app, but I'm going > > > > way over my daily free quota and it is not cheap. Maybe I should be > > > > using Phython and not Java? > > > > > > On Aug 6, 10:52 pm, SC2PlayerFinder <[email protected]> > wrote: > > > > > Application was running ok yesturday. Still had about 10% > > > > > HardDeadlineExceededError, but the same task that executed > yesturday > > > > > are not failing almost 100% of the time right now. I even cut back > > > > > the amount of records it is processing from 100 to 10 and it is > still > > > > > failing. > > > > > > > How do I get ahold of someone at Google, to see if there is > something > > > > > wrong with the server my app is on or something. Why would the same > > > > > proccess consume more CPU time one day than another? > > > > > > -- > > > > You received this message because you are subscribed to the Google > Groups > > > > "Google App Engine for Java" group. > > > > To post to this group, send email to > > > > [email protected]. > > > > To unsubscribe from this group, send email to > > > > [email protected]<google-appengine-java%[email protected]><google-appengine-java%2B > [email protected]> > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine-java?hl=en. > > > > > -- > > > Ikai Lan > > > Developer Programs Engineer, Google App Engine > > > Blog:http://googleappengine.blogspot.com > > > Twitter:http://twitter.com/app_engine > > > Reddit:http://www.reddit.com/r/appengine > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine-java%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
