Hi Robert, Sorry my ramblings were hard to parse. Basic question is the first you mentioned, "Why is there the TQ option for DEFAULT_APP_VERSION, and when is it best used?"
There was a discussion related to a priority TQ a year or so ago. Someone (Vlad?) initiated a request for a priority TQ, and I did star it at the time. Cheers, Steve On Jan 27, 10:16 pm, Robert Kluin <[email protected]> wrote: > Hey Steve, > I'm a bit confused about what you're asking / commenting about here. Are > you asking about the default queue, why there is a > taskqueue.DEFAULT_APP_VERSION variable, or whether it is better to send > tasks to a different version of your app? > > I generally prefer to use the same version of my apps for both tasks and > user requests. The biggest reason queues slow down is because there aren't > enough instances. Splitting user requests and tasks up will probably just > make that problem worse. > > My tasks generally run quite fast, but they do occasionally back up when I > font have enough instances. The taskqueue team recently added a new header > you can use to research the execution latency of your tasks. > > I know several others have asked for the ability to define a high priority > push queue as well. I agree that this would be a very powerful feature. > Particularly if you could define the relative priority to user requests and > other queues. I'm not sure if there is an existing feature request for > this or not, but if you make one I'll star it. > > Robert > > > > > > > > > > On Friday, January 27, 2012, stevep <[email protected]> wrote: > > Thanks Robert, > > > I knew about the addition of the "default" TQ call, but am not sure > > about why it should be implemented. Last I checked, there was only one > > sentence in the docs about this new feature. > > > Since I use TQs heavily, my main concern is ensuring in the manner > > Brandon is investigating the issue of avoiding "minutes or hours to > > task initiation" is important for me. Some tasks are not time- > > sensitive (e.g. stats updates), however others such as what Brandon is > > describing I would like to run as immediately as possible. > > > My initial thought about sending tasks to a the default app from a > > different numbered version was that it would enable this "highly > > reliable" queue. If all the only resource user in the default app is > > my TQ, does this ensure more reliable and faster task processing vs. > > sending the task to a TQ in a shared front-end handler environment? > > > That's my personal take on using the default queue feature -- but > > influenced by my bias for wanting a highly reliable queue. To date, > > I've seen nothing in the forums or docs about why the default queue > > should be used. I know there is sound rationale for it since someone > > took the time to develop it. > > > BTW Brandon: be sure to look at pull queues also. May have great > > potential for your never-ending pursuit to run GAE for Jeff for Free. > > View the video session title "Putting Task Queues to Work" at the > > link below. First 15 minutes shows how to reduce datastore writes by a > > factor of 1,000 -- something I really needed recently. Link: > >http://code.google.com/appengine/docs/videoresources.html > > > Thanks, > > stevep > > > "Task queues: the red-headed child in a family of blondes" > > > On Jan 26, 11:26 pm, Robert Kluin <[email protected]> wrote: > >> Hey Steve, > >> They actually added a default version "target" recently. You can > >> use that to send a task back to the default version of your app, just > >> as you would to send a task to a numbered version. Just set the > >> target to taskqueue.DEFAULT_APP_VERSION. > > >> Robert > > >> On Tue, Jan 24, 2012 at 11:25, stevep <[email protected]> wrote: > >> > Very helpful. Any ideas about the what+why of sending tasks to the > >> > default version when running a numbered version? thx-stevep > > >> > On Jan 23, 12:53 pm, Jeff Schnitzer <[email protected]> wrote: > >> >> Task Queues execute on frontend instances just like normal requests. > >> >> They will only spin up a 2nd instance if the additional task traffic > >> >> crosses the normal latency/idle threshold. > > >> >> Tasks are processed just like normal requests; they share the same > >> >> memcache, instance memory, etc as your normal application. > > >> >> Tasks are, however, separate requests so if you have some thread local > >> >> data it won't be available in the later task execution thread. Also > >> >> sometimes the task queues get backed up so it's hard to absolutely > >> >> predict when a task will be run. Usually it's pretty instantaneous > >> >> though. > > >> >> One thing that makes tasks easier is the Deferred feature, available > >> >> in both Python and Java. > > >> >> The task queue rocks. I lean on it pretty heavily. Also, you can > >> >> enlist some number of tasks (5, I think) transactionally, which > >> >> creates a good way to work around 2pc issues. > > >> >> Jeff > > >> >> On Mon, Jan 23, 2012 at 2:58 PM, Brandon Wirtz <[email protected]> > wrote: > >> >> > Apparently nothing. But having only been doing python for a year, > I was > >> >> > trying to do it the PHP way. > > >> >> > Now that I know... This looks great, and it appears that what I > should have > >> >> > done with ASync writes should have been done with task queues. > > >> >> > Do Task Queues Execute on the Front end instances? Or will adding > tasks spin > >> >> > up a second instance? > > >> >> > I'm currently serving 30k visitors a day per instance, and I don't > know if > >> >> > tasks would take some users from 1 instance to 2 instances. > > >> >> > -Brandon > > >> >> > -----Original Message----- > >> >> > From: [email protected] > >> >> > [mailto:[email protected]] On Behalf Of Jeff > Schnitzer > >> >> > Sent: Monday, January 23, 2012 11:15 AM > >> >> > To: [email protected] > >> >> > Subject: Re: [google-appengine] 'Flush' Like "I'm tapping out but > keep > >> >> > working" in Python? > > >> >> > On Mon, Jan 23, 2012 at 1:49 PM, Brandon Wirtz <[email protected]> > wrote: > > >> >> >> But can I say "Your process will be completed shortly" close the > >> >> >> output, and then Log some stuff, make some updates to the data > store, > >> >> >> but let the user get on with life? > > >> >> > How would this be different than shunting any remaining work off to > the task > >> >> > queue? > > >> >> > Jeff > > >> >> > -- > >> >> > You received this message because you are subscribed to the Google > Groups > >> >> > "Google App Engine" group. > >> >> > To post to this group, send email to > > [email protected]. > > >> >> > To unsubscribe from this group, send email to > >> >> > [email protected]. > >> >> > For more options, visit this group at > >> >> >http://groups.google.com/group/google-appengine?hl=en. > > >> >> > -- > >> >> > You received this message because you are > > -- > ------ > Robert Kluin > Ezox Systems, LLC -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
