Hey Steve, The DEFAULT_APP_VERSION var was added so that you could send a task to whatever version is currently marked as default. Without it there was no way to target a task at the current default version of your app. You'd want to use it if you have, for example, a backend that inserts tasks that need to run on the current default version. There are a number of cases where this arises, when you hit one you will know.
Yeah I'd love to see the ability to give task queues a priority relative to user requests and relative to each other. That would be pretty cool. Of course if they implement that then we could probably get even finer control over the scheduler -- which could be handy as well! Robert On Sat, Jan 28, 2012 at 18:47, stevep <[email protected]> wrote: > Hi Robert, > > Sorry my ramblings were hard to parse. Basic question is the first you > mentioned, "Why is there the TQ option for DEFAULT_APP_VERSION, and > when is it best used?" > > There was a discussion related to a priority TQ a year or so ago. > Someone (Vlad?) initiated a request for a priority TQ, and I did star > it at the time. > > Cheers, > Steve > > On Jan 27, 10:16 pm, Robert Kluin <[email protected]> wrote: >> Hey Steve, >> I'm a bit confused about what you're asking / commenting about here. Are >> you asking about the default queue, why there is a >> taskqueue.DEFAULT_APP_VERSION variable, or whether it is better to send >> tasks to a different version of your app? >> >> I generally prefer to use the same version of my apps for both tasks and >> user requests. The biggest reason queues slow down is because there aren't >> enough instances. Splitting user requests and tasks up will probably just >> make that problem worse. >> >> My tasks generally run quite fast, but they do occasionally back up when I >> font have enough instances. The taskqueue team recently added a new header >> you can use to research the execution latency of your tasks. >> >> I know several others have asked for the ability to define a high priority >> push queue as well. I agree that this would be a very powerful feature. >> Particularly if you could define the relative priority to user requests and >> other queues. I'm not sure if there is an existing feature request for >> this or not, but if you make one I'll star it. >> >> Robert >> >> >> >> >> >> >> >> >> >> On Friday, January 27, 2012, stevep <[email protected]> wrote: >> > Thanks Robert, >> >> > I knew about the addition of the "default" TQ call, but am not sure >> > about why it should be implemented. Last I checked, there was only one >> > sentence in the docs about this new feature. >> >> > Since I use TQs heavily, my main concern is ensuring in the manner >> > Brandon is investigating the issue of avoiding "minutes or hours to >> > task initiation" is important for me. Some tasks are not time- >> > sensitive (e.g. stats updates), however others such as what Brandon is >> > describing I would like to run as immediately as possible. >> >> > My initial thought about sending tasks to a the default app from a >> > different numbered version was that it would enable this "highly >> > reliable" queue. If all the only resource user in the default app is >> > my TQ, does this ensure more reliable and faster task processing vs. >> > sending the task to a TQ in a shared front-end handler environment? >> >> > That's my personal take on using the default queue feature -- but >> > influenced by my bias for wanting a highly reliable queue. To date, >> > I've seen nothing in the forums or docs about why the default queue >> > should be used. I know there is sound rationale for it since someone >> > took the time to develop it. >> >> > BTW Brandon: be sure to look at pull queues also. May have great >> > potential for your never-ending pursuit to run GAE for Jeff for Free. >> > View the video session title "Putting Task Queues to Work" at the >> > link below. First 15 minutes shows how to reduce datastore writes by a >> > factor of 1,000 -- something I really needed recently. Link: >> >http://code.google.com/appengine/docs/videoresources.html >> >> > Thanks, >> > stevep >> >> > "Task queues: the red-headed child in a family of blondes" >> >> > On Jan 26, 11:26 pm, Robert Kluin <[email protected]> wrote: >> >> Hey Steve, >> >> They actually added a default version "target" recently. You can >> >> use that to send a task back to the default version of your app, just >> >> as you would to send a task to a numbered version. Just set the >> >> target to taskqueue.DEFAULT_APP_VERSION. >> >> >> Robert >> >> >> On Tue, Jan 24, 2012 at 11:25, stevep <[email protected]> wrote: >> >> > Very helpful. Any ideas about the what+why of sending tasks to the >> >> > default version when running a numbered version? thx-stevep >> >> >> > On Jan 23, 12:53 pm, Jeff Schnitzer <[email protected]> wrote: >> >> >> Task Queues execute on frontend instances just like normal requests. >> >> >> They will only spin up a 2nd instance if the additional task traffic >> >> >> crosses the normal latency/idle threshold. >> >> >> >> Tasks are processed just like normal requests; they share the same >> >> >> memcache, instance memory, etc as your normal application. >> >> >> >> Tasks are, however, separate requests so if you have some thread local >> >> >> data it won't be available in the later task execution thread. Also >> >> >> sometimes the task queues get backed up so it's hard to absolutely >> >> >> predict when a task will be run. Usually it's pretty instantaneous >> >> >> though. >> >> >> >> One thing that makes tasks easier is the Deferred feature, available >> >> >> in both Python and Java. >> >> >> >> The task queue rocks. I lean on it pretty heavily. Also, you can >> >> >> enlist some number of tasks (5, I think) transactionally, which >> >> >> creates a good way to work around 2pc issues. >> >> >> >> Jeff >> >> >> >> On Mon, Jan 23, 2012 at 2:58 PM, Brandon Wirtz <[email protected]> >> wrote: >> >> >> > Apparently nothing. But having only been doing python for a year, >> I was >> >> >> > trying to do it the PHP way. >> >> >> >> > Now that I know... This looks great, and it appears that what I >> should have >> >> >> > done with ASync writes should have been done with task queues. >> >> >> >> > Do Task Queues Execute on the Front end instances? Or will adding >> tasks spin >> >> >> > up a second instance? >> >> >> >> > I'm currently serving 30k visitors a day per instance, and I don't >> know if >> >> >> > tasks would take some users from 1 instance to 2 instances. >> >> >> >> > -Brandon >> >> >> >> > -----Original Message----- >> >> >> > From: [email protected] >> >> >> > [mailto:[email protected]] On Behalf Of Jeff >> Schnitzer >> >> >> > Sent: Monday, January 23, 2012 11:15 AM >> >> >> > To: [email protected] >> >> >> > Subject: Re: [google-appengine] 'Flush' Like "I'm tapping out but >> keep >> >> >> > working" in Python? >> >> >> >> > On Mon, Jan 23, 2012 at 1:49 PM, Brandon Wirtz <[email protected]> >> wrote: >> >> >> >> >> But can I say "Your process will be completed shortly" close the >> >> >> >> output, and then Log some stuff, make some updates to the data >> store, >> >> >> >> but let the user get on with life? >> >> >> >> > How would this be different than shunting any remaining work off to >> the task >> >> >> > queue? >> >> >> >> > Jeff >> >> >> >> > -- >> >> >> > You received this message because you are subscribed to the Google >> Groups >> >> >> > "Google App Engine" group. >> >> >> > To post to this group, send email to >> >> [email protected]. >> >> >> >> > To unsubscribe from this group, send email to >> >> >> > [email protected]. >> >> >> > For more options, visit this group at >> >> >> >http://groups.google.com/group/google-appengine?hl=en. >> >> >> >> > -- >> >> >> > You received this message because you are >> >> -- >> ------ >> Robert Kluin >> Ezox Systems, LLC > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
