Sorry about my terrible lack of proof reading. Here is what I meant to say in that terribly mangled sentence from the previous post:
Since I use TQs heavily in the manner Brandon is investigating, my main concern is ensuring the issue of avoiding "minutes or hours to task initiation". On Jan 27, 8:58 am, stevep <[email protected]> wrote: > Thanks Robert, > > I knew about the addition of the "default" TQ call, but am not sure > about why it should be implemented. Last I checked, there was only one > sentence in the docs about this new feature. > > Since I use TQs heavily, my main concern is ensuring in the manner > Brandon is investigating the issue of avoiding "minutes or hours to > task initiation" is important for me. Some tasks are not time- > sensitive (e.g. stats updates), however others such as what Brandon is > describing I would like to run as immediately as possible. > > My initial thought about sending tasks to a the default app from a > different numbered version was that it would enable this "highly > reliable" queue. If all the only resource user in the default app is > my TQ, does this ensure more reliable and faster task processing vs. > sending the task to a TQ in a shared front-end handler environment? > > That's my personal take on using the default queue feature -- but > influenced by my bias for wanting a highly reliable queue. To date, > I've seen nothing in the forums or docs about why the default queue > should be used. I know there is sound rationale for it since someone > took the time to develop it. > > BTW Brandon: be sure to look at pull queues also. May have great > potential for your never-ending pursuit to run GAE for Jeff for Free. > View the video session title "Putting Task Queues to Work" at the > link below. First 15 minutes shows how to reduce datastore writes by a > factor of 1,000 -- something I really needed recently. > Link:http://code.google.com/appengine/docs/videoresources.html > > Thanks, > stevep > > "Task queues: the red-headed child in a family of blondes" > > On Jan 26, 11:26 pm, Robert Kluin <[email protected]> wrote: > > > > > > > > > Hey Steve, > > They actually added a default version "target" recently. You can > > use that to send a task back to the default version of your app, just > > as you would to send a task to a numbered version. Just set the > > target to taskqueue.DEFAULT_APP_VERSION. > > > Robert > > > On Tue, Jan 24, 2012 at 11:25, stevep <[email protected]> wrote: > > > Very helpful. Any ideas about the what+why of sending tasks to the > > > default version when running a numbered version? thx-stevep > > > > On Jan 23, 12:53 pm, Jeff Schnitzer <[email protected]> wrote: > > >> Task Queues execute on frontend instances just like normal requests. > > >> They will only spin up a 2nd instance if the additional task traffic > > >> crosses the normal latency/idle threshold. > > > >> Tasks are processed just like normal requests; they share the same > > >> memcache, instance memory, etc as your normal application. > > > >> Tasks are, however, separate requests so if you have some thread local > > >> data it won't be available in the later task execution thread. Also > > >> sometimes the task queues get backed up so it's hard to absolutely > > >> predict when a task will be run. Usually it's pretty instantaneous > > >> though. > > > >> One thing that makes tasks easier is the Deferred feature, available > > >> in both Python and Java. > > > >> The task queue rocks. I lean on it pretty heavily. Also, you can > > >> enlist some number of tasks (5, I think) transactionally, which > > >> creates a good way to work around 2pc issues. > > > >> Jeff > > > >> On Mon, Jan 23, 2012 at 2:58 PM, Brandon Wirtz <[email protected]> > > >> wrote: > > >> > Apparently nothing. But having only been doing python for a year, I > > >> > was > > >> > trying to do it the PHP way. > > > >> > Now that I know... This looks great, and it appears that what I should > > >> > have > > >> > done with ASync writes should have been done with task queues. > > > >> > Do Task Queues Execute on the Front end instances? Or will adding > > >> > tasks spin > > >> > up a second instance? > > > >> > I'm currently serving 30k visitors a day per instance, and I don't > > >> > know if > > >> > tasks would take some users from 1 instance to 2 instances. > > > >> > -Brandon > > > >> > -----Original Message----- > > >> > From: [email protected] > > >> > [mailto:[email protected]] On Behalf Of Jeff Schnitzer > > >> > Sent: Monday, January 23, 2012 11:15 AM > > >> > To: [email protected] > > >> > Subject: Re: [google-appengine] 'Flush' Like "I'm tapping out but keep > > >> > working" in Python? > > > >> > On Mon, Jan 23, 2012 at 1:49 PM, Brandon Wirtz <[email protected]> > > >> > wrote: > > > >> >> But can I say "Your process will be completed shortly" close the > > >> >> output, and then Log some stuff, make some updates to the data store, > > >> >> but let the user get on with life? > > > >> > How would this be different than shunting any remaining work off to > > >> > the task > > >> > queue? > > > >> > Jeff > > > >> > -- > > >> > You received this message because you are subscribed to the Google > > >> > Groups > > >> > "Google App Engine" group. > > >> > To post to this group, send email to [email protected]. > > >> > To unsubscribe from this group, send email to > > >> > [email protected]. > > >> > For more options, visit this group at > > >> >http://groups.google.com/group/google-appengine?hl=en. > > > >> > -- > > >> > You received this message because you are subscribed to the Google > > >> > Groups "Google App Engine" group. > > >> > To post to this group, send email to [email protected]. > > >> > To unsubscribe from this group, send email to > > >> > [email protected]. > > >> > For more options, visit this group > > >> > athttp://groups.google.com/group/google-appengine?hl=en. > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Google App Engine" group. > > > To post to this group, send email to [email protected]. > > > To unsubscribe from this group, send email to > > > [email protected]. > > > For more options, visit this group > > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
