you should try to not use Task Dispatching. Rather if you are doing this once a day, have a periodic task run every 24 hours. In the same module as the thread try doing something like what is at the bottom of this page http://docs.python.org/2/library/queue.html . It is a queue thread daemon architecture. You will process a 100k in no time at all. Make sure to have proper exception handling and recovery however. And as always just use the docs to learn from but know there are better ways to go about it and improve the threading model.
On Sunday, April 21, 2013 9:40:29 AM UTC-4, Michael Hernandez wrote: > > Sorry darn phone changed daemon to farmingn > On Apr 21, 2013 9:39 AM, "Michael Hernandez" < > michael.hernandez1...@gmail.com> wrote: > >> I have been working with heavy reporting and analytics lately. I would >> suggest use thread queuing with many farming threads. I use python thread >> but Greenlets it some else should be better >> On Apr 21, 2013 5:48 AM, "sparky" <cfspa...@gmail.com> wrote: >> >>> One last thing to add, the task it's self does not seems to be the >>> issue, 'got message from broker' is the 3-4 second wait I can see. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Django users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to django-users+unsubscr...@googlegroups.com. >>> To post to this group, send email to django-users@googlegroups.com. >>> Visit this group at http://groups.google.com/group/django-users?hl=en. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >>> >> -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
