Hey Javier, Thanks for the reply. My problem with the logs in the past was that they tended to make the task even slower (due to recursion) but I guess that's probably because I didn't call the logging from the appropriate places in the code.
To make things a little more complicated, the task involves writing a large amount of data to a temp database, handling it and then saving some resulting queries to the permanent DB. This makes it a tad harder to analyze what goes on in the first part of the code. I'll try logging again over the weekend and see how that works.. I just wish there were third party apps and tools for debugging this sort of problem. Thanks again for your input. Cheers, AT On Apr 3, 2012 3:47 PM, "Javier Guerra Giraldez" <jav...@guerrag.com> wrote: > > On Tue, Apr 3, 2012 at 8:21 AM, Andre Terra <andrete...@gmail.com> wrote: > > I have some complex and database intensive asynchronous tasks running under > > celery which take a LONG time to complete and I'd just love to be able to > > keep track of the queries they generate in order to optimize and possibly > > remove the biggest bottlenecks. > > the easiest would be to write detailed logs, which _can_ be analysed > in real time, not only 'after the fact'. > > my second idea would be to hack the log output so instead of writing > to a file, it would store messages (probably with some structure) to > some comfortable database. I'd use Redis, but i guess MongoDB or even > an SQL-based DB could work too. then you can easily filter and > aggregate times according to task type, when it happened, etc. > > -- > Javier > > -- > You received this message because you are subscribed to the Google Groups "Django users" group. > To post to this group, send email to django-users@googlegroups.com. > To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/django-users?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
