so it seems that what you need is not to handle http requests asynchronously, but to process async jobs requested by http. if that is the case celery is a popular option and relatively easy to setup, I personally would use it for such things.
buton the other hand you are saying that the task in hand will be done once a year, so it has nothing to do with http request, the job is activated by time. I wouldn't setup all this overhead for a once a year task, you don't even need async infrastructure, just set the command on cron and run On Fri, May 6, 2016 at 3:37 PM, luke lukes <lordluk...@gmail.com> wrote: > Hi everyone, > I need to generate a PDF report for each entry of a django queryset. > There'll be between between 30k and 40k entries. > > The PDF is generated through an external API. Since currently is generated > on demand, this is handled synchronously via an HTTP request/response. That > will be different for this task, since I think I'll use a django management > command to loop through the queryset and perform the PDF generation. > > Which approach should I follow for this task? I thought about 3 possibile > solutions, although are technologies that I never used: > > 1) Celery > <http://docs.celeryproject.org/en/latest/userguide/remote-tasks.html>: > assign a task (http request with a different payload) to a worker, then > retrieve it once it's done; > > 2) request-futures <https://github.com/ross/requests-futures>: using > requests in a non-blocking way; > > 3) multiprocessing > <https://docs.python.org/2/library/multiprocessing.html> package, e.g. > assigning a Pool of workers. > > > the goal is to use the API concurrently (e.g. send 10 or 100 http requests > simultaneously, depending on how many concurrent requests the API can > handle). > > the project runs on Python 2.7 and this task will occur approximately once > a year. > > > Anybody here that handled a similar task and can give advices on how to > proceed on this? > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-users+unsubscr...@googlegroups.com. > To post to this group, send email to django-users@googlegroups.com. > Visit this group at https://groups.google.com/group/django-users. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/a10436c0-2493-4889-b829-b4fc725f7e8e%40googlegroups.com > <https://groups.google.com/d/msgid/django-users/a10436c0-2493-4889-b829-b4fc725f7e8e%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFWa6tJ0AAFR8OW43jVMEFPUL%3DbudTujMFeXzhcj2Vw9%3DUJyQQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
