Hi Guys,
I have a "for" loop which goes thru every row in the table (1000 rows), do
some calculation and update the row. But whole process takes around 2-3 sec
for one row as i'm connecting to few external databases to collect data. So
for 1k rows it takes ages to accoplish all those tasks.
In pure python i used Threading module, but it seems not to work in web2py:
def thr(item):
query = db(db.table.id > 0).select()
for item in query:
t = threading.Thread(target=my_function, args=(item,))
threads.append(t)
for thr in threads:
thr.start()
def my_function(item):
## do some stuff, then update the row
item.update_row(row1=value1,row2=value2)
It looks like i go to my_function, do caluculation, but item is not updated
in the table at the end.
Not sure if i can use threading with web2py so i tried to use Scheduler to
acomplish the same task.
def thr(item):
query = db(db.table.id > 0).select()
for item in query:
### with Scheduler tasks
scheduler.queue_task('checks',[item.id])
def my_function(item):
item = db(db.table.id=item).select().first()
## do some stuff, then update the row
item.update_row(row1=value1,row2=value2)
It seems to work, task is created but web2py wait till previous task is
completed to start next task from the queue and I want them to run the same
time (or 100 tasks at the same time).
And also after loop is finished i have 1000 items in scheduler table.
Should i remove them somehow from web2py controller level after task is
completed? I don't want to add 1k task each time i run a loop.
How can i acoplish multithreading with web2py? Should i use threading
module (if so, why it doesn't update records in the table?) or Scheduler
(if so, how to run multiple tasks the same time and how to remove tasks
from scheduler db once task is completed.
Thanks.
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
