You can use web2py cron and trigger the processing immediately after
the action that queues the tasks returns.
Massimo
On Jul 10, 8:11 am, kralin <andrea.pierle...@gmail.com> wrote:
> I finally made it, at least for now.
>
> There is a database table where all the processes are stored, both to
> be processed and completed ones.
>
> db.define_table('work',
> SQLField('name','string'),
> SQLField('status','string'),
> SQLField('priority','integer'),
> SQLField('input','text'),
> SQLField('results','text'),
> SQLField
> ('date_submitted','datetime',default=datetime.datetime.now()),
> SQLField('date_completed','datetime'),
> )
>
> than there is a cron that each minute look a the table and if there
> are jobs to process runs the first one.
>
> def process_job():
> queue=db(db.work.status=='to be done').select
> (orderby=db.work.priority)
> if queue:
> job=queue[0]
> #do stuff like:
> input_string=job.input
> output_string='Procesed by cron worker : '+str(input_string)
> #submit results
> db.work[job.id]=dict
> (status='completed',results=output_string,date_completed=datetime.datetime.now
> ())
> db.commit()
> return
>
> the problem with this is that I have to wait one minute for each table
> lookup, and that I'm not sure it will work correctly under WSGI, since
> I do not want to rely on system cron.
> The fastest improvement would be to run all the process in queue each
> time the controller is called by the cron,
> one by one, but I'm still convinced that using cron is not the best
> way to do this, even if it works.
>
> Am I the only one with this kind of issues? If not maybe I can post
> the "solution" to AlterEgo...
>
> On Jul 10, 3:19 am, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > You can use the databae as long the objects are serializable
>
> > On Jul 9, 6:00 pm, kralin <andrea.pierle...@gmail.com> wrote:
>
> > > thanks massimo, very elegant...
> > > I have to check for the lock thing, otherwise a list is more tha
> > > sufficient for thequeue:
>
> > >queue=cache.ram('queue',lambda:[],10**10)
> > >queue.append(o)
> > >queue.pop(0)
>
> > > it should be also very easy to use a priority schedule by appending
> > > [int(priority), task] to thequeuelist.
>
> > > however I still need to cope with the guy that's going to do the heavy
> > > job.
> > > Is it possibile to set a Worker in web2py that is able to read the
> > > input from thequeueand put the results in a database, or do I have
> > > to launch a cron each minute to check for thequeue?
>
> > > On 10 Lug, 00:00, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > > You can do something like
>
> > > > classQueue:
> > > > forever = 10**10
> > > > def __init__(self):
> > > > import thread
> > > > self.q=[]
> > > > self.lock=thread.allocate_lock()
> > > > def enque(self,o):
> > > > self.lock.acquire()
> > > > self.q.append(o)
> > > > self.lock.release()
> > > > def dequeue(self):
> > > > self.lock.acquire()
> > > > o = self.q[0]
> > > > del q[0]
> > > > self.lock.release()
> > > > return o
> > > > def __len__(self):
> > > > self.lock.acquire()
> > > > ell=len(self.q)
> > > > self.lock.release()
> > > > return ell
>
> > > >queue=cache.ram('queue',lambda:Queue,Queue.forever)
> > > >queue.enqueue('object')
> > > > print len(queue)
> > > > o =queue.dequeue()
>
> > > > If you define it in a model it will be visible everywhere, including
> > > > cron scripts. I am not sure if the lock is really necessary (because
> > > > of the global interpreter lock) but it is safe.
>
> > > >queue.append(
>
> > > > On Jul 9, 4:07 pm, kralin <andrea.pierle...@gmail.com> wrote:
>
> > > > > Hi All,
> > > > > I'm diving into web2py that at the moment gives me the power and the
> > > > > time do a lot of cool things. so thanks a lot to you guys.
> > > > > I was wondering if any of you ever had the need to set aqueuefor a
> > > > > heavy process.
> > > > > Let's suppose I've got one heavy process that takes 2 mins to run on a
> > > > > singol-processor machine.
> > > > > now if 10 user submit the process at (more or less) the same time, is
> > > > > there a way to schedule the process based on the first coming, first
> > > > > to have response rule? each of this user should be able to see their
> > > > > result once completed or see something like"wait, 8 jobs still in
> > > > >queue"
>
> > > > > I know it can be done easily with external job sheduler, but it will
> > > > > require to use a separate process, not beeing portable, and a little
> > > > > bit more headhache...
>
> > > > > a quick and dirty solution wuld be to run a cron each, let's say, 2
> > > > > mins and if a process isqueue, execute it, but I feel that can be
> > > > > done in a much more elegant way.
>
> > > > > what do youthink?
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"web2py Web Framework" group.
To post to this group, send email to web2py@googlegroups.com
To unsubscribe from this group, send email to
web2py+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/web2py?hl=en
-~----------~----~----~----~------~----~------~--~---
