Re: parsing in python
Depending on what you want to do, you might try looking at the pyparsing
module. I have used it to successfully parse sentences looking for
keywords and structures.
On 8/3/2011 9:26 AM, Jayron Soares wrote:
Hi folks,
I've created a simple method to grab files texts from directory by
words random, however I figure out that I need extract the content
inside of each file, in fact I believe I have to create a parsing,
nonetheless I don't know how to create a parser.
Please some could share some tips to do it?
Cheers
Jayron
here is the code:
# -*- conding: utf-8 -*-
from subprocess import Popen, PIPE
pesquisa = raw_input(" Digite a pesquisa de interesse: ")
def Pesquisar(pesquisa):
p = Popen(["search", pesquisa],stdout=PIPE)
resultado = p.communicate()[0]
return resultado
print Pesquisar(pesquisa)
--
/" A Vida é arte do Saber...Quem quiser saber tem que *Estudar*!"/
http://bucolick.tumblr.com
Multiprocessing.Process Daemonic Behavior
Greetings, I'm trying to understand the behavior of the multiprocessing.Process daemonic attribute. Based on what I have read, if a Process ( X ) is created, and before it is started ( X.start() ), I should be able to set the process as daemonic using X.daemon=True. Once this attribute is set, if the parent process terminates, all daemonic process should also terminate. To experiment with this, I have written the following script. #-- BEGIN SCRIPT --# #!/usr/bin/env python from time import sleep from os import getpid from multiprocessing import Process class worker(Process): def __init__(self, id): Process.__init__(self) self.id = id def run(self): try: print 'Starting thread %s (pid:%s)' % (self.id, getpid()) while True: print 'Hello from thread %s (pid:%s)' % (self.id, getpid()) sleep(1) except KeyboardInterrupt: print '** KeyboardInterrupt captured. Shutting down thread %d (pid:%s) ' % (self.id, getpid()) if __name__ == '__main__': print 'Starting main (pid:%s)' % ( getpid()) t = [] try: for i in range(2): w = worker(i) w.daemon=True if w._daemonic: print 'thread %d is daemonic' % i t.append(w) try: for i in range(len(t)): t[i].start() loop = True while loop: loop = False for i in range(len(t)): if t[i].is_alive(): loop = True if not t[i]._daemonic: print 'thread %d is NOT daemonic' % i sleep(1) except KeyboardInterrupt: raise except KeyboardInterrupt: print 'KeyboardInterrupt captured. Shutting down main (pid:%s)' % getpid() print 'Shutting down' sleep(10) print 'Done.' # -- END SCRIPT -- # From the command line, if I run the script and then enter ctrl-C the program and daemonic children terminate as I would expect. ~/temp$ ./threadtest.py Starting main (pid:13395) thread 0 is daemonic thread 1 is daemonic Starting thread 0 (pid:13396) Hello from thread 0 (pid:13396) Starting thread 1 (pid:13397) Hello from thread 1 (pid:13397) Hello from thread 0 (pid:13396) Hello from thread 1 (pid:13397) Hello from thread 0 (pid:13396) Hello from thread 1 (pid:13397) Hello from thread 0 (pid:13396) Hello from thread 1 (pid:13397) Hello from thread 0 (pid:13396) Hello from thread 1 (pid:13397) ^C KeyboardInterrupt captured. Shutting down main (pid:13395) Shutting down ** KeyboardInterrupt captured. Shutting down thread 0 (pid:13396) ** KeyboardInterrupt captured. Shutting down thread 1 (pid:13397) Done. Likewise, if i use a 'kill -2 13395', the results are the same (13395 being the parent pid). If I use a 'kill -2 13396 13397', the daemonic children terminate, and the main closes as one would expect (13396 and 13397 being the children pids). However, if I use a 'kill 13395' (or something other then a SIGINT) the parent process terminates and the daemonic children CONTINUE TO PROCESS. The children essentially become zombies merrily meandering around the system. Really, is there anything worse then a daemonic zombie? I would have expected the daemonic children processes to terminate with the parent process, regardless of how the parent process terminates, either normally or forcefully. I would love to be enlightened Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing.Process Daemonic Behavior
On 3/15/2011 11:19 PM, James Mills wrote: On Wed, Mar 16, 2011 at 12:34 PM, John L. Stephens wrote: I would have expected the daemonic children processes to terminate with the parent process, regardless of how the parent process terminates, either normally or forcefully. As I understand it. If you forcibly kill the parent process with the KILL signal then any child procesases that were created become zombies. You also can't handle the KILL signal in your application (nor can the multiprocessing library) and so it therefore cannot cleanup and terminate any child processes in the normal way. cheers James So as I have contemplated this in the wee hours of the morning (risking zombie status myself I might add), my rationale for the behavior I am seeing is thus: As the parent process terminates 'normally' (either through normal termination or SIGINT termination), mulitprocessing steps in and performs child process cleanup via the x.terminate() method. If the parent terminates any other way, multiprocessing doesn't have the opportunity to cleanup. I would have thought (silly me) the child process would also monitor his parent process, and when the child can no longer 'see' or 'sense the presence' of its parent, it too would terminate. I can certainly make the children monitor the parent as well, I was just hoping I wouldn't have to put in additional code and overhead. John -- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing Pool workers cannot create subprocesses
On 3/18/2011 7:54 PM, Jason Grout wrote: Right; thanks. Let me rephrase my questions: 1. Why is important that the multiprocessing Pool worker processors have daemon=True (I think this is the same as asking: why is it important that they be terminated with terminate() rather than join() )? 2. Is it safe for us to reset a Pool worker process to have daemon=False before starting a subprocess from that worker, like in the code from the original message? Thanks, Jason Jason, I just happen to be dealing with a project that uses multiprocessing. What I have learned is this... If a child thread (pool worker) is not set to daemon (daemon=False), if for some reason the parent thread terminates either normally or abnormally and the worker thread has not completed its task, the child thread will terminate by throwing all sorts of nasty errors. However, in daemon mode, multiprocessing will terminate the child thread 'cleanly'. That's not to say that the worker has a chance to complete its work or shut itself down. Multiprocessing will absorb the exceptions and not pass them along. You may terminate a child thread using join(). That is probably the safest way to do it. terminate() will just kill the worker thread immediately without any regard to whether or not it has completed its tasks. I believe multiprocessing uses terminate() as well to kill a daemon thread if the parent thread disappears. join() will, however, block until the task has competed and returned. If you want to continue doing work in the parent thread while the child thread is off doing its thing, then another means of syncing up the parent and children threads is needed. As for allowing children threads to spawn off children of its own using subprocess runs the risk of creating a little army of zombie 'grandchildren' if either the parent or child threads terminate before the subprocess completes and returns. Hope that helps John -- http://mail.python.org/mailman/listinfo/python-list
