Re: Please help with Threading

2013-06-02 Thread Jurgens de Bruin
On Saturday, 18 May 2013 10:58:13 UTC+2, Jurgens de Bruin wrote: > This is my first script where I want to use the python threading module. I > have a large dataset which is a list of dict this can be as much as 200 > dictionaries in the list. The final goal is a histogram for each dict 16 > h

Re: Please help with Threading

2013-05-20 Thread Steven D'Aprano
On Tue, 21 May 2013 05:53:46 +0300, Carlos Nepomuceno wrote: > BTW, why I didn't find the source code to the sys module in the 'Lib' > directory? Because sys is a built-in module. It is embedded in the Python interpreter. -- Steven -- http://mail.python.org/mailman/listinfo/python-list

RE: Please help with Threading

2013-05-20 Thread Carlos Nepomuceno
> On Tue, May 21, 2013 at 11:44 AM, 8 Dihedral > wrote: >> OK, if the python interpreter has a global hiden print out >> buffer of ,say, 2to 16 K bytes, and all string print functions >> just construct the output string from the format to this string >> in an efficient low level way, then the

RE: Please help with Threading

2013-05-20 Thread Carlos Nepomuceno
e source code to the sys module in the 'Lib' directory? > Date: Tue, 21 May 2013 11:50:17 +1000 > Subject: Re: Please help with Threading > From: ros...@gmail.com > To: python-list@python.org > > On Tue, May 21, 2013 at 11:44 A

Re: Please help with Threading

2013-05-20 Thread Chris Angelico
On Tue, May 21, 2013 at 11:44 AM, 8 Dihedral wrote: > OK, if the python interpreter has a global hiden print out > buffer of ,say, 2to 16 K bytes, and all string print functions > just construct the output string from the format to this string > in an efficient low level way, then the next qu

Re: Please help with Threading

2013-05-20 Thread 88888 Dihedral
Chris Angelico於 2013年5月20日星期一UTC+8下午5時09分13秒寫道: > On Mon, May 20, 2013 at 6:35 PM, Cameron Simpson wrote: > > > _lock = Lock() > > > > > > def lprint(*a, **kw): > > > global _lock > > > with _lock: > > > print(*a, **kw) > > > > > > and use lprint() everywhere? > > > >

Re: Please help with Threading

2013-05-20 Thread Fábio Santos
I didn't know that. On 20 May 2013 12:10, "Dave Angel" wrote: > Are you making function calls, using system libraries, or creating or deleting any objects? All of these use the GIL because they use common data structures shared among all threads. At the lowest level, creating an object requires

Re: Please help with Threading

2013-05-20 Thread Chris Angelico
=On Mon, May 20, 2013 at 8:46 PM, Ned Batchelder wrote: > On 5/20/2013 6:09 AM, Chris Angelico wrote: >> >> Referencing a function's own name in a default has to have one of >> these interpretations: >> >> 1) It's a self-reference, which can be used to guarantee recursion >> even if the name is re

Re: Please help with Threading

2013-05-20 Thread Dave Angel
On 05/20/2013 03:55 AM, Fábio Santos wrote: My use case was a tight loop processing an image pixel by pixel, or crunching a CSV file. If it only uses local variables (and probably hold a lock before releasing the GIL) it should be safe, no? Are you making function calls, using system libraries

Re: Please help with Threading

2013-05-20 Thread Ned Batchelder
On 5/20/2013 6:09 AM, Chris Angelico wrote: Referencing a function's own name in a default has to have one of these interpretations: 1) It's a self-reference, which can be used to guarantee recursion even if the name is rebound 2) It references whatever previously held that name before this def

Re: Please help with Threading

2013-05-20 Thread Chris Angelico
On Mon, May 20, 2013 at 7:54 PM, Cameron Simpson wrote: > On 20May2013 19:09, Chris Angelico wrote: > | On Mon, May 20, 2013 at 6:35 PM, Cameron Simpson wrote: > | > _lock = Lock() > | > > | > def lprint(*a, **kw): > | > global _lock > | > with _lock: > | > print(*a, **kw) > |

RE: Please help with Threading

2013-05-20 Thread Carlos Nepomuceno
> Date: Mon, 20 May 2013 18:35:20 +1000 > From: c...@zip.com.au > To: carlosnepomuc...@outlook.com > CC: python-list@python.org > Subject: Re: Please help with Threading > > On 20May2013 10:53, Carlos Nepomuceno wrote: > | I just g

Re: Please help with Threading

2013-05-20 Thread Cameron Simpson
On 20May2013 19:09, Chris Angelico wrote: | On Mon, May 20, 2013 at 6:35 PM, Cameron Simpson wrote: | > _lock = Lock() | > | > def lprint(*a, **kw): | > global _lock | > with _lock: | > print(*a, **kw) | > | > and use lprint() everywhere? | | Fun little hack: | | def print(*ar

Re: Please help with Threading

2013-05-20 Thread Fábio Santos
It is pretty cool although it looks like a recursive function at first ;) On 20 May 2013 10:13, "Chris Angelico" wrote: > On Mon, May 20, 2013 at 6:35 PM, Cameron Simpson wrote: > > _lock = Lock() > > > > def lprint(*a, **kw): > > global _lock > > with _lock: > > print(*a, **kw

Re: Please help with Threading

2013-05-20 Thread Chris Angelico
On Mon, May 20, 2013 at 6:35 PM, Cameron Simpson wrote: > _lock = Lock() > > def lprint(*a, **kw): > global _lock > with _lock: > print(*a, **kw) > > and use lprint() everywhere? Fun little hack: def print(*args,print=print,lock=Lock(),**kwargs): with lock: print(*args,**

Re: Please help with Threading

2013-05-20 Thread Cameron Simpson
On 20May2013 10:53, Carlos Nepomuceno wrote: | I just got my hands dirty trying to synchronize Python prints from many threads. | Sometimes they mess up when printing the newlines. | I tried several approaches using threading.Lock and Condition. | None of them worked perfectly and all of them ma

Re: Please help with Threading

2013-05-20 Thread Fábio Santos
My use case was a tight loop processing an image pixel by pixel, or crunching a CSV file. If it only uses local variables (and probably hold a lock before releasing the GIL) it should be safe, no? My idea is that it's a little bad to have to write C or use multiprocessing just to do simultaneous c

RE: Please help with Threading

2013-05-20 Thread Carlos Nepomuceno
> Date: Mon, 20 May 2013 17:45:14 +1000 > From: c...@zip.com.au > To: fabiosantos...@gmail.com > Subject: Re: Please help with Threading > CC: python-list@python.org; wlfr...@ix.netcom.com > > On 20May2013 07:25, Fábio Santos wrote:

RE: Please help with Threading

2013-05-20 Thread Carlos Nepomuceno
> Date: Sun, 19 May 2013 13:10:36 +1000 > From: c...@zip.com.au > To: carlosnepomuc...@outlook.com > CC: python-list@python.org > Subject: Re: Please help with Threading > > On 19May2013 03:02, Carlos Nepomuceno wrote: > | Just be

Re: Please help with Threading

2013-05-20 Thread Cameron Simpson
On 20May2013 07:25, Fábio Santos wrote: | On 18 May 2013 20:33, "Dennis Lee Bieber" wrote: | > Python threads work fine if the threads either rely on intelligent | > DLLs for number crunching (instead of doing nested Python loops to | > process a numeric array you pass it to something lik

Re: Please help with Threading

2013-05-19 Thread Fábio Santos
On 18 May 2013 20:33, "Dennis Lee Bieber" wrote: > Python threads work fine if the threads either rely on intelligent > DLLs for number crunching (instead of doing nested Python loops to > process a numeric array you pass it to something like NumPy which > releases the GIL while crunching

Re: Please help with Threading

2013-05-19 Thread Dave Angel
On 05/19/2013 05:46 PM, Dennis Lee Bieber wrote: On Sun, 19 May 2013 10:38:14 +1000, Chris Angelico declaimed the following in gmane.comp.python.general: On Sun, May 19, 2013 at 10:02 AM, Carlos Nepomuceno wrote: I didn't know Python threads aren't preemptive. Seems to be something really o

Re: Please help with Threading

2013-05-19 Thread Chris Angelico
On Mon, May 20, 2013 at 7:46 AM, Dennis Lee Bieber wrote: > On Sun, 19 May 2013 10:38:14 +1000, Chris Angelico > declaimed the following in gmane.comp.python.general: >> With interpreted code eg in CPython, it's easy to implement preemption >> in the interpreter. I don't know how it's actually do

Re: Please help with Threading

2013-05-18 Thread Cameron Simpson
On 19May2013 03:02, Carlos Nepomuceno wrote: | Just been told that GIL doesn't make things slower, but as I | didn't know that such a thing even existed I went out looking for | more info and found that document: | http://www.dabeaz.com/python/UnderstandingGIL.pdf | | Is it current? I didn't know

Re: Please help with Threading

2013-05-18 Thread Chris Angelico
On Sun, May 19, 2013 at 10:02 AM, Carlos Nepomuceno wrote: > I didn't know Python threads aren't preemptive. Seems to be something really > old considering the state of the art on parallel execution on multi-cores. > > What's the catch on making Python threads preemptive? Are there any ongoing >

RE: Please help with Threading

2013-05-18 Thread Carlos Nepomuceno
> To: python-list@python.org > From: wlfr...@ix.netcom.com > Subject: Re: Please help with Threading > Date: Sat, 18 May 2013 15:28:56 -0400 > > On Sat, 18 May 2013 01:58:13 -0700 (PDT), Jurgens de Bruin > decla

Re: Please help with Threading

2013-05-18 Thread Dave Angel
On 05/18/2013 04:58 AM, Jurgens de Bruin wrote: This is my first script where I want to use the python threading module. I have a large dataset which is a list of dict this can be as much as 200 dictionaries in the list. The final goal is a histogram for each dict 16 histograms on a page ( 4x

Re: Please help with Threading

2013-05-18 Thread Peter Otten
Jurgens de Bruin wrote: > I will post code - the entire scripts is 1000 lines of code - can I post > the threading functions only? Try to condense it to the relevant parts, but make sure that it can be run by us. As a general note, when you add new stuff to an existing longish script it is alw

Re: Please help with Threading

2013-05-18 Thread Jurgens de Bruin
I will post code - the entire scripts is 1000 lines of code - can I post the threading functions only? -- http://mail.python.org/mailman/listinfo/python-list

Re: Please help with Threading

2013-05-18 Thread Peter Otten
Jurgens de Bruin wrote: > This is my first script where I want to use the python threading module. I > have a large dataset which is a list of dict this can be as much as 200 > dictionaries in the list. The final goal is a histogram for each dict 16 > histograms on a page ( 4x4 ) - this already w

Please help with Threading

2013-05-18 Thread Jurgens de Bruin
This is my first script where I want to use the python threading module. I have a large dataset which is a list of dict this can be as much as 200 dictionaries in the list. The final goal is a histogram for each dict 16 histograms on a page ( 4x4 ) - this already works. What I currently do is

Re: Help with Threading

2005-01-25 Thread Philip Smith
<[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] >I use threading.Thread as outlined in this recipe: > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65448 >Thanks -- http://mail.python.org/mailman/listinfo/python-list

Re: Help with Threading

2005-01-24 Thread [EMAIL PROTECTED]
I use threading.Thread as outlined in this recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65448 -- http://mail.python.org/mailman/listinfo/python-list

Re: Help with Threading

2005-01-24 Thread Pierre Barbier de Reuille
Philip Smith a écrit : Hi I am fairly new to Python threading and my needs are simple(!) I want to establish a number of threads each of which work on the same computationally intensive problem in different ways. I am using the thread module rather than the threading module. My problem is I can't

Help with Threading

2005-01-23 Thread Philip Smith
Hi I am fairly new to Python threading and my needs are simple(!) I want to establish a number of threads each of which work on the same computationally intensive problem in different ways. I am using the thread module rather than the threading module. My problem is I can't see how (when one t

Re: need some help with threading module...

2004-12-30 Thread M.E.Farmer
Yacine, I didn't run it from Idle . I don't use Idle ! I wrote my own IDE when I first started programming in Python, that is what I used(it does execute scripts from the shell ;) What you are seeing is exactly what I was talking about threads and.. dare I say it... bugs . Be sure to look at t

Re: need some help with threading module...

2004-12-30 Thread chahnaz.ourzikene
"Steve Holden" <[EMAIL PROTECTED]> a écrit dans le message de news: [EMAIL PROTECTED] > Could be the OP is using Cygwin, which won't support threading by > default and will give very confusing results > > just-a-guess-ly y'rs - steve Nice try :), but nope :). Yacine Chaouche -- France. -- h

Re: need some help with threading module...

2004-12-30 Thread chahnaz.ourzikene
"M.E.Farmer" <[EMAIL PROTECTED]> a écrit dans le message de news: [EMAIL PROTECTED] > What did you expect? This is what it did on win 2000/python 2.2.3 > ##> controller waiting... 0 loops > ##> controller waiting... 1 loops > Subject : the counter is now 0 > ##> controller waiting..

Re: need some help with threading module...

2004-12-30 Thread Steve Holden
M.E.Farmer wrote: Steve Holden wrote: [snip] Could be the OP is using Cygwin, which won't support threading by default and will give very confusing results Thanks Steve, Well your guess was better then mine :) I didn't know Cygwin did not support threads by default , I will have to remember that.

Re: need some help with threading module...

2004-12-30 Thread M.E.Farmer
Steve Holden wrote: [snip] >Could be the OP is using Cygwin, which won't support threading by >default and will give very confusing results Thanks Steve, Well your guess was better then mine :) I didn't know Cygwin did not support threads by default , I will have to remember that. Why do you suppo

Re: need some help with threading module...

2004-12-30 Thread Steve Holden
M.E.Farmer wrote: chahnaz.ourzikene wrote: Hi, I fixed the code, it runs under Linux but not under windows 0_o ??! i guess windows and Linux do not handle threads the same way. However, i don't have the result i excpect. What did you expect? This is what it did on win 2000/python 2.2.3 ##> cont

Re: need some help with threading module...

2004-12-30 Thread M.E.Farmer
chahnaz.ourzikene wrote: > Hi, > > I fixed the code, it runs under Linux but not under windows 0_o ??! i guess > windows and Linux do not handle threads the same way. > > However, i don't have the result i excpect. What did you expect? This is what it did on win 2000/python 2.2.3 ##> controller

Re: need some help with threading module...

2004-12-30 Thread chahnaz.ourzikene
Hi, I fixed the code, it runs under Linux but not under windows 0_o ??! i guess windows and Linux do not handle threads the same way. However, i don't have the result i excpect. Please have a look here : ## In this little program, i'm trying to find a way to yield data from a thread within anot

Re: need some help with threading module...

2004-12-27 Thread M.E.Farmer
chahnaz.ourzikene wrote: > "M.E.Farmer" <[EMAIL PROTECTED]> a écrit dans le message de news: > [EMAIL PROTECTED] > > > Just a warning! > > Threads and newbies don't mix well, > > many pitfalls and hard to find bugs await you. > > I would avoid using threads if at all possible. > > Indeed :). But ho

Re: need some help with threading module...

2004-12-27 Thread chahnaz.ourzikene
"M.E.Farmer" <[EMAIL PROTECTED]> a écrit dans le message de news: [EMAIL PROTECTED] > Just a warning! > Threads and newbies don't mix well, > many pitfalls and hard to find bugs await you. > I would avoid using threads if at all possible. Indeed :). But how will i learn using threads if i avoid

Re: need some help with threading module...

2004-12-27 Thread chahnaz.ourzikene
Hi everybody, "Daniel Bickett" <[EMAIL PROTECTED]> a écrit dans le message de news: [EMAIL PROTECTED] > Instead of having the Controller query the Subject (not exactly > plausible), I had it wait for a signal (threading.Event) as set by the > Subject. You could also have it query a queue, as that

Re: need some help with threading module...

2004-12-26 Thread Daniel Bickett
I found your object-oriented approach, while admirable, a little muddled. So rather than modify your code, I simply took the paragraph you wrote describing the scenario and wrote my own.[1] Instead of having the Controller query the Subject (not exactly plausible), I had it wait for a signal (thre

Re: need some help with threading module...

2004-12-26 Thread M.E.Farmer
Just a warning! Threads and newbies don't mix well, many pitfalls and hard to find bugs await you. I would avoid using threads if at all possible. Now we have all that over lets see some code. py> import threading py> class Test(threading.Thread): ...def run(self): ... x = 0

Re: need some help with threading module...

2004-12-26 Thread chahnaz.ourzikene
I think it is more suitable in this form... from threading import * class Subject(object): def __init__(self) : self.counter = 0 t = Timer(0.1,self.doIteratingStuff) t.start() def incrementCounter(self,n=1) :

Re: need some help with threading module...

2004-12-26 Thread M.E.Farmer
> Thanks for all and merry x-mas and blablablah There is no X in Christmas, and blablablah should read Happy New Year! >Of course, the indentations are lost since i copied this code from my emacs >on linux, and then paste it on my outlook express (i don't have the net on >linux :(...). I have had

need some help with threading module...

2004-12-26 Thread chahnaz.ourzikene
Hi all, This is the first i post in this newsgroup, i hope my english is not too bad... Let's get straight to the point ! I have a little probleme using threads in my little training example : I wish to create two threads in my application, one thread (the subject) will increment a variable, and a