Re: memory leak with re.match

2017-07-05 Thread Peter Otten
Mayling ge wrote: > Sorry. The code here is just to describe the issue and is just pseudo > code, That is the problem with your post. It's too vague for us to make sense of it. Can you provide a minimal example that shows what you think is a "memory leak"? Then we can either help you avo

Re: memory leak with re.match

2017-07-05 Thread Mayling ge
Mayling ge Sent: Tuesday, July 4, 2017 9:01 AM To: python-list Subject: memory leak with re.match Hi, My function is in the following way to handle file line by line. There are multiple error patterns defined and need to apply to each line. I use

Re: memory leak with re.match

2017-07-05 Thread Albert-Jan Roskam
From: Python-list on behalf of Mayling ge Sent: Tuesday, July 4, 2017 9:01 AM To: python-list Subject: memory leak with re.match      Hi,    My function is in the following way to handle file line by line. There are    multiple error patterns  defined and  need to apply  to each  line. I

Re: memory leak with re.match

2017-07-05 Thread Mayling ge
Thanks. I actually comment out all handling code. The loop ends with the re_pat.match and nothing followed. Sent from Mail Master On 07/05/2017 08:31, [1]Cameron Simpson wrote: On 04Jul2017 17:01, Mayling ge wrote: > My function is in the following way to handle file lin

Re: memory leak with re.match

2017-07-04 Thread Cameron Simpson
On 04Jul2017 17:01, Mayling ge wrote: My function is in the following way to handle file line by line. There are multiple error patterns defined and need to apply to each line. I use multiprocessing.Pool to handle the file in block. The memory usage increases to 2G for a 1G file. A

memory leak with re.match

2017-07-04 Thread Mayling ge
Hi, My function is in the following way to handle file line by line. There are multiple error patterns defined and need to apply to each line. I use multiprocessing.Pool to handle the file in block. The memory usage increases to 2G for a 1G file. And stays in 2G even after