Oh good! I'm glad you found the root cause.
Later I anticipate the following will happen, which may help:
* the test event system is arranged to capture stdout/stderr from each test
method within the test events.
* the stdout/stderr from the inferior dotest.py will become far less
important and t
Zach had the clue that found the problem. Python on Windows uses the stdio
from CRT. On Windows, the default limit for open file descriptors is 512.
When you have 40 logical cores and the parent process spends several FDs
communicating with each one, you get real close to that number.
https://ms
Adding a printing destructor to threading.Event seems to aggravate timing
problems, causing several tests to fail to make their inferiors and that
seemingly keeps us below the open file limit. That aside, the destructor
did fire many hundreds of times, so there's not a general problem stopping
all
Okay.
A promising avenue might be to look at how Windows cleans up the
threading.Event objects. Chasing that thread might yield why the events
are not going away (assuming those are the events that are lingering on
your end). One thing you could consider doing is patching in a replacement
destru
Python 2.7.10 made no difference. I'm dealing with other issues this
afternoon, so I'll probably return to this on Wednesday. It's not critical
since there are workarounds.
On Tue, Oct 6, 2015 at 9:41 AM, Todd Fiala wrote:
>
>
> On Mon, Oct 5, 2015 at 3:58 PM, Adrian McCarthy
> wrote:
>
>> Di
On Mon, Oct 5, 2015 at 3:58 PM, Adrian McCarthy wrote:
> Different tools are giving me different numbers.
>
> At the time of the error, Windbg says there are about 2000 open handles,
> most of them are Event handles, not File handles. That's higher than I'd
> expect, but not really concerning.
>
Different tools are giving me different numbers.
At the time of the error, Windbg says there are about 2000 open handles,
most of them are Event handles, not File handles. That's higher than I'd
expect, but not really concerning.
Process Explorer, however, shows ~20k open handles per Python proc
It's possible. However, I was monitoring actual open files during the
course of the run (i.e. what the kernel thought was open for the master
driver process, which is the only place that makes sense to see leaks
accumulate) in both threading and threading-pool (on OS X), and I saw only
the handful
I'm poking around with some SysInternals tools. Over the course of test
run, there are about 602k opens (CreateFiles) and 405k
closes (CloseFiles) system-wide.
I'm looking for a way to stop it once the error happens, so I can see how
many files each process has open. As it stands, the OS cleans
On OS X, I'm also not seeing growth in the --test-runner-name
threading-pool (the one you were using on Windows).
Perhaps you can dig into if you're experiencing some kind of file leak on
Windows. It's possible you're hitting a platform-specific leak? I recall
Ed Maste hitting a FreeBSD-only lea
Hmm, on OS X the file handles seem to be well behaved on the
--test-runner-name threading. I'm not seeing any file handle growth beyond
the file handles I expect to be open.
I'll see if the threading-pool behaves differently. (That is similar to
threading but uses the multiprocessing.pool mechan
Interesting, okay..
This does appear to be an accumulation issue. You made it most of the way
through before the issue hit. I suspect we're leaking file handles. It
probably doesn't hit the per-process limit on multiprocessing because the
leaked files get spread across more processes.
(All spe
Thanks for the ideas.
With `--test-runner-name threading-pool`, I get too many open files.
With `--test-runner-name multiprocessing-pool`, the suite runs fine.
My machine has 40 logical cores.
With `--threads=20`: SUCCESS (and perhaps _faster_).
With `--threads=30`: SUCCESS.
With `--threads
(swapped out the lldb list for the newer one)
On Fri, Oct 2, 2015 at 5:37 PM, Todd Fiala wrote:
> Hmm, sounds suspicious.
>
> Can you try running the tests with two options and see if you get
> different results?
>
> # should be equivalent for the default on Windows, thus should match your
> abo
14 matches
Mail list logo
