[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Changes by Charles Cazabon : Removed file: http://bugs.python.org/file14987/worker-lifetime-python2.6.2.patch ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Charles Cazabon added the comment: No problem, Jesse, I realize you're busy. I've updated the patch very slightly to handle an issue I had where the worker maintainer hung once. -- Added file: http://bugs.python.org/file15940/worker-lifetime-python2.6.2.patch ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Charles Cazabon added the comment: Thanks, Jesse -- it looks good. If there are bugs remaining in the patch, they're mine and not yours. -- ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7970] email.Generator fails to flatten message parsed by email.Parser
New submission from Charles Cazabon :
email.Generator fails to flatten a message parsed by email.Parser; it throws an
exception with an odd (but apparently legal) message. First, the exception:
File "/usr/local/lib/python2.6/email/generator.py", line 84, in flatten
self._write(msg)
File "/usr/local/lib/python2.6/email/generator.py", line 109, in _write
self._dispatch(msg)
File "/usr/local/lib/python2.6/email/generator.py", line 135, in _dispatch
meth(msg)
File "/usr/local/lib/python2.6/email/generator.py", line 266, in
_handle_message
g.flatten(msg.get_payload(0), unixfrom=False)
File "/usr/local/lib/python2.6/email/message.py", line 189, in get_payload
raise TypeError('Expected list, got %s' % type(self._payload))
TypeError: Expected list, got
The oddity of the message its handling is that it's a single-part MIME message,
where the content-type is declared as message/rfc822. Most MUAs, when
forwarding message, create a multipart MIME message with a single attachment
part, and have the attachment be message/rfc822. But Groupwise, when
configured to forward messages to another address (without specifying an
additional text to insert with the forwarded message) creates these slightly
odd messages.
I've not seen them before, but a couple of getmail users have run into this
issue. I've confirmed the problem is the same in Python 2.3, 2.4, 2.5, and
2.6, and I presume it's the same in 2.7 but haven't tested yet.
I'll attach a minimal test script and a datafile which is a minimal message
showing the problem.
--
components: Library (Lib)
files: testcase.py
messages: 99606
nosy: charlesc
severity: normal
status: open
title: email.Generator fails to flatten message parsed by email.Parser
type: behavior
versions: Python 2.5, Python 2.6
Added file: http://bugs.python.org/file16262/testcase.py
___
Python tracker
<http://bugs.python.org/issue7970>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30161] Using `with` statement causes dict to start papering over attribute errors
New submission from Charles Cazabon:
This is a weird one. I've reproduced it with 3 versions of 2.7, including the
latest 2.7.13. I didn't find an open bug about this, but I had difficulty
crafting a search string for it, so I may have missed something.
Basically, using a `with` statement (maybe any such statement, but using an
open file definitely does it, even when I do nothing with it) causes the
built-in dict class to stop raising AttributeErrors, which can result in odd
bugs.
Example:
Python 2.7.13 (default, Apr 25 2017, 10:12:36)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> with sys.stderr as foo:
... pass
...
>>> {}.nosuchattribute
>>> {}.nosuchattribute is None
>>>
I haven't tried the latest 3.x, but it's definitely still there in 3.2.3:
Python 3.2.3 (default, Nov 17 2016, 01:04:00)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> with sys.stderr as foo:
... pass
...
>>> {}.nosuchattribute
>>> {}.nosuchattribute is None
>>>
--
components: Interpreter Core
messages: 292270
nosy: charlesc
priority: normal
severity: normal
status: open
title: Using `with` statement causes dict to start papering over attribute
errors
type: behavior
versions: Python 2.7
___
Python tracker
<http://bugs.python.org/issue30161>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30161] Using `with` statement causes dict to start papering over attribute errors
Charles Cazabon added the comment: oh ffs ;) -- ___ Python tracker <http://bugs.python.org/issue30161> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
New submission from Charles Cazabon : Worker processes with multiprocessing.Pool live for the duration of the Pool. If the tasks they run happen to leak memory (from a C extension module, or from creating cycles of unreachable objects, etc) or open files or other resources, there's no easy way to clean them up. Similarly, if one task passed to the pool allocates a large amount of memory, but further tasks are small, that additional memory isn't returned to the system because the process involved hasn't exited. A common approach to this problem (as used by Apache, mod_wsgi, and various other software) is to allow worker processes to exit (and be replaced with fresh processes) after completing a specified amount of work. The attached patch (against Python 2.6.2, but applies to various other versions with some fuzz) implements this as optional new behaviour in multiprocessing.Pool(). An additional optional argument is specified for the maximum number of tasks a worker process performs before it exits and is replaced with a fresh worker process. -- components: Library (Lib) files: worker-lifetime-python2.6.2.patch keywords: patch messages: 92971 nosy: charlesc severity: normal status: open title: Add worker process lifetime to multiprocessing.Pool - patch included type: feature request versions: Python 2.7, Python 3.1 Added file: http://bugs.python.org/file14946/worker-lifetime-python2.6.2.patch ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Changes by Charles Cazabon : Removed file: http://bugs.python.org/file14946/worker-lifetime-python2.6.2.patch ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Changes by Charles Cazabon : Added file: http://bugs.python.org/file14947/worker-lifetime-python2.6.2.patch ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Charles Cazabon added the comment: Alright, I'll add those. Thanks for looking at it; first pass was mostly to ensure it wouldn't be wasted work. -- ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Changes by Charles Cazabon : Removed file: http://bugs.python.org/file14947/worker-lifetime-python2.6.2.patch ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Charles Cazabon added the comment: Updated patch attached; handles some of the Pool methods that weren't handled before. Now includes documentation and unit test additions as well. -- Added file: http://bugs.python.org/file14987/worker-lifetime-python2.6.2.patch ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Charles Cazabon added the comment: Jesse: this is ready for your review now. Thanks, Charles -- ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Charles Cazabon added the comment: Can someone review this patch? I believe it's sufficient for inclusion now, as it includes docs and unit tests, but if anything about it requires further attention I'd be happy to listen to change requests. We'd like get this into mainline where everyone can benefit rather than continuing to maintain it out-of-tree. -- ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Charles Cazabon added the comment: Okay, thanks, Jesse. Didn't realize the Con was on. -- ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Charles Cazabon added the comment: Hi Jesse -- Any chance you'll be able to review this in time for it to make it into trunk for the 2.7 alpha release? Charles -- ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6963] Add worker process lifetime to multiprocessing.Pool - patch included
Charles Cazabon added the comment: Ping... two alphas into 2.7. Have you had a chance to review this functionality? Thanks. -- ___ Python tracker <http://bugs.python.org/issue6963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
