[Python-Dev] semantics of subclassing things from itertools
Hi I would like to know what are the semantics if you subclass something from itertools (e.g. islice). Right now it's allowed and people do it, which is why the documentation is incorrect. It states "equivalent to: a function-or a generator", but you can't subclass whatever it is equivalent to, which is why in PyPy we're unable to make it work in pure python. I would like some clarification on that. Cheers, fijal ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] semantics of subclassing things from itertools
On 10.09.15 10:23, Maciej Fijalkowski wrote: I would like to know what are the semantics if you subclass something from itertools (e.g. islice). Right now it's allowed and people do it, which is why the documentation is incorrect. It states "equivalent to: a function-or a generator", but you can't subclass whatever it is equivalent to, which is why in PyPy we're unable to make it work in pure python. I would like some clarification on that. There is another reason why itertools iterators can't be implemented as simple generator functions. All iterators are pickleable in 3.x. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Can't post to bugs.python.org
I can't neither post a message to existing issue nor open a new issue. The irker854 bot on IRC channel #python-dev cites my message and the tracker updates activity time of existing issue, but doesn't show my message and doesn't reflect changes of status. Posting via e-mail doesn't work as well. Web server replies: An error has occurred A problem was encountered processing your request. The tracker maintainers have been notified of the problem. E-mail server replies: Subject: Failed issue tracker submission You are not a registered user. Please register at: http://bugs.python.org/user?@template=register ...before sending mail to the tracker. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] semantics of subclassing things from itertools
On Thu, Sep 10, 2015 at 10:26 AM, Serhiy Storchaka wrote: > On 10.09.15 10:23, Maciej Fijalkowski wrote: >> >> I would like to know what are the semantics if you subclass something >> from itertools (e.g. islice). >> >> Right now it's allowed and people do it, which is why the >> documentation is incorrect. It states "equivalent to: a function-or a >> generator", but you can't subclass whatever it is equivalent to, which >> is why in PyPy we're unable to make it work in pure python. >> >> I would like some clarification on that. > > > There is another reason why itertools iterators can't be implemented as > simple generator functions. All iterators are pickleable in 3.x. maybe the documentation should reflect that? (note that generators are pickleable on pypy anyway) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can't post to bugs.python.org
On Thu, 10 Sep 2015 13:27:42 +0300, Serhiy Storchaka wrote: > I can't neither post a message to existing issue nor open a new issue. > The irker854 bot on IRC channel #python-dev cites my message and the > tracker updates activity time of existing issue, but doesn't show my > message and doesn't reflect changes of status. Posting via e-mail > doesn't work as well. > > Web server replies: > > An error has occurred > A problem was encountered processing your request. The tracker > maintainers have been notified of the problem. > > E-mail server replies: > > Subject: Failed issue tracker submission > > You are not a registered user. Please register at: > > http://bugs.python.org/user?@template=register > > ...before sending mail to the tracker. I haven't tried the email gateway, but I can update issues. I see the irker message from when you tried to create one and indeed that issue does not exist. However, there are no messages among those sent to the tracker administrators that show errors around that time or for a few hours before it. If this continues to plague you, we'll probably need to do some live debugging. You can ping me (bitdancer) on IRC, I should be on for the next 8 hours or so. --David ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can't post to bugs.python.org
On Thu, 10 Sep 2015 09:02:01 -0400, "R. David Murray" wrote: > If this continues to plague you, we'll probably need to do some live > debugging. You can ping me (bitdancer) on IRC, I should be on for the > next 8 hours or so. This turns out to have been specific to Serhiy (or any issue on which he was nosy) and was due to a keyboard error on my part. It is now fixed. --David ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Yet another "A better story for multi-core Python" comment
On Wed, Sep 09, 2015 at 04:33:49PM -0400, Trent Nelson wrote: PyObjects, loads a huge NumPy array, and has a WSS of ~11GB. [...] I've done a couple of consultancy projects now that were very data science oriented (with huge data sets), so I really gained an appreciation for how common the situation you describe is. It is probably the best demonstration of PyParallel's strengths. This problem is also common in well-heeled financial services places, many of which are non-Windows. There might be some good opportunities there. Trent. Martin pgpEhY6ahMoN9.pgp Description: PGP signature ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] semantics of subclassing things from itertools
On 9/10/2015 3:23 AM, Maciej Fijalkowski wrote: Hi I would like to know what are the semantics if you subclass something from itertools (e.g. islice). I believe people are depending on an undocumented internal speed optimization. See below. Right now it's allowed and people do it, which is why the documentation is incorrect. It states "equivalent to: a function-or a generator", I believe Raymond has said that 'equivalent' should be taken as 'equivalent in essential function for iterating' rather than 'exactly equivalent for all operations'. The results of type() and isinstance() are not relevant for this. The itertools doc begins with "This module implements a number of iterator building blocks ..." After listing them, it says "The following module *functions* all construct and return iterators." (These part of the doc are unchanged from the original in 2.3. I added the emphasis.) The generator functions are mathematically equivalent if they produce equivalent iterators for the same inputs. The iterators are equivalent if they produce the same stream of objects when iterated. If they do, the doc is correct; if not, the doc is buggy and should be fixed. I see the undocumented fact that the module *functions* are implemented as C classes as an internal implementation detail to optimize speed. I believe Raymond intentionally used 'function' rather than 'class' and intended the equivalents to be usable by other implementations. Ask Raymond directly (he is not currently active on pydev) if I am correct. but you can't subclass whatever it is equivalent to, which is why in PyPy we're unable to make it work in pure python. You could write equivalent iterator classes in Python, but the result would be significantly slower. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.5): whatsnew/3.5: Editorialization pass on library section
On 11.09.15 00:36, yury.selivanov wrote: https://hg.python.org/cpython/rev/3265f33df731 changeset: 97874:3265f33df731 branch: 3.5 parent: 97872:70c97a626c41 user:Yury Selivanov date:Thu Sep 10 17:35:38 2015 -0400 summary: whatsnew/3.5: Editorialization pass on library section Patch by Elvis Pranskevichus io -- -* New Python implementation of :class:`io.FileIO` to make dependency on - ``_io`` module optional. - (Contributed by Serhiy Storchaka in :issue:`21859`.) +:class:`io.FileIO` has been implemented in Python which makes C implementation +of :mod:`io` module entirely optional. (Contributed by Serhiy Storchaka +in :issue:`21859`.) Unfortunately this is not true. Python implementation is required to make the _io module optional, but it is not enough. io and _pyio still depend from _io (issue17984). May be it is not worth to mention this change at all. End users unlikely will encounter Python implementation. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.5): whatsnew/3.5: Editorialization pass on library section
On 2015-09-10 6:16 PM, Serhiy Storchaka wrote: On 11.09.15 00:36, yury.selivanov wrote: https://hg.python.org/cpython/rev/3265f33df731 changeset: 97874:3265f33df731 branch: 3.5 parent: 97872:70c97a626c41 user:Yury Selivanov date:Thu Sep 10 17:35:38 2015 -0400 summary: whatsnew/3.5: Editorialization pass on library section Patch by Elvis Pranskevichus io -- -* New Python implementation of :class:`io.FileIO` to make dependency on - ``_io`` module optional. - (Contributed by Serhiy Storchaka in :issue:`21859`.) +:class:`io.FileIO` has been implemented in Python which makes C implementation +of :mod:`io` module entirely optional. (Contributed by Serhiy Storchaka +in :issue:`21859`.) Unfortunately this is not true. Python implementation is required to make the _io module optional, but it is not enough. io and _pyio still depend from _io (issue17984). May be it is not worth to mention this change at all. End users unlikely will encounter Python implementation. Alright, I dropped it. Thanks! Yury ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] semantics of subclassing things from itertools
> On Sep 10, 2015, at 3:23 AM, Maciej Fijalkowski wrote: > > I would like to know what are the semantics if you subclass something > from itertools (e.g. islice). > > Right now it's allowed and people do it, which is why the > documentation is incorrect. It states "equivalent to: a function-or a > generator", but you can't subclass whatever it is equivalent to, which > is why in PyPy we're unable to make it work in pure python. > > I would like some clarification on that. The docs should say "roughly equivalent to" not "exactly equivalent to". The intended purpose of the examples in the itertools docs is to use pure python code to help people better understand each tool. It is not is intended to dictate that tool x is a generator or is a function. The intended semantics are that the itertools are classes (not functions and not generators). They are intended to be sub-classable (that is why they have Py_TPFLAGS_BASETYPE defined). The description as a function was perhaps used too loosely (in much the same way that we tend to think of int(3.14) as being a function when int is really a class). I tend to think about mapping, filtering, accumulating, as being functions while at the same time knowing that they are actually classes that produce iterators. The section called "itertools functions" is a misnomer but is also useful because the patterns of documenting functions better fit the itertools and because documenting them as classes suggest that they should each have a list of methods on that class (which doesn't make send because the itertools are each one trick ponies with no aspirations to grow a pool of methods). When I get a chance, I'll go through those docs and make them more precise. Sorry for the ambiguity. Raymond ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
