[issue4565] Rewrite the IO stack in C

Changes by Benjamin Peterson : -- dependencies: -possible deadlock in python IO implementation ___ Python tracker ___ ___ Python-bugs-

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: And the io-c branch has been merged in r70152. -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: 2009/3/3 Daniel Diniz : > A couple of typos in the Python implementation. Thanks for taking a look! Fixed these things in r70135. > http://codereview.appspot.com/22061/diff/1/11#newcode844 > Line 844: self._reset_read_buf() > Setting "_read_buf" and "_read_

[issue4565] Rewrite the IO stack in C

Daniel Diniz added the comment: A couple of typos in the Python implementation. http://codereview.appspot.com/22061/diff/1/11 File Lib/_pyio.py (right): http://codereview.appspot.com/22061/diff/1/11#newcode266 Line 266: fp is closed after the suite of the with statment is complete: statment ->

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: Reviewers: , Description: The diff between the py3k and io-c branches, for whoever wants to review it. Please review this at http://codereview.appspot.com/22061 Affected files: Doc/library/io.rst Lib/_pyio.py Lib/importlib/__init__.py Lib/importli

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: These StringIO bugs should be dealt with: #5264 #5265 #5266 ___ Python tracker ___ ___ Python-bugs-list m

[issue4565] Rewrite the IO stack in C

Changes by Benjamin Peterson : -- dependencies: +Duplicate UTF-16 BOM if a file is open in append mode, utf-16 BOM is not skipped after seek(0) ___ Python tracker ___ ___

[issue4565] Rewrite the IO stack in C

Changes by Benjamin Peterson : -- dependencies: +possible deadlock in IO library (Lib/io.py) ___ Python tracker ___ ___ Python-bugs-lis

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: I just fixed the last failing test_io. (I'm listing as dependencies issues we can close after the branch is merged.) -- dependencies: +BufferedWriter non-blocking overage, io.TextIOWrapper calls buffer.read1() _

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: Ok, so the ABC stuff is done now. Remaining: - fix the test failures with the Python implementation - the _ssl bug ___ Python tracker ___ ___

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: On Wed, Feb 25, 2009 at 10:15 AM, Antoine Pitrou wrote: > > Antoine Pitrou added the comment: > > I just took a quick look at Lib/abc.py and there's no way *I*'ll > reimplement it in C :) I don't blame you for that. :) > > The only workable approach would

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: I just took a quick look at Lib/abc.py and there's no way *I*'ll reimplement it in C :) The only workable approach would be: 1. rename the current would-be ABCs (IOBase, RawIOBase, etc.) with a leading underscore (_IOBase, _RawIOBase, etc.) 2. call abc.ABCMeta(

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: > We also have to figure out how to make the C IOBase a ABC, so people can > implement it. Mmmh, I know absolutely nothing about the ABC implementation. ___ Python tracker

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: We also have to figure out how to make the C IOBase a ABC, so people can implement it. ___ Python tracker ___

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: On Mon, Feb 23, 2009 at 2:26 PM, Antoine Pitrou wrote: > test_largefile is done. Thanks. > One more question: what shall we do with _pyio.OpenWrapper? > Should it become the default exported "open" object? No, I think it was just meant to be used when _py

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: test_largefile is done. One more question: what shall we do with _pyio.OpenWrapper? Should it become the default exported "open" object? ___ Python tracker _

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: Oh, and test_largefile and test_debussy as well :) Le dimanche 22 février 2009 à 23:00 +, Antoine Pitrou a écrit : > Antoine Pitrou added the comment: > > There's also test_univnewlines, I think. ___ Python tracker

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: There's also test_univnewlines, I think. ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: On Sun, Feb 22, 2009 at 1:50 PM, Antoine Pitrou wrote: > > Antoine Pitrou added the comment: > > What should we do about test_fileio, test_file and test_bufio? I changed test_file and test_bufio to test the open() implementations of each library. test_file

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: What should we do about test_fileio, test_file and test_bufio? ___ Python tracker ___ ___ Python-bugs-list ma

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: Ok. I've fixed all the tests except PyBufferedRandomTest.testFlushAndPeek and the garbage collections ones. ___ Python tracker ___ ___

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: Ok. I've split the Python io implementation into the _pyio module and rewritten the tests. All the C ones are passing, but some Python implementation ones are failing. ___ Python tracker

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: The StringIO rewrite is finished now. ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: It seems the decision of Python-dev is to keep both implementations. We'll stuff the python one in _pyio and rewrite the tests to test both. I'll see if I can get to this this weekend. ___ Python tracker

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: > +1 to setting it up so that unit tests are always run against both and > keeping both. If this is the way forward I recommend putting the pure Python versions into a separate module, eg pyio.py (although the name is not very elegant). It will make the separa

[issue4565] Rewrite the IO stack in C

Gregory P. Smith added the comment: +1 to setting it up so that unit tests are always run against both and keeping both. -- nosy: +gregory.p.smith ___ Python tracker ___ ___

[issue4565] Rewrite the IO stack in C

Raymond Hettinger added the comment: > Basically, my point is that maintaining C and Python > versions is *cheaper* than just maintaining the C > version alone. Well said. ___ Python tracker _

[issue4565] Rewrite the IO stack in C

Jean-Paul Calderone added the comment: Hi Antoine, > > Surely the majority of the burden is imposed by the C implementation. I expect that 90% of the time spent fixing bugs will be spent fixing them in C. > Hmm, it depends. It's probably true in general, but I suspect a fair amount of work al

[issue4565] Rewrite the IO stack in C

Raymond Hettinger added the comment: [Benjamin Peterson] > I think we should just drop the Python implementations. There's no point > in trying to keep two implementations around. I disagree. I've found great value in keeping a pure python version around for things I've converted to C. The f

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: Hello JP, > Surely the majority of the burden is imposed by the C implementation. I > expect that 90% of the time spent fixing bugs will be spent fixing them > in C. Hmm, it depends. It's probably true in general, but I suspect a fair amount of work also went

[issue4565] Rewrite the IO stack in C

Jean-Paul Calderone added the comment: > We don't maintain any other features in two languages for those purposes. IMO, it will just be more of a burden to fix bugs in two different places as compared to the advantages you mention. Surely the majority of the burden is imposed by the C implement

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: On Thu, Feb 19, 2009 at 1:57 PM, Jean-Paul Calderone wrote: > > Jean-Paul Calderone added the comment: > >> Oh, and "what to do of the now unused pure Python implementations in > io.py"? Easiest would be to dump them, as they will probably get > hopelessly

[issue4565] Rewrite the IO stack in C

Jean-Paul Calderone added the comment: > Oh, and "what to do of the now unused pure Python implementations in io.py"? Easiest would be to dump them, as they will probably get hopelessly out of sync, but perhaps there's some genuine portability/educational advantage to keep them? The test suite

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: I think we should just drop the Python implementations. There's no point in trying to keep two implementations around. Besides, if we don't backport IO in C, we can maintain them in the trunk. :) ___ Python tracker

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: Oh, and "what to do of the now unused pure Python implementations in io.py"? Easiest would be to dump them, as they will probably get hopelessly out of sync, but perhaps there's some genuine portability/educational advantage to keep them? __

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: After "rewrite the rest of StringIO in C", there's "sanitize the destructor behaviour of IOBase (if at all possible)". ___ Python tracker ___ ___

[issue4565] Rewrite the IO stack in C

Benjamin Peterson added the comment: This is basically going to be the killer feature in 3.1 ;). Therefore, these are steps I think we need before we can merge the branch: - Fix the dependencies. (#4967) - Resolve all outstanding issues with the IO lib on the io-c branch. - Rewrite the rest of

[issue4565] Rewrite the IO stack in C

Changes by Benjamin Peterson : -- nosy: +benjamin.peterson ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: Issue depends on #4967 which blocks use of memoryview objects with the _ssl module. -- assignee: amaury.forgeotdarc -> dependencies: +Bugs in _ssl object read() when a buffer is specified ___ Python tracker

[issue4565] Rewrite the IO stack in C

Antoine Pitrou added the comment: The work to rewrite the IO stack in C will solve this problem as it will probably solve most performance-related IO problems in py3k. Amaury and I have been progressing a lot, the rewrite is now a real branch in SVN at branches/io-c/. On this very issue, it is