[issue12175] FileIO.readall() read the file position and size at each read

Changes by STINNER Victor : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue12175] FileIO.readall() read the file position and size at each read

Roundup Robot added the comment: New changeset 72c89daace57 by Victor Stinner in branch 'default': Issue #12175: FileIO.readall() now only reads the file position and size once. http://hg.python.org/cpython/rev/72c89daace57 New changeset 3c7792ec4547 by Victor Stinner in branch 'default': Issue

[issue12175] FileIO.readall() read the file position and size at each read

Antoine Pitrou added the comment: > So, do you think that fileio_readall.patch and > bufferedreader_readall-2.patch should be commited, or these changes > are useless? They should be committed. Thank you :) -- ___ Python tracker

[issue12175] FileIO.readall() read the file position and size at each read

Changes by STINNER Victor : Removed file: http://bugs.python.org/file22108/rawiobase_readall.patch ___ Python tracker ___ ___ Python-bugs-list

[issue12175] FileIO.readall() read the file position and size at each read

Changes by STINNER Victor : Removed file: http://bugs.python.org/file22107/fileio_readall_closed.patch ___ Python tracker ___ ___ Python-bugs-

[issue12175] FileIO.readall() read the file position and size at each read

Changes by STINNER Victor : Removed file: http://bugs.python.org/file22109/bufferedreader_readall.patch ___ Python tracker ___ ___ Python-bugs

[issue12175] FileIO.readall() read the file position and size at each read

Changes by STINNER Victor : Added file: http://bugs.python.org/file22118/bufferedreader_readall-2.patch ___ Python tracker ___ ___ Python-bugs

[issue12175] FileIO.readall() read the file position and size at each read

Changes by STINNER Victor : Removed file: http://bugs.python.org/file22117/bufferedreader_readall-2.patch ___ Python tracker ___ ___ Python-bu

[issue12175] FileIO.readall() read the file position and size at each read

STINNER Victor added the comment: > You must Py_DECREF(all) first. > Also, you should check that "all" is either None or a bytes object. Right, fixed in bufferedreader_readall-2.patch. By the way, thanks for your reviews. So, do you think that fileio_readall.patch and bufferedreader_readall-2

[issue12175] FileIO.readall() read the file position and size at each read

STINNER Victor added the comment: > Attached patch [fileio_readall.patch] fixes both problems. > Looks ok to me. Did you test under Windows? Yes, test_io pass on Windows Vista 64 bits. > Did you run some benchmarks? Yes, see msg136853. Do you mean that I should not touch FileIO.readall() if

[issue12175] FileIO.readall() read the file position and size at each read

Roundup Robot added the comment: New changeset 4a7118cff1d3 by Victor Stinner in branch '3.1': Issue #12175: RawIOBase.readall() now returns None if read() returns None. http://hg.python.org/cpython/rev/4a7118cff1d3 New changeset fb29dc650d24 by Victor Stinner in branch '3.2': (Merge 3.1) Issue

[issue12175] FileIO.readall() read the file position and size at each read

Roundup Robot added the comment: New changeset f2414bb35c96 by Victor Stinner in branch '2.7': Issue #12175: FileIO.readall() now raises a ValueError instead of an IOError if http://hg.python.org/cpython/rev/f2414bb35c96 -- ___ Python tracker

[issue12175] FileIO.readall() read the file position and size at each read

Roundup Robot added the comment: New changeset 742ff94cdd20 by Victor Stinner in branch '3.1': Issue #12175: FileIO.readall() now raises a ValueError instead of an IOError if http://hg.python.org/cpython/rev/742ff94cdd20 New changeset 745e373c0b8e by Victor Stinner in branch '3.2': (Merge 3.1)

[issue12175] FileIO.readall() read the file position and size at each read

Antoine Pitrou added the comment: About rawiobase_readall.patch: why do you use PyObject_Size() if you know chunks is a list? PyList_GET_SIZE is much more efficient. About bufferedreader_readall.patch: +PyBytes_Concat(&data, all); +if (data == NULL) +re

[issue12175] FileIO.readall() read the file position and size at each read

STINNER Victor added the comment: Number of syscalls without the patch -> with the patch: - read the README file, text mode: 17 -> 12 syscalls (-5) - read the README file, binary mode: 15 -> 11 syscalls (-4) - read a binary file of 10 MB: 17 -> 12 syscalls (-5) Quick benchmark open('README').r

[issue12175] FileIO.readall() read the file position and size at each read

Antoine Pitrou added the comment: > bufferedreader_readall.patch: BufferedReader.read(None) calls > raw.readall() if available. Have you run any benchmarks? -- ___ Python tracker _

[issue12175] FileIO.readall() read the file position and size at each read

STINNER Victor added the comment: bufferedreader_readall.patch: BufferedReader.read(None) calls raw.readall() if available. bufferedreader_readall.patch requires fileio_readall_closed.patch and rawiobase_readall.patch fixes. -- Added file: http://bugs.python.org/file22109/bufferedrea

[issue12175] FileIO.readall() read the file position and size at each read

STINNER Victor added the comment: RawIOBase.readall() fails if .read() returns None: fix attached. -- Added file: http://bugs.python.org/file22108/rawiobase_readall.patch ___ Python tracker ___

[issue12175] FileIO.readall() read the file position and size at each read

Antoine Pitrou added the comment: +raw = f +if hasattr(raw, 'buffer'): +raw = raw.buffer +if hasattr(raw, 'raw'): +raw = raw.raw f.close() self.assertRaises(ValueError, f.flush) self.assert

[issue12175] FileIO.readall() read the file position and size at each read

STINNER Victor added the comment: Oh, FileIO.readall() doesn't raise a ValueError if the file is closed => patch attached to fix this. -- Added file: http://bugs.python.org/file22107/fileio_readall_closed.patch ___ Python tracker

[issue12175] FileIO.readall() read the file position and size at each read

Antoine Pitrou added the comment: > FileIO.readall() reads the file position and size before each call to > read(), to adjust the buffer size. > > Moreover FileIO.readall() calls lseek() on Windows: it should use > _lseeki64() instead, to handle correctly file bigger than 2 GB (or > maybe 4 GB?

[issue12175] FileIO.readall() read the file position and size at each read

Antoine Pitrou added the comment: > BufferedReader.read() calls FileIO.read() until FileIO.read() returns > an empty byte string. Why not calling FileIO.read() only once? BufferedReader doesn't call FileIO.read, it calls .read(). The latter can be e.g. a socket and call recv(). If you want to r

[issue12175] FileIO.readall() read the file position and size at each read

New submission from STINNER Victor : FileIO.readall() reads the file position and size before each call to read(), to adjust the buffer size. Moreover FileIO.readall() calls lseek() on Windows: it should use _lseeki64() instead, to handle correctly file bigger than 2 GB (or maybe 4 GB? I don't