[issue25997] Tarfile.add with bytes path is failling
New submission from Patrik Dufresne: With python 3.4, Tarfile doesn't properly support adding a files with bytes path. Only unicode is supported. It's failing with exception similar to: tar.add(os.path.join(dirpath, filename), filename) File "/usr/lib/python3.4/tarfile.py", line 1907, in add tarinfo = self.gettarinfo(name, arcname) File "/usr/lib/python3.4/tarfile.py", line 1767, in gettarinfo arcname = arcname.replace(os.sep, "/") TypeError: expected bytes, bytearray or buffer compatible object It uses os.sep, and u"/". Instead, it should use something like posixpath.py:_get_sep(path). -- components: Unicode messages: 257355 nosy: Patrik Dufresne, ezio.melotti, haypo priority: normal severity: normal status: open title: Tarfile.add with bytes path is failling type: behavior versions: Python 3.4 ___ Python tracker <http://bugs.python.org/issue25997> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24110] zipfile.ZipFile.write() does not accept bytes arcname
Patrik Dufresne added the comment: I'm converting my project into python3. I'm encountering issue with zipfile encoding. Look like, it only support unicode path. This is a huge issue since path are, by definition, bytes. You may store a file name with an invalid character without issue on the filesystem. As such, arcname should support bytes. Like, Tar, zip file format doesn't define a specific encoding. You may store filename as bytes. ------ nosy: +Patrik Dufresne ___ Python tracker <http://bugs.python.org/issue24110> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10757] zipfile.write, arcname should be allowed to be a byte string
Patrik Dufresne added the comment: This bug is very old, any development on the subject. This issue is hitting me trying to port my project (rdiffweb) to python3. It received a lot of broken filename with invalid encoding and I need to create a meaningful Zip archive with it. Currently, it just fail because zipfile doesn't accept arcname as bytes. -- nosy: +Patrik Dufresne ___ Python tracker <http://bugs.python.org/issue10757> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25997] Tarfile.add with bytes path is failing
Patrik Dufresne added the comment: > Is the tarfile module designed to support bytes for file names in general? > The documentation doesn’t seem to mention bytes anywhere relevant. This seems > more like a new feature rather than a bug to me. I'm using bytes in Unix to represent a path. From `os.path` docs : The path parameters can be passed as either strings, or bytes. Applications are encouraged to represent file names as (Unicode) character strings. Unfortunately, some file names may not be representable as strings on Unix, so applications that need to support arbitrary file names on Unix should use bytes objects to represent path names. Vice versa, using bytes objects cannot represent all file names on Windows (in the standard mbcs encoding), hence Windows applications should use string objects to access all files. As such, I'm expecting to use bytes to represent a path with tarfile. Also, tar file format doesn't define any specific encoding for filename. I'me xpecting to but any kind of bytes data for a given filename... since this was wokring in tarfile with py2. > Does using a surrogateescape encoded filename work? (You won't get the error > you report...my question is, does that do the right thing when building the > archive?) I will need to have further look into surrogateescape. I read somewhere it was an experimental feature, so I didn't try it. Thanks both for your quick feedback in this holidays. -- ___ Python tracker <http://bugs.python.org/issue25997> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25997] Tarfile.add with bytes path is failing
Patrik Dufresne added the comment: It's a bit tricky, but with help of surrogateescape I get the expected result. I'm closing this bug. Thanks -- status: open -> closed ___ Python tracker <http://bugs.python.org/issue25997> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23252] Add support of writing to unseekable file in zipfile
Patrik Dufresne added the comment: While this path provide support to create Zip file for non-seekable stream. It doesn't support reading a file from a non-seekable stream. -- nosy: +Patrik Dufresne ___ Python tracker <http://bugs.python.org/issue23252> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24110] zipfile.ZipFile.write() does not accept bytes arcname
Patrik Dufresne added the comment: Manage to work around this issue by using surrogateescape for arcname and filename. For me it's no longer an issue. -- ___ Python tracker <http://bugs.python.org/issue24110> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26253] tarfile in stream mode always set zlib compression level to 9
New submission from Patrik Dufresne: When using tarfile.open(mode='w|gz'), the compression level is hard-coded to 9. Seed _Stream._init_write_gz(): self.zlib.compressobj(9, 1. In regards to zlib, I would start by replacing the value of 9 by zlib.Z_DEFAULT_COMPRESSION. This is the default value and zipfile is using it. Why using something different. 2. Then, I would also love to control the compression level when calling tarfile.open() -- components: Library (Lib) messages: 259304 nosy: Patrik Dufresne priority: normal severity: normal status: open title: tarfile in stream mode always set zlib compression level to 9 type: behavior versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6 ___ Python tracker <http://bugs.python.org/issue26253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26269] zipfile should call lstat instead of stat if available
New submission from Patrik Dufresne: To mirror the behavior in tarfile, zipfile should make a call to stat instead of lstat if available. Failure to do so resolved into an "IOError No such file or directory" if the filename is a symbolik link being broken. As such, the creation of the archive fail. -- components: Library (Lib) messages: 259436 nosy: Patrik Dufresne priority: normal severity: normal status: open title: zipfile should call lstat instead of stat if available versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6 ___ Python tracker <http://bugs.python.org/issue26269> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14119] Ability to adjust queue size in Executors
Patrik Dufresne added the comment: Any update on this subject ? Also had to monkey patch the implementation to avoid consuming all the system memory. -- nosy: +Patrik Dufresne ___ Python tracker <http://bugs.python.org/issue14119> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21009] Potential deadlock in concurrent futures when garbage collection occurs during Queue.get
Patrik Dufresne added the comment: I've encounter this issue. To easily avoid this issue, I've change `queue.put(None)` to `queue.put(None, block=False)` to work around this. -- nosy: +Patrik Dufresne ___ Python tracker <http://bugs.python.org/issue21009> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27978] Executor#shutdown with timeout
New submission from Patrik Dufresne: Would be nice to add a new parameter to timeout if the shutdown take too long. def shutdown(self, wait=True, timeout=None): with self._shutdown_lock: self._shutdown = True self._work_queue.put(None) if wait: for t in self._threads: start = time.time() t.join(timeout) timeout = timeout + start - time.time() -- components: Library (Lib) messages: 274599 nosy: Patrik Dufresne priority: normal severity: normal status: open title: Executor#shutdown with timeout type: enhancement versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue27978> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
