[issue36975] csv: undocumented UnicodeDecodeError on malformed file
New submission from alter-bug-tracer : UnicodeDecodeError is thrown instead of csv.Error when parsing malformed inputs. Examples: 1. file0 UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 0: invalid continuation byte Traceback (most recent call last): File "csv_parser.py", line 6, in for row in reader: File "/usr/local/lib/python3.8/csv.py", line 111, in __next__ row = next(self.reader) File "/usr/local/lib/python3.8/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) 2. file1 UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 51: invalid start byte Traceback (most recent call last): File "csv_parser.py", line 6, in for row in reader: File "/usr/local/lib/python3.8/csv.py", line 110, in __next__ self.fieldnames File "/usr/local/lib/python3.8/csv.py", line 97, in fieldnames self._fieldnames = next(self.reader) File "/usr/local/lib/python3.8/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) (file0, file1 and csv_parser.py attached) -- files: csv.zip messages: 342939 nosy: alter-bug-tracer priority: normal severity: normal status: open title: csv: undocumented UnicodeDecodeError on malformed file type: behavior versions: Python 3.8 Added file: https://bugs.python.org/file48339/csv.zip ___ Python tracker <https://bugs.python.org/issue36975> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36976] email: AttributeError
New submission from alter-bug-tracer :
The 'lower' method is called on a Header object when trying to parse the
attached file.
Code:
import email
import sys
with open(sys.argv[1], "rb") as f:
msg = email.message_from_binary_file(f)
print (len(msg))
Traceback:
msg = email.message_from_binary_file(f)
File "/usr/lib/python3.5/email/__init__.py", line 62, in
message_from_binary_file
return BytesParser(*args, **kws).parse(fp)
File "/usr/lib/python3.5/email/parser.py", line 110, in parse
return self.parser.parse(fp, headersonly)
File "/usr/lib/python3.5/email/parser.py", line 57, in parse
feedparser.feed(data)
File "/usr/lib/python3.5/email/feedparser.py", line 178, in feed
self._call_parse()
File "/usr/lib/python3.5/email/feedparser.py", line 182, in _call_parse
self._parse()
File "/usr/lib/python3.5/email/feedparser.py", line 322, in _parsegen
if (self._cur.get('content-transfer-encoding', '8bit').lower()
AttributeError: 'Header' object has no attribute 'lower'
--
files: file0.zip
messages: 342940
nosy: alter-bug-tracer
priority: normal
severity: normal
status: open
title: email: AttributeError
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file48340/file0.zip
___
Python tracker
<https://bugs.python.org/issue36976>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36976] email: AttributeError
alter-bug-tracer added the comment: It is actually the exact same bug. At that time we were not sure if it security related or not, so we disclosed it privately and discussed it with Mark. We were not aware that he has submitted it for us. -- ___ Python tracker <https://bugs.python.org/issue36976> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36986] tarfile: unexpected IsADirectoryError on extraction
New submission from alter-bug-tracer :
The following code creates a new directory 'output' and extracts the tar
archive given as argv in it. When a malformed archive (attached) is given as
argv, a IsADirectoryError is thrown, as opposed to extracting the file
contained in the archive to the 'output' directory.
Code:
import tarfile
import sys
import os
def extract(archive_path, destination_path):
if not tarfile.is_tarfile(archive_path):
print ('Not a tar file')
return
tar = tarfile.open(archive_path, 'r')
for info in tar.getmembers():
tar.extractall(destination_path, members=[tar.getmember(info.name)])
destination_path = 'output'
os.mkdir(destination_path)
extract(sys.argv[1], destination_path)
Result:
Traceback (most recent call last):
File "code.py", line 15, in
extract(sys.argv[1], destination_path)
File "code.py", line 11, in extract
tar.extractall(destination_path, members=[tar.getmember(info.name)])
File "/usr/lib/python3.6/tarfile.py", line 2010, in extractall
numeric_owner=numeric_owner)
File "/usr/lib/python3.6/tarfile.py", line 2052, in extract
numeric_owner=numeric_owner)
File "/usr/lib/python3.6/tarfile.py", line 2122, in _extract_member
self.makefile(tarinfo, targetpath)
File "/usr/lib/python3.6/tarfile.py", line 2163, in makefile
with bltn_open(targetpath, "wb") as target:
IsADirectoryError: [Errno 21] Is a directory: 'output'
--
files: input0.tar
messages: 342983
nosy: alter-bug-tracer
priority: normal
severity: normal
status: open
title: tarfile: unexpected IsADirectoryError on extraction
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48341/input0.tar
___
Python tracker
<https://bugs.python.org/issue36986>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36988] zipfile: string IndexError on extract
New submission from alter-bug-tracer : The following code throws an IndexError when attempting to extract a malformed archive (attached): import zipfile import sys zf = zipfile.ZipFile(sys.argv[1]) for info in zf.infolist(): zf.extract(info.filename) Result: Traceback (most recent call last): File "code.py", line 6, in zf.extract(info.filename) File "/usr/lib/python3.6/zipfile.py", line 1507, in extract return self._extract_member(member, path, pwd) File "/usr/lib/python3.6/zipfile.py", line 1572, in _extract_member if member.is_dir(): File "/usr/lib/python3.6/zipfile.py", line 531, in is_dir return self.filename[-1] == '/' IndexError: string index out of range ------ files: file0.zip messages: 343006 nosy: alter-bug-tracer priority: normal severity: normal status: open title: zipfile: string IndexError on extract type: behavior versions: Python 3.6, Python 3.7, Python 3.8 Added file: https://bugs.python.org/file48345/file0.zip ___ Python tracker <https://bugs.python.org/issue36988> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36975] csv: undocumented UnicodeDecodeError on malformed file
alter-bug-tracer added the comment: Shouldn't all of them be documented? Either that, or converted to csv.Error? Take, for example, the C++ std. -- ___ Python tracker <https://bugs.python.org/issue36975> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36991] zipfile: AttributeError on extract
New submission from alter-bug-tracer : The following code throws an AttributeError when attempting to extract a malformed archive (attached): import zipfile import sys zf = zipfile.ZipFile(sys.argv[1]) for info in zf.infolist(): zf.extract(info.filename) Result: Traceback (most recent call last): File "code.py", line 6, in zf.extract(info.filename) File "/usr/local/lib/python3.8/zipfile.py", line 1607, in extract return self._extract_member(member, path, pwd) File "/usr/local/lib/python3.8/zipfile.py", line 1677, in _extract_member with self.open(member, pwd=pwd) as source, \ File "/usr/local/lib/python3.8/zipfile.py", line 1548, in open return ZipExtFile(zef_file, mode, zinfo, zd, True) File "/usr/local/lib/python3.8/zipfile.py", line 801, in __init__ self._decompressor = _get_decompressor(self._compress_type) File "/usr/local/lib/python3.8/zipfile.py", line 708, in _get_decompressor return bz2.BZ2Decompressor() AttributeError: 'NoneType' object has no attribute 'BZ2Decompressor' -- files: attr0.zip messages: 343035 nosy: alter-bug-tracer priority: normal severity: normal status: open title: zipfile: AttributeError on extract type: behavior versions: Python 3.8 Added file: https://bugs.python.org/file48346/attr0.zip ___ Python tracker <https://bugs.python.org/issue36991> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36992] zipfile: AttributeError on extract (LZMA)
New submission from alter-bug-tracer : The following code throws an AttributeError when attempting to extract a malformed archive (attached): import zipfile import sys zf = zipfile.ZipFile(sys.argv[1]) for info in zf.infolist(): zf.extract(info.filename) Result: Traceback (most recent call last): File "code.py", line 6, in zf.extract(info.filename) File "/usr/local/lib/python3.8/zipfile.py", line 1607, in extract return self._extract_member(member, path, pwd) File "/usr/local/lib/python3.8/zipfile.py", line 1679, in _extract_member shutil.copyfileobj(source, target) File "/usr/local/lib/python3.8/shutil.py", line 198, in copyfileobj buf = fsrc_read(length) File "/usr/local/lib/python3.8/zipfile.py", line 903, in read data = self._read1(n) File "/usr/local/lib/python3.8/zipfile.py", line 986, in _read1 data = self._decompressor.decompress(data) File "/usr/local/lib/python3.8/zipfile.py", line 635, in decompress self._decomp = lzma.LZMADecompressor(lzma.FORMAT_RAW, filters=[ AttributeError: 'NoneType' object has no attribute 'LZMADecompressor' -- files: attr1.zip messages: 343036 nosy: alter-bug-tracer priority: normal severity: normal status: open title: zipfile: AttributeError on extract (LZMA) type: behavior versions: Python 3.8 Added file: https://bugs.python.org/file48347/attr1.zip ___ Python tracker <https://bugs.python.org/issue36992> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36993] zipfile: tuple IndexError on extract
New submission from alter-bug-tracer : The following code throws an IndexError when attempting to extract a malformed archive (attached): import zipfile import sys zf = zipfile.ZipFile(sys.argv[1]) for info in zf.infolist(): zf.extract(info.filename) Result: Traceback (most recent call last): File "code.py", line 4, in zf = zipfile.ZipFile(sys.argv[1]) File "/usr/local/lib/python3.8/zipfile.py", line 1230, in __init__ self._RealGetContents() File "/usr/local/lib/python3.8/zipfile.py", line 1353, in _RealGetContents x._decodeExtra() File "/usr/local/lib/python3.8/zipfile.py", line 480, in _decodeExtra self.file_size = counts[idx] IndexError: tuple index out of range -- files: index_tuple.zip messages: 343038 nosy: alter-bug-tracer priority: normal severity: normal status: open title: zipfile: tuple IndexError on extract versions: Python 3.6, Python 3.7, Python 3.8 Added file: https://bugs.python.org/file48348/index_tuple.zip ___ Python tracker <https://bugs.python.org/issue36993> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36988] zipfile: string IndexError on extract
alter-bug-tracer added the comment: Hi, The zip is corrupted on purpose. I agree, every input should be checked before doing stuff with it. -- ___ Python tracker <https://bugs.python.org/issue36988> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36993] zipfile: tuple IndexError on extract
alter-bug-tracer added the comment: @berker.peksag, first of all sorry for the late reply. We are not sure that we know how to do that. Our tests are generated automatically. What we can do is retest the lib with your temporary fixes in place, to see if they fix all the problems our software can detect. Would that help you? -- ___ Python tracker <https://bugs.python.org/issue36993> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
