[issue30034] csv reader chokes on bad quoting in large files

2017-04-10 Thread Keith Erskine
New submission from Keith Erskine: If a csv file has a quote character at the beginning of a field but no closing quote, the csv module will keep reading the file until the very end in an attempt to close out the field. It's true this situation occurs only when the quoting in a csv fi

[issue30034] csv reader chokes on bad quoting in large files

2017-04-10 Thread Keith Erskine
Keith Erskine added the comment: Perhaps I should add what I would prefer the csv reader to return in my example above. That would be: ['a', 'b', 'c'] ['d', 'e,f'] ['g', 'h', 'i'] Yes, the second line is s

[issue30034] csv reader chokes on bad quoting in large files

2017-04-10 Thread Keith Erskine
Keith Erskine added the comment: The csv reader already handles a certain amount of bad formatting. For example, using default behavior, the following file: a,b,c d,"e"X,f g,h,i is read as: ['a', 'b', 'c'] ['d', 'eX', 'f&

[issue30034] csv reader chokes on bad quoting in large files

2017-04-10 Thread Keith Erskine
Keith Erskine added the comment: As you say, David, however much we would like the world to stick to a given CSV standard, the reality is that people don't, which is all the more reason for making the csv reader flexible and forgiving. The csv module can and should be used for more than

[issue30034] csv reader chokes on bad quoting in large files

2017-04-11 Thread Keith Erskine
Keith Erskine added the comment: The csv reader already supports bad CSV - that's what I believe "strict" is for - but only in one specific scenario. My request is to make that "strict" attribute a bit more useful. Thank you for your suggestion, Peter. I have toyed

[issue30034] csv reader chokes on bad quoting in large files

2017-04-11 Thread Keith Erskine
Keith Erskine added the comment: I should have said, Peter, an odd number of quotes does not necessarily mean the quoting is bad. For example, a line of: a,b",c will parse fine as ['a', 'b"', 'c']. Figuring out bad quoting is not easy, but if we know t

[issue30034] csv reader chokes on bad quoting in large files

2017-04-15 Thread Keith Erskine
Keith Erskine added the comment: OK Terry. Thank you everybody for your thoughts and suggestions. -- ___ Python tracker <http://bugs.python.org/issue30