Re: csv module and NULL data byte

2018-03-11 Thread Andrew McNamara
>Any idea why it might throw an exception on encountering a NULL in the >input stream? It accepts all other 255 byte values. Was this behaviour >intended? Perhaps a comment should be added to the docs. >Thanks for your work on the module anyway. The original module was like this - it comes about

Re: csv module and NULL data byte

2018-03-07 Thread John Pote
On 07/03/2018 07:59, Andrew McNamara wrote: Last time I read the documentation, it was recommended that the file be opened in BINARY mode ("rb"). It recommends binary mode, but seems to largely work fine with text/ascii mode or even arbitrary iterables. I've not seen the rationale behin

Re: csv module and NULL data byte

2018-03-07 Thread Andrew McNamara
>> Last time I read the documentation, it was recommended that >> the file be opened in BINARY mode ("rb"). > >It recommends binary mode, but seems to largely work fine with >text/ascii mode or even arbitrary iterables. I've not seen the >rationale behind the binary recommendation, but in 10+

Re: csv module and NULL data byte

2018-03-01 Thread Tim Chase
On 2018-03-01 23:57, John Pote wrote: > On 01/03/2018 01:35, Tim Chase wrote: > > While inelegant, I've "solved" this with a wrapper/generator > > > >f = file(fname, …) > >g = (line.replace('\0', '') for line in f) > I wondered about something like this but thought if there's a way > of a

Re: csv module and NULL data byte

2018-03-01 Thread John Pote
On 01/03/2018 01:35, Tim Chase wrote: While inelegant, I've "solved" this with a wrapper/generator f = file(fname, …) g = (line.replace('\0', '') for line in f) I wondered about something like this but thought if there's a way of avoiding the extra step it would keep the execution speed u

Re: csv module and NULL data byte

2018-03-01 Thread John Pote
On 01/03/2018 02:38, Dennis Lee Bieber wrote: On Wed, 28 Feb 2018 23:40:41 +, John Pote declaimed the following:     with open( fname, 'rt', encoding='iso-8859-1' ) as csvfile: Pardon? Has the CSV module changed in the last year or so? Python 3.6 docs say csv reader has to be gi

Re: csv module and NULL data byte

2018-03-01 Thread Neil Cerutti
On 2018-03-01, Tim Chase wrote: > On 2018-02-28 21:38, Dennis Lee Bieber wrote: >> >     with open( fname, 'rt', encoding='iso-8859-1' ) as csvfile: >> >> Pardon? Has the CSV module changed in the last year or so? >> >> Last time I read the documentation, it was recommended that >> t

Re: csv module and NULL data byte

2018-02-28 Thread Terry Reedy
On 2/28/2018 8:35 PM, Tim Chase wrote: While inelegant, I've "solved" this with a wrapper/generator f = file(fname, …) g = (line.replace('\0', '') for line in f) reader = csv.reader(g, …) for row in reader: process(row) I think this is elegant in that is cleans the input strea

Re: csv module and NULL data byte

2018-02-28 Thread Tim Chase
On 2018-02-28 21:38, Dennis Lee Bieber wrote: > >     with open( fname, 'rt', encoding='iso-8859-1' ) as csvfile: > > Pardon? Has the CSV module changed in the last year or so? > > Last time I read the documentation, it was recommended that > the file be opened in BINARY mode ("rb")

Re: csv module and NULL data byte

2018-02-28 Thread Tim Chase
While inelegant, I've "solved" this with a wrapper/generator f = file(fname, …) g = (line.replace('\0', '') for line in f) reader = csv.reader(g, …) for row in reader: process(row) My actual use at $DAYJOB cleans out a few other things too, particularly non-breaking spaces coming from

csv module and NULL data byte

2018-02-28 Thread John Pote
I have a csv data file that may become corrupted (already happened) resulting in a NULL byte appearing in the file. The NULL byte causes an _csv.Error exception. I'd rather like the csv reader to return csv lines as best it can and subsequent processing of each comma separated field deal with