On Tue, 06 Oct 2015 01:34:17 +1100, Chris Angelico wrote:
> On Tue, Oct 6, 2015 at 1:06 AM, Tim Chase
> <python.l...@tim.thechases.com> wrote:
>> That way, if you determine by line 3 that your million-row CSV file has
>> no blank columns, you can get away with not processing all million
>> rows.
>
> Sure, although that effectively means the entire job is moot. I kinda
> assume that the OP knows that there are some blank columns (maybe lots
> of them). The extra check is unnecessary unless it's actually plausible
> that there'll be no blanks whatsoever.
>
> Incidentally, you have an ordered_headers list which is the blank
> columns in order; I think the OP was looking for a list of the
> _non_blank columns. But that's a trivial difference, easy to tweak.
>
> ChrisA
Thanks to you all. I got it this far. But while writing back to another
csv file, I got this error - "ValueError: dict contains fields not in
fieldnames: None". Here is my code.
rdr = csv.DictReader(fin, delimiter=',')
header_set = set(rdr.fieldnames)
for r in rdr:
header_set = set(h for h in header_set if not r[h])
if not header_set:
break
for r in rdr:
data = list(r[i] for i in header_set)
dw = csv.DictWriter(fout, header_set)
dw.writeheader()
dw.writerows(data)
Also, there is difference between len(header_set) and len(data[0].keys).
Why is so?
Thanks again for all your help.
Thanks.
--
https://mail.python.org/mailman/listinfo/python-list
