On Fri, 09 Nov 2007 02:51:10 -0800, Michel Albert wrote:
> Obviously this won't work as you cannot access a slice of a csv-file.
> Would it be possible to subclass the csv.reader class in a way that
> you can somewhat efficiently access a slice?
An arbitrary slice? I guess not as all records before must have been read
because the lines are not equally long.
> The obvious way is to do the following:
>
> buffer = []
> for line in reader:
> buffer.append(line)
> if len(buffer) == 1000:
> f = job_server.submit(calc_scores, buffer)
> buffer = []
With `itertools.islice()` this can be written as:
while True:
buffer = list(itertools.islice(reader, 1000))
if not buffer:
break
f = job_server.submit(calc_scores, buffer)
--
http://mail.python.org/mailman/listinfo/python-list
