This does not solve the problem unfortunately because s.read() still
reads everything in memory.
I am thinking about an hypotetical action that does:
def export():
s=StringIO.StringIO()
db(query).select().export_to_csv_file(s)
return s.getvalue()
and replace it with
def export():
Here is at least the tempfile part of it. Not sure how you would do
the pagination, since its exporting an already selected Rows object.
Johann could you look and see if using a tempfile instead works? I
attached the hg export diff.
-Thadeus
On Sat, Jan 30, 2010 at 3:32 PM, mdipierro wrote:
I'd just rewrite it to use tempfile and use add an option to do
pagination (the counterindidation with pagination is that it may cause
data loss or duplication although very rarely).
On Jan 30, 3:03 pm, Thadeus Burgess wrote:
> Rewrite or provide an alternative to ?
>
> -Thadeus
>
> On Sat, Jan 3
Rewrite or provide an alternative to ?
-Thadeus
On Sat, Jan 30, 2010 at 2:51 PM, mdipierro wrote:
> I think we should consider rewriting the export function using a
> tmpfile instead of StringIO and doing pagination.
>
> On Jan 30, 2:49 pm, Thadeus Burgess wrote:
>> Perhaps you need to writ
I think we should consider rewriting the export function using a
tmpfile instead of StringIO and doing pagination.
On Jan 30, 2:49 pm, Thadeus Burgess wrote:
> Perhaps you need to write your own csv export. Save a csv file to /tmp
> instead and then when your finished creating your csv file then
Perhaps you need to write your own csv export. Save a csv file to /tmp
instead and then when your finished creating your csv file then stream
it for download.
This might reduce the memory requirements? Though not sure what you
can do about the fact web2py cache's your select into memory, maybe do
The problem is that all records are copied into memory before being
serialized + the entire csv is copied into memory. We need to find a
better way when exporting a large of records.
On Jan 30, 12:19 pm, Johann Spies wrote:
> It happened more than once that a csv-export using appadmin resulted
>
7 matches
Mail list logo
