On Wed, Apr 9, 2008 at 8:45 AM, [EMAIL PROTECTED] < [EMAIL PROTECTED]> wrote:
> > I'm using a non-Django form for a very large file upload ( I've never > used FileField, because they used to give me problems, and this code > has worked forever for smaller files ). > > It seems to work great with files 32MB and smaller... but anything > over that seems to throw this mod_python MemoryError. I am all for > using FTP, but the powers that be ( my boss/project manager ) would > like people to be able to upload a 150MB file via form post ( I know > it takes over an hour..nuts ). > > This is my code: > fout = file > (os.path.join(getUploadDir(), filename), 'wb',10000) > while 1: > chunk = > fileitem.file.read(10000) > if not chunk: break > fout.write (chunk) > fout.close() > > > Anyone been successful w/ Django + mod_python and really large file > uploads? I saw something about a streaming file upload patch, but > I'm assuming I'd have to switch to FileField for that, and wanted to > see if anyone had tried it first... > The code in Django that causes out-of-memory errors for large file uploads is the code that populates request.FILES -- this is code that runs before your view is even called so it doesn't matter if you are using a newforms.FileField or rolling your own form to handle uploaded files. As it stands today, before ever calling your view Django will try to read all of the uploaded file data into memory, and that is where you run into out-of-memory problems. So, I believe you will need the patch for large streaming uploads ( http://code.djangoproject.com/ticket/2070) if you want to be able to handle uploading very large files. This is a big patch and I'm not sure how close it is to getting into the code base, but I gave it (the last of the many patches listed in the ticket) a quick try and it worked fine for my app. Given where this code hooks in I don't know that you would need to switch to using FileField, but I can't guarantee your current code will work since in my case I am using a newforms.FileField to upload files (though I am not using models.FileField). Personally, though, I'd rather get whatever problems I had with FileField fixed than work around them, but since you didn't give any details on what the problems you ran into were it's hard to provide any guidance on how to do that. Karen --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
