Many thanks for the detailed explanation, Russ! Really appreciate it. Now I understand much better why django 1.7 enforces this. I will follow your advice to re-structure my code to make it work on django 1.7.
Best regards, Hong On Monday, December 15, 2014 6:55:04 PM UTC-5, Russell Keith-Magee wrote: > > > On Tue, Dec 16, 2014 at 4:53 AM, Hong Yi <hon...@renci.org <javascript:>> > wrote: > >> Hello, >> >> I am new to Django and have implemented two views and their corresponding >> template pages and they are working well in Django 1.6. However, when >> migrating to Django 1.7, I got an error "I/O operation on closed file" when >> getting to the second view function. I found out the root cause for this >> error is that on Django 1.7, InMemoryUploadedFile object retrieved from >> request.FILES got automatically closed at the end of the request by Django. >> I am trying to find a solution to work around this migration issue. >> Specifically, the first view and its template page ask users to browse >> files, and the files can be retrieved from request.FILES and saved to a >> global variable for use by the second view/request. Since the file got >> closed automatically at the end of the first request, when the second view >> tries to operate on this file, that "I/O operation on closed file" error >> results. >> >> This is my first post, and I am trying to get some >> recommendations/suggestions on how to handle this use case to work around >> this new security feature implemented in Django 1.7 (i.e., automatically >> close file at the end of each request). Any help and suggestions are >> greatly appreciated. >> >> > Hi Hong, > > In this case, the solution isn't to work around Django - it's to work out > why Django is getting in your way. The cause is a fundamental > misunderstanding about how you should be looking at the web. > > Each request on a web site should be completely independent - you can't > rely on shared state between one request and the "next" request. Two > examples for why this is important: > > 1) The second request might be served by an entirely different server. > Once you get into any sort of non-trivial deployment, you will have more > than one web server to ensure availability; if a file has been uploaded on > one server, it won't be available on the other server unless you're > providing some sort of independent storage. > > 2) There's no guarantees that a single user will be responsible for two > requests in a row. It's easy to think of a situation where there are two > users using your website at the same time; if your website code makes any > assumptions about request ordering, it makes a big difference whether the > requests are handled as AABB or ABAB. If A and B are normal users, this > might just be an annoying bug; if B is an attacker, then A's data could be > stolen or compromised. > > You might claim that your website will never be big enough to require (1), > or have enough users that (2) will be a problem - and that might be true - > but it doesn't change the fact that web frameworks are built on the > assumption that both (1) and (2) are going to happen, and the > infrastructure they put in place will ensure that those two cases don't > cause problems. > > So - you shouldn't be using a global variable to store *anything*. (That's > generally good advice for programming anyway, but it's doubly important for > web programming). You also shouldn't be loading data into memory on the > assumption that it will be used in the "next" request. If I were an > attacker, and I found out that you were doing this, I would make a whole > stack of the first requests, and then never make the second request - and > I've just starved your server of memory. > > If you need to have a 2 step process where one view selects a file, and > the second view "handles" the file, then what you should be passing around > is a filename, or some other reference that lets you retrieve a file. Then, > the two views should be built so that they are completely independent. View > 1 provides a way for a user to select a file. View 2 opens, reads, > processes and closes the file. > > Memory usage shouldn't be a concern here. When you open a file, you don't > have to load the whole thing into memory to pass it around. Operating > systems are really good at handling file pointers, which are just an index > into an open file. You don't have to store any more data than the character > you're currently pointing at. Most file formats are developed so that you > don't need to read the *entire* file into memory in one pass - they're > optimised to allow you to read them "on demand". > > In short - don't work around this problem; fix the underlying cause of the > problem, and restructure your app. > > Hope this has been helpful! > > Yours > Russ Magee %-) > > > > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/fec8842d-e914-48db-9910-059136520650%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
