On Aug 31, 1:09 am, dave b <db.pub.m...@gmail.com> wrote: > /me rolls eyes. > You have a valid point re /tmp, sorry I am used to mounting /tmp as > /tmpfs - my mistake :) > Ok lets be *really* clear the security problem still exists. > An attack can in the limits set on the maximum post by the httpd / > module in use upload a large file.
Using /tmp or /var is potentially a silly place to put uploads anyway. This is because they are partitions dependent upon by other applications. If you fill up those partitions you will impact those other applications and they could fail and stuff up your system in other ways. A decent production setup would put uploads in a dedicated partition where use can be properly controlled. > > I don't actually use Django so not 100% sure, but yes there possibly > > isn't an equivalent of LimitRequestBody definable within Django unless > > can be done with middleware. > > Ok so you don't even use django, ok... > You know I think I missed your presentation at pycon-au. > > > So, yes it may make sense for Django to have a fail safe and allow you > > to specify a maximum on upload size if it doesn't already, but that is > > only of use where you haven't set up your production web server > > properly to protect you from abuses, something you should be doing. > > Yes and imho it should be in django by default, not up to end django > users to figure out. > Secure by default please! It isn't that simple though. It may be quite unrealistic to have a single global value. This is because in practice, what is a valid maximum for an form post/upload depends on the URL. This is why using front end web server like Apache is preferred, because its Location directive allows you to implement more fine grained control than what a single value allows. For example, you could have a default of: LimitRequestBody 1000000 but then for a specific URL designed to handle large uploads, you could have: LimitRequestBody 100000000 Allowing this in Django itself would be harder. The fact still remains that you should do these things using what is the most appropriate tool and Django isn't going to be the right place. It certainly would be very dangerous to rely purely on Django defaults in a production setting and if that is the approach you take for your systems, I would be quite concerned as through ignoring the abilities of the web server to protect you, you may be opening yourself up to even more danger. You always want to be cutting off things as early as possible where it can be done efficiently. Doing it in your Python web application is the worst place to do it as you still clog up the pipeline and are also having to do it with a programming language which isn't going to be as efficient in dealing things as pure C code in the web server. > > Anyway, I would have to agree with Russell, you are simply not making > > yourself clear enough and to added to that seem to keep echoing > > statements that have been refuted. > > If you say so. I was pushing some other(more aggressive) impacts in > exotic configurations with custom httpd etc. . > > > For the third time I ask you whether you have actually gone and tested > > your hypothesis and can provide a working test case that demonstrates > > the problem. > > Ok. Look. You don't use django. > 1. Try this - go to the django > websitehttp://docs.djangoproject.com/en/dev/intro/tutorial01/ Be careful. I may not use Django, but I am not ignorant. I would probably know more than most about the internal workings of Django and Python web frameworks in general, and most definitely would know more in regard to the interface between web server and web framework/ application. I am also well capable of reading code to get an understanding things. > 2. and follow the tutorial 1 (and also do 2 ) when it says put the > poll file like this: > ... > Ok still following? > well you finish the tutorial(s) now and then you try to upload a file right? > So you start uploading the file. Now because (I assume you are still > using the django built in webserver) why don't you play with this a > bit, start uploading say 10 1gb files(all at once) then stop them(all) > at around say 700mb~ in. The Django development server is single threaded so it isn't going to do what I expect you think it is. If I cut them all off while still processing the first, the latter requests wouldn't even have been accepted by the web server at that point and so do nothing. Using the development server as a guide is also stupid. If you use a real server, even one that can handle concurrent requests, if the client connection is cut off, the application should se an exception for the broken connection prior to all content being received. Presumably in that case the upload file will be removed as wasn't complete. Also, if you do manage to fill up the disk partition then the write of data to file will fail and again an exception will occur and file should be removed. If Django isn't removing files when exceptions occur while writing it I would be quite surprised. Thus impacts of filling up disk using a rogue upload should in some respects be transitory as it should recover. So, does this mean you have actually run this test and proven conclusively that there is an issue, or again is this just speculation? If you are finding that Django isn't cleaning up failed uploads due to truncated client connection or running out of disk space, then that may be an issue that needs solving, but you are not be clear enough about the actual problems to know whether it is all speculation or is instead fact. > Have fun! (obviously you should go further than this and try with > apache setup etc.). Why should I be trying to prove your speculation? I also don't care about that specific case, I am taking you to task about your other hypothetical cases around trying to stuff down more content than specified by content length, use of mod_deflate etc. That is there area where I get the impression you haven't actually done any testing. Also, you posted patches previously. One was: +++ django/core/files/uploadhandler.py 2010-08-29 14:01:15.000000000 +1000 @@ -153,7 +153,7 @@ """ # Check the content-length header to see if we should # If the post is too large, we cannot use the Memory handler. - if content_length > settings.FILE_UPLOAD_MAX_MEMORY_SIZE: + if content_length is None or content_length > settings.FILE_UPLOAD_MAX_MEMORY_SIZE: self.activated = False else: self.activated = True Did you actually even test to see whether this was required. If you look at the code of Django where it uses those classes, such as in 'django/http/multipartparser.py', you will see: content_length = int(META.get('HTTP_CONTENT_LENGTH', META.get('CONTENT_LENGTH',0))) So, the users of those upload handler have already handled the case of there being no CONTENT_LENGTH, making it 0 if not supplied. The assumption there in code is therefore that you will always supply a valid content length, and not None. Thus the patch serves no purpose. The other patch is also not well thought out either. You have: @@ -170,6 +170,7 @@ """ if self.activated: self.file.write(raw_data) + self.file.truncate(settings.FILE_UPLOAD_MAX_MEMORY_SIZE) else: return raw_data That truncation is being done on the handling of every block which is going to be inefficient for a start. It is also bad for other reasons as well. First is that it silently throws away data in excess of a value totally unrelated to the content length. Thus higher level code will believe the upload worked quite okay, yet what has actually happened is a loss of data. Second is that this isn't actually avoiding the Django application having to read the data in in the first place. > > FWIW, there are much simpler ways to bring a site down than this. I > > suggest you go research Slowloris. > > I know about this attack, but I can use my attack against those who > are not using apache. > What do you say to this ? Think beyond Slowloris. The Slowloris attack as it is named, is targeting slow delivery of bytes making up the original request headers. It therefore is attacking the initial web server code. Granted that it mainly affects process/thread based web servers and not asynchronous servers, but think what happens if you do the same slow delivery of data for the actual request content. In this latter case you are hitting the actual web application itself and because WSGI applications require a process/thread blocking model, then applying this type of attack to actual request content will cause problems for a Django application no matter what web server you are using. Some web servers used in front of an application may help to avoid this. For example, when using nginx as a proxy front end it will buffer up to 1MB of data in memory before actually deciding to proxy the request through to backend. In other words, it attempts to only hand over a request when pretty sure have the data needed to satisfy the request. Because nginx is asynchronous, it itself will not suffer too much from the problem except if it buffers in memory and memory use starts increasing. You can still defeat it though by saying content length is going to be more than 1MB, because in that case nginx will proxy immediately and stream data and so backend can still be attacked. Anyway, enough said, I am not going to post about this again as little I say seems to be sinking in at all. If you still think there is a problem, submit a ticket to the Django issue tracker with clear description, a test script to prove it along with results from test script, plus your patch, plus your patch for that test script, plus the documentation. Good luck. Graham -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
