Django serving multiple subdomain sites -- okay to use thread-locals?
We're adapting our Django powered video site to be an open video platform for any user to create their own "site" using subdomains (eg mysite.vidplatform.com) on one Django server instance. Each "site" would obviously have content associated with only that site in addition to template and navigation customizations. Adapting the ``django-multisite`` app seems best and is working well on our dev server: http://github.com/shestera/django-multisite However, a number of Django developers have expressed concerns using thread-locals: http://code.djangoproject.com/wiki/CookBookThreadlocalsAndUser I don't see any other option besides using thread-locals to dynamically update SITE_ID per request. What would be another way of accomplishing this goal without thread- locals if indeed this is not best practice? Thanks in advance for your input. Kieran -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Django serving multiple subdomain sites -- okay to use thread-locals?
Thanks, Graham. We'll keep using this method and I'll continue this thread if we run into problems. I appreciate the insight re: Python's implementation of thread locals. I tend to agree re: security -- thread locals only appears to be a security threat if a system has already been seriously compromised, at which point there'd be easier attacks to execute. Kieran On May 31, 9:17 pm, Graham Dumpleton wrote: > On Jun 1, 1:04 pm, Kieran Farr wrote: > > > > > > > We're adapting our Django powered video site to be an open video > > platform for any user to create their own "site" using subdomains (eg > > mysite.vidplatform.com) on one Django server instance. Each "site" > > would obviously have content associated with only that site in > > addition to template and navigation customizations. > > > Adapting the ``django-multisite`` app seems best and is working well > > on our dev server:http://github.com/shestera/django-multisite > > > However, a number of Django developers have expressed concerns using > > thread-locals:http://code.djangoproject.com/wiki/CookBookThreadlocalsAndUser > > > I don't see any other option besides using thread-locals to > > dynamically update SITE_ID per request. > > > What would be another way of accomplishing this goal without thread- > > locals if indeed this is not best practice? > > Irrespective of whether you use thread locals, in a multithreaded > system you are going to have that risk of one thread in your web > server potentially accessing or modifying the current state of another > thread in the system. That is simply the nature of multithreaded > systems and more so with Python where one can access just about > anything of the environment of the executing Python process, including > the stack frames of other executing threads. > > If anything I would say that thread locals if used correctly are going > to be more secure than you having your own global dictionary which is > caching information keyed on the request or thread ID. This is because > in modern Python versions it would be much harder to actually get > access to the thread local instance for another thread. This is > because the thread local object is actually kept in C data structures > for the current thread and you only get given back the object instance > relevant to variable for your specific thread. To get access to that > for another thread you may require a C extension module which > traverses all the C thread state objects for other threads. > > So, not sure why that warning has been given. I could understand it > for older Python versions where one may have been using Python > implementation of threading locals, ie., the threading_locals module, > as in that case data for other threads was exposed, but for latest > Python versions believe it is a bit harder to get at thread locals for > another thread. If this is not the case and it is easy to get at > thread locals for another thread, can someone post Python code as to > how. > > Note that someone can take advantage of this presupposes that someone > has managed to inject code into your application from outside and have > it executed. If they can do that you probably have bigger problems to > contend with than this. What for example is there to stop someone > looking at DJANGO_SETTINGS_MODULE to work out name of settings module > and then getting that module and working out what your database > password is. They could even simply access the database using active > database handle and make arbitrary queries/changes to the database, > including stuff related to users personal details. > > So, I would say that if someone has got as far as to being able to > even attempt to take advantage of it, ie., compromised your system and > can perform code injection, you are already stuffed in many many ways > and this is going to be the least of your concerns and that there is > likely much easier ways of stealing data. > > Graham -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Error in form validation with choices
I just ran into this as well. Very odd that earlier versions didn't raise the same error. On May 18, 1:38 pm, Jori wrote: > Thanks, you're correct. I don't know how I didn't notice but then > again it worked just fine with 1.1.1. > > -Jori > > On May 18, 11:03 pm, Daniel Roseman wrote: > > > > > It's an integer field, but the choices are all strings. The first > > value in each tuple should be an integer, to match the field. > > -- > > DR. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Django users" group. > > To post to this group, send email to django-us...@googlegroups.com. > > To unsubscribe from this group, send email to > > django-users+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/django-users?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-us...@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/django-users?hl=en. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: HTTP POST sent from app to Django Server returns 403 Forbidden
Hi, were having the exact same problem. We're integrating with a third-party API that sends "pings" via simple POST requests to our server to give us updates re: status video conversion processes. We're running Django 1.2.1 with Apache mod_python on Ubuntu 9.04. I've disabled ALL csrf related middleware for debugging purposes in settings.py, although we'll want that enabled on production. I've been attempting to debug with wget. When I hit the listener URL, it works fine if no post data is sent. However, as soon as I add the -- post-data tag with ANY text, a 403 is always returned. Even tried @csrf_exempt and @csrf_response_exempt AND adding csrf_exempt = True to the response object. This is quite frustrating as we're following all the docs for disabling CSRF and this is getting in the way of what would be a very simple task. Further, debugging 403s is nearly impossible. It'd be very helpful when in DEBUG mode to reveal who/what/why raised the 403. Any ideas? Kieran On Jul 26, 8:30 am, etone wrote: > Hi there, > > I'm trying to sent a HTTP POST from a client application to my Django > app. Django does indeed receive the HTTP POST as I do hit > _HandleRequest(), however it returns a 403 Forbidden, instead of > hitting my handler function. I experimented and sent a HTTP GET from > my client application and in this case I am able to hit my handler > function. I would like to use HTTP POST as I want to upload some data > to my Django app. > > What am I doing wrong/missing? > > Here is my settings.py in my django app: > > try: > from djangoappengine.settings_base import * > has_djangoappengine = True > except ImportError: > has_djangoappengine = False > DEBUG = True > TEMPLATE_DEBUG = DEBUG > > import os > > INSTALLED_APPS = ( > 'djangotoolbox', > # 'django.contrib.auth', > 'django.contrib.contenttypes', > 'django.contrib.sessions', > ) > > if has_djangoappengine: > INSTALLED_APPS = ('djangoappengine',) + INSTALLED_APPS > > ADMIN_MEDIA_PREFIX = '/media/admin/' > MEDIA_ROOT = os.path.join(os.path.dirname(__file__), 'media') > TEMPLATE_DIRS = (os.path.join(os.path.dirname(__file__), > 'templates'),) > > ROOT_URLCONF = 'urls' -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: HTTP POST sent from app to Django Server returns 403 Forbidden
This is intended not to be protected by auth, so this page is publicly accessible, no login required. Here's the function in its entirety: 255 @csrf_response_exempt 256 def DistroHeySpreadListener(request): 257 if request.method == 'POST': 258 post = request.POST 259 raw = request.raw_post_data 260 remote_media_id = post.get("video_id", "") 261 # Try to fetch the distros for this remote_media_id 262 distro_list = Distro.objects.filter(remote_media_id__exact=remote_media_id) 263 264 # Make sure there's at least one 265 if len(distro_list) < 1: 266 raise Http404 267 268 # For each distro instance, check to see if there's a response in this api call 269 for distro in distro_list: 270 # Prepare logging 271 log = distro.log 272 log += "Raw data from POST" + str(raw) 273 distro.log = log 274 distro.save() 275 276 # Get distro short name 277 dest_short_name = distro.credential.destination.short_name 278 279 # From short_name, return either error or link starting with http:// 280 response = post.get('link[%s]' % dest_short_name, False) or post.get('error[%s]' % dest_short_name, False) 281 282 if response.startswith('http://'): 283 # If response starts with http then success, populate as link 284 distro.destination_url = response 285 distro.state = TRANSCODE_SUCCESS 286 log += '\nSUCCESS, url = %s' % response 287 else: 288 # Else, change state to error and log error message 289 log += '\nERROR, error = %s' % response 290 distro.state = TRANSCODE_ERROR 291 292 distro.log = log 293 distro.save() 294 295 296 h = HttpResponse("Thanks, HeySpread! We love you.", mimetype="text/plain", status=200) 297 h.csrf_exempt = True 298 return h 299 # return HttpResponse("Thanks, HeySpread! We love you.", mimetype="text/plain", status=200) 300 else: 301 return HttpResponse("Oops, something went wrong.", mimetype="text/plain", status=200) On Jul 27, 11:30 am, raj wrote: > Most probably it has something to do with permissions. Go thru the > exact code block which tries to post the data. Is the login successful > and does the logged-in user have got all necessary permissions > required? Post that part of the code if you can't find out. > > Rajeesh. > > On Jul 27, 11:20 pm, Kieran Farr wrote: > > > > > Further, debugging 403s is nearly impossible. It'd be very helpful > > when in DEBUG mode to reveal who/what/why raised the 403. > > > > What am I doing wrong/missing? -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: HTTP POST sent from app to Django Server returns 403 Forbidden
Raj sorry I misread your question. This initial response is in re: my listener. The "sender" of this POST request is a third party server that we have no control over. However, even when debugging with wget we receive the 403. Here is the wget command I've been testing with: $ wget http://www.url.com/subdir/listener/ --post-data "?test=test" -U "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us) AppleWebKit/ 533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16" --2010-07-27 20:06:36-- http://www.url.com/subdir/listener/ Resolving [URL] ... [IP ADDRESS] Connecting to [URL]|[IP ADDRESS]|:80... connected. HTTP request sent, awaiting response... 403 Forbidden 2010-07-27 20:06:37 ERROR 403: Forbidden. However, if I send without the post-data, here's the response: $ wget http://www.url.com/subdir/listener/ --2010-07-27 20:31:14-- http://www.url.com/subdir/listener/ Resolving [URL]... [IP ADDRESS] Connecting to [URL|[IP ADDRESS]|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/plain] Saving to: `index.html.1' [ <=> ] 27 --.-K/s in 0s 2010-07-27 20:31:14 (1.84 MB/s) - `index.html.1' saved [27] On Jul 27, 11:34 am, Kieran Farr wrote: > This is intended not to be protected by auth, so this page is publicly > accessible, no login required. > > Here's the function in its entirety: > > 255 @csrf_response_exempt > 256 def DistroHeySpreadListener(request): > 257 if request.method == 'POST': > 258 post = request.POST > 259 raw = request.raw_post_data > 260 remote_media_id = post.get("video_id", "") > 261 # Try to fetch the distros for this > remote_media_id > 262 distro_list = > Distro.objects.filter(remote_media_id__exact=remote_media_id) > 263 > 264 # Make sure there's at least one > 265 if len(distro_list) < 1: > 266 raise Http404 > 267 > 268 # For each distro instance, check to see if > there's a response in this api call > 269 for distro in distro_list: > 270 # Prepare logging > 271 log = distro.log > 272 log += "Raw data from POST" + str(raw) > 273 distro.log = log > 274 distro.save() > 275 > 276 # Get distro short name > 277 dest_short_name = > distro.credential.destination.short_name > 278 > 279 # From short_name, return either error or > link starting with http:// > 280 response = post.get('link[%s]' % > dest_short_name, False) or post.get('error[%s]' % dest_short_name, > False) > 281 > 282 if response.startswith('http://'): > 283 # If response starts with http > then success, populate as link > 284 distro.destination_url = response > 285 distro.state = TRANSCODE_SUCCESS > 286 log += '\nSUCCESS, url = %s' % > response > 287 else: > 288 # Else, change state to error and > log error message > 289 log += '\nERROR, error = %s' % > response > 290 distro.state = TRANSCODE_ERROR > 291 > 292 distro.log = log > 293 distro.save() > 294 > 295 > 296 h = HttpResponse("Thanks, HeySpread! We love > you.", mimetype="text/plain", status=200) > 297 h.csrf_exempt = True > 298 return h > 299 # return HttpResponse("Thanks, HeySpread! We love > you.", mimetype="text/plain", status=200) > 300 else: > 301 return HttpResponse("Oops, something went wrong.", > mimetype="text/plain", status=200) > > On Jul 27, 11:30 am, raj wrote: > > > > > Most probably it has something to do with permissions. Go thru the > > exact code block which tries to post the data. Is the login successful > > and does the logged-in user have got all necessary permissions > > required? Post that part of the code if you can't find out. > > > Rajeesh. > > > On Jul 27, 11:20 pm, Kieran Farr wrote:
Re: HTTP POST sent from app to Django Server returns 403 Forbidden
Further research shows that CSRF is enabled regardless of my settings.py if we use Django's built-in auth. Obviously, we need to still use Django's auth, so we can't just disable CSRF site-wide like this hack: http://stackoverflow.com/questions/1650941/django-csrf-framework-cannot-be-disabled-and-is-breaking-my-site ...or this hack... http://djangosnippets.org/snippets/2069/ Obviously also, we have no control over the third-party server sending the POST request, so we can't resort to this hack: http://stackoverflow.com/questions/2405353/having-a-postable-api-and-djangos-csrf-middleware The decorator @csrf_exempt does not work as described in the docs as our view always returns a 403 when any content is POSTed. Very confusing! Kieran On Jul 27, 1:32 pm, Kieran Farr wrote: > Raj sorry I misread your question. This initial response is in re: my > listener. > > The "sender" of this POST request is a third party server that we have > no control over. > > However, even when debugging with wget we receive the 403. > > Here is the wget command I've been testing with: > > $ wgethttp://www.url.com/subdir/listener/--post-data "?test=test" -U > "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us) AppleWebKit/ > 533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16" > > --2010-07-27 20:06:36-- http://www.url.com/subdir/listener/ > Resolving [URL] ... [IP ADDRESS] > Connecting to [URL]|[IP ADDRESS]|:80... connected. > HTTP request sent, awaiting response... 403 Forbidden > 2010-07-27 20:06:37 ERROR 403: Forbidden. > > However, if I send without the post-data, here's the response: > > $ wgethttp://www.url.com/subdir/listener/ > > --2010-07-27 20:31:14-- http://www.url.com/subdir/listener/ > Resolving [URL]... [IP ADDRESS] > Connecting to [URL|[IP ADDRESS]|:80... connected. > HTTP request sent, awaiting response... 200 OK > Length: unspecified [text/plain] > Saving to: `index.html.1' > > [ <=> > ] > 27 --.-K/s in 0s > > 2010-07-27 20:31:14 (1.84 MB/s) - `index.html.1' saved [27] > > On Jul 27, 11:34 am, Kieran Farr wrote: > > > > > This is intended not to be protected by auth, so this page is publicly > > accessible, no login required. > > > Here's the function in its entirety: > > > 255 @csrf_response_exempt > > 256 def DistroHeySpreadListener(request): > > 257 if request.method == 'POST': > > 258 post = request.POST > > 259 raw = request.raw_post_data > > 260 remote_media_id = post.get("video_id", "") > > 261 # Try to fetch the distros for this > > remote_media_id > > 262 distro_list = > > Distro.objects.filter(remote_media_id__exact=remote_media_id) > > 263 > > 264 # Make sure there's at least one > > 265 if len(distro_list) < 1: > > 266 raise Http404 > > 267 > > 268 # For each distro instance, check to see if > > there's a response in this api call > > 269 for distro in distro_list: > > 270 # Prepare logging > > 271 log = distro.log > > 272 log += "Raw data from POST" + str(raw) > > 273 distro.log = log > > 274 distro.save() > > 275 > > 276 # Get distro short name > > 277 dest_short_name = > > distro.credential.destination.short_name > > 278 > > 279 # From short_name, return either error or > > link starting with http:// > > 280 response = post.get('link[%s]' % > > dest_short_name, False) or post.get('error[%s]' % dest_short_name, > > False) > > 281 > > 282 if response.startswith('http://'): > > 283 # If response starts with http > > then success, populate as link > > 284 distro.destination_url = response > > 285 distro.state = TRANSCODE_SUCCESS > > 286 log += '\nSUCCESS, url = %s' % > > response > > 287 else: > > 288 # Else, change state to error and > > log error message > > 289
Re: HTTP POST sent from app to Django Server returns 403 Forbidden
Finally figured out what's going on. My hypothesis: An HTTP request with POST method sent to a Django view wrapped with @csrf_exempt will still raise a 403 CSRF error if the wrapped view raises an Http404 exception. In this specific case, we raise 404 when there is no valid video_id passed in the POST request: 264 # Make sure there's at least one 265 if len(distro_list) < 1: 266 raise Http404 Raising a 403 was obviously not the result behavior we expected, instead we would have expected the 404 to be raised without CSRF interfering given the initial view was properly wrapped with @csrf_exempt. Further contributing, this was rather difficult to diagnose due to the opaque 403 error message even when DEBUG = True. Is this expected Django behavior? If not, can we please address this in a future release? Is there a better way to provide debugging information for 403 errors raised from the built-in CSRF methods? All the best and thanks for a great framework. Kieran On Jul 27, 6:54 pm, Kieran Farr wrote: > Further research shows that CSRF is enabled regardless of my > settings.py if we use Django's built-in auth. > > Obviously, we need to still use Django's auth, so we can't just > disable CSRF site-wide like this > hack:http://stackoverflow.com/questions/1650941/django-csrf-framework-cann... > ...or this hack...http://djangosnippets.org/snippets/2069/ > > Obviously also, we have no control over the third-party server sending > the POST request, so we can't resort to this > hack:http://stackoverflow.com/questions/2405353/having-a-postable-api-and-... > > The decorator @csrf_exempt does not work as described in the docs as > our view always returns a 403 when any content is POSTed. > > Very confusing! > > Kieran > > On Jul 27, 1:32 pm, Kieran Farr wrote: > > > > > Raj sorry I misread your question. This initial response is in re: my > > listener. > > > The "sender" of this POST request is a third party server that we have > > no control over. > > > However, even when debugging with wget we receive the 403. > > > Here is the wget command I've been testing with: > > > $ wgethttp://www.url.com/subdir/listener/--post-data"?test=test"; -U > > "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us) AppleWebKit/ > > 533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16" > > > --2010-07-27 20:06:36-- http://www.url.com/subdir/listener/ > > Resolving [URL] ... [IP ADDRESS] > > Connecting to [URL]|[IP ADDRESS]|:80... connected. > > HTTP request sent, awaiting response... 403 Forbidden > > 2010-07-27 20:06:37 ERROR 403: Forbidden. > > > However, if I send without the post-data, here's the response: > > > $ wgethttp://www.url.com/subdir/listener/ > > > --2010-07-27 20:31:14-- http://www.url.com/subdir/listener/ > > Resolving [URL]... [IP ADDRESS] > > Connecting to [URL|[IP ADDRESS]|:80... connected. > > HTTP request sent, awaiting response... 200 OK > > Length: unspecified [text/plain] > > Saving to: `index.html.1' > > > [ <=> > > ] > > 27 --.-K/s in 0s > > > 2010-07-27 20:31:14 (1.84 MB/s) - `index.html.1' saved [27] > > > On Jul 27, 11:34 am, Kieran Farr wrote: > > > > This is intended not to be protected by auth, so this page is publicly > > > accessible, no login required. > > > > Here's the function in its entirety: > > > > 255 @csrf_response_exempt > > > 256 def DistroHeySpreadListener(request): > > > 257 if request.method == 'POST': > > > 258 post = request.POST > > > 259 raw = request.raw_post_data > > > 260 remote_media_id = post.get("video_id", "") > > > 261 # Try to fetch the distros for this > > > remote_media_id > > > 262 distro_list = > > > Distro.objects.filter(remote_media_id__exact=remote_media_id) > > > 263 > > > 264 # Make sure there's at least one > > > 265 if len(distro_list) < 1: > > > 266 raise Http404 > > > 267 > > > 268 # For each distro instance, check to see if > > > there's a response in this api call > > > 269 for distro in distro_list: > > > 270 # Prepare logging > > > 271
Re: HTTP POST sent from app to Django Server returns 403 Forbidden
Thanks, David -- you're right on, now I just return an HttpResponse with result code 404. Benedict, could you post the view for /management/statistics/top/user/ yearly/ that is causing the 403? Kieran On Jul 28, 6:47 am, steven314 wrote: > @etone: has this discussion of CSRF enabled you to hunt down your > problem? > > Steven. > > On Jul 26, 4:30 pm, etone wrote: > > > > > Hi there, > > > I'm trying to sent a HTTP POST from a client application to my Django > > app. Django does indeed receive the HTTP POST as I do hit > > _HandleRequest(), however it returns a 403 Forbidden, instead of > > hitting my handler function. I experimented and sent a HTTP GET from > > my client application and in this case I am able to hit my handler > > function. I would like to use HTTP POST as I want to upload some data > > to my Django app. > > > What am I doing wrong/missing? > > > Here is my settings.py in my django app: > > > try: > > from djangoappengine.settings_base import * > > has_djangoappengine = True > > except ImportError: > > has_djangoappengine = False > > DEBUG = True > > TEMPLATE_DEBUG = DEBUG > > > import os > > > INSTALLED_APPS = ( > > 'djangotoolbox', > > # 'django.contrib.auth', > > 'django.contrib.contenttypes', > > 'django.contrib.sessions', > > ) > > > if has_djangoappengine: > > INSTALLED_APPS = ('djangoappengine',) + INSTALLED_APPS > > > ADMIN_MEDIA_PREFIX = '/media/admin/' > > MEDIA_ROOT = os.path.join(os.path.dirname(__file__), 'media') > > TEMPLATE_DIRS = (os.path.join(os.path.dirname(__file__), > > 'templates'),) > > > ROOT_URLCONF = 'urls' -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.