I can't figure it out: "manage.py dbshell" gives "Access denied for user 'ODBC'@'localhost'"
Hey All, I've had this problem for a while and still can't figure it out. I have django setup and working on my windows machine. I can run manage.py syncdb and other commands, but for some reason the dbshell command doesn't work. It just gives this error: > manage.py dbshell > ERROR 1045 (28000): Access denied for user 'ODBC'@'localhost' (using > password: NO) I modified django/db/backends/mysql/client.py like another poster did to see what credentials were being passed, and I got this: client.py: ... print "self.executable_name:", self.executable_name print "args:", args os.execvp(self.executable_name, args) output: > self.executable_name: mysql > args: ['', '--user=test', '--password=test', 'test'] > ERROR 1045 (28000): Access denied for user 'ODBC'@'localhost' (using > password: NO) I also tried the following lines, with the same results: print "os.execlp:" os.execlp(self.executable_name, "-?") print "os.execvp:" os.execvp(self.executable_name, ["-?"]) These lines should simple call "mysql.exe" and print out the help file, but instead it seems like mysql.exe is always running without any arguments. Does anyone have a clue why mysql.exe isn't seeing the arguments? I'd love to figure this out and be able to use dbshell from now on. Thanks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: I can't figure it out: "manage.py dbshell" gives "Access denied for user 'ODBC'@'localhost'"
Alright, after some more searching I figured out my answer. First
off, the test statements I posted were wrong. The first argument
passed to execlp is the executable name, so:
> os.execlp(self.executable_name, "-?")
Should be
> os.execlp(self.executable_name, self.executable_name, "-?")
That works now. So I applied the same thing to the original client.py
file:
def runshell(self):
-args = ['']
+args = [self.executable_name]
db = settings.DATABASE_OPTIONS.get('db',
settings.DATABASE_NAME)
user = settings.DATABASE_OPTIONS.get('user',
settings.DATABASE_USER)
...
print "self.executable_name:", self.executable_name
print "args:", args
os.execvp(self.executable_name, args)
And I get this:
> self.executable_name: mysql
> args: ['mysql', '--user=test', '--password=test', 'test']
>
> Welcome to the MySQL monitor. Commands end with ; or \g.
> Your MySQL connection id is 212
> Server version: 5.0.51a-community-nt-log MySQL Community Edition (GPL)
>
> Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
>
> mysql>
So it looks like passing '' as the first argument to execvp() on
Windows for mysql doesn't work. If using self.executable_name as the
first arg works on other platforms, it looks like this should be
changed to fix this bug and make client.py more cross-platform.
Can anyone see if it works on unix? Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
Figuring out my memory usage for django + wsgi + apache prefork
I'm running a bunch of django apps on my shared host with an 80mb memory limit. I have a bunch of very low-traffic sites I want to keep running, but use as little memory as possible, so my goal is to minimize the memory usage for each of these very small sites. I'm just starting to learn about apache/python/django memory usage, so I figure there's a lot I just don't know right now. I'm running on webfaction with: Apache 2.2.3 with the Prefork MPM Python 2.5.2 Django 1.0.2 (as stated in webfaction's automatic setup, though django/ __init__.py shows VERSION = (1, 1, 0, 'beta', 1)) I'm measuring against the RSS. A sample of one of my apps: RSSCommand 3624 .../apache2/bin/httpd -f .../apache2/conf/httpd.conf -k start 15716 .../apache2/bin/httpd -f .../apache2/conf/httpd.conf -k start In this case, I've got one apache supervisor process using a small amount of memory (3.5MB) and a single worker process using a good amount (15.3MB). When I first start apache, the worker process starts small (<4MB), but increases with the first page access (>12MB) and stays there until the next restart. Ideally, I'd like for the worker threads to release all the memory they're using when idle to get back down to their initial memory usage. I realize that there will be some performance tradeoffs for this, but in this case these sites have very low requirements so it seems worth the tradeoff. I've already played with apache's httpd.conf settings: ServerLimit 1 StartServers 1 MinSpareServers 1 In an effort to remove the second worker thread (so I have one process using 13mb instead of two). However, at this point apache still spawns another process when usage spikes slightly, and then the second thread remains until restart. I've tried investigating apache MPM's, apache settings, and python memory profilers, but haven't made much progress. Some things I'd like to try to figure out: -What is using memory in my apache worker threads? Is that the baseline django memory usage, or are there ways to decrease this minimum? -What python techniques could I be using to decrease the long-term memory usage? (not leaks, just persistent memory usage) -What apache/wsgi config changes should I be looking into? Different MPM? Any comments/suggestions are appreciated, I know I'm probably missing some obvious things here. Thanks. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Using the @login_required() decorator with #hash url's
Currently, the @login_required() decorator does not preserve any #hash part of the URL. So if I go to /page#section1 without being logged in, I'm redirected to /login?next=/page#section1. After logging in, I'm redirected to /page (without the hash part). I read in another thread here that the hash is not sent as part of the request headers, so the django server doesn't have this information. However, I still need to be able to preserve the hash part of URL's when using the login feature, since the hash can alter the page in AJAX apps. My question: does anyone have any ideas how to go about implementing this? My first thought is to modify the @login_required() decorator, so that it spits out some javascript to 1) capture the hash of the url and then 2) redirect to the login page. This may break the back button in browsers. Does anyone else have ideas of how to do something like this? Thanks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: New to unit testing, need advice
> besides the testing issues (which are certainly a heated debate!), i have to > say that my Django projects became far better organized and a lot more > flexible when i learned to put most of the code on the models, and not on the > views. This is a pretty interesting topic. Coming from a PHP background, I started out with very messy code (each .php file contained it's own program logic at the top). From there I went to Javabeans (tomcat) and got a crash course in MVC & OO web apps, almost to an extreme I wouldn't want to duplicate. Now that I'm in django, I have a nice middle ground. There are certain ideals (ie. programming paradigms) that might make unit testing easier. One is separating the different parts of MVC (model, view, controller). Controller in our case is mostly the django code (if I'm not wrong). The views are our views.py functions. The models are our models.py file. But there's the unspoken part of business logic. By default, most django apps seem to have very dumb models.py files (the classes are simply wrappers around database rows & tables), and all the logic is in views.py. However, the business logic should really be pulled out of views.py, whether that goes into heavy models.py objects (like Javier has), or a separate business_logic.py file. It helps to keep asking yourself, what is this line of code doing? For example, in views.py: def handle_add_item_form(request): form = AddItemForm(request.POST) if form.is_valid(): item = Item.objects.create( name=form.cleaned_data['name'], price=form.cleaned_data['price'], status=form.cleaned_data['status'], ) if item.status == 'out of stock': # Order 100 more items for later order_more_items(item, 100) email_admin_about_oos(settings.ADMINS) This handles a simple form to create an item. However, there's clearly business logic here... the whole "status == 'out of stock'" sticks out like a sore thumb. It'd be better like this: in views.py: def handle_add_item_form(request): form = AddItemForm(request.POST) if form.is_valid(): item = ItemManager.create( name=form.cleaned_data['name'], price=form.cleaned_data['price'], status=form.cleaned_data['status'], ) and in managers.py: class ItemManager: def create(name, price, status): item = Item.objects.create( name=form.cleaned_data['name'], price=form.cleaned_data['price'], status=form.cleaned_data['status'], ) if item.status == 'out of stock': # Order 100 more items for later order_more_items(item, 100) email_admin_about_oos(settings.ADMINS) You have to type a bit more (to create all those managers, or service objects, or whatever you call them), but the benefit is that business logic is now in one place. Views.py should only be the dumb glue that connects the HTTP requests with the service objects, and returns the results. Like Daniel said, models.py are usually so simple/dumb that you don't need to test them much. Your views.py should be the same way. They're simply glue, so the only things that will go wrong are simple logic errors and typos. All the complicated stuff is in the business logic objects (the managers.py here), and that's what you spend a lot of time unit testing. In this example, you can use the ItemManager.create() function anywhere you create items... adding new ones, copying existing ones, importing items from a file, etc. In the old example, you'd have to duplicate the views.py code in each view that handles this situation and test each one. You could put all these functions into the models.py file if you wanted, you'd just end up with huge models.py objects, and you might also run into cases where there is not an obvious place to put a bit of business logic. Not every business logic function corresponds exactly to one model. Another great change... with all your logic separated like this, if you have to change something very complicated (like the way the whole system handles out of stock, ordering more inventory, permissions, etc), you have one place to change & test. It's great. So do I use this? No, all my code is in wrapped up in my views.py and I have no unit testing. I try to break out duplicated functionality into their own functions, but that's more for convenience than true separation. I'd recommend starting with the "keep them separated" mentality, since it gets harder and harder as a project goes on. Sorry for the long rant, but it touched on something I've been observing in my own code. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to th
Re: Using the @login_required() decorator with #hash url's
On Jul 16, 11:05 am, Javier Guerra wrote: > maybe urlencode()'ing the '/page#section1' parameter? The first problem is that the django server only receives the '/page' part of the URL. The browser itself holds onto the '#hash' part and doesn't transmit that to the django server at all, so the login_required() decorator calls request.get_full_path() and gets '/ page', so that's what it uses. It doesn't look like there's a simple way to get the '#hash' from django. Once I get it, yes I can use urlencode(full_path_with_hash) and use that instead. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Using the @login_required() decorator with #hash url's
> The only way I know around this involves leaning on JavaScript to
> get the URL, split off the #hash bit from it, and sneak it in as
> a hidden element on the login form.
Yeah, I ended up doing this on the /login page. Basically:
$(function() {
$('#url-hash').attr('value', window.location.hash);
});
...
This works for firefox. The bigger problem? Internet Explorer does
not preserve the hash when following redirects!!! So in this case,
you have *no opportunity* to save the hash via javascript on the login
page. So instead of sending the 'location' header to redirect, you'd
have to send something like this HTML page:
window.location = '/login?next=' + escape(window.location);
Besides being totally annoying and hackish, it breaks the back
button. You can't save the hash with javascript on the login form
with IE, because to get to the login form you had to use a redirect,
and when you use a redirect the hash is lost... this means the login
page never sees the hash in any way when using a redirect. It's
amazing that something seemingly simple and fundamental ends up being
such a hack.
So besides my horrible back-button-breaking hack, does anyone else see
a way to do this?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
How do I minimize memory usage with WSGI and Apache?
I'm hosting a bunch of django apps on a shared host with 80MB of application memory (Webfaction in this case). I've got a bunch of apps that are very infrequently used, but need to be online. I've already changed the httpd.conf: ServerLimit 1 Instead of the default "ServerLimit 2". With the default, each app spawns 3 apache processes; one manager, 2 worker processes. With ServerLimit 1, there is only a manager and a single worker. This cuts down on a lot for low-traffic sites. However, I'd like to see what the bare minimum memory usage can be. I've made sure DEBUG = False. I also separated the static media to a separate server with MEDIA_ROOT and MEDIA_URL. When I first start apache, the manager and worker processes use about 3MB (RSS) each. After the first access with a browser, the manager stays the same but the worker uses 16MB and stays there until apache is restarted. Ideally, I'd love it if the memory usage go back to the initial state (3MB) after a period of inactivity. I'm curious; what is being stored in memory? Are there other things I can do to reduce memory usage? The one thing I've seen is the "MaxRequestsPerChild" apache setting. As far as I can tell, it'll recreate the worker process after X requests. But this seems like it might adversely affect performance, since I need something time-based, and not # requests based. With MaxRequestsPerChild, I'm guessing if I set it to 10 but only had 9 requests, it would sit there using up memory until the tenth request, which is not ideal. With MaxRequestsPerChild set to 1, I'd be guaranteed low memory usage but would also recreate the worker process for each request, which seems like a bad idea. My goal is to have many apps running at bare-minimum memory usage, and only the frequently accessed sites taking up ~16MB memory each. And when the active sites are dormant (late at night, etc), I'd like those to lower their memory use as well. Is there any way to do this? Thanks! (and apologies if I've missed some obvious documentation... I've gone through a few guides, but nothing seems to have helped). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: How do I minimize memory usage with WSGI and Apache?
Wow, lots of good feedback here. Thanks! On Aug 9, 6:17 pm, Graham Dumpleton wrote: > Replace use of mod_python with mod_wsgi. I'm using mod_wsgi for everything already, forgot to say. When I researched, it seemed like the newer/better way to do django. > Ensure you use prefork MPM > for Apache and delegate Django instances to run in mod_wsgi daemon > mode process. > Sorry, meant to say 'worker' rather than 'prefork'. DO NOT USE > PREFORK. Hmmm, that was a bad stuff up. :-) I just started using the worker MPM for one of my apps. It didn't seem to give any performance or memory savings, since my app is so small/simple. I'll keep an eye on it for later. Using daemon mode gives me auto-reloading by touching the .wsgi file (yay), but it uses more memory. Instead of supervisor (3MB) and apache worker/WSGI/Django (16MB) processes, now I have supervisor (3MB), apache worker (3MB), and wsgi/django (15MB). Not a huge gain, but just a bit more. I'm willing to live with that for the auto- reloading benefit, however. > Having done that, you can use inactivity-timeout option > for mod_wsgi daemon processes to have the process restarted and > returned to low memory footprint idle process after no use for a > while. This is exactly what I wanted!!! It's funny how when I finally read the description for something, it matches exactly what I'm looking for. I just didn't know where to look. I just tried it, and yes my memory baloons up to 16MB, then comes back down later. The only strange thing is that I set it to 10s (very short) to test out, but it takes a while for the process to actually be reclaimed. If I put inactivity-timeout=10, then load the page, then close the browser, memory usage should be reclaimed in 10 seconds, right? It seems more like 1 minute wait, which is fine but strange to me. Maybe there's only polling at certain intervals (checks every 1 minute), so the resolution isn't that fine. Maybe it's dependent on the deadlock timeout? Don't know. > Note, just ensure you don't keep loading mod_python after having > switched to mod_wsgi as mod_python will still consume some extra > memory when not in use. Yeah, never touched mod_python, so no problem here. On Aug 10, 6:44 am, Dj Gilcrease wrote: > I just went though this process on Webfaction as well, so here is my > apache conf, it might help... Thanks for the post, good to see another webfaction user here. It always helps to see other people's real-world configs, since examples usually leave out the key bits. On Aug 10, 10:25 am, Dj Gilcrease wrote: > >> MaxRequestsPerChild 5000 > > Don't see much point for MaxRequestsPerChild. > This just restarts the child after 5k requests, which helps clear out > some memory on occasion (I am still trying to get rrd or something run > on on Webfaction so I can actually map memory usage over time, and > start tuning Apache to my actual usage needs better) I experimented with using this as a dumb alternative to inactivity- timeout. Set MaxRequestsPerChild to 1, means the WSGI process will restart after each HTTP request. Very slow, but gives immediate memory savings. Only viable if the MEDIA_URL is set to a separate, static server (which webfaction recommends and makes easy). However, inactivity-timeout is still better. > >> WSGIPythonOptimize 1 > > Am dubious that turning on optimisation helps and can cause problems > > if set to '2' as doc strings will be stripped. > I doubt it is much improvement, but I just turned it on as a test and > did read that 2 is a bad idea which is why I stuck with 1 I'd love to see some benchmarks for this config with real apps. The docs make sense... don't expect magic switches to help if you haven't optimized your code, so I really shouldn't even worry about this option yet. > >> > >> ServerName demo.digitalxero.net > >> DocumentRoot /home/digitalxero/webapps/wsgi/demo > >> WSGIDaemonProcess demo.digitalxero.net threads=5 > >> maximum-requests=1 inactivity-timeout=3600 > >> display-name=wsgi:demo_site > > > That is 1 hour for inactivity timeout. They possibly want something > > less than that. > > Ya, I picked an hour for starters and will drop it down or increase it > as my needs dictate (one I get something to monitory usage better then > guessing) Their default DJango installs are either mod_python or mod_wsgi. The mod_wsgi is embedded mode, and has no inactivity-timeout and is StartServers 2, so by default one Django app takes 3MB + 16MB + 16MB of memory (35MB) out of your allotted 80MB. That means only 2.5 django apps before you run out of memory. They have a "how to save memory with django" howto, but it doesn't have any of these settings. I'll try to send it updates, they'll probably appreciate it. > > If you application does leak Python objects due to inability to break > > reference count cycles, there shouldn't really be a need for maximum > > requests. How would we go about finding out if there are memory leaks? I'm
Re: How do I minimize memory usage with WSGI and Apache?
On Aug 10, 2:24 pm, dartdog wrote: > Big help!! Had just started getting using too much memory notices from > Webfaction yesterday evening... Yeah, I didn't even pay attention till I hit about 120-140MB on my 80MB plan and got everything turned off. Otherwise I would have just let it grow, not paying attention. > The other item is using the crontab -e command from ssh to comment out > apps you don't need to have running For some reason could not find > that when I was looking for it and one of my sites just kept > restarting after .stop, which I understand but did not know how to > stop the behavior! Good catch! I noticed that they restart themselves, didn't bother finding out why. Here's my complete writeup and howto: http://forum.webfaction.com/viewtopic.php?pid=11096 Thanks to those in this thread for helping me make this work. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Django dev server (runserver) takes 5 seconds after reloading new code?
I have a strange problem that's popped up just recently, and it's puzzling to me. I'm running the "manage.py runserver" dev server for development. Normally, right after I save a file, the server reloads the new code and is ready to go right away. However, recently one of my apps has a strange problem where it'll reload the code immediately, but then the server is unresponsive for exactly 5 seconds. So it goes like this: -Modify code & save -Django dev server reloads, says "Validating models...", then "0 errors found" and is ready. -Attempting to connect via the browser just stalls, then times out. -5 seconds after the dev server reloads, it starts accepting connections from the browser. So there's just a 5 second waiting time between when the dev server reloads, and when it will accept connections. It only happens with one of my apps, and has just recently started. Does anyone know why this might be happening? I just wanted to ask before I start diving through django source code. Thanks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
ManyToManyField() causes bugs when using 'ModelName' instead of 'self'
Summary: django should throw an exception when a model has its own
name as the 'to' field of a ManyToMany() insead of 'self'.
I had a field like this:
def Item(Model):
name = CharField(max_length=100)
related_items = ManyToManyField('Item', blank=True)
But this caused some very strange bugs. For example:
item1 = Item.objects.create(name='First')
item1.save()
item2 = Item.objects.create(name='Second', related_items=[item1])
item2.save()
print item2.related_items
> []
print len(item2.related_items)
> 0
The correct way, as stated in the docs, is to use 'self' for a
recursive relationship.
def Item(Model):
name = CharField(max_length=100)
related_items = ManyToManyField('self', blank=True)
(NOTE: This is pseudo/test code, may contain typos)
When I use this method, it works fine. However, if the first method
is incorrect and produces functionally invalid results, then there
should be a model validation check for this and an exception if it's
found. Basically, if a model has a ManyToManyField() with its own
name as the 'to' parameter, an exception should be raised.
I understsand that the current functoinality & docs are working as
they're supposed to. However, it would also be nice for django to
raies an exception instead of silently failing and producing invalid
results.
Have I missed anything? I may be misunderstanding the problem here,
but I have managed to fix my problem and figure this could save others
from the same trap. I'd like to get some input before submitting a
ticket.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
Model.objects.values('some_column').distinct() returns the wrong results
I have a model with a field, and I wanted to pull all the unique
values of that field. So I do this:
distinct_values = Model.objects.values('some_column').distinct()
print 'distinct_values.count()', distinct_values.count()
print 'len(distinct_values)', len(distinct_values)
Which gives:
distinct_values.count()4
len(distinct_values) 137
This seems wrong. If I do this:
print distinct_values
or
for value in distinct_values:
print value
I get the entire list of values from every row in the database. It
does not print only the distinct values, as I'd assume it would.
The closest I've found is this closed ticket from a year ago:
http://code.djangoproject.com/ticket/2939
But that is ticket is different: 1) it shows the correct results when
printing the results, and the len() and count() results are swapped
from this example.
Is anyone else seeing this? Maybe I just have my code mixed up.
Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
Re: Starting a daemon thread from Django view
I'm in the same boat here. My plan is similar. >From views.py, record the job in the DB. Start the daemon thread. Return a "Processing..." response to the browser. When the daemon thread is done, it records it in the DB. The browser page uses AJAX to query the DB and update the status. When the DB changes, the browser page then shows "Done." and the results. One unanswered question is whether updating the DB with the models is thread safe. I realize that Django itself is not guaranteed to be thread safe, but I don't yet see that there would be any problems inserting & updating models (db rows) so far. I know I'll have to worry about things like race conditions, but I suspect that actual operation of updating a model would not be affected. Hopefully that's true and I don't run into crazy threading bugs. Your method will be safer, since calling a web service to signal the end of the job is concurrency, but avoids the threading problem. It's no different than multiple users calling the server at once. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Unable to filter a query on >1 ManyToMany relationships
I have a class like this:
class Node:
name = models.CharField()
parents = models.ManyToManyField('self', symmetrical=False,
related_name='child_nodes', null=True, blank=True)
This allows a basic parent(s)/children hierarchy. A node can have >=
0 parents. A parent can have >= 0 children.
Say I have these nodes:
A - parent node
B - parent node
C - child of both A & B
So I can do this:
Node.objects.filter(name='C')
> returns ['C' node].
Node.objects.filter(name='C', parents__name='A')
> returns ['C' node].
Node.objects.filter(name='C', parents__name='B')
> returns ['C' node].
Node.objects.filter(Q(name='C'), Q(parents__name='A'))
> returns ['C' node].
Node.objects.filter(Q(name='C'), Q(parents__name='B'))
> returns ['C' node].
Node.objects.filter(Q(name='C'), Q(parents__name='A'),
Q(parents__name='B'))
> returns []
Node.objects.filter(Q(name='C') & Q(parents__name='A') &
Q(parents__name='B'))
> returns []
Node.objects.filter(Q(parents__name='A'), Q(parents__name='B'))
> returns []
Why do those last two return an empty set? I can even do this:
Node.objects.filter(Q(name='C'),
Q(parents__name='A')).filter(Q(parents__name='B'))
> returns ['C' node].
That works. But filtering on two parents (AND, not OR... I want the
child nodes of both parents) returns the empty set. It doesn't make
sense to me why using two separate filters would work, but combining
them into one statement gives different results.
Anyone have an idea about this? Is this a bug in the django query
system, or am I looking at this wrong? Thanks.
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
Re: What does an ideal django workflow setup look like?
This is such a great thread! I'd love to see a wiki page detailing some good setup ideas. It's such a leap forward from the nuts & bolts of "How do I install django" and "How do I run a complex query" to this... "How do I use django more effectively, end to end?" I agree - most of these tips will be generic to project dev & deployment, but it seems there are a few django specific ones. My thoughts... - Use VCS. First step. I use SVN, but others are probably better. Even if you're a solo dev, even for small projects, it's worth it. - Definitely have local, test, and prod servers. Local is where the development happens (on your machine). Testing is where you can test the latest commits. Ideally you'd have continuous integration as well, which would also run on the latest revision. Prod is the live site. If you're a single dev on a small project, you can get away with local & prod. - +1 on using "settings.py" and "local_settings.py". The problem with keeping local settings files in the VCS is when you have multiple servers with different settings. You could store each file as local_settings_test_server.py and so on. Then you could have something like "settings_server_name.py" that's not in VCS, and it only has: SERVER_NAME = 'prod' And in your settings.py: import * from settings_server_name if SERVER_NAME == 'prod': import * from settings_prod_server elif SERVER_NAME == 'test': ... etc That way almost everything is in VCS, and you only have one file to write for each server. - Try to develop & test on servers similar to your PROD server. It stinks to develop and test, then realize you had python X and PROD server is python X - 0.1, which means some features/libraries aren't available (or syntax is wrong). - Use "south" for database changes! It takes just a bit to get used to, but will make database changes (both schema and data) so much easier to keep track of. This is basically what Kenneth Gonsalves described, but in reliable/convenient/automated form. It's amazing. Go south! Use south! (don't think about dmigrations. It does not exist.) - Use some kind of deployment script. I wrote my own and it's simple and awesome. You can also use something like fabric, which seems complicated but is very useful. Add things like "svn update force" to update a repsitory, "touch *.wsgi" to reload the app, "python manage.py migrate" to update any DB changes (using south), etc. Anything you'd do on the server, create scripts for. Even stuff like "copy PROD DB to TEST server", so you have real data to play with. That's what I can think of at the moment. It'd be nice to have a "best practices" page on the django site, since a lot of these suggestions will be repeated for most users. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: ManytoMany initial values
Giancarlo, I've run into this trouble myself. The problem here is the widget itself. It does exactly what you described: -Shows all possible values for the ManyToMany field. -Highlights the "active" or "selected" values. These correspond to items in that list/array/queryset. For any more than a couple of values, it's horribly confusing. Even though I'm used to it, I still find myself assuming that listed values are selected (especially when you have a few hundred options). It's just a bad widget for this kind of data. Take a look at the Admin control panel widgets. Specifically, view a User in the django control panel and look at the permissions widget. That's what you want. Something where it shows all available (unselected) options on one side, all selected options on the other side, and a way to filter the available options for the ones you want. Much better than the simple SELECT MULTIPLE="" that is used by default. I haven't looked into it, but there may be a way to use that admin widget for your own forms. This really should be the default widget anyway. So bottom line - you need a different widget, maybe one you have to write yourself. Good luck! -James -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: ManytoMany initial values
On Mon, Oct 25, 2010 at 2:23 PM, Giancarlo Razzolini wrote: > James, > First of all thank you for the fast response. I'll take a look into the > admin forms widgets. I'm already looking for third party applications field > types and widgets to see if I find one that does what I want. I was even > considering creating views on the underlying database to accomplish this. I > think that django should implement on the forms a better way to show granular > output. My django application is completely user dependent, and only let a > user to view/add/edit/remove information that belongs to him. Only the > superuser can deal with all objects. I was using a custom for to accomplish > that, but since I'm using a ManyToMany relation, it was a lot of work to > validate it myself and save the relationship. I guess I'll have to turn back > to it if I can't use the admin form you mentioned. Thank you again. > My regards, > Giancarlo Razzolini No problem. For anything user-facing, I'd recommend not using admin forms. You might be able to get away with it, but it seems each time you add some business logic (permissions, validation, etc) it makes it less and less a fit with the built-in forms. It's relatively trivial to create your own forms, so it shouldn't be too hard to ignore the admin pages. It's just annoying to have to do the work to create templates, logins, etc. But depending on your requirements, you'll probably find that it's easier to just create your own forms. You'll still have to figure out the widget issue, but at least you'll have better control over the validation. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Unable to filter a query on >1 ManyToMany relationships
Tom, Thanks for the link, I read through that and it makes sense. So if both conditions are in the same filter, they are both applied simultaneously. Since I wanted two separate conditions, I needed two separate filter() statements. In other words, my method, is "Find a node who's parent is both A and B". What I wanted, and what you recommended, is "Find a node who's parents include A and include B". Very interesting. I get it now, and yes it is documented. For some reason I have a really hard time navigating the docs, even though I've read the page you mentioned about a dozen times. I think a lot of it has to do with the horrible heading / lack of indents in the docs. But I digress. Thanks tom so much, this answers my question! -James -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
How do I filter/exclude custom queryset fields created by .extra()?
I have something like this:
class Node(Model):
widgets = ForeignKey(Widget)
And I want to retrieve all nodes that have >0 widgets. I have
an .extra() query like this:
nodes = Node.objects.all().extra(
select={
'num_widgets': 'SELECT COUNT(*) FROM appname_node_widgets
WHERE appname_node_widgets.node_id = appname_node.id',
}
)
This works fine, since I can now do this:
print nodes[0].num_widgets
But I can't filter or exclude on this field. I try this:
nodes = nodes.exclude(num_widgets=0)
Gives a "FieldError: Cannot resolve keyword 'num_widgets' into
field." I understand that the fields created by the .extra() call are
not "real" fields attached to the model, but they are fields attached
to the QuerySet. Is there a way to use fields created with .extra()
in subsequent filter/exclude calls?
My next guess is to drop down to custom SQL and write the whole query
out by hand, but I thought I'd check to see if I'm doing something
wrong here. I thought I remembered being able to use .extra() fields
in later .filter() statements, but I might be imagining that.
Any thoughts? Thanks!
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
Re: adding field to query set
What's your motive? Are you worried about performance?
If performance is not an issue, you can just do this:
for project in Project.objects.all():
print project.user.first_name + ' ' + project.user.last_namea
This might speed things up:
projects = Project.objects.all().select_related('user')
for project in projects:
print project.user.first_name + ' ' + project.user.last_namea
If that's not fast enough, you may want to do something using .extra()
or .raw(). I'm not sure how to do this effeciently with .extra().
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
Re: adding field to query set
If it were my project, I'd probably add this kind of custom field to
your mapping:
mapping = {
# list the normal columns first... then:
'full_name': ['user__first_name', 'user__last_name'],
}
Then your render code would be something like this:
def render(self):
for row in self._data:
for name, value in self._mapping.items():
print 'name: %s' % name
if hasattr(value, '__iter__'):
final_value = []
for value2 in value:
final_value.append(getattr(row, value2))
final_value = ' '.join(final_value)
else:
final_value = getattr(row, value)
print 'final_value: %s' % final_value
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
Aggregates are giving the wrong numbers for .annotate()? What am I doing wrong?
Hi all,
I've been working in django for a while now, but am still wrapping my
head around the more complex queryset features.
I have something like this:
class Foo:
name = CharField()
bars = ForeignKey(Bar)
widgets = ForeignKey(Widget)
I can do this:
Foo.objects.extra(select={
'num_bars': 'SELECT COUNT(*) FROM app_bar WHERE app_bar.foo_id =
app_foo.id',
'num_widgets': 'SELECT COUNT(*) FROM app_widget WHERE
app_widget.foo_id = app_foo.id',
})
That gives me what I want... a list of all foo's with counts of how
many bars & widgets it's linked to.
NameNum BarsNum Widgets
First 1 4
Second 2 3
However, I wanted to use annotations. So I do this:
Foo.objects.annotate(num_bars=Count(bar), num_widgets=Count(widget))
But I get increased counts:
NameNum BarsNum Widgets
First 19 19
Second 5 5
When I spit out the str(queryset.query) var, I get SQL like this:
SELECT
app_foo.name,
COUNT(app_bar.id) AS num_bars,
COUNT(app_widget.id) AS num_widgets
FROM app_foo
LEFT OUTER JOIN app_bar ON ...
LEFT OUTER JOIN app_widget ON ...
WHERE ...
GROUP BY app_food.id
This seems to fail because each LEFT OUTER JOIN gives multiple rows
for each match in the resulting table. Then when COUNT() goes to
work, it will count each of the duplicate rows as another entry. Then
all the COUNT() vars end up being a) larger, and b) the same as each
other, since they're all counting the end-result rows.
I've tried .distinct(), but it does not fix the problem. Am I
misunderstaning .annotate()? Is this a bug? Thanks!
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
Re: How do I filter/exclude custom queryset fields created by .extra()?
>From a raw SQL standpoint, I've figured out that these constraints need to be in the HAVING clause, not the WHERE clause. But it looks like HAVING is not exposed through the .extra() function, which would seems logical. It appears you can create extra fields, but due to this lack you can't filter on them. Is there another way to customize the HAVING field? -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Reloading a single model object from .py file... keeps getting cached results?
I want to run a script outside of the django server for maintenance purposes, but it still accesses some of the models. The problem is that I can't seem to get the model object to reload it's data from the DB - it always seems cached. # Setup django. import project.settings from django.core.management import setup_environ setup_environ(project.settings) from project import models import time while True: obj = models.Object.objects.get(id=1) print 'obj.status: %s' % obj.status time.sleep(1) Theoretically the line "obj = models.Object.objects.get(id=1)" should reload a new copy of the object with id=1 each time. But when I run this script, the values never change - they are whatever they were when the object was first loaded, and do not change (regardless of what's in the DB) until the next time I run the program. My understanding is that this is how you reload an object after changing it (after .save(), etc). But it's clearly not working in my case. Am I doing something wrong? How do I force the object to be reloaded from the database? Thanks! -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: absolute url in template and url name
I always do something like this:
abs_activate_url =
request.build_absolute_uri(reverse('account_activate', args=user.id))
That's an absolute URL, so it's like "http://domain.com/account/
activate/2" instead of the normal reverse() or {% url %} which gives a
relative URL like "/account/activate/2". Then you can use it just
like others have stated:
from django.template.loader import render_to_string
email_body = render_to_string('email_template.html', {
'abs_activate_url': abs_activate_url,
}
Jason Mayfield's "current site" method seems easier, since you don't
have to make & pass all the absolute URLs first.
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
Re: absolute url in template and url name
As a followup, I had to do something similar and ended up using this:
# Gets the http://domain.com without the trailing /
base_url = request.build_absolute_uri('/')[:-1]
And in the template I can do this:
...
The benefits:
-You only have to create the base_url in the view, the rest you can do
in the template itself.
-The URLs are absolute & contain the domain (ie. http://www.domain.com/page/1,
https://www.domain.com:8008/view2, etc).
-Includes http & https.
-Simple.
This the method I'm using from now on.
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
How do I rename a FileField() that's attached to a model?
I've gone through the documentation a ton, been trying different techniques, and I know I'm just missing something obvious. I have a model like this: class MyModel(Model): file1 = FileField(...) So I can do this: model = MyModel.objects.get(id=1) print mode.file1.name But I want to rename the file on the filesystem. I want something like this: model.file1.rename(new_filename) But that doesn't exist. The closest I can find is somethig like this: contents = model.file1.read() model.file1.delete() model.file1.save(new_filename, contents) But this gives an exception since the FileField save() method requires a "contents" parameter that has a ".chunks()" method. Right now it's just a string. I can't use a python tempfile.TemporaryFile() object, since that then gives the exception "Unable to determine the file's size". All of this seems like I'm jumping through hoops that don't make sense. How do I just rename the file? Is there an easier way? Thanks so much! -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: How do I rename a FileField() that's attached to a model?
> you could do it manually. first use os.rename to rename the file on the disk, > and then update your model instance with the new name and save it. I think this is what I'm missing. What's the code for this? Can I just do: os.rename(model.file1.name, new_filename) model.file1.name = new_filename model.save() Is that enough? I'll go try that now. The problem is that the documentation leaves it unclear whether simply changing the name and saving the model is kosher. I guess I'd expect a more complicated method, but if that works then great. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: How do I rename a FileField() that's attached to a model?
Yes, it was that easy. I don't know why, but I've been banging my head against the wall on this for a while and never thought of this solution. One thing that would really help a ton is more/better examples about file-handling in django. I've spent more hours on this single feature reading & rereading the docs trying to figure out how things work. A few good examples would have saved me tons of time. Who do I talk to about possibly contributing to the docs? On Mar 14, 3:10 am, Jumpfroggy wrote: > > you could do it manually. first use os.rename to rename the file on the > > disk, > > and then update your model instance with the new name and save it. > > I think this is what I'm missing. What's the code for this? Can I > just do: > > os.rename(model.file1.name, new_filename) > model.file1.name = new_filename > model.save() > > Is that enough? I'll go try that now. The problem is that the > documentation leaves it unclear whether simply changing the name and > saving the model is kosher. I guess I'd expect a more complicated > method, but if that works then great. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
form.is_valid() changes a ModelForm's associated instance now?
I discovered the recent changes to the "is_valid()" model form method in 1.2, and I'd love to hear thoughts on the rationale behind this change. I have this kind of code: item = Item.objects.get(id=1) print 'item.value: %s' % item.value form = ItemModelForm(request.POST, instance=item) old_item = item if form.is_valid(): print 'old_item.value: %s' % old_item.value print "form.cleaned_data['value']: %s" % form.cleaned_data['value'] new_item = form.save(commit=False) print 'new_item.value: %s' % new_item.value And I'd expect this output: item.value: old_value old_item.value: old_value form.cleaned_data['value']: new_value new_item.value: new_value But instead I get this: item.value: old_value old_item.value: new_value form.cleaned_data['value']: new_value new_item.value: new_value It seems like the "is_valid()" function not only checks for validation errors and creates the "form.cleaned_data" members, it also modifies the instance of the model attached to this form. I just looked over the docs, and I can now see that this is implied but not really explicitly referenced here: http://docs.djangoproject.com/en/1.2/topics/forms/modelforms/ The first time you call is_valid() or access the errors attribute of a ModelForm has always triggered form validation, but as of Django 1.2, it will also trigger model validation. This has the side-effect of cleaning the model you pass to the ModelForm constructor. For instance, calling is_valid() on your form will convert any date fields on your model to actual date objects. The problem is not that the instance is being "cleaned". It's that the instance is being modified, then cleaned. It would help if the docs specified this as well, so we know what the side effects of "is_valid()" really are. Also, doesn't this make this code obsolete? item = form.save() And: item = form.save(commit=False) item.save() We could instead just do this: form = ItemModelForm(request.POST, instance=item) if form.is_valid(): item.save() (assuming that we're modifying an existing instance, and not creating a new one) Since the original "item" is modified, I could just save that as well, right? Is that what we truly want to happen? Seems like the modification of the original instance in this case is an unintentional side effect of needing to clean the model for validation. It seems like it would be better if is_valid() created a copy of the instance, and then modified & cleaned that copy instead of the original. Otherwise it seems tricky to have a validation method secretly modifying seemingly unrelated data. I just want to wrap my head around these new changes, and maybe understand the rationale behind them. Thanks! PS. The solution to my specific problem above is to do this: item = Item.objects.get(id=1) print 'item.value: %s' % item.value form = ItemModelForm(request.POST, instance=item) # NOTE: This fixes the problem: old_item = copy.deepcopy(item) if form.is_valid(): print 'old_item.value: %s' % old_item.value print "form.cleaned_data['value']: %s" % form.cleaned_data['value'] new_item = form.save(commit=False) print 'new_item.value: %s' % new_item.value -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
How do I run test and get results programmatically? (django, south, fixtures, sqlite)
I have an app that I'm starting to write tests for. The app uses
south for all migrations and mysql for the db.
I wrote some tests and also created an initial_data.json fixture to
provide some default data. If I run "python manage.py test", the
database is created via sqlite, the tests are run, and then the report
is spit out. This works fine.
However, I want to automate this where I can programmatically run
tests, find the failures, and do some custom logging/saving of the
failures. How do I run the tests in python?
I've tried a few variations of the below code, but the problem is the
database. When I run the "python manage.py test" command, it
automatically uses sqlite3 as the backend, creates the database, and
uses it for all tests. However, when I run my own tests, it uses the
mysql backend. If I force it to use sqlite, is has problems with the
fixtures because the migrations are not run before installing the
fixtures.
How do I run tests manually using the same database setup that happens
when using the manage.py test command?
My code:
import my_application.settings
# NOTE: Use this to force the testing database to use the sqlite
engine.
my_application.settings.DATABASE_ENGINE = 'sqlite3'
from django.core.management import setup_environ
setup_environ(my_application.settings)
from django.conf import settings
from django.test.utils import get_runner
verbosity = 1
interactive = False
failfast = True
TestRunner = get_runner(settings)
test_labels = ()
test_runner = TestRunner(verbosity=verbosity,
interactive=interactive, failfast=failfast)
failures = test_runner.run_tests(test_labels)
And this is the error I get:
calling TestRunner.run_tests()
verbosity: 1
interactive: False
failfast: True
test_labels: ()
connection.settings_dict: {'ENGINE': 'django.db.backends.sqlite3',
'TEST_MIRROR': None, 'NAME': 'my_application', 'TEST_CHARSET': None,
'TIME_ZONE': 'America/New_York', 'TEST_COLLATION': None, 'OPTIONS':
{}, 'HOST': '', 'USER': 'my_application', 'TEST_NAME': None,
'PASSWORD': 'my_application', 'PORT': ''}
test_databases: {('', '', 'django.db.backends.sqlite3',
'my_application'): ['default']}
Creating test database 'default'...
Syncing...
Creating table django_admin_log
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table auth_message
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table south_migrationhistory
Installing index for admin.LogEntry model
Installing index for auth.Permission model
Installing index for auth.Group_permissions model
Installing index for auth.User_user_permissions model
Installing index for auth.User_groups model
Installing index for auth.Message model
No fixtures found.
Synced:
> django.contrib.admin
> django.contrib.auth
> django.contrib.contenttypes
> django.contrib.sessions
> django.contrib.sites
> south
Not synced (use migrations):
- my_application
(use ./manage.py migrate to migrate these)
test_db_name: ':memory:'
Problem installing fixture 'C:\virtualenv-my_application\lib
\site-packages\django\contrib\auth\fixtures\authtestdata.json':
Traceback (most recent call last):
File "C:\virtualenv-my_application\lib\site-packages\django\core
\management\commands\loaddata.py", line 174, in handle
obj.save(using=using)
File "C:\virtualenv-my_application\Lib\site-packages\django\core
\serializers\base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "C:\virtualenv-my_application\Lib\site-packages\django\db
\models\base.py", line 566, in save_base
created=(not record_exists), raw=raw)
File "C:\virtualenv-my_application\Lib\site-packages\django
\dispatch\dispatcher.py", line 172, in send
response = receiver(signal=self, sender=sender, **named)
File "C:\my_application\..\my_application\models.py", line 1175,
in _create_profile_for_user
profile.save()
File "C:\virtualenv-my_application\Lib\site-packages\django\db
\models\base.py", line 456, in save
self.save_base(using=using, force_insert=force_insert,
force_update=force_update)
File "C:\virtualenv-my_application\Lib\site-packages\django\db
\models\base.py", line 549, in save_base
result = manager._insert(values, return_id=update_pk,
using=using)
File "C:\virtualenv-my_application\Lib\site-packages\django\db
\models\manager.py", line 195, in _insert
return insert_query(self.model, values, **kwargs)
File "C:\virtualenv-my_application\Lib\site-packages\django\db
\models\query.py", line 1518, in insert_query
return query.ge
Re: How do I run test and get results programmatically? (django, south, fixtures, sqlite)
@gladys, While that didn't solve my original question, it did solve another related problem. Before, I had issue with migrations failing since they depended on pre-existing data (which did not exist in the blank testing database). But if I added an initial_data.json fixture, it gets run for every migration - during prod and testing. This is bad. So I went with "SOUTH_TESTS_MIGRATE = False", then I manually load the "test_data" fixture during the first test. Works great. I still can't figure out why running "python manage.py test" and running the above code have such different results. I'd love to figure out why, since it would be easier to use the API calls and look in the "failures" var than it is to run the "python manage.py test" command and parse through the output. But for now, the latter gets things working. So thanks! My original question is still open, but I'm un-stuck and can continue working for now. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
How can I integrate performance measurements with unit tests?
I have django testing setup, and it runs a few tests. I would like to measure the time each test takes to run, and record that in a data file I can access later. How do I record the time each test takes? If this is not (easily) possible with unit tests, then is there another easy way to test view times? My goal is to be able to run a performance test for each revision of the code, and be able to track performance improvements / regressions as development happens. Thanks! -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Just started Django - please help clarify web site configuration strategy
> Basically, the question is - when a web > application starts there is a number of things that needs to be > available any time any request. Some of them are really static in > nature - for example, pagination parameters; If these are static values, a good place to put them is in the settings.py file. I put things like APP_NUMBER_POSTS_PER_PAGE = 10 in settings.py. As a bonus, they can be overridden with your local_settings.py file (if you use one). Alternately, you could have an "app_settings.py" file where you store a bunch of static values, and just import that file into anywhere it's needed (like your views.py). > some of them are more > dynamic - for example, menu options. If these menu options are application wide, but change often, then I'd create models to represent them, and store them in the database. If these are server wide config settings that only change once in a while, then I'd add them to settings.py (since you probably want those hardcoded and recorded in your source-control system. If those settings are per-user, then they'd either be cookies (browser-based), session vars (session-based), or a settings stored in the user's profile (ie. stored in DB as a model, the most permanent of the 3). > Things get more interesting when such data are read from the database. > So far what I see is that all that stuff should be places in the > request context (hopefully cached). It can be done more elegantly if > custom context processor(s) are created. But we are still rely on > request contexts. Unless I am missing something? In django, contexts are tied to requests. There really isn't an "application context". If you have a scenario that doesn't fit into a) settings.py vars, or b) database-backed settings, give us more details and we can figure out how best to store them. Context processors are convenient ways to automatically run code for each request, but I normally just use those to automatically add certain settings.py vars to each context (like MEDIA_URL, current user, etc). > It seems that although request contexts combined with caching can > help, it is way too complex for the purposes of initializing the > application. What's the caching for? > Which leads me to another question - is there any callback mechanism > when Django application starts? I guess what I am looking is a place > to make one-time queries, store data in a static member - and viola, I > have my application context. There is no callback mechanism. You can add code to any file (models.py is a good place), and you can do some tricky python-fu to make sure it only runs once per process, but that's a bit of a hack. The real question is - do you really need this? Is there a better way? The most helpful thing would be an example - what is a real-world value you'd like to store in an "application context", and how would you use it? Then we can give you some pointers on how to do things the django way. As an aside, I know asp (and other web servers) run as if you have a single process, with a single set of static vars, and you can pretend that everything is running as a single application. In django, it's often better to think about things as "stateless"... ie. there's no guarantee that there is a single, persistent app running. There's just your code that runs when needed. For example, if you run apache + django with multiple processes, then they are separate. Changes to static vars in one process will not affect the same var in other processes. So forget all about static classes/objects/variables, as they're probably inappropriate for most use cases. Also, you might be over-optimizing. If you're worried about the performance hit of loading a set of values for each request (such as the settings.py values above), you should profile / measure it. The performance hit of loading such vars would be almost completely unmeasurable in most cases. So you could just load up such settings for each request instead of keeping them in static memory, which really is an optimization (ie. trading memory space vs. CPU & disk access). -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
