Re: dynamic upcast

[Carl Meyer]

Hi harold,

On Oct 23, 9:27 am, dadapapa <[EMAIL PROTECTED]> wrote:
> If by "save a derived object from a parent instance" you mean that
> the method that saves the object is defined in the parent class,
> than this should not cause a problem, since type(self) will
> dynamically identify the object as being of the derived type,
> so final_type gets initialized correctly. Then again, I might have
> misunderstood your problem.

Sorry, I wasn't clear.  You may have a BaseClass instance that is
"really" a DerivedClass, and if you ever call save() from that
BaseClass instance (without casting it to a DerivedClass first), the
recipe will break.  Code is clearer:

>>> d = DerivedClass.objects.create()
>>> d.final_type

>>> d2 = BaseClass.objects.all()[0]
>>> d2.final_type

>>> d2.save()
>>> d2.final_type


The fix is just to add an "if not self.id" in the save() method, so
the "final_type" is only set once, at creation time, not on every
save:

def save(self, *args, **kwargs) :
if not self.id:
self.final_type =
ContentType.objects.get_for_model(type(self))
super(BaseClass, self).save(*args)

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: from mysite.polls.models import Poll, Choice

[Carl Meyer]

On Oct 23, 3:30 pm, "James Bennett" <[EMAIL PROTECTED]> wrote:
> It's documented that way because it means we can have a tutorial that
> doesn't immediately dump import-path configuration issues onto people
> who are brand-new to Python; doing things the way the tutorial does
> them, everything automatically ends up on the Python path thanks to
> manage.py.

Things work just as magically with the dev server for apps inside the
project dir if you simply import by the app name and omit the project
name.  manage.py puts both the project dir itself and its parent dir
onto the Python path.  So AFAICT, this particular reasoning is
baseless.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: from mysite.polls.models import Poll, Choice

[Carl Meyer]

On Oct 24, 9:30 am, Carl Meyer <[EMAIL PROTECTED]> wrote:
> manage.py puts both the project dir itself and its parent dir
> onto the Python path.

Sorry, this is not accurate.  manage.py only adds the project's parent
dir explicitly.  Importing by app name works because generally you'd
run manage.py from within the project dir ("./manage.py"), thus it is
the current directory, which is on the Python path.

Nonetheless, the point stands.  Since the tutorial only advises
running manage.py from in the project dir, everything in the tutorial
would work just as well if the project name were removed from every
app import statement.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Default custom settings

[Carl Meyer]

On Nov 18, 10:43 am, mdnesvold <[EMAIL PROTECTED]> wrote:
> The problem with using getattr(settings, 'MY_SETTING', default) is
> that I'd have to specify the default value every time I access the
> value; seems fairly un-DRY-ish, and if I ever want to change the
> default, I'd have to track down every pieces of code that uses it.

The pattern I use is to provide a settings.py file in my application,
which looks like:

from django.conf import settings

MY_SETTING = getattr(settings, 'MY_SETTING', 'default_value')
... more settings ...

Then elsewhere in my application, I always import app-specific
settings from my_app.settings, not django.conf.settings.  This way the
default values are all in one place and it provides an easy reference
for anyone wanting to know what extra settings the app uses.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: configuring production server for static media

[Carl Meyer]

Please stop the excessive top-posting; it makes reading the list
digest painful.  Quote only what you need to, and locate the quotes
appropriately in your response.  Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Importing standalone apps in a project

[Carl Meyer]

On Nov 20, 10:22 am, "Saurabh Agrawal" <[EMAIL PROTECTED]> wrote:
> Using the Python IDLE, I am able to issue command "import coltrane" without
> problem. (coltrane is the name of standalone app)
>
> However, I am not able to do so using Django configured shell "python
> manage.py shell".

It would seem that Django is using a different Python path than IDLE
for some reason (two different versions of Python installed?).  Try
the following once under IDLE and once under django shell and examine
for differences:

>>> import sys; sys.path

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Regional administrator

[Carl Meyer]

On Nov 20, 10:21 am, Huy Dinh <[EMAIL PROTECTED]> wrote:
> Each user is allowed to make new posts and edit their own posts. They
> are not allowed to create a post for a region that's not their own.

The admin has hooks to allow you to do this sort of thing.  This
snippet[1] demonstrates the basic technique; you should be able to
modify it to your needs.

Carl

[1] http://www.djangosnippets.org/snippets/1054/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: display of datefield breaks admin template

[Carl Meyer]

On Sep 30, 7:29 am, Gerard Petersen <[EMAIL PROTECTED]> wrote:
> I get this trace:http://paste.pocoo.org/show/86579/
>
> This is the model in question:http://paste.pocoo.org/show/86580/

Remove the strftime() call from your default= setting for the
DateField.

Django model fields automatically handle converting appropriate Python
values to/from DB backend values.  For a DateField, the appropriate
value is the Python date/datetime object, not a string.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Using forloop counter to index into a query set

[Carl Meyer]

On Sep 30, 11:05 am, SnappyDjangoUser <[EMAIL PROTECTED]> wrote:
> > set up the data structures
> > you pass to your view a bit differently so that you can loop over the
> > forms and the products simultaneously (that is, pull apart the formset
> > forms and zip them together with the product entries in the view).
>
> This is exactly what I like to do!  I am still a bit confused on exact
> implementation, however, because I have not found a way in Django to
> loop through 2 structures at once in a template.

{% for x,y in zipped %}

You need to set up "zipped" as an iterable of tuples in your view, of
course.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Translation of object.attribute in template

[Carl Meyer]

On Sep 30, 2:58 pm, Gerard Petersen <[EMAIL PROTECTED]> wrote:
> It is indeed stored, I'm using a state machine 
> (http://www.djangosnippets.org/snippets/737/). Setting this up multilingual 
> could result in unusable orders states, when users change their locale during 
> the state/transition of an order.
>
> The tricky thing is that the states (used for control) also have a meaning to 
> the user displayed in the template. So I would want to translate them at the 
> last moment, and not store them as such.

Use a choices array with numerical keys and just translate the values
(which are never stored in the DB).
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: display of datefield breaks admin template

[Carl Meyer]

On Sep 30, 2:19 pm, Gerard Petersen <[EMAIL PROTECTED]> wrote:
> That did the trick. It does, however, leave me with a previous problem. When 
> I edit the object via a form (in my app) it displays the field value like 
> this "2008-09-30". And due to field validation the form widget only accepts 
> this format: "30-09-2008".
>
> Any clue on how a datefield contents then would be correctly presented in a 
> form when the format deviates from how it's stored in a mySQL database?

How it's stored in a MySQL database is completely irrelevant to you,
as Django abstracts that away.  The form widget receives a Python
datetime.date object, for which the default string representation is
the form "2008-09-30".  If you use a stock forms.DateField and don't
mess with the validation, it will work just fine (and accept a variety
of date formats from user input[1]).  If you override the validation
to require a different format, you could use a custom widget that
overrides the render() method to coerce the date object to the string
representation you prefer.

This should be easier to customize; for more info see ticket 3672 [2]

[1] 
http://docs.djangoproject.com/en/dev/ref/forms/fields/#django.forms.DateTimeField.input_formats
[2] http://code.djangoproject.com/ticket/3672
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Translation of object.attribute in template

[Carl Meyer]

On Sep 30, 3:29 pm, Gerard Petersen <[EMAIL PROTECTED]> wrote:
> You lost me. You mean make the states numerical and in the array related its 
> (numeric) keys to the, then translatable, 'words'?
>
> If so, the problem would be that the ease of use in coding would stop to 
> exist. The statemachine dynamically creates methods so you can use stuff like 
> order.bill() and order.is_billed()

That's right.  The state machine snippet you're using stores text
strings in the database, and you're trying to use those same strings
as user-facing text, which leaves you trying to dynamically translate
arbitrary strings coming out of the DB, a problem for which there is
no sensible solution.  You can't have your cake and eat it too; if
state names are dynamic and could be anything, you can't prepare
translations for them.  If they are known in advance, then you can a)
abandon that particular state machine snippet and use static choices,
or b) write a similar state machine snippet that uses static choice
arrays, or c) continue using this snippet and define a reference
dictionary mapping (internal) state names to user-facing strings,
which can be translated in the normal way because they aren't stored
in the DB.  Option c is probably the least work for you at this point,
but also arguably the least sensible and maintainable (though better
than what you were originally trying to do, which was essentially the
same thing but directly in the .po file).

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: How to invoke a request to an URL inside a web app without making a real request?

[Carl Meyer]

On Sep 30, 8:00 pm, maverick <[EMAIL PROTECTED]> wrote:
> Hi, I have a django web application, I want to invoke a request of one
> URL inside this application, e.g.  "/getdata/", and capture the output
> of such URL and do some further processing.
>
> Of course I can do that by make a real request, however I feel this
> may not be the best solution, is there any internal way to invoke a
> request ?

With no more information than this, this request gives off a bit of a
smell of "doing it wrong".  Why not refactor your code so that the
functionality you need is encapsulated in a model method or a utility
function that isn't coupled tightly to the URL (or the view function,
since that depends on a request object)?  Then just write a simple
standalone script or a management command to call the functionality
directly, and run that from the command line.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: automatically generated field in admin

[Carl Meyer]

On Oct 1, 8:34 am, Ramashish Baranwal <[EMAIL PROTECTED]>
wrote:
> I have a model in which a field should be generated automatically. So
> it shouldn't be shown in the admin add page. Also in the change page
> it should be shown but should be non-editable. As an example-

Pending ticket 342, another way to do this is to make a custom admin
change_form template for the particular model (meaning you create
admin/appname/modelname/change_form.html), inherit from the main
change_form template (admin/change_form.html), and define the
"after_field_sets" block.  You can refer to your model instance via
the variable "original" and display whatever attribute(s) you want.
The disadvantage of this method is that you can't control where it
appears in the page, it will appear after the fieldset box.

Carl


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: SlugField & prepopulate_from

[Carl Meyer]

On Sep 30, 9:27 pm, "Keith Eberle" <[EMAIL PROTECTED]> wrote:
> Yea... I didn't know for sure, but I thought it used to.  That's why I
> asked.  I wonder why that functionality wasn't there by default...

Because in many cases it would be a usability problem.  What if you've
manually adjusted the slug - do you really want your manual changes
automatically erased anytime you touch the prepopulate_from field?

If you really want a fully-automatic, non-editable slug field,
consider doing it on the server side with one of the many snippets on
djangosnippets.org.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Simple Invoices & Django

[Carl Meyer]

On Oct 3, 2:37 am, Gour <[EMAIL PROTECTED]> wrote:
> I'm quite new with Python & Django but enthusiastic to work on 'porting'
> Simple Invoices - PHP invoicing system - to Django 
> (seehttp://simpleinvoices.org/).

Hi Gour,

The django-developers list is solely for work on developing Django
itself.  Your query would be suited for the django-users list.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Simple Invoices & Django

[Carl Meyer]

Whoops, sorry Gour!  Got mixed up on which list I was reading.  As you
were...
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: check if file exists before uploading | adding a validator to a field

[Carl Meyer]

Hi jelle,

On Oct 6, 5:34 am, jelle <[EMAIL PROTECTED]> wrote:
> Now, what I'd like to do is to write a validator which will check if a
> ( renamed ) file is already existing. Writing the validator itself
> isnt too hard; I can just use pre_save.connect and call a function
> from here. What I do not know is how I can hook up this validator such
> that it informs the user that the file is already uploaded.

The pre_save signal does not provide any way to "veto" the save that
is about to happen, so it's not useful for validation that needs to
abort the save and alert the user.  For this you'll need to write your
validation in the usual way in a Form class [1].  If you're doing the
uploading in the admin, you'll need to hook in a custom ModelForm
there.[2]

Carl

 [1] http://docs.djangoproject.com/en/dev/ref/forms/validation/
 [2] 
http://docs.djangoproject.com/en/dev/ref/contrib/admin/#adding-custom-validation-to-the-admin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Django Admin - Does not see templates, images or templates.

[Carl Meyer]

On Oct 8, 3:28 am, NoviceSortOf <[EMAIL PROTECTED]> wrote:
> These 3 directories seem to contain many redundant files.
>
> /var/www/Django-1.0/django/
> /var/www/django-trunk/django/
>
> /usr/lib/python2.3/site-packages/django/
>
> How do I determine if one of these directories
> contains a duplicate installation and which
> one is the latest?

Probably the one under site-packages is a symbolic link to one of the
first two, and whichever one it's pointing to is probably the one your
system is actually using.

Of the other two, one claims to be Django 1.0 and one claims to be
Django SVN trunk.  Assuming they are what they claim to be, which one
is the latest depends on whether the trunk checkout is from before or
after 1.0.  You might get a clue by looking at django/__init__.py in
both and checking the value of VERSION.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Organizing apps without using projects

[Carl Meyer]

On Oct 7, 10:28 pm, Brent Hagany <[EMAIL PROTECTED]> wrote:
> My apologies if this has been brought up before on this group - my
> searching did not turn up anything, but my search skills are
> notoriously suspect.
>
> I have been reorganizing my code so that it does not depend on the
> project (mostly with success) after watching James Bennet's talk at
> Djangocon about reusable apps.  I've chosen to follow the model James
> talks about with respect to Ellington - that is, I have a bunch of
> interdependent apps that are all under the umbrella of one python
> module.  Right now, I actually have one umbrella "app", with a bunch
> of proper apps underneath it, so that I only have to put the umbrella
> module in INSTALLED_APPS, rather than each sub-app.  I'm not quite
> sure if that's the best way to do this, but more on that in a bit.

This can't be working as you describe it, unless you're doing some
other magic you aren't telling us about to get your models loaded.
The same problem you're having with tests you should be having with
models as well; Django will look for your models in umbrella.models,
not umbrella.app.models, because you just have "umbrella" in
INSTALLED_APPS.

The solution to both problems is to include "umbrella.*" in
INSTALLED_APPS.  This will include every app inside the umbrella
package, without you having to list them all separately.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Branching proj/myapp into branch/0.1/myapp breaks

[Carl Meyer]

On Oct 8, 2:08 am, Gerard Petersen <[EMAIL PROTECTED]> wrote:
> When the project dir is named 0.1 instead of djapp I get a trace (see below) 
> that it has no module 0.1. Where and why does django look at the name of the 
> parent (project) dir?

In Python, you can happily put any directory into the PYTHONPATH (even
if the final path component of it would be an invalid module name),
and then load modules (with valid names) from within that directory.
You might reasonably expect the same to work here if you put the
project directory itself into the PYTHONPATH and then import settings,
urlconf, and apps relative to that.  But when Django sets up your
environment (in django.core.management.setup_environ) it does a bit of
magic to figure out the parent dir of your project and make the
project directory itself importable as a module.  In other words,
Django requires that your project directory itself be a valid Python
module name, even if you never import it yourself.

This is not just "normal Python" behavior; arguably it should be noted
in the docs.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Extend contrib.comments?

[Carl Meyer]

On Oct 7, 10:49 pm, Eric Abrahamsen <[EMAIL PROTECTED]> wrote:
> from django.contrib import comments
>
> def my_get_form():
>      return MyCommentForm
>
> comments.get_form = my_get_form

This works, but it should be noted that it's an unfortunate
monkeypatch that is necessary because contrib.comments doesn't
properly support customization yet.  Once ticket 8630 [1] is resolved,
the proper way to do this will be to create your own app with a
get_form() function in __init__.py, and then set COMMENTS_APP to point
to your app.

Carl

 [1] http://code.djangoproject.com/ticket/8630
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: i18n in templates and views

[Carl Meyer]

On Oct 8, 12:22 am, J <[EMAIL PROTECTED]> wrote:
> How would I include the "date" field from the first model ("Post") in
> the queryset created from "PostI18N", using something similar to:
>
> queryset =
> PostI18N.objects.filter(lang=get_language()).order_by('post__date')
>
> Would that date field already be included? How would I access it from
> the template?

That query will work fine.  In the background it will join the two
tables for the purposes of filtering your result set, but will not
select any data from the Post table.  You can still do this in your
template (if post_i18n is a member of the above queryset):

{{ post_i18n.post.date }}

But each time you do that it will generate a new database query to
look up the Post info.

To avoid the extra queries, you'd use the select_related() method on
your queryset:

queryset =
PostI18N.objects.filter(lang=get_language()).order_by('post__date').select_related('post')

This will select all the data from both tables initially, so a single
query gets you everything you need.

Carl

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Access a model's original get_absolute_url

[Carl Meyer]

On Oct 8, 5:29 am, Erik Allik <[EMAIL PROTECTED]> wrote:
> I'm writing a content management framework which stores content in a  
> tree of nodes. Nodes can be of different content types and I want to  
> be able to add new content types by simply adding apps that provide  
> models to INSTALLED_APPS. And I don't want these apps to be forced to  
> be aware of the existence of my content framework. Anyway, apps  
> normally define URLs/views for models, and that means I won't be able  
> to pull these models into my own URL hierarchy. So for that reason I  
> need to be able to override URLs of other apps' models.
>
> And here comes the problem I'm trying to solve: to be able to override  
> a models URL, I still need to know which view/args/kwargs it maps to  
> so I need to take the model's original URL, resolve it using the  
> model's containing app's URLconf, and then dynamically create a URL  
> pattern for the view/args/kwargs that the model's original URL  
> resolved to. But, by the time my custom URL resolver kicks in, the  
> ABSOLUTE_URL_OVERRIDES setting has already taken effect and there's no  
> way to find out which view/args/kwargs a model maps to any more.

You could abandon ABSOLUTE_URL_OVERRIDES and just monkeypatch these
other models' get_absolute_url methods yourself.  Kind of ugly, but
much better than putting the ugly hack into your Django source tree.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Organizing apps without using projects

[Carl Meyer]

On Oct 8, 9:51 am, Brent Hagany <[EMAIL PROTECTED]> wrote:
> I'm not in front of my code at the moment, but I'm pretty sure there's
> no magic involved, and it works just fine.  When I do a runserver from
> my (completely decoupled) project, the root urlconf sends everything
> to umbrella.urls, which then delegates once again to
> umbrella.app.urls, which knows where to find the views, and the views
> know where to find the models.  Everything just looks for the parts
> they need, relative to umbrella.  That's not magic, is it?

No, but it's only half- working.  Ever try a "syncdb" since you did
this rearrangement?  It won't find your models.  And if you try to use
the admin, it won't find them either.  Bottom line is, it's not
correctly configured.  A properly configured app needs to have a
models module, and needs to be in INSTALLED_APPS.  "umbrella" is not
an app, and your other apps aren't actually installed because they
aren't in INSTALLED_APPS.

> In any case, I will try the umbrella.* thing.  Will that try to put
> things like umbrella.urls and umbrella.templates into INSTALLED_APPS,
> too?

Assuming umbrella.urls is a urls.py file, not a urls/ package
directory, it won't try to load that (it only loads subdirectories).
I haven't tested, but it probably will try to load your templates
directory, and may fail somewhere down the line because it isn't a
Python module.  If you use umbrella.*, it's really not advisable to
have anything but the apps themselves in the umbrella directory.

Also, if the apps are really so tightly coupled that they could never
be used apart from each other, I'd question the value of having them
be separate apps at all.  It sounds more like you have one app.  If it
has too many e.g. models to want to put them all in one models.py, you
could look at splitting up your models.py into a models/ package.  The
point of splitting out separate reusable apps is that they address a
single well-defined concern, and you can actually reuse them
independently of each other.

Carl

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Multiple projects, model inheritance, and one common database

[Carl Meyer]

On Oct 8, 12:47 pm, barbara shaurette <[EMAIL PROTECTED]> wrote:
> We're building a couple of different projects - one social network-y
> site, and one that's bloglike.  So each has its own unique database,
> but they do share one common set of content.

You'll almost certainly have better luck having the two sites share
the same database, and use the Sites framework where needed to
distinguish content for only one or the other, than trying to hack
some solution with two separate DBs and a third shared one.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Branching proj/myapp into branch/0.1/myapp breaks

[Carl Meyer]

On Oct 8, 1:07 pm, Dan Ellis <[EMAIL PROTECTED]> wrote:
> So what about all those Django apps with hyphens in their names?

You won't find a Django app that asks you to put a directory with a
hyphen into your Python path.  Usually the pattern is for an app's
human-readable name to look like "django-tagging" or "django-
extensions".  The top-level directory will contain external stuff like
docs, setup.py, etc, and then contain a subdirectory like "tagging" or
"django_extensions" that you actually link into your Python site-
packages.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Branching proj/myapp into branch/0.1/myapp breaks

[Carl Meyer]

On Oct 8, 1:09 pm, Gerard Petersen <[EMAIL PROTECTED]> wrote:
> Can somebody tell me how one would best store django apps and projects when 
> starting branches and tags under svn?

If an app is reusable between projects I store it in its own separate
VC repo (bazaar, in my case).  If an app is special-purpose for one
project (which I avoid as much as possible), I put it in the project
directory.

> And how portable are apps between projects?

Depends how you write them :-)  Best practice is to write reusable
apps that live directly on the Python path and minimize dependencies
on a particular project or (as much as possible) other apps.

> So would tagging an app seperately from a project be a good idea?

Probably, if it's reusable.

> Or do you pro's only do a checkout of let's say the branch you're working on 
> instead of the 'full tree' with trunks, tags and branches?

Generally I would only checkout a single branch or tag (and use "svn
switch" to switch between branches).

HTH,

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Access a model's original get_absolute_url

[Carl Meyer]

On Oct 8, 2:30 pm, Erik Allik <[EMAIL PROTECTED]> wrote:
> I've thought about this but I'm not sure how. Is there a hook/signal  
> that gets fired when all models have been loaded but  
> ModelBase._prepare hasn't run?

No, there isn't, but it doesn't seem like you'd need that.  It seems
like listening for class_prepared would be one option.  There's no
need to do anything before ModelBase._prepare runs; the only thing
that does is handle looking up ABSOLUTE_URL_OVERRIDES, which you won't
be using anyway.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Password Change Usage

[Carl Meyer]

On Oct 8, 11:56 pm, "Chris Spencer" <[EMAIL PROTECTED]> wrote:
> Yeah, this has been a big let down. What I'd hoped would be a few
> minutes setting up a basic site framework has turned into several
> hours wasted on boilerplate. The lack of docs, templates, and strange
> bugs like these leads me to think that django-registration isn't fully
> baked. I'm simply writing my own at this point, since I can't find a
> single example through Google of someone who's used that portion of
> registration successfully.

django-registration works great for me out of the box, with all the
features.  You'll want the latest SVN version, as there are a number
of bugs fixed since the last release (0.6).

You also shouldn't need to write your own form, as the password_change
view (just like all the others) gives you a template variable "form"
that you can just render normally (with form.as_ul or siblings, or
just iterating over the fields).

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: newbie question about breadcrumbs

[Carl Meyer]

Hi Mirto,

On Oct 9, 9:04 am, Mirto Silvio Busico <[EMAIL PROTECTED]> wrote:
> I'm confused because the admin app seems unaware of living inside a
> project that is inside an apache site: the home link always point to the
> first admin app page

The admin breadcrumbs are never aware of anything outside the admin,
and that is by design.  In order to change that you'll need to do
quite a bit of poking around in the admin templates, override the
proper block(s) and do your own breadcrumb logic.

> In the myappX pages (tacking the breadcrumbs block from the base.html of
> the admin app) the construted breadcrumbs contain always 2 entries: an
> Home (pointing to / - so the apache site) an a not acrive entry with the
> page name.

The breadcrumbs for your own apps should not involve pulling anything
from the admin.  Just put the top two breadcrumbs (/ and /mysite) in
your site-wide base.html, then pull that into your app templates using
{{ block.super }} and append the app-specific breadcrumbs.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Password Change Usage

[Carl Meyer]

On Oct 10, 7:44 am, "Chris Spencer" <[EMAIL PROTECTED]> wrote:
> That might be the problem then. I dislike the default markup, so I'm
> using my own. How do you find if the form as errors? 
> Usinghttp://docs.djangoproject.com/en/dev/topics/forms/as a guide, I've
> tried using form.has_errors, form.errors and form.message.errors, but
> Registration doesn't seem to use any of those.

RegistrationForm is a normal Form instance.  Most errors will be
attached to specific fields, and thus will be in
form.field_name.errors, as documented.  Errors not associated with any
particular field are available from form.non_field_errors(), as
documented here [1].

HTH,
Carl

[1] http://docs.djangoproject.com/en/dev/ref/forms/validation/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: dynamic upcast

[Carl Meyer]

On Sep 12, 2:57 pm, dadapapa <[EMAIL PROTECTED]> wrote:
>     class BaseClass(models.Model) :
>         final_type = models.ForeignKey(ContentType)
>
>         def save(self,*args) :
>             self.final_type =
> ContentType.objects.get_for_model(type(self))
>             super(BaseClass,self)save(*args)
>
>         def cast(self) :
>             return
> self.final_type.get_object_for_this_type(id=self.id)
>
>     class DerivedClass(ParentClass) :
>         pass

Just ran into this issue myself (first time using non-abstract
inheritance).  The catch with this recipe is that you can never save a
derived object from a parent instance, or you break the final_type
field (it will then contain the parent class content type).

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Admin view - merge two tables? (potentially bad db design)

[Carl Meyer]

This database design will result in lots of joins, but it's perfectly
normalized and very flexible.  Fine if you expect low traffic and you
need the flexibility.

The other downside is the extra administration complexity, which is
the meat of your question.  Fortunately Django's admin can help you
out, and it's quite easy: look into inlines[1].

Carl

[1] 
http://docs.djangoproject.com/en/dev/ref/contrib/admin/#inlinemodeladmin-objects
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: object_detail 404 (queryset outdated?)

[Carl Meyer]

Hi Simon,

On Feb 2, 7:36 am, Simon Westphahl  wrote:
> ###
> class OfferManager(models.Manager):
>     def get_query_set(self):
>         return super(OfferManager, self).get_query_set().exclude
> (start_date__gt=datetime.now).exclude(end_date__lt=datetime.now)
> ###
>
> I'm using a queryset filtered with a callable (datetime.now) which
> appears to be set only once/on load time?!

I think you've got the right idea of what's going wrong, but you're
looking for the problem in the wrong place.  This method isn't module-
level code; it will be executed anew each time you query on this
manager.  More likely is that you're storing a reference to the
returned queryset in module-level code; probably in a dictionary of
kwargs you're passing to the object_detail generic view (most likely
in your urls.py).  It's that queryset that is never getting updated;
the call to your manager method is happening only once, when the
urls.py module is first loaded.

The solution is to wrap the object_detail view with a very lightweight
view of your own that calls the manager method and passes the
resulting queryset in to object_detail.  Then the queryset will be re-
created for each request.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Admin view - merge two tables? (potentially bad db design)

[Carl Meyer]

On Feb 2, 6:44 pm, Zbigniew Braniecki 
wrote:
> How can I overwrite the way Questions are selected so that when they
> are returned by Question.objects.get('x') or Question.objects.filter()
> or Question.objects.all() they are already joined with
> QuestionProperties values?

Try select_related() [1].

Carl

[1] http://docs.djangoproject.com/en/dev/ref/models/querysets/#id4
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: auth.contrib silently catching TypeError. Bug?

[Carl Meyer]

Hi Tamas,

On Apr 23, 3:25 am, Tamas Szabo  wrote:
> For example if I have a Token based authenticator
> (authenticate(token)) that is configured to be after my User/Password
> based authenticator (authenticate(user, password)) in settings.py the
> current code will call the user/password based authenticator and it
> will pass in the token as a username (password will be null). If I
> have an authenticator with 2 string arguments, the same will happen.

I don't think you're understanding how keyword arguments work in
Python.  Python automatically raises a TypeError if the wrong keyword
arguments are passed to a function; based on argument name, not type.
So if "username" and "password" are passed to authenticate(), your
hypothetical "authenticator with 2 string arguments" will still raise
a TypeError unless its two arguments are actually named "username" and
"password" - in which case they should be exactly what you're
expecting.  And if only "token" is passed to authenticate(), calling
the standard auth backend will raise TypeError, it won't "pass in the
token as username" at all.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: How to make a foreign key field of type other than int?

[Carl Meyer]

Hi,

On Oct 24, 10:53 am, Kayode Odeyemi  wrote:
> I'm sorry but it doesn't just seem to work for me. I have tried
>
> possibilities like:
>
> self.fields['branch_name'].choices =
> Branch.objects.all().values('name_branch', 'name_branch')
> self.fields['branch_name'].choices =
> Branch.objects.all().values('name_branch')

I don't think you've read Tom's responses carefully. You're still
trying to use ".values()" here, resulting in a ValuesQuerySet which
returns dictionaries instead of a regular QuerySet which returns model
objects. That simply won't ever work. And that is the reason for the
"need more than one value to unpack" error; it has to do with the
format of the results returned, nothing to do with how many results
are returned by the query.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ORM help

[Carl Meyer]

Hi Daniel,

On Apr 22, 5:57 pm, Daniel Gerzo  wrote:
> I have a following models:
>
> class Movie(models.Model):
>      title = models.CharField(max_length=255)
>      year = models.IntegerField(max_length=4)
>      rating = models.DecimalField(max_digits=2, decimal_places=1, default=0)
>
> class Request(models.Model):
>      movie = models.ForeignKey(Movie)
>      user = models.ForeignKey(User)
>      language = models.CharField(max_length=7, db_index=True)
>      notes = models.CharField(max_length=255, blank=True, null=True)
>      added = models.DateTimeField(default=datetime.now)
>
> I want to get all Requests, with count per each pair of Movie and language.
>
> So far I got this:
> Request.objects.values('movie', 'language').annotate(Count('language'))
>
> That seems to give me almost what I want, but that way I get a
> ValueQuerySet. Is there a way to get or convert this to a QuerySet? I'd
> like to get an easy way to access Movie attributes from the above query
> as I will need to display movie title and rating in the template for
> each result from the above query.
>
> So, is there some better way of what am I trying to accomplish? If not,
> how should I proceed in retrieving the Movie attributes? Will I need to
> fetch those with something like Movie.objects.filter(pk__in=[list of
> ids]) and then manually map those?

That's one way, if you actually get full Movie objects out. However,
if you just need certain fields from the movie, you should be able to
do this:

Request.objects.values("movie__title",
"language").annotate(Count("language"))

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Tests and temp tables

[Carl Meyer]

Hi Larry,

On 02/05/2015 06:57 AM, Larry Martell wrote:
> On Thu, Feb 5, 2015 at 7:29 AM, Jani Tiainen  wrote:
>> On Tue, 3 Feb 2015 19:38:31 -0500
>> Larry Martell  wrote:
>>
>>> I have a django app that uses a temp table. In the real world this is
>>> no issue, as each invocation of the app runs in its own MySQL session
>>> so there cannot be any conflict with the temp tables. But in my tests
>>> there are multiple requests sent, and apparently they are all in the
>>> same session, as on the second request I get an error because the temp
>>> table already exists. I tried logging out between requests, and I
>>> tried creating a new Client instance for each request, but I still got
>>> the error. Then I tried deleting the Client object, but I got Client
>>> object has no attribute __del__.
>>>
>>> What I can do so that each request in a test has its own MySQL session?
>>
>>
>> Instead of Django standard TestCase (which internally wraps all in single 
>> transaction and makes transactions as a no-op), you should use 
>> TransactionalTestCase.
>>
>> https://docs.djangoproject.com/en/1.7/topics/testing/tools/#transactiontestcase
> 
> Thanks for the reply Jani. We're using 1.5 and I don't think this is
> available in that version. We'll probably be moving to 1.6 soon
> though, and it is there.

TransactionTestCase has been around for quite a long time (since 1.1,
IIRC). It's definitely in 1.5.

> But I'm not sure how this will be useful to
> me. The docs say "A TransactionTestCase resets the database after the
> test runs by truncating all tables."  My test code is something like
> this:
> 
> # load test data
> # login
> response = self.client.get('...')
> self.assertEqual(response.status_code, 200)
> # collect results
> 
> response = self.client.get('...')
> self.assertEqual(response.status_code, 200)
> # collect results
> 
> and so on. I need the temp table to get dropped between each get call.
> The best way to do this is if each is in its own MySQL session. Having
> the table truncated doesn't really accomplish this. I think I will
> have to modify the code to drop the temp table after it's used. I hate
> to muck with working production code to accommodate a test, but I also
> don't want the test branch to have a different code base from the
> production branch. I'll probably go with the former option.

I agree, I don't think TransactionTestCase will help with this situation.

I find the production design a bit questionable, but taking it as a
given that it meets your needs adequately and you don't want to change
it, you could also address this problem by simply dropping the temp
table in a tearDown() method, no?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54D39205.5090800%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Tests and temp tables

[Carl Meyer]

On 02/05/2015 11:20 AM, Larry Martell wrote:
> On Thu, Feb 5, 2015 at 10:53 AM, Carl Meyer  wrote:
>> TransactionTestCase has been around for quite a long time (since 1.1,
>> IIRC). It's definitely in 1.5.
> 
> I thought it was not in 1.5 because I went to
> https://docs.djangoproject.com/en/1.5/topics/testing/tools/#transactiontestcase
> and got a 404.

Ah, yeah; I think the docs just moved around.

>>> and so on. I need the temp table to get dropped between each get call.
>>> The best way to do this is if each is in its own MySQL session. Having
>>> the table truncated doesn't really accomplish this. I think I will
>>> have to modify the code to drop the temp table after it's used. I hate
>>> to muck with working production code to accommodate a test, but I also
>>> don't want the test branch to have a different code base from the
>>> production branch. I'll probably go with the former option.
>>
>> I agree, I don't think TransactionTestCase will help with this situation.
>>
>> I find the production design a bit questionable,
> 
> Why do you say that? I recently added the temp table for performance.
> I have a query that was taking over an hour. When I broke it into 2
> queries using a temp table it then ran in 10 seconds. If I've
> introduced something bad in my app, I'd really like to know about
> that.

I don't know if it's bad or not, given your needs, but introducing the
constraint "every request must run in its own MySQL session which is
shut down after the request" would give me pause, since reusing
connections (using the built-in Django connection-reuse feature, which
isn't in 1.5 yet I don't think, or using an external connection pooler)
is often a quick and easy performance win. And because having every
request leave the connection in an unusable state makes testing
harder/slower, too. It just feels wrong :-)

I don't know exactly why restructuring the query made it faster. In
PostgreSQL I would usually use a CTE (WITH clause) to allow the same
type of query decomposition without needing an actual temp table; not
sure if MySQL supports those. (I recently reworked an ORM query using
joins to raw SQL with CTEs and saw a similar many-orders-of-magnitude
performance improvement, because the version with joins was internally
generating a result-set that grew with the square of the size of a
particular table.) But if the temp table is the only approach that
works, I would probably prefer to have my production code clean it up to
make session reuse possible. (Ideally, by encapsulating the creation and
destruction of the temp table in a database stored procedure, so it
becomes what it ought to be, a query implementation detail, not
something that application logic needs to be concerned about.)

>>  but taking it as a
>> given that it meets your needs adequately and you don't want to change
>> it, you could also address this problem by simply dropping the temp
>> table in a tearDown() method, no?
> 
> A tearDown() method would be run after the test, right? I am sending
> multiple requests within one test.

Can you restructure your tests to avoid that? The more focused a test
is, usually the better.

If not, then I think you just need a utility method to destroy the temp
table, which you either call explicitly from your tests after each
request, or perhaps build into a subclass of the test client so it's
automated. Although a bit ugly, I think that's still better than trying
to reconnect to the database for each test; that'll kill your
test-running speed. (But I'd still prefer fixing the problem at the source.)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54D3CFA0.2070101%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Logging transaction statements?

[Carl Meyer]

Hi Erik,

On 02/10/2015 10:28 AM, Erik Cederstrand wrote:
> Hmm, I'm beginning to think I don't understand Django transactions. I
> have the following two snippets, boiled down from my original code. The
> first one calls cache methods via helper functions, the other one calls
> the cache methods directly. If I run the code in parallel in two
> different processes, then the first example asserts almost instantly,
> while the second example survives several minutes. Can someone explain this?

I assume you're using the 'db' cache backend? Otherwise, it wouldn't
make sense to expect transactions to affect cache calls at all.

> import random
> from django.db import transaction
> from django.core.cache import cache
> 
> def get_val(key):
> cache.get(key)
> 
> def set_val(key, val):
> cache.add(key, val)
> 
> def del_val(key):
> cache.delete(key)
> 
> 
> # First example, fails
> while True:
> with transaction.atomic():
> if not get_val('foo'):
> print('no key found')
> time.sleep(random.random())
> if set_val('foo', 'bar'):
> print('key added')
> time.sleep(random.random())
> else:
> assert False
> del_val('foo')
> print('key deleted')
> time.sleep(random.random())
> 
> 
> # Second example, runs correctly
> while True:
> with transaction.atomic():
> if not cache.get('foo'):
> print('no key found')
> time.sleep(random.random())
> if cache.add('foo', 'bar'):
> print('key added')
> time.sleep(random.random())
> else:
> assert False
> cache.delete('foo')
> print('key deleted')
> time.sleep(random.random())

The difference between the snippets is that you are doing a conditional
branch based on the return value of the cache add, but your set_val
helper method always returns None, it doesn't return the return value of
cache.add().

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54DA4136.8020105%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Logging transaction statements?

[Carl Meyer]

On 02/10/2015 01:27 PM, Erik Cederstrand wrote:
>> Den 10/02/2015 kl. 18.34 skrev Carl Meyer > <mailto:c...@oddbird.net>>:
>>
>> I assume you're using the 'db' cache backend? Otherwise, it wouldn't
>> make sense to expect transactions to affect cache calls at all.
> 
> Yes, I'm using the db backend.
> 
>> The difference between the snippets is that you are doing a conditional
>> branch based on the return value of the cache add, but your set_val
>> helper method always returns None, it doesn't return the return value of
>> cache.add().
> 
> Wow, *that* was embarrassing :-) You see what you want to see, I guess...
> 
> I'm still having issues with this, though. I reduced my own code to the
> following. The intent of the code is to use the cache to implement
> multiprocess/multiserver locking:
> 
> from django.db import transaction
> from django.core.cache import cache
> 
> class CacheError(Exception):
> pass
> 
> cache.delete('foo')
> while True:
> print('Trying to cache foo')
> try:
> with transaction.atomic():
> if cache.get('foo'):
> raise CacheError()
> print('adding foo to cache')
> assert cache.add(key='foo', value='bar')
> print('foo cached')
> time.sleep(random.random())
> with transaction.atomic():
> if cache.get('foo'):
> cache.delete('foo')
> except CacheError:
> print('Failed to cache foo')
> time.sleep(random.random())
> 
> 
> Running this in parallel in two processes on the same machine returns
> this after a while:
> 
> Process A:
> Trying to cache foo
> 2015-02-10 21:02:25,781 DEBUG(0.001) SELECT cache_key, value,
> expires FROM "dispatch_cache" WHERE cache_key = 'foo'; args=['foo']
> adding foo to cache
> 2015-02-10 21:02:25,782 DEBUG(0.000) SELECT COUNT(*) FROM
> "dispatch_cache"; args=None
> 2015-02-10 21:02:25,782 DEBUG(0.000) SAVEPOINT
> "s140735261451008_x1"; args=None
> 2015-02-10 21:02:25,783 DEBUG(0.000) SELECT cache_key, expires FROM
> "dispatch_cache" WHERE cache_key = 'foo'; args=['foo']
> 2015-02-10 21:02:25,784 DEBUG(0.000) INSERT INTO "dispatch_cache"
> (cache_key, value, expires) VALUES ('foo', 'gASVBwCMA2JhcpQu',
> '-12-31 23:59:59'); args=['foo', 'gASVBwCMA2JhcpQu',
> '-12-31 23:59:59']
> 2015-02-10 21:02:25,784 DEBUG(0.000) RELEASE SAVEPOINT
> "s140735261451008_x1"; args=None
> foo cached
> 2015-02-10 21:02:26,771 DEBUG(0.000) SELECT cache_key, value,
> expires FROM "dispatch_cache" WHERE cache_key = 'foo'; args=['foo']
> 2015-02-10 21:02:26,772 DEBUG(0.000) DELETE FROM "dispatch_cache"
> WHERE cache_key = 'foo'; args=['foo']
> 
> 
> Process B:
> Trying to cache foo
> 2015-02-10 21:02:25,782 DEBUG(0.000) SELECT cache_key, value,
> expires FROM "dispatch_cache" WHERE cache_key = 'foo'; args=['foo']
> adding foo to cache
> 2015-02-10 21:02:25,783 DEBUG(0.000) SELECT COUNT(*) FROM
> "dispatch_cache"; args=None
> 2015-02-10 21:02:25,784 DEBUG(0.000) SAVEPOINT
> "s140735261451008_x1"; args=None
> 2015-02-10 21:02:25,784 DEBUG(0.000) SELECT cache_key, expires FROM
> "dispatch_cache" WHERE cache_key = 'foo'; args=['foo']
> 2015-02-10 21:02:25,791 DEBUG(0.007) INSERT INTO "dispatch_cache"
> (cache_key, value, expires) VALUES ('foo', 'gASVBwCMA2JhcpQu',
> '-12-31 23:59:59'); args=['foo', 'gASVBwCMA2JhcpQu',
> '-12-31 23:59:59']
> 2015-02-10 21:02:25,792 DEBUG(0.000) ROLLBACK TO SAVEPOINT
> "s140735261451008_x1"; args=None
> Traceback (most recent call last):
>   File "tmp.py", line 30, in 
> assert cache.add(key='foo', value='bar')
> AssertionError
> 
> 
> I don't see how this is possible when I'm using transaction.atomic().

I believe you're running into the difference between the transaction
isolation levels REPEATABLE READ vs READ COMMITTED. The former is the
default for MySQL, the latter for PostgreSQL.

In REPEATABLE READ isolation level, once you enter a transaction your
view of the world will never change, except due to your own actions,
until the end of that t

Re: Logging transaction statements?

[Carl Meyer]

On 02/10/2015 02:46 PM, Carl Meyer wrote:
> On 02/10/2015 01:27 PM, Erik Cederstrand wrote:
>>> Den 10/02/2015 kl. 18.34 skrev Carl Meyer >> <mailto:c...@oddbird.net>>:
>>>
>>> I assume you're using the 'db' cache backend? Otherwise, it wouldn't
>>> make sense to expect transactions to affect cache calls at all.
>>
>> Yes, I'm using the db backend.
>>
>>> The difference between the snippets is that you are doing a conditional
>>> branch based on the return value of the cache add, but your set_val
>>> helper method always returns None, it doesn't return the return value of
>>> cache.add().
>>
>> Wow, *that* was embarrassing :-) You see what you want to see, I guess...
>>
>> I'm still having issues with this, though. I reduced my own code to the
>> following. The intent of the code is to use the cache to implement
>> multiprocess/multiserver locking:
>>
>> from django.db import transaction
>> from django.core.cache import cache
>>
>> class CacheError(Exception):
>> pass
>>
>> cache.delete('foo')
>> while True:
>> print('Trying to cache foo')
>> try:
>> with transaction.atomic():
>> if cache.get('foo'):
>> raise CacheError()
>> print('adding foo to cache')
>> assert cache.add(key='foo', value='bar')
>> print('foo cached')
>> time.sleep(random.random())
>> with transaction.atomic():
>> if cache.get('foo'):
>> cache.delete('foo')
>> except CacheError:
>> print('Failed to cache foo')
>> time.sleep(random.random())

You might find it useful to look at the implementation of Django's
QuerySet.get_or_create() method, if you want to see how to (mostly)
safely implement a get-or-create pattern like this without race
conditions under READ COMMITTED isolation level.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54DA7D14.9030208%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Logging transaction statements?

[Carl Meyer]

Hi Erik,

On 02/10/2015 03:30 PM, Erik Cederstrand wrote:
> Den 10/02/2015 kl. 21.27 skrev Erik Cederstrand
> :
>> 
>> Running this in parallel in two processes on the same machine
>> returns this after a while:
>> 
>> Process A: [...]
>> 
>> Process B: [...] Traceback (most recent call last): File "tmp.py",
>> line 30, in  assert cache.add(key='foo', value='bar') 
>> AssertionError
>> 
>> 
>> I don't see how this is possible when I'm using
>> transaction.atomic().
> 
> Phew. This situation is actually possible since "Read Committed" is
> the default isolation level in PostgreSQL, which means that
> non-repeatable reads are possible within a transaction.

Yes, that's what I said :-)

> My code relies on isolation level "Serializable" for a cache.get()
> followed by cache.add() to be reliable, but Django uses the default
> from PostgreSQL.

I don't think you need full Serializable, Repeatable Read should be
sufficient. Serializable is dangerous, has a tendency to cause deadlocks
(as I think you've discovered). Even with Repeatable Read, you'll need
be a bit careful about any long-running processes that might hold open a
transaction.

> The isolation level is supposed to be configurable with the
> 'isolation_level' setting in OPTIONS
> (https://docs.djangoproject.com/en/1.7/ref/databases/#isolation-level).
> Except it doesn't work because
> django/db/backends/postgresql_psycopg2/base.py::_set_isolation_level()
> is never called anywhere, AFAICS.

But `self.connection.set_isolation_level()` is called in
`_set_autocommit()`, and that should be sufficient. (The
`_set_isolation_level()` method is dead code in 1.7 and has since been
removed, but what really matters is that
`self.connection.set_isolation_level()` is called.)

> I tried disabling autocommit (the docs are wrong BTW,
> https://docs.djangoproject.com/en/1.7/ref/databases/#autocommit-mode
> says to put it in OPTIONS, but django/db/backends/__init__.py (line
> 123) looks at settings_dict['AUTOCOMMIT'], not in OPTIONS)

OPTIONS['autocommit'] has been ignored since Django 1.6, as the docs you
linked clearly state: "This configuration is ignored and can be safely
removed."

You're looking for
https://docs.djangoproject.com/en/1.7/ref/settings/#autocommit instead.

(It would be good for the former to have a link to the latter, though.)

 and
> hacking postgresql_psycopg2/base.py::_set_autocommit() to call
> _set_isolation_level(), but now process A blocks process B entirely,
> even when process A is outside the atomic() context manager.
> 
> Has anyone got isolation_level "Serializable" to work in Django 1.7?

I strongly recommend against using the Serializable isolation level, or
turning off autocommit mode. The transaction.atomic() API will not work
correctly or predictably with autocommit turned off. See
https://docs.djangoproject.com/en/1.7/topics/db/transactions/#deactivate-transaction-management

You could switch to Repeatable Read (but leave AUTOCOMMIT on). Or (what
I would probably do) you could leave the isolation level at the default
(since it's the default for Django, some parts of Django, such as
QuerySet.get_or_create(), may not work correctly under a different
isolation level) and simply update your locking logic to guard against
that race condition (if the add fails, assume that another process has
grabbed the lock in the interim).

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54DA8A07.9020704%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Logging transaction statements?

[Carl Meyer]

On 02/10/2015 04:03 PM, Erik Cederstrand wrote:
>> But `self.connection.set_isolation_level()` is called in 
>> `_set_autocommit()`, and that should be sufficient. (The 
>> `_set_isolation_level()` method is dead code in 1.7 and has since
>> been removed, but what really matters is that 
>> `self.connection.set_isolation_level()` is called.)
> 
> It's only called if self.psycopg2_version < (2, 4, 2), at least in
> the file I'm looking at. My version is 2.5.2.

You're totally right. That's a bug. Thanks for catching it! I've filed
https://code.djangoproject.com/ticket/24318

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54DA969C.7040509%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: how to handle race conditions?

[Carl Meyer]

Hi Abraham,

On 02/17/2015 10:01 PM, Abraham Varricatt wrote:
> I'm trying to make an app where folks can order X quantity of an item.
> The condition is that the order should only be made if inventory exists.
> Assume that we have stock of Y items. This means that only if Y >= X
> should we allow the sale to go through. And once we accept an order, the
> inventory should be updated so that Y = Y - X. 

In general, the construct you need is called a "transaction", which
ensures that a series of database operations will either all be
committed together, or rolled back if they can't be successfully
completed. The Django API for that is "django.db.transaction.atomic".

> The way I've currently implemented this is with a pair of getter/setter
> methods. I have a model 'ShopItem' which has two fields called
> 'quantity' and 'name' (this is unique). I've made a utility class
> (outside model source file) where I've made the following 2 functions;
> 
> def get_quantity(name):
>   stuff_left = 0
>   try:
> item = ShopItem.objects.get(name=name)
> stuff_left = item.quantity
>   except ShopItem.DoesNotExist:
> pass # return zero
>   return stuff_left
> 
> def set_quantity(name, stuff_left):
>   item = ShopItem.objects.get(name=name)
>   item.quantity = stuff_left
>   item.save()
> 
> 
> Elsewhere in my project, if I need to display the remaining quantity of
> an item in a view, I'll pass the result from get_quantity(). At sale
> time here is how I update things;
> 
> def customer_buy(name, number):
>   temp = get_quantity(name)
>   if (temp >= number):
> temp = temp - number
> set_quantity(name, temp)
> // do other things
>   else:
> // sale failed, do something else
> 
> I tested this on my system and things appear to work. But I'm concerned
> about race conditions. What if two different customers came in to buy
> the same item? As in; if I only have 7 items in stock, but one customer
> came to buy 6nos and another 5nos. There is a chance (as per my
> understanding) that both orders will be accepted - even worse, my
> inventory will not accurately reflect the updated situation. 
> 
> Any ideas on how this issue can be resolved? I've come across this
> - 
> https://docs.djangoproject.com/en/1.6/ref/models/instances/#updating-attributes-based-on-existing-fields
> , but not sure how to apply it to the current scenario. 

If we're only considering the changes to your ShopItem model, you don't
even need an explicit transaction to avoid race conditions, because (as
the docs you linked show) the operation can be completed in a single
database query, which is inherently atomic. This is how it would look to
do it in a single query:

from django.db.models import F

def customer_buy(name, number):
ShopItem.objects.filter(
name=name).update(quantity=F('quantity')-number)

You want this to fail if someone tries to purchase a larger quantity
than are available. The best way to do this is via a database-level
"check constraint" on the column, such that the database itself will
never permit a negative quantity. If you make 'quantity' a
PositiveIntegerField (the name is wrong, it actually allows zero too) on
your model, and your database is PostgreSQL (or Oracle), Django will add
this constraint for you automatically. Then if someone tries to purchase
more than are available, you'll get an IntegrityError, which you'd want
to catch and handle in some way:

from django.db import IntegrityError
from django.db.models import F

class InsufficientInventory(Exception):
pass

def customer_buy(name, number):
try:
ShopItem.objects.filter(
name=name).update(quantity=F('quantity')-number)
except IntegrityError:
# signal to the calling code that the purchase failed - the
# calling code should catch this exception and notify the
# user that the purchase failed due to lack of inventory,
# and tell them the updated available quantity
raise InsufficientInventory()

You also want this function to handle the case where the given product
name doesn't exist. To help with this case, the `update` method returns
the number of rows updated:

from django.db import IntegrityError
from django.db.models import F

class InsufficientInventory(Exception):
pass

def customer_buy(name, number):
try:
updated = ShopItem.objects.filter(
name=name).update(quantity=F('quantity')-number)
except IntegrityError:
raise InsufficientInventory()
if not updated:
# Here we reuse Django's built-in DoesNotExist exception;
# you could define your own exception class instead.
raise ShopItem.DoesNotExist()

With this code, you've solved the bad race conditions -- quantity will
never go negative, and a sale will never appear to succeed when it
should have failed, because of two user

Re: how to handle race conditions?

[Carl Meyer]

Hi Abraham,

On 02/19/2015 03:41 AM, Abraham Varricatt wrote:
> I've heard of the term 'atomic operations' , but didn't realize it had
> support in django. Thank you for the detailed explanation!

You're welcome.

> It's an interesting idea - to mark the field in the database as a
> positive integer, but since it doesn't apply to all DBs, I think I'll
> keep it aside for now. 

This is a bit of a tangent, but when you get to the point of worrying
about things like race conditions and transactional behavior, your life
will be easier if you simply choose a database for your project and
don't worry about making everything cross-database portable.

And, frankly, if you're concerned about data integrity, your life will
be easier if the database server you choose is PostgreSQL.

Your ability to use Django effectively is much greater if you know your
specific database well, and aren't afraid to use database-specific
features and write some raw SQL now and then. Limiting yourself to
cross-database-portable ORM features is a significant handicap, one
which I think only separately-distributed reusable apps should take on.

In your particular case, without a database-level check constraint on
the quantity field, it becomes much trickier to ensure that quantity
can't go negative.

> @Waitman,
> 
> Don't think I've ever come across the term 'backorder', but its an
> interesting concept. And very appreciated, from the business angle. I
> need to think more on this.

Yes - I should have said more clearly that my answer was intended as a
primer on basic atomicity and locking concepts, not a treatise on
shopping cart user experience. It's quite likely that for a production
shopping cart you'd want significantly more sophisticated behavior than
I covered.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54E61D6D.2030006%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Question about form security.

[Carl Meyer]

On 02/19/2015 04:03 PM, Chen Xu wrote:
> what if I have two addresses, id is 1 and 2, and I want to update 2, how
> I should distinguish 2 from 1, I mean, if I have addressid in the form,
> and someone modify it from 2 to 1, and that will update 1, right? How to
> prevent this?

Your server-side code must handle all security-related validation, and
must assume that no data sent from the client can be trusted. So in this
case you describe, your server-side code must have some way to know
which address(es) the currently logged-in user has the authority to edit.

Usually in this type of case I would make the address ID being edited a
part of the URL, not a form parameter (because it's used to query the
instance being edited by the ModelForm), and I would perform the access
check at the very top of the view function, often returning a 404 if the
user should not have access to that address.

If for some reason the address ID must be in the form rather than a URL
parameter, you could pass `request.user` to the form and have the form
validation check that that user is permitted to edit the given address ID.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54E66EC8.70808%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrations force me to have, forever, any name/reference I use in field validators=, choices=, whatever, valid forever (even if the requirements change).

[Carl Meyer]

On 02/26/2015 08:41 AM, Carsten Fuchs wrote:
> Hi all,
> 
> Am 26.02.2015 um 13:54 schrieb Tim Graham:
>> Yes, it's expected behavior. Please see the documentation on the topic:
>> https://docs.djangoproject.com/en/stable/topics/migrations/#historical-models
>>
> 
> I have not yet tried this, but won't squashing migrations as a side
> effect also get us rid of dependencies of historical models?

Yes, squashmigrations is the right way to deal with this problem.

If you have RunPython/RunSQL migrations that can also safely go away,
you'll need to manually excise them before squashing in order to get a
complete squash, since they are opaque to squashmigrations and won't be
optimized across. See https://code.djangoproject.com/ticket/24109

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54EF40B4.6020305%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Problem installing and using smart_selects

[Carl Meyer]

Hi Michael,

On 03/01/2015 01:11 AM, Michael Ackary wrote:
> Smart Selects looked so simple and promising for chained selects
> (https://github.com/digi604/django-smart-selects).  However, no matter
> how I install it, it doesn't seem to work.  Here is what I've tried:
> 
> 
> *Attempt #1 - putting it in django.contrib
> *
> Copied the smart_selects folder to /django/contrib/smart_selects
> 
> Added "django.contrib.smart_selects" to INSTALLED_APPS in settings.py
> 
> But within Django shell I got this error:
> 
  from django.contrib import smart_selects
  smart_selects.urls 
> AttributeError: 'module' object has no attribute 'urls'

You don't ever want to do this. The "django.contrib" module is for
modules distributed with Django; it's not a place to add your own
additional code.

It also doesn't work, because smart_select expects to be able to import
itself as "smart_select", not as "django.contrib.smart_select."

> *Attempt #2 - installed using pip
> *
> I removed the smart_selects folder from /django/contrib  folder (above)
> and then installed using pip... which put smart_selects in the Python
> site-packages folder.
> 
> But then trying to use smart_selects within urls.py produced an ugly
> "ImportError: cannot import name simplejson" which suggests
> smart_selects has some outdated dependencies. (I"m using Django 1.7).

This is the right way to make use of third-party code. The problem here,
as you say, is just that smart_select has some outdated code. It's
actually been fixed in master, just not posted to PyPI yet, so you'll
need to install it directly from git, as mentioned in the fourth comment
here: https://github.com/digi604/django-smart-selects/issues/70

> *Attemp #3 - putting smart_packages in the root folder of my Django project
> *
> When smart_selects is placed here, again I can import smart_selects from
> the Django shell  but not access smart-selects.urls :
> 
> Type "help", "copyright", "credits" or "license" for more information.
> (InteractiveConsole)
 import smart_selects
 dir(smart_selects)
> ['__builtins__', '__doc__', '__file__', '__name__', '__package__',
> '__path__', 'models']
 smart_selects.urls
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: 'module' object has no attribute 'urls'
> 
> 
> Finally it works if i do this:
> 
 import smart_selects.urls
 smart_selects.urls
>  '/Users/chas/Projects/switch/djangoroot/smart_selects/urls.pyc'>

The difference between the "doesn't work" and "works" versions here
doesn't have anything to do with how you've installed the code, it's
simply how submodule imports work in Python. Submodules are never
automatically imported unless you import them explicitly.

> However, I really don't like putting apps here since they clutter up my
> own development (and would rather keep 3rd party apps in django/contrib). 

Yes, this installation technique will work fine, but it clutters things
up and makes it harder to upgrade in the future.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54F3C467.7080203%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Fixing a poorly written Django website (no unit tests)

[Carl Meyer]

On 03/06/2015 11:23 AM, Ilya Kazakevich wrote:
> You may start from highest level testing: 
> 1) create "usage scenarios" for your website. Like "customer opens page
> 'foo', and should see 'bar'". You use such scenarios for manual testing,
> right?
> 2) code such scenarios against Django project. You may use BDD testing
> tools (like lettuce or behave), or do it in pure python. You may call
> Django views and templates directly (using django unittesting support or
> lettuce/harvest), or do it through the browser using Selenium. One
> scenario -- one method. Yes, there is a lot of boilerplate code, but you
> only need to write it once. BDD tools are used to simplify this process
> by writing scenarios on DSL called Gherkin, which is easier. 
> 
> This is some kind of "acceptance testing", the one that helps you to be
> sure website works like customer expects it to work. Every project
> should have one, I believe.
> 
> After it, you may find that some units of your system are complex and
> error prone. What is unit? Unit is module, function or even package with
> _clear interface_. You then create unit tests against this module. Only
> units with complex logic need to be tested. No need to test simple view
> that just returns render(): this view is tested as part of high level
> testing. But if you have function "calc_user_amount" and complex
> business logic stands behind it, you may create unit test for it.
> 
> There are 3 mistakes people do about testing:
> 1) Testing each function, even private function, even 1 line function.
> It takes a lot of time, and such tests are fragile. You throw'em away
> after simple refactoring.
> 2) Testing "in the middle of abstraction": I mean testing functions with
> out of clear interface, like private functions. If you need to read
> function code before writing test (pydoc is not enough), you should not
> test this function. Try a higher level of abstraction.
> 3) Skipping high level testing: even if all your code is covered with
> low-level unit-tests, you still need high level testing like the one
> with Selenium to make sure everything works correctly, and when you
> refactor your code you use such testing to make sure you did not break
> anything.
> 
> So, you start with high level, and then cover _some_ units with unit
> test, if you believe they may contain error.

This is really excellent advice.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54F9FA6E.8050001%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Using the eval() command

[Carl Meyer]

Hi Henry,

On 03/10/2015 03:25 PM, Henry Versemann wrote:
> I have a new dictionary that I want to build, using data from another
> dictionary. I have a view which is receiving a single key/value pair
> from the original dictionary. Then in the view I've defined the new
> dictionary like this:
> 
> innerDict = {}  
> 
> Now I want to make this as dynamic as possible so I'm trying to use the
> "eval()" statement below to add the new key/value pair to the new
> dictionary, which is declared above. Will the following code work to
> actually add the new key/value pair to the new dictionary?
> 
> innrDictCmnd = "innerDict['"+newinnrkey+"'] = newinnrval"
> eval(innrDictCmnd)
> 
> If not why not, and in lieu of the statements above not working, then
> how would I do it?

It doesn't work, because eval() only accepts expressions; assignment is
a statement. Using exec() instead of eval() will work (though the way
you have it written, it will always assign the string "newinnrval" --
perhaps you meant to end innrDictCmnd with '... = ' + newinnrval).

But regardless, you should not use either eval() or exec().

Since you say this code is in a view, I assume that newinnrkey comes
from request data (user input). Imagine what happens if I am a malicious
user and I call this view with newinnrkey set to:

'] = 0; import os; os.rm('/'); d = {}; d['

Oops.

Both exec() and eval() should be avoided. They are very rarely
necessary, they usually make code less readable and maintainable, and if
you ever accidentally pass user input to them, you've opened up a
security hole in your application that someone could drive a truck through.

For your case, what's wrong with just writing `innerDict[newinnerkey] =
newinnerval`? It's every bit as dynamic as the version using eval or
exec - the eval/exec gains you nothing.

Carl


-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54FF66E3.5050408%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django 1.6's lifespan for security updates?

[Carl Meyer]

Hi Christian,

On 03/10/2015 01:44 PM, Christian Hammond wrote:
> According to
> https://docs.djangoproject.com/en/1.7/internals/release-process/#supported-versions,
> it appears that when Django 1.8 is released, Django 1.6 will no longer
> receive security updates. I wanted to verify that this is true, and ask
> whether there's a possibility of an extension on this timeframe.

It is true, and I don't think it should be extended.

> I'll explain the situation we're in.
> 
> I manage Review Board, a code review tool currently in use by several
> thousand companies/organizations, many of whom (according to stats we
> have available) are on Python 2.6. From conversations we've had, many of
> these companies are on LTS releases of Linux distributions that bundle
> Python 2.6 by default (including their mod_wsgi support, etc.), and are
> likely to remain on it for the next year or two. Not to mention Amazon
> Linux and other variants are all sticking with 2.6 for now as well.
> 
> This puts us in a difficult position where we are unable to drop Python
> 2.6 support without affecting a large number of installs out there (12%
> of our base, or over 700 installs), meaning we haven't yet been able to
> make the transition to Django 1.7/1.8 (as much as we want it). (It also
> makes the lives of packagers easier who are trying to support software
> stuck in this situation, from what I'm being told, as they're
> responsible for security updates.)
> 
> As Django 1.6 is the last release to support Python 2.6, it would be
> very nice to have a longer-term security release plan while companies
> transition over. We see this happening, but slowly.
> 
> Is there any possibility of treating Django 1.6 as a special LTS release?

I sympathize with your situation, but Python 2.6 reached end-of-life on
October 29, 2013 (a year and a half ago now), and since then has been
unsupported and not receiving security updates. I don't think the Django
core team should set a precedent of extended support for Python versions
which are themselves unsupported by the core Python developers.

If some Linux distributions are backporting Python security patches to
2.6 themselves in order to extend its lifetime in their distribution,
perhaps it would make sense to ask them whether they will also backport
Django security patches to Django 1.6. (I would guess that some of them
may already be planning to do so, and may even have already done so for
previous Django releases in the past.)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54FF6991.7080909%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django 1.6's lifespan for security updates?

[Carl Meyer]

Hi Stephen,

On 03/11/2015 06:28 AM, Stephen Gallagher wrote:
> So, here's the basic problem. The distributions that are packaging
> python 2.6 are, basically Red Hat Enterprise Linux 6 and its clones
> (CentOS, Scientific Linux, Oracle, etc.) and SUSE Linux Enterprise
> Server 11. These two distributions make up a significant percentage of
> the enterprise deployment space. Neither of these distributions ships
> Django itself. For RHEL, the Fedora Project provides the EPEL add-on
> repository which is unsupported and *may* carry Django (though it has
> proven to be difficult, more on that in a minute). For SLES, there is
> OpenSUSE which acts similarly. These are community-sponsored and
> generally limited to only packaging what upstream provides, since with
> no corporate backing and engineering support, backporting patches from
> newer releases is unlikely to happen.
> 
> The reason that neither of these distributions carries Django is
> specifically because of the short lifecycle of Django. The LTM releases
> (1.4 and the upcoming 1.8) are somewhat better aligned with the goals of
> enterprise distributions, but even those approximately three-year
> lifespans are significantly shorter than the ten years that the
> enterprise distributions want to maintain. So the chances of Django ever
> being shipped by a company capable of supporting it for that length of
> time is basically zero. (Not unless Django someday becomes a critical
> part of the standard Linux platform).
> 
> In the Enterprise software space, there's an average of a two-year
> period between a new version of software becoming available and
> companies actually deploying it. This essentially means that an LTM
> release needs to be at minimum four years long, since it will require
> half of the supported lifecycle for consumers to convert over to it and
> they'll need the other half of it in order to migrate to the next
> version. (Also, most companies don't like constant migrations like this).

It certainly sounds like there's an opportunity here for someone to
provide extra-extended security-backport support for certain Django
releases (beyond the ~3.5 years we'll typically support an LTS release
under current policy), to accommodate these enterprise users on dead (to
upstream) Python versions.

I remain quite unconvinced that this "someone" should be the Django core
team. I think the core team's limited energies are better spent
continuing to move Django forward.

> So, I realize that Django 1.6 is not an LTM release, and so asking for
> an extension on it is probably unlikely to happen. I'd like to see 1.4
> supported for at least the two-year migration period to 1.8, though
> (with 1.8 planned to accommodate the future migration as well).

I think the current plan is for a 6-month support window for 1.4 after
the release of 1.8. (Tim can probably correct me if I'm wrong - I know
there was discussion about 3 months vs 6 months; I thought we settled on
six.)

> Also, is there any chance at all that python 2.6 support could be
> reintroduced to Django 1.8? That would make it plausible to migrate
> existing users to an LTM release at least, buying time to figure further
> plans out.

I think that's extremely unlikely.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55006656.7070305%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Using the eval() command

[Carl Meyer]

Hi Henry,

On 03/10/2015 07:37 PM, Henry Versemann wrote:
> So how does an expression like you suggested above (
> innerDict['+newinnrkey+'] = newinnrval ) work then?
> It seems like it wouldn't work without enclosing the expression
> with quotes or double-quotes, and even then it seems like it would only
> become some kind of string instead of a statement which would be
> automatically executed to produce a result. Please explain or point me
> to some documentation explaining this type of code or coding.
> Thanks.

I didn't suggest that line of code, Bill did. As Tom said, it was
probably a typo. The line of code I suggested, which is what you should
use, is simply:

innerDict[newinnrkey] = newinnrval

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/550067A9.6030903%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Testing template context processors

[Carl Meyer]

Hi Gergely,

On 03/17/2015 10:52 AM, Gergely Polonkai wrote:
> I wrote a context processor which makes some global variables of mine
> available for templates. I cannot find a good (cough… any) article on
> how to test such processors. Could you give me some directions on where
> to start with writing one?

Writing a unit test for a context processor is quite simple. Construct a
fake request (you can use RequestFactory, though if your context
processor only accesses one or two attributes of the request, it might
be simpler to just build a custom fake or use a library like pretend
[1]), then call the context processor function in your test, passing in
the fake request, and assert that the returned dictionary contains what
you expect.

If your templates require the values from the context processor in order
to render correctly, you can also write a higher-level integration or
functional test by using the test client to access a view that renders a
template that needs the context processor, and then assert the right
value is found in the rendered HTML.

Carl

  [1] https://pypi.python.org/pypi/pretend

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55085D7A.5040007%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django 1.6's lifespan for security updates?

[Carl Meyer]

On 03/19/2015 01:01 AM, Russell Keith-Magee wrote:
> On Thu, Mar 19, 2015 at 12:05 PM, James Bennett  > wrote:
> Way, way, *way* back in the day, there was unofficial bugfix support
> for pre-magic-removal versions of Django run on the basis of "some
> people who need it step up and do all the work". It was solely done
> in SVN and never involved issuing new releases, so anyone running
> off the maintenance branch had to just update their checkout
> whenever a fix landed in it.
> 
> I'm not opposed to allowing that again -- and in fact I was one of
> the people who did extended maintenance for pre-m-r Django because
> my employer at the time needed it -- but I think we should carefully
> consider how it's going to work before trying it, since we no longer
> have the ability to give partial commit bits
> 
> 
> Agreed that this is an approach we could take - but I don't think the
> lack of a partial commit bit isn't a major issue IMHO.
> 
> At a technical level, Git gives anyone the ability to fork Django's
> repository. There's no *technical* reason that someone couldn't maintain
> a 1.6 branch *right now* - the only restrictions are that they couldn't
> call that repository official (as per BSD licensing terms), and they
> wouldn't have access to security patches until everyone else did.
> 
> However, both of these are solvable problems. 
> 
> The only thing that makes Django's Github repository "official" is
> because the core team says it is. If someone wants to take on the task
> of back porting all the security patches to 1.6, we (as a project) can
> point to that repository if we want. If we're interested in doing 1.6
> support as an "unofficial" subproject, then it seems appropriate to me
> that the repository *isn't* Django's core repository; picking a
> third-party vendor's repo sounds like the right way to do it. 
> 
> As for getting the patches ahead of time - I'd be more than happy for an
> appropriately qualified and trustworthy individual/company to be added
> to the security pre-notification list.
> 
> So - it really just comes down to someone having the enthusiasm to
> maintain the branch. Christian has been around the Django project for a
> long time (he was a contributor to Django Evolution from very early on,
> and took over maintenance of the project when I moved on). If he's got a
> commercial interest in continued 1.6 support, and he can spare a couple
> of hours to do a backport when a security issue is announced, I'd have
> no problem adding him to the pre-notification list, and adding some
> documentation pointing at his fork of Django as an "unofficial 1.6
> repository". If Christian hasn't got the time to do this, but he (and/or
> anyone else) can throw some funds at Tim, Tim is already on the security
> notification list.

I agree with all of this. I think that the Django core team will not
provide this extended support for 1.6 (because it gives the impression
that Python 2.6 is a supported and secure platform, which is only true
for the subset of our user base on certain extended-support distros),
and I don't think that extended-support releases of 1.6 should or will
ever appear on pypi.python.org/pypi/Django.

But I think that anyone who feels the need for this extended support can
feel free to raise the resources to make it happen and do it in a github
fork. If someone planning to maintain such a fork applied for
pre-notification of security releases, and could demonstrate significant
support for their effort, I would be in favor of that application.

And I think that if "that person" ends up being Tim Graham, as an
individual outside of his duties as Django Fellow, that's his choice and
I don't see any problem with it. We've already established a precedent,
several times over, for members of the core team to be paid for certain
tasks that parts of the community need or want that aren't getting done
otherwise. I would not support the DSF or Django core team sponsoring
the fundraising for this effort, but if a part of the community raises
the funds and Tim wants to accept said funds and do the work, more power
to him, as long as it doesn't detract from his other work as Django Fellow.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/550AF6BD.8000703%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Re-run only tests that failed last time

[Carl Meyer]

Hi Gergely,

On 03/24/2015 04:27 AM, Gergely Polonkai wrote:
> I have a pretty extended test suite for my application, which can run
> for about 15 minutes. Now when I introduce an error, I will be notified
> of it pretty late, and a one-liner fix needs another 15 minutes to
> check. Of course if only one of my tests fail, I can add the name of the
> test case as a parameter to manage.py test, but in case of many failures
> it’s pretty hard to do the same.
> 
> Is there a way to run only the tests that failed last time, even if I
> have to install a separate app/module for that? A quick Google search
> gave me no usable results.

I use py.test [1], another test runner which has plugins that offer this
feature. The pytest-django [2] plugin provides the necessary integration
to make py.test able to run a normal Django test suite. Then the
pytest-cache [3] plugin offers an ``--lf`` (last-failed) option to run
only the tests that failed in the last run, and the pytest-xdist [4]
plugin offers the even-more-advanced ``--looponfail`` option, which "run
your tests repeatedly in a subprocess. After each run py.test waits
until a file in your project changes and then re-runs the previously
failing tests. This is repeated until all tests pass after which again a
full run is performed."

If you're interested in advanced Python testing, I think py.test is near
the top of the list of tools you should at least take for a spin.

Carl

[1] http://pytest.org/latest/
[2] https://pypi.python.org/pypi/pytest-django/
[3] https://pypi.python.org/pypi/pytest-cache
[4] https://bitbucket.org/pytest-dev/pytest-xdist

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5511E627.8090409%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: QueryDict and its .dict() method - why does it do that?

[Carl Meyer]

Hi Gabriel,

On 03/27/2015 01:34 PM, Gabriel Pugliese wrote:
> @Masklinn
> That input is from jQuery's default serializer. It sends the data with
> contentType 'application/x-www-form-urlencoded; charset=UTF-8'. I just
> need to pass a data parameter with my JS Object (JSON) and it does the
> magic.

Yes, jQuery has chosen to adopt the same non-standard that Rails and PHP
use for smuggling nested lists and objects through a flat querystring.

> I understand I can use .getlist() or any other non-optimal solution.

request.GET.getlist('key') is the correct solution in Django for getting
a list of values from a querystring.

> I
> also understand what's going on about keys here (that's why I asked
> about laziness nature of QueryDict).

I don't understand what you think is lazy about a QueryDict, or how that
would be related to this issue. QueryDicts are not lazy; they parse the
querystring immediately on instantiation.

> But following DRY, KISS and other
> philosophies, couldn't Django do the same as other frameworks assuming
> it's widely adopted that way? What is the advantage of doing things so
> differently about that?

Django's philosophy here follows KISS. We give you the querystring in a
parsed format that allows you to easily get the data out of it by the
actual key names found in the querystring. We don't assume that you are
using jQuery on the client side, and we don't add any magical
non-standard features to smuggle nested objects through the querystring.
Such behavior might "do the expected thing" for some new users of
Django, at the expense of being very confusing to other new users of
Django, who understand HTTP and HTML but haven't been exposed to the
jQuery/Rails/PHP behavior.

> Why should I need to implement a lib that re-iterates over the same
> string that QueryDict already parses

You don't need to go back to the raw querystring. It would not be hard
to implement a small function that would take a QueryDict, iterate over
it and interpret it in the jQuery/Rails/PHP style, and return a
dictionary. I'd guess that someone may have already done this, though
I'm not sure what terms I'd use to search for it. If nobody has, then
you could be the first! Put it on PyPI, advertise it, and see what kind
of demand there is.

> to achieve the result that is
> expected to everyone that is new to Django?

I am a counter-example to your assertion that the Rails/PHP/jQuery
behavior is "expected to everyone that is new to Django." I had never
heard of the Rails/PHP/jQuery behavior when I was new to Django, and
Django's handling made perfect sense to me. The first time I ran across
jQuery's behavior, I was very confused.

> Anyway, thanks for the clear answer and information. And sorry about any
> grammar errors, English is not my first language.

No problem. I hope my answer was clear and informative.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5515B49F.3010603%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: QueryDict and its .dict() method - why does it do that?

[Carl Meyer]

Hi Gabriel,

On 03/27/2015 02:10 PM, Gabriel Pugliese wrote:
> I perfectly understand what you are saying. It was very clear and
> informative, but do not agree with the design chosen here. Below is just
> an opinion and you do not have to agree with it:

That's good :-) I probably won't continue with the design discussion
past this email, because I don't think we will reach a conclusion which
convinces the Django core team to change the design.

> My buddies have given PHP and Rails examples, but there are other
> frameworks from other languages that do that the same way.

I'm not personally aware of them, but I believe you. If this behavior
ever becomes part of an accepted RFC, I'm sure Django will implement it
in core :-)

> I mean,
> what's the advantage here doing differently from others?

I already tried to explain that: simple, predictable behavior that
exposes the underlying querystring data directly to the user, with no
munging of values which might be unexpected by some users.

Django provides the simple, predictable basic behavior. If you want this
extended special behavior, it's easy to implement it on top of what
Django provides.

> And I don't agree it follows KISS if I need to re-iterate on the result
> again to get a dict from it (one clear example usage is destructuring as
> named function parameters).

If you just want a dict, you don't need to iterate, you can just use the
.dict() method (or other options already explained earlier in this thread).

If you want a dict interpretation of the querystring *that handles keys
with brackets in their names in a special way*, then yes, you have to
implement that yourself. It wouldn't be hard to implement, and then that
code could be shared by all who want it.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5515BB4F.3010502%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Help with Dynamically Adding Forms to Formset!!

[Carl Meyer]

Hi Stephanie,

On 03/30/2015 02:37 PM, Stephanie Socias wrote:
> I'm having trouble creating a formset where I can dynamically add and
> delete items. I've tried many different approaches and can't get
> anything to work. I'm new to Django and have been at this for days. Can
> someone please help me!? I can send my code, post it, or do a screen
> share. Please let me know!! Any help would be greatly appreciated.

I'm afraid I don't have time in my day for a screen-share, but I'd be
happy to try to help, if you can link to a gist or paste showing your
current best effort, and explaining how it is not working.

FWIW, I've usually used this library for the JavaScript side of things:
https://github.com/jgerigmeyer/jquery-django-superformset

If you'd find real-time assistance more helpful, you could visit the
#django channel on FreeNode IRC; I and others are available to help out
there.

This is a challenging task you've tackled, even for experienced Django
developers, so it's not surprising that it's taking some time if you're
new to Django.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5519BD9B.9080105%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Problem after Virtualenv Setup

[Carl Meyer]

Hi Kishan,

On 04/02/2015 09:41 AM, Kishan Mehta wrote:
> I have created my django project and worked on it . Recently I came to
> know about virtualenv and the problem it solves.
> I have installed virtualenv and activated that environment.
> Now django command is not recognized.
> * Will I have to install django again ? Or is there any other efficient
> way.*
> Thanks for help.

You will have to reinstall Django inside the virtual environment.

You should probably also uninstall the global Django you installed
(should just be a matter of "pip uninstall Django" without the
virtualenv activated; may need to preface that with "sudo " depending on
your OS and configuration.)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/551D63D9.8070005%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: How do I manage a Non-default Django PostgreSQL database

[Carl Meyer]

On 04/09/2015 07:12 AM, Stephen M wrote:
> If Django migrations with PostgreSQL is a bit flaky (as suggested by
> S.O. comment) which backend database is reliably supported by the Django
> migrations mechanism?

The premise is false. Postgres is almost certainly the best-supported
database for Django migrations.

I'm not entirely sure what's going on in your example, but the issue is
most likely related to multiple-databases, not related to Postgres. Did
you read the documentation at
http://django.readthedocs.org/en/latest/topics/db/multi-db.html#synchronizing-your-databases
(really you should carefully read that whole page if you're using
multiple databases)? Did you use the `--database` option to the
`migrate` command?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55269ADD.7090608%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrations During Development

[Carl Meyer]

On 04/25/2015 06:48 AM, Jorge Andrés Vergara Ebratt wrote:
> Are you using version control like GIT? I save the migration folder with
> the __init__.py in GIT, nothing else, because al the migrations will be
> diferent in the diferent servers.

This is not a good idea. Migrations should be committed to git, and
should be the same on all deployments. Otherwise they do not serve their
purpose of ensuring that all deployments end up with the same database
schema.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/553C1F55.10904%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrations During Development

[Carl Meyer]

On 04/25/2015 11:15 AM, Javier Guerra Giraldez wrote:
> On Sat, Apr 25, 2015 at 12:08 PM, Timothy W. Cook 
> wrote:
>> 
>> On Sat, Apr 25, 2015 at 9:48 AM, Jorge Andrés Vergara Ebratt
>>  wrote:
>>> 
>>> Are you using version control like GIT? I save the migration
>>> folder with the __init__.py in GIT, nothing else, because al the
>>> migrations will be diferent in the diferent servers.
>>> 
>> 
>> This is probably what I should have done. I have been saving them
>> all to the git repo.
> 
> 
> with South i used to save them to the repo too and push whole to 
> production.  haven't tried Django migrations yet; is there any 
> difference that make this more appropriate?
> 
> if you don't store migrations, then you have to generate them on
> each production update, right?  if so, wouldn't they count as
> untested code?  how about manually tweaked, or data migrations?

Migrations should be committed to VCS in both South and in Django
migrations.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/553C1F83.60105%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrations During Development

[Carl Meyer]

On 04/25/2015 11:28 AM, Timothy W. Cook wrote:
> if you don't store migrations, then you have to generate them on each
> production update, right?  if so, wouldn't they count as untested
> code?  how about manually tweaked, or data migrations?
> 
> 
> ​That will certainly be true once you start using a staging or
> production server.  
> But before they they are just a kind left over creative ideas.  :-)  Not
> really useful for anything in the future, AFAICS. 

If you only have one deployment (your local machine), you can erase all
your migrations and "start over" anytime with a new initial migration,
with no problems.

Once you have additional deployments (either other developers' machines,
or staging or production deployments), you shouldn't remove or change
any migrations that have been pushed to those other deployments.

You can use the "squash" feature [1] to squash a set of historical
migrations down to a single migration. This requires a phased
deployment, where you first push out a version including both the new
squashed migration and all the old replaced migrations, and then later
(once you are confident all deployments are up to date) you can safely
remove the old replaced migrations.

It's also true that if you have a series of new migrations locally that
haven't yet been pushed to any other deployment, you can delete and
replace them with a single migration before pushing.

It's very much like VCS commits: it's ok to edit your local history, but
you have to be a lot more careful with history that you've already
shared with others.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/553C20AE.9080609%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrations During Development

[Carl Meyer]

On 04/25/2015 05:18 PM, Carl Meyer wrote:
> You can use the "squash" feature [1] to squash a set of historical
> migrations down to a single migration.

Forgot this link:
https://docs.djangoproject.com/en/1.8/topics/migrations/#squashing-migrations

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/553C210A.1050209%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: forcing user model to return first_name last_name if exists

[Carl Meyer]

On 04/28/2015 09:10 AM, Ilya Kazakevich wrote:
> I have many places in my app where user (from user model) is displayed:
> templates, forms, fliters, tables etc.
> It is displayed as username everywhere.
> 
> I want it to be displayed as first_name/last_name.
> 
> I can do that with monkey patching:
> 
> @receiver(request_started)
> def patch(*args, **kwargs):
> get_user_model().__str__ = my_smart_str_func
> 
> I use "request_started" to make sure all models are loaded. (BTW, does
> there is something like "models loaded" signal?).

Not a signal; the proper place for code like this in Django 1.7+ is in
an AppConfig.ready() method.

> How ever, I feel it a little bit hacky. There should be more pythonic
> way to do that. Am I miss something?

Of course, the best way to do this is a custom User model so you don't
have to monkeypatch at all. Sadly, switching to a custom User model is
difficult if the project didn't start with one.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/553FA98A.4060301%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django runserver needs restart on psql data changes

[Carl Meyer]

Hi Arnab,

On 04/29/2015 03:13 PM, Arnab Banerji wrote:
> Hi Javier - thanks for the response. Yes - I am storing the data in a
> dictionary - then having django_tables2 API render the dictionary as a
> table. How do I force a read of that variable for every browser refresh
> on the client side? 

Using a variable is fine; the problem is that (I'm guessing) you are
creating that variable at the top level of the module. Such code is only
run once, when the module is first imported.

You need to make sure that all database queries are run only inside view
functions (or methods of view classes), never at the top-level of a module.

carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5541568D.1060808%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: RemovedInDjango19Warning: Model class %s doesn't declare an explicit app_label and either isn't in an application in INSTALLED_APPS ..

[Carl Meyer]

Hi,

On 04/29/2015 11:13 AM, Bumyong Choi wrote:
> I am trying to understand this warning better. I understand that we want
> to make sure that a model has an explicit app_label or it is in
> INSTALLED_APPS. I spent some time trying to remove warnings from my code
> because I was getting these warning messages even though my model was
> defined in an app listed in INSTALLED_APPS. 
> 
> After scratching my head for quite some time, I put a breakpoint on the
> following line of the code in django.db.models.base:
> 
> # Look for an application configuration to attach the model to.
> 
> app_config =apps.get_containing_app_config(module)
> 
> And I checked to see what I get for "app_configs.values()" and I
> realized that the app is in INSTALLED_APPS but not yet been loaded. This
> was because one of the third party framework I use was trying to import
> models before they were loaded. 
> 
> My question is the following:
> 
> Is Django trying to prevent a model from being imported before this
> model is actually defined? If this is the case, is it possible to
> improve the warning so that it would check "INSTALLED_APPS" and warn the
> user that even if the app is listed there some other app is trying to
> import the model before the app is loaded?"

Yes, that is the current intention in Django 1.7+: you may not import a
models.py file before the app cache has been prepared. (I personally
think this is unfortunate, but to change it while maintaining consistent
and predictable semantics would require a deep reworking of how relation
fields are initialized, which nobody has so far attempted).

I'm surprised you are getting that particular error though - if a
models.py file is imported too soon, I would expect an "App registry not
ready" error instead. So it may be a bug that you are getting this error
message instead of that one; hard to know without seeing code to
reproduce the issue.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/554157B1.5000702%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Why so complicated :O??

[Carl Meyer]

Hi Vesko,

On 05/06/2015 12:47 PM, Vesko Nikolov wrote:
> Hello everyone. So i have been trying to learn Django for the last
> couple of months but have been seeing it quite hard. I have a decent set
> of skills in PHP but i wanted to learn something newer and that is what
> brought me here. So on with the question. Where can i get a BASIC
> tutorial of how django works and how to perform simple tasks. All the
> tutorials I find are DATABASES MODELS TEMPLATING. I don't care about any
> of that sh*t (at least not at this stage). I want to make django
> calculate some math for me or some equally simple tasks so i can get an
> idea how basics work before i interact with databases and make really
> dynamic projects. 

Sure, that's reasonable. I would actually prefer it if the official
Django tutorial tackled things in the order you suggest, covering simple
views and HTTP request/response handling before covering models, the
admin, etc.

I think you'll find the starting point you're looking for in part 3 of
the official tutorial, at
https://docs.djangoproject.com/en/1.8/intro/tutorial03/#write-your-first-view

You'll still need to follow part 1 to get your project and app created,
but you can skip over the database and models stuff and come back to
that later, if you want.

Good luck!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/554A7E19.6040708%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Ensure an object is saved even when the atomic bock rollsback

[Carl Meyer]

Hi Marc,

On 05/08/2015 07:15 AM, Marc Aymerich wrote:
> I'm using atomic requests, but one of my views needs to save a model
> regardless of wheter the current transaction rolls back or not.
> 
> I'm confused about how to proceed with the current autocommit behaviour.
> Do I need to use a different db connection? or perhaps there is some way
> of telling django to ignore autocommit for some particular block ? 

You'd need to use a different db connection, since what you're trying to
do violates the very nature of a database transaction. If you were just
trying to run some raw SQL, you could establish the separate connection
yourself manually, but if you're trying to save a Django model, you'll
probably need a second connection defined in your DATABASES setting.

Possible alternatives:

- If you're using a task queuing system (like Celery or rq), queue a
task to perform the save; you can do this without making it conditional
on the transaction committing, as long as your task queue is using a
store other than your primary database (e.g. Redis or RabbitMQ). Usually
when people do this it's unintentional and a bug (they really wanted the
task to execute only if the current transaction committed), but in your
case it could be a feature.

- Switch from ATOMIC_REQUESTS to explicit use of transaction.atomic() in
your views, so that you can place this "no matter what" save in its own
transaction.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/554CDB37.7060503%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django code of conduct - last line

[Carl Meyer]

Hi Graham,

On 05/08/2015 01:43 PM, Graham Oliver wrote:
> Thank you, so how about
> 
> 'Don’t forget that it is human to err and blaming each other doesn’t get
> us anywhere.
> Instead, focus on helping to resolve issues and learning from mistakes'

I think this is an improvement in clarity and style over the current
wording. Please see https://www.djangoproject.com/conduct/changes/ for
the process to make changes to the Code (short version: send a pull
request to http://github.com/django/djangoproject.com).

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/554D1DA9.1090401%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Invalid HTTP_HOST, can someone explain why I am getting this?

[Carl Meyer]

On 05/15/2015 09:14 AM, Andreas Kuhne wrote:
> You are getting it because someone has come to your server by
> entering www.g3suprimentos.com.br  in
> a browser. If you don't have that address in the allowed hosts setting,
> you will get the error you have received. 

Correct.

> If you don't want your server to show something
> for www.g3suprimentos.com.br , then
> the problem is not on your end, but rather the clients (it can be an
> incorrect DNS entry, or that someone has edited their hosts file). So
> you can't do anything to fix it.

Not true - in most cases you can fix it on the server side, but the
details depend on your server configuration. You need to configure
whatever front-end server is listening for the web requests to ignore
any requests coming in for a host name that you don't want your server
to respond to. If you're using Apache, this meand making sure that your
Django site is running in a name-based virtualhost (and not the default
one). If you're using nginx, it just means setting the server_name of
the server block to the hostname(s) you actually want to serve. Etc.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5556116D.5090806%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Invalid HTTP_HOST, can someone explain why I am getting this?

[Carl Meyer]

> On May 16, 2015, at 8:12 AM, Gerald Klein  wrote:
> 
> This looks like a cross domain request that is being received by your site 
> and the requestor is not listed in your sites allowed sites list, Sites 
> conforming with CORS will have a list of sites that can request data via http 
> from them, this is stating the requestor  is not on the list

No, this is not related to the problem. Django's ALLOWED_HOSTS setting has 
nothing to do with CORS or the domain the request is coming from, it's about 
the server's domain(s) and the Host header in the request. 

Carl

> Hope that helps
> 
>> On Sat, May 16, 2015 at 8:49 AM, bobhaugen  wrote:
>> I've gotten this error now and then when I have updated the software on a 
>> webfaction-hosted site and restarted Apache. Then it goes away after maybe a 
>> couple more restarts. It's like the software has not quite gotten itself 
>> together yet. I have no idea why this should be, and since it goes away, I 
>> have not opened a ticket on webfaction. 
>> 
>>> On Wednesday, May 13, 2015 at 5:45:43 PM UTC-5, Russell Keith-Magee wrote:
>>> 
>>> Is there a setting to turn of a warning that someone is attempting to 
>>> access your site in a potentially malicious way? No.
>>> 
>>> When you get a warning like this, you investigate the cause, and fix the 
>>> problem. You don't just silence the warning.
>>> 
>>> Yours,
>>> Russ Magee %-)
>>> 
 On Thu, May 14, 2015 at 3:44 AM, frocco  wrote:
 Thanks for getting back to me.
 I am on django 1.5. Is there a setting I can use to avoid getting these 
 emails?
 I get at least 5 a week.
 
 
 
> On Tuesday, March 17, 2015 at 6:58:00 PM UTC-4, Russell Keith-Magee wrote:
> Hi,
> 
> It's possible that you're getting this error for the exact reason that 
> the check was added - someone is submitting requests to your site with a 
> manipulated HTTP_HOST header. This may not be malicious - it could just 
> be a badly configured robot making web requests.
> 
> It might also point at a problem with your hosting provider - especially 
> if you're sharing an IP address with g3suprimentos.com.br, or if 
> g3suprimentos.com.br once had the same IP that you're currently using. 
> 
> Without more details, it's hard to say for certain. But is someone trying 
> to hack your site? Almost certainly not. If you were being hacked, you 
> wouldn't have described the problem as "I see this error from time to 
> time" - you would have said "I just received a thousand of these in the 
> last 10 minutes". Attackers aren't noted for their subtlety - they will 
> generally try every possible backdoor and exploit that they know of, all 
> at once. 
> 
> Yours,
> Russ Magee %-)
> 
>> On Tue, Mar 17, 2015 at 11:45 PM, frocco  wrote:
>> I am concerned because I have a value in ALLOWED_HOSTS, but it is not 
>> this value.
>> 
>> 
>>> On Tuesday, March 17, 2015 at 9:17:15 AM UTC-4, frocco wrote:
>>> SuspiciousOperation: Invalid HTTP_HOST header (you may need to set 
>>> ALLOWED_HOSTS): www.g3suprimentos.com.br
>>> 
>>> I keep getting this error from time to time.
>>> 
>>> Is someone trying to hack my site?
>>> 
>>> Thanks
>> 
>> -- 
>> You received this message because you are subscribed to the Google 
>> Groups "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send 
>> an email to django-users...@googlegroups.com.
>> To post to this group, send email to django...@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/5550dde0-4230-4068-a8b5-196f0bd844fc%40googlegroups.com.
>> 
>> For more options, visit https://groups.google.com/d/optout.
> 
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 "Django users" group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to django-users...@googlegroups.com.
 To post to this group, send email to django...@googlegroups.com.
 Visit this group at http://groups.google.com/group/django-users.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/django-users/ab908cd5-8c5b-4d6e-9033-80fca60415f7%40googlegroups.com.
 
 For more options, visit https://groups.google.com/d/optout.
>>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-users.
>> To view this discussion on the web vi

Re: Adding a one field to user model: trying not to create custom one)

[Carl Meyer]

Hello Ilya,

On 05/18/2015 04:58 PM, Ilya Kazakevich wrote:
> I want to add just a one field to my model, and this field participates
> in user representation ( I use monkeypatch to change __str__).
> 
> Django manual tells me to create profile with one-to-one relation in
> this case. Well, it may work. But if I have  with 250 users, I
> face 250 "SELECT" queries to my db (N+1). I need to tell "UserManager"
> somehow to use "select_related". But how can I do that with out of new
> ugly monkeypatching?
> Does there is an official (recommended) way to do that? If no, how  can
> I use one-to-one user profile if I have "list of users" webpage with out
> of N+1? How to add one field to user model not extending it?

I am not aware of a good solution to this problem other than manually
adding the .select_related() to your query on the list-of-users page.

> I am really unhappy with idea of using custom user model.

Why?

If it's because this is an existing project and the prospect of
migrating your existing data to a custom user model is daunting, I
totally understand that. It's doable, but hard.

If this is a new project, I think that there is no good reason to avoid
a custom user model. Every new project should always use a custom user
model, even if to begin with it just inherits from AbstractUser and
doesn't change or add anything. This way in the future you have control
over your user model and can modify it as needed without monkeypatching.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/555A72A0.2070105%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Adding a one field to user model: trying not to create custom one)

[Carl Meyer]

On Tuesday, May 19, 2015 at 8:18:24 AM UTC-6, Ilya Kazakevich wrote:
>
> > I am really unhappy with idea of using custom user model. 
>>
>> Why? 
>>
>> If it's because this is an existing project and the prospect of 
>> migrating your existing data to a custom user model is daunting, I 
>> totally understand that. It's doable, but hard. 
>>
> Yes, this project already deployed and has some data. Sure I can solve 
> this, but I feel that changing user model will make my app less reusable. 
>

I don't think "will make my app less reusable" is a valid reason to avoid a 
custom User model. A lot of thought went into the custom User model system 
to ensure that it preserves reusability. That's why things like 
`django.contrib.auth.get_user_model()` and friends exist.
 

> But I will probably stay with new model. There is something wrong with 
> Django in this area. There should be some easy and elegant way to add one 
> field to auth_user. I wonder why Django developers do not care about it.
>

Django developers do care about the problem, and the preferred solution 
(after many years of discussion) is the custom User model. I believe that 
it's a better solution than introducing a hacky way to monkeypatch fields 
onto a built-in model. I think every Django project should use a custom 
User model.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/d2f5e133-47a4-4255-a016-03d4b8a14d5d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Why the extra minute in PostgreSQL when using time zone info?

[Carl Meyer]

Hi aRkadeFR,

On 05/29/2015 02:09 AM, aRkadeFR wrote:
> Indeed, the problem comes from the representation on PG,
> cause I have only good results (0/30 for the minutes) from
> the python function.
[snip]

The OP already found and posted the solution (see below) and it is not
related to Postgres. Here's a fuller explanation:

A pytz timezone class does not represent a single offset from UTC, it
represents a geographical area which, over the course of history, has
probably gone through several different UTC offsets. The oldest offset
for a given zone, representing the offset from before time zones were
standardized (in the late 1800s, most places) is usually called "LMT"
(Local Mean Time), and it is often offset from UTC by an odd number of
minutes.

When you create a timezone using `pytz.timezone('Europe/London')`, pytz
doesn't yet know what datetime you plan to attach the timezone object
to, so it doesn't know which historical period's offset it should use,
and it defaults to the first one in the zoneinfo database, which is
usually LMT:

>>> import pytz
>>> uktz = pytz.timezone('Europe/London')
>>> uktz


You can see that this timezone object is Europe/London _LMT_, and you
can also see that it is one minute offset from UTC (that's the "23:59:00
STD" bit).

The _wrong_ way to attach a pytz timezone to a naive Python datetime is
by passing it to the `tzinfo` argument of the datetime constructor (or
by using `.replace(tzinfo=...)`). When you do this, pytz literally
attaches the exact timezone offset (London LMT, dating from the
mid-1800s) to your datetime:

>>> from datetime import datetime
>>> lmt_dt = datetime(2015, 6, 11, 13, 30, tzinfo=uktz)
datetime.datetime(2015, 6, 11, 13, 30, tzinfo=)

If you convert this to UTC, you'll see the effect of that mysterious one
minute offset:

>>> lmt_dt.astimezone(pytz.utc)
datetime.datetime(2015, 6, 11, 13, 31, tzinfo=)

The _right_ way to attach a pytz timezone to a naive Python datetime is
to call `tzobj.localize(dt)`. This gives pytz a chance to say "oh, your
datetime is in 2015, so I'll use the offset for Europe/London that's in
use in 2015, rather than the one that was in use in the mid-1800s":

>>> good_dt = uktz.localize(datetime(2015, 6, 11, 13, 30))
>>> good_dt
datetime.datetime(2015, 6, 11, 13, 30, tzinfo=)

You can see that this datetime object is in BST (one hour offset from
UTC, at least in June) instead of LMT - a much better choice for this
century. And we can convert it to UTC and get a more sensible result:

>>> good_dt.astimezone(pytz.utc)
datetime.datetime(2015, 6, 11, 12, 30, tzinfo=)

Since we're on the Django list here, I should also point out that
`django.utils.timezone` has a `make_aware` method which handles this
correctly for you automatically.

Interesting historical sidenote: the 1-minute offset for London LMT is
based on historical observations such as this [1]:

# From Peter Ilieve (1994-07-06):
#
# On 17 Jan 1994 the Independent, a UK quality newspaper, had a piece about
# historical vistas along the Thames in west London. There was a photo
# and a sketch map showing some of the sightlines involved. One paragraph
# of the text said:
#
# 'An old stone obelisk marking a forgotten terrestrial meridian stands
# beside the river at Kew. In the 18th century, before time and longitude
# was standardised by the Royal Observatory in Greenwich, scholars observed
# this stone and the movement of stars from Kew Observatory nearby. They
# made their calculations and set the time for the Horse Guards and
Parliament,
# but now the stone is obscured by scrubwood and can only be seen by walking
# along the towpath within a few yards of it.'
#
# I have a one inch to one mile map of London and my estimate of the stone's
# position is 51 degrees 28' 30" N, 0 degrees 18' 45" W. The longitude
should
# be within about +-2". The Ordnance Survey grid reference is TQ172761.
#
# [This yields GMTOFF = -0:01:15 for London LMT in the 18th century.]


Carl

[1] https://github.com/eggert/tz/blob/master/europe#L107


>> On Thu, May 28, 2015 at 11:42 PM, Daniel Grace > > wrote:
>>
>> I used this function, seemed to tidy up the problem:
>>
>> from datetime import datetime
>> from pytz import timezone
>>
>> def utcDtTm(self, dt_tm):
>> tz = timezone('Europe/London')
>> return
>> tz.normalize(tz.localize(dt_tm)).astimezone(timezone('UTC'))

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55688D0F.9040602%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description:

Re: Help with customizing Django authentication

[Carl Meyer]

Hello Carlos,

On 05/29/2015 03:19 PM, Carlos Ribas wrote:
> Hello,
> 
> I have to confess that I did not understand your suggestion. How this
> will help me to reverse the logic of my system? I mean, instead of User
> with or without a profile (the Person class, in my case), I want a
> Person with or without a User.
> 
> Thanks anyway
> 
> Em quarta-feira, 27 de maio de 2015 12:54:59 UTC-3, Carlos Ribas escreveu:
> 
> Hello All,
> 
> I am currently extending the existing User model to store additional
> information. So, basically I have:
> 
> # models.py
> class Person(models.Model):
> user = models.OneToOneField(User, verbose_name=_('User'))
> zipcode = models.CharField(_('Zip Code'), max_length=9,
> blank=True, null=True)
> street = models.CharField(_('Address'), max_length=255,
> blank=True, null=True)
> ...
> 
> #admin.py
> class PersonAdmin(admin.StackedInline):
> model = Person
> ...
> 
> class UserAdmin(UserAdmin):
> inlines = (PersonAdmin, )
> ...
> 
> admin.site.unregister(User)
> admin.site.register(User, UserAdmin)   
> 
> 
> The problem is that my system should be able to register a new
> person, but this person may not need an account on the system (I
> just need to have his/her information). Right now, I can not create
> a person without an account. Besides that, first_name and last_name
> fields are not in the person class. 
> 
> I am wondering what is the best solution for my case. Probably, I
> will need to move the first_name and last_name fields to the Person
> class, and to do that, I will have to create custom users, is that
> right? 

Yes, the best solution for your case (and for all Django projects) is to
use a custom User model.

In order to have every User linked to a Person, but not all Persons
linked to Users, you need to place the OneToOneField on the User model
pointing to Person, rather than the other way around. And in order to do
that, you need to have control over the User model.

(You _could_ sort of do it without a custom User model by having a
ManyToMany relationship between User and Person, but that allows a wide
variety of situations you don't want to allow.)

> If that is the case, may I just copy and paste the lines shown here
> 
> (https://github.com/django/django/blob/master/django/contrib/auth/models.py
> 
> )
> and remove the lines about first_name and last_name? 

No, you should follow the documentation on custom User models.

You'll want to inherit from AbstractBaseUser instead of AbstractUser, so
as to not have first_name and last_name fields. You'll need to add the
PermissionsMixin and probably a few other fields (including a username
field). The docs should explain what you need to know.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5568DFAC.8020800%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Bug? Infinite migrations with empty help_text and ugettext_lazy

[Carl Meyer]

Hi Andy,

On 06/04/2015 09:44 AM, Andy Gimblett wrote:
> I've come across something which might possibly be a Django bug, but I wanted 
> to raise it here first in case I'm missing something, or this is expected 
> behaviour.
> 
> There's a fuller explanation and easily runnable example here:
> 
> https://github.com/gimbo/django-ugettext_lazy_migrations
> 
> In short, given the following model (note the empty help_text string),
> 
> from django.db import models
> from django.utils.translation import ugettext_lazy as _
> 
> class X(models.Model):
> y = models.CharField(help_text=_(''), max_length=10)
> 
> repeated calls of "manage.py makemigrations" will, after the initial 
> migration, produce an infinite series of new migrations, each of which 
> contains an AlterField, but which does nothing.  The expected behaviour would 
> be to produce an initial migration and then no further migrations, of course.
> 
> The problems goes away if you do any of the following:
> 
> * Remove the help_text parameter
> * Use a non-empty help_text
> * Don't internationalise the help_text value
> * Use ugettext instead of ugettext_lazy (which, I understand, you shouldn't)
> 
> I've reproduced it on all released 1.8.x Django versions up to 1.8.2, but it 
> doesn't exhibit on any 1.7.x versions I've tried.
> I haven't tried any development versions.
> 
> Can anyone provide any insight?  Is this worth raising a bug over?

That is most certainly a bug. It's not a super-high-priority bug
(because an empty translated help_text is an odd thing to do), but it's
a bug nonetheless and should be filed at
https://code.djangoproject.com/newticket and fixed.

I would guess the problem has to do with a careless `if help_text:`
somewhere in the migration detection code.

It would be worth checking whether the same problem exists with any
other similar field parameters (e.g. `label`).

Thanks for finding and reporting the bug!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55708551.1010702%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Bug in 1.7-1.8 migrations

[Carl Meyer]

Hi!

On 06/05/2015 05:29 AM, Владислав Пискунов wrote:
> In 1.7 auto-created migration with makemigrations. it contains: 
> 
> |
> related_name=b''
> |
> 
> Now in 1.8 while migrate, django tries Render model state and fails with:
> 
[snip]

>   File
> "/home/vladislav/.virtualenvs/furskru/local/lib/python2.7/site-packages/django/db/models/fields/related.py",
> line 1355, in is_hidden
> return self.related_name is not None and self.related_name[-1] == '+'
> IndexError: string index out of range
> |

Certainly looks like a bug! Would you be willing to report it at
https://code.djangoproject.com/newticket ?

Thanks!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5571E8BC.2000403%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Upgrade Confusion with ForeignKey

[Carl Meyer]

Hi Tim,

On 06/05/2015 03:44 PM, Tim Sawyer wrote:
> I've just upgraded from a very old version of Django (1.4) up to 1.7.8,
> and some behaviour isn't how I expect.  I may have missed something -
> I've not been keeping an eye on the framework changes.

I would strongly recommend that you upgrade version by version (that is,
first 1.4.x to 1.5.12, then 1.5.12 to 1.6.11, then 1.6.11 to 1.7.8),
each as a separate step, making sure that everything works as you expect
between each step, and that you carefully read the release notes (if you
haven't) for each version as you update to it.

Django tries to maintain backwards compatibility where we can, but we do
have to change and remove features sometimes. Those changes are
documented in the release notes for each version.

If you've upgraded all the way from 1.4 to 1.7 without reading the
release notes, which is what it sounds like, then I will be very
surprised if the below is the only problem you have.

> Two models, Venue and VenueAlias.  VenueAlias has a ForeignKey to Venue,
> one venue has many VenueAliases. (Full models pasted below.)
> 
> My problem is with accessing venuealias_set on my Venue, it doesn't seem
> to be limiting to Venue how it used to in the earlier version.
> 
> Here's an interactive session that shows the problem.  Venue 5854 has
> three aliases.
> 
 from bbr.contests.models import Venue
 lVenue = Venue.objects.filter(id=5854)[0]
 lVenue
>  Manchester, Greater Manchester, England, UK>
 lVenue.exact
> False
 lAliases = lVenue.venuealias_set.all()
 len(lAliases)
> 402
 print lAliases.query
> SELECT "venues_venuealias"."id", "venues_venuealias"."last_modified",
> "venues_venuealias"."created", "venues_venuealias"."name",
> "venues_venuealias"."alias_start_date",
> "venues_venuealias"."alias_end_date", "venues_venuealias"."venue_id",
> "venues_venuealias"."lastChangedBy_id", "venues_venuealias"."owner_id"
> FROM "venues_venuealias" INNER JOIN "contests_venue" ON (
> "venues_venuealias"."venue_id" = "contests_venue"."id" ) WHERE
> "contests_venue"."exact" = True ORDER BY "venues_venuealias"."name" ASC
> 
> Where's the SQL to limit by the foreign key to venue.id=5854?  Why is
> there a reference in here to contests_venue.exact?

That is very strange. I've never seen that, and I'm not aware of any
changes to Django that would cause that. Nor do I see anything in the
models code you pasted below that looks suspicious to me as a possible
cause.

I have to guess that there's some other code in your project causing
this problem, but I'm afraid I don't know what that might be.

I'm afraid the best I can advise is to go back to Django 1.4 and do the
upgrade again methodically, and see if you can track down at precisely
which version this problem first manifests.

Good luck!

Carl


> Approaching the same problem from the other direction works fine, and I
> get the correct number of aliases returned.
> 
 from bbr.venues.models import VenueAlias
 lVenueAliases = VenueAlias.objects.filter(venue_id=5854)
 len(lVenueAliases)
> 3
 print lVenueAliases.query
> SELECT "venues_venuealias"."id", "venues_venuealias"."last_modified",
> "venues_venuealias"."created", "venues_venuealias"."name",
> "venues_venuealias"."alias_start_date",
> "venues_venuealias"."alias_end_date", "venues_venuealias"."venue_id",
> "venues_venuealias"."lastChangedBy_id", "venues_venuealias"."owner_id"
> FROM "venues_venuealias" WHERE "venues_venuealias"."venue_id" = 5854
> ORDER BY "venues_venuealias"."name" ASC
> 
> On my old server, running the old django 1.4 code, both these approaches
> work as expected.  Venue and VenueAlias are actually in different
> models.py files, for historical reasons.
> 
> What have I missed?  Thanks for any input/education!

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55721CB8.9070202%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Upgrade Confusion with ForeignKey

[Carl Meyer]

Hi Tim,

On 06/05/2015 04:26 PM, Tim Sawyer wrote:
> Thanks. :-)
> 
> Did that, read the release notes, made sure it worked after each upgrade
> stage.  This is an esoteric corner of my app, and I didn't notice it not
> working.  The other foreign key stuff I have seems to work ok.

Ah, good! In that case, sorry for the unnecessary admonitions. :-)

> Looks like I have more investigation to do...any more clues over what
> *could* be the cause appreciated!

Well, I can give you a rundown of some things that crossed my mind as I
was thinking through what might cause that symptom. Some of them don't
make much sense, and none of them satisfy me as an explanation given the
data I have, but maybe they'll remind you of something in your codebase:

* Was your shell session pasted exactly as you ran it? No possibility
that queryset came from somewhere other than the reverse related manager?

* Any custom model managers in use on either of the relevant models?
(Didn't look like it from the code you pasted).

* Both the apps are in INSTALLED_APPS?

* No `to_field` on any relevant ForeignKeys?

Beyond that, if I were debugging this I would use PDB to step through
the entire process of accessing the reverse-relation attribute,
generating the related-manager, getting a queryset from it, etc, and
compare that to the same process for a reverse-related-set that's
working properly.

Good luck!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/557223E8.3060401%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Have mercy please!

[Carl Meyer]

Hi Jon,

Thanks for sharing your experience.

On 06/09/2015 07:25 PM, Jon Foster wrote:
> I've been involved with Django, on and off, since v0.96 or so. I love
> Django and think its the most productive way to build rich websites with
> custom defined content types. Throw in Django-CMS and things get pretty
> darn cool.
> 
> I was thrilled when Django reached v1.0 since it came with the promise
> of having a consistent and stable API, specifically a lack of backwards
> incompatible changes, unless required by security concerns.
> Unfortunately this promise is routinely violated for various excuses.
> The bottom line is none of the apps I've written have been able to
> survive any 2 point upgrades (v+0.2). Single point upgrades usually only
> cause minor breakage.

This last confuses me, so I'd like to get clarity. If "single point
upgrades usually only cause minor breakage", then a two point upgrade is
just two single point upgrades, one after the other, correct?

Trying to upgrade directly from e.g. 1.6 to 1.8 without first going
through 1.7 - by which I mean "having a fully working site that passes
its tests with no deprecation warnings in 1.7" -- is definitely not
recommended.

So is the problem here that you've been trying to do a two-version
upgrade directly, instead of version-by-version? Or that "upgrade with
minor breakage" + "upgrade with minor breakage" = "too much for the
project to survive"?

> I realize the desire to grow things and I applaud it. But there is a
> business issue here. I can't, in good conscience recommend Django as a
> site platform to many of my small clients as they simply could not
> afford the upkeep of a Django powered site. Especially if the site is
> e-commerce related, where PCI, and responsible site operation, will
> require that we stay current. In order to do so would require staying up
> with the constant flow of backwards incompatible changes, combined with
> the time and effort to reverse engineer and maintain contributed apps,
> which aren't keeping pace either.
> 
> With the current method of development on the Django platform, if I had
> just a dozen sites of moderate complexity, it would become a full time
> job just keeping them updated. Its complicated enough just finding the
> apps that will actually work with each other to construct a site. But
> the carefully constructed house of cards is virtually guaranteed to
> break with the next update.
> 
> So I ask, PLEASE return to and stick with the promise of API stability?
> You promised and routinely point to that statement, while making
> backwards incompatible changes. I want to spend more time working with
> Django, but I need to know that my clients can rely on painless and cost
> effective upgrades.
> 
> Thanks for reading my complaint,
> Jon
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-users@googlegroups.com
> .
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/11e78690-2b5f-4e99-a377-62c19b74e333%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/557886D1.9030608%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Best Practices for Migrations in Reusable Apps?

[Carl Meyer]

Hi Jonathan,

On 06/10/2015 12:24 AM, Jonathan Barratt wrote:
> I'm interested to know what the community's opinions are on how best
> to package migrations with reusable apps (I'm thinking of the
> built-in migrations, but it seems the same principles would apply for
> pre-1.8 south migrations).
> 
> My current approach is to squash all schema migrations each time a
> new one is generated so that new installs are smooth;

You can do this, but you should keep the old migrations around as well,
for the sake of your existing upgrading users. You should only remove
replaced migrations once you are confident that all your users (or
whatever percentage of them you care about) have upgraded.

The migration-squashing system is designed to make it easy to squash
migrations for new users but also keep the old ones around for upgrading
users; it automatically takes care of making sure the right migrations
are used in each case.

> we've seen
> issues arise from just keeping the individual migrations that
> accumulate the long way (in terms of both speed and bugs).

Speed can be an issue with lots of migrations; squashing migrations (but
keeping the old ones) addresses this.

Not much that can be said about bugs without specific examples. The
migration system has certainly had its share of bugs! Almost all the
ones we know about have been fixed. Your old set of migrations should
result in the exact same schema as the new squashed one. If it doesn't,
that's certainly a problem that needs to be fixed, in Django and/or in
one or more of the migration files.

> I figure existing users who upgrade can be expected to run their own
> make migrations if necessary, but perhaps I'm placing the burden of
> work on the wrong party...

This is placing the burden on the wrong party (and on many parties,
instead of one party), and it actually puts your upgrading users in
quite a pickle. They can't add migrations directly to your codebase
without forking it, which means their only recourse to add the necessary
migration is to set MIGRATION_MODULES to ignore your migrations entirely
and replace them with their own.

MIGRATION_MODULES is intended for rare cases, not routine use. Your app
should contain the necessary migrations for all your users, new and
upgrading. Failing to provide migrations that will work for upgrading
users means you miss out on all the benefits of migrations; you're
essentially returning to the old syncdb world, where no provision is
made for upgrades that include schema changes.

> If initial data is necessary, then it gets its own separate data
> migration - this seems to keep the code (and git history) cleaner
> IMO.
> 
> I'd greatly appreciate and criticism, suggestions or alternate
> approaches that occurs to you.
> 
> Thanks in advance, Jonathan

Hope that helps,

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55788921.2020406%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Squashed migration

[Carl Meyer]

On 06/12/2015 06:32 AM, aRkadeFR wrote:
> You need to delete your old migrations so it uses only the squashed
> one after.

No, the squashed migration should be used in place of the old ones for
any new database, even if the old ones are still present. This is the
point of the squashmigrations feature; that you can keep the old ones
around (which is necessary for any deployments that may not yet have
applied all of them) while still gaining the benefit of the new squashed
migration for new deployments (and tests).

I know this works, because I just did it recently myself. It sounds like
Cherie was able to get it working too, though we didn't get any
clarification on why it didn't seem to work originally.

> In the documentation:
> "This enables you to squash and not mess up systems currently in
> production that aren’t fully up-to-date yet. The recommended process is
> to squash, keeping the old files, commit and release, wait until all
> systems are upgraded with the new release (or if you’re a third-party
> project, just ensure your users upgrade releases in order without
> skipping any), and then remove the old files, commit and do a second
> release."

That's right. The length of time you need to wait before removing can
vary widely. For a third-party app, it may be a full release cycle or
two (as long as you can tell your users to upgrade version-by-version).
For a project with only a few deployments, all under your control, it
may be the same day. But regardless, the squashed migration will still
be used in tests immediately, before you remove the old migrations.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/557AFDF3.1090900%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Squashed migration

[Carl Meyer]

Hi Cherie,

On 06/12/2015 06:32 AM, Cherie Pun wrote:
> Thanks for your replies! I am still on Django 1.7 but we are hoping to
> upgrade to 1.8 soon. I manage to use the squashed migrations when
> creating the test database now,

Do you know why it didn't work initially? That'd be useful to know, in
case it's something that might trip up other users, too.

> but there's syntax error in the
> automatically generated squash migration. The error occurs on the line
> on which it reference the user-defined functions in other migrations. It
> seems fine when it ran the first methods though.

That sounds like a probably-fixable bug (though it may already be fixed
in 1.8 and/or master). If you're able to reproduce the bug on master,
please do file a ticket for it.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/557AFE4C.7060403%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django 1.8 Transaction Begin

[Carl Meyer]

Hi Kate,

On 06/18/2015 04:05 AM, Katarzyna Łabędź wrote:
> Maybe i didn't understand it correctly. 
> 
> I know Django support transactions, but isn't it do it on commit? 
> I just tried to find the BEGIN statement inside them and i could not
> find it - searched through internet and what I found was a bunch of
> people asking the same question - where is the begin in the
> transactions? So i just figured i ask here. 
> Why Django don't execute Begin statement ? Isn't it necessary to have a
> fully correct transaction?

You are right to be confused, because this is confusing territory.

The default behavior of all Python database adapters is "implicit
transactions", meaning the underlying database adapter library itself
(Python-MySQL, psycopg2, etc) sends a BEGIN automatically as soon as you
send your first query. (At least if the library is compliant with PEP
249 [1], the Python database adapter standard, which all adapters used
by built-in Django backends are.)

Django overrides this and places the underlying connection into
autocommit mode (no transactions) by default. But when you request a
transaction (by using an 'atomic' block), Django doesn't need to
explicitly send BEGIN, it just puts the connection back into the default
PEP 249 "implicit transactions" mode, which means the database adapter
will send BEGIN automatically as soon as you send a query. So Django
never sends a BEGIN itself, just COMMIT or ROLLBACK.

In practice, though, you can ignore this implementation detail, and just
assume that a BEGIN is sent at the start of any (outer) 'atomic' block
(and COMMIT or ROLLBACK) is sent at the end, and you'll be correct.

I wrote a blog post [2] which discusses this in more detail. It's about
PostgreSQL instead of MySQL, but the part near the beginning about PEP
249 and Django applies equally well to MySQL (I think; I'm not very
familiar with transactions on MySQL).

Carl

[1] https://www.python.org/dev/peps/pep-0249/
[2] http://oddbird.net/2014/06/14/sqlalchemy-postgres-autocommit/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5582E88F.7090709%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Inconsistency when importing views and models

[Carl Meyer]

Hi,

On 06/19/2015 08:32 PM, Peith Vergil wrote:
> It depends on how your project files are structured. If you have a file
> structure that looks like this:
> 
> app/
> __init__.py
> models.py
> views.py
> urls.py
> 
> Then, you can simply use relative imports.
> 
> In views.py:
> from models import *
> 
> In urls.py:
> from views import *

This style of import is called "implicit relative imports", and you
should never use it. It is deprecated in Python 2, and doesn't work at
all in Python 3. (You can opt in to the Python 3 behavior in Python 2
with "from __future__ import absolute_import".)

The reason it is "implicit" is because it is ambiguous - the "models"
and "views" modules you are importing from could be relative to the
current module location, or "absolute" (directly on sys.path), and it
opens the possibility of accidentally shadowing a top-level installed
module (even a stdlib module) with a local module (e.g. if you put a
`datetime.py` or a `django.py` into your package), causing hard-to-debug
issues.

(Also, as has been mentioned, you shouldn't use `import *` because it
leads to ambiguity as to the source of names in your code.)

The right way to do relative imports is with the explicit syntax:

from .models import MyModel
from .views import some_view

or to import a whole module:

from . import models
from . import views

Using the `.` make it explicit that you are importing from a local
module relative to your current module, not from a top-level module.

Carl

> If, for example, your models live in a separate Django app (i.e. a
> separate Python package):
> 
> app1/
> __init__.py
> views.py
> urls.py
> app2/
> __init__.py
> models.py
> 
> Then, you have to import your models using its full path so Python will
> know where to look for it.
> 
> In app1/views.py:
> from app2.models import *
> 
> You should familiarize yourself on how Python imports work since this is
> more of a Python issue rather than a Django issue.
> 
> On Jun 20, 2015 09:10,  > wrote:
> 
> This is mostly a cosmetic question, and I could be completely wrong
> because I'm fairly new to Django, or it could be that there is a
> perfectly logical explanation for this, but here goes:
> 
> It seems the code required to import views in urls.py and models in
> views.py is inconsistent (and in the case of urls.py perhaps redudant).
> 
> To import views in urls.py I have to use
> 
> |
> fromimportviews
> |
> 
> ...while in views.py I can simply write
> 
> |
> frommodels import*
> |
> 
> Why do I need to reference the appname in urls.py but not in
> views.py? Or is there a reason for this?
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to django-users+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-users@googlegroups.com
> .
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/django-users/03eb4740-6503-4757-ae09-9183ac2b8502%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-users@googlegroups.com
> .
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAMHY-KdkaS0RBdS4F4mi%3DaGHTP9uZThJVCesXM9%3DGbVbZoteaw%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55876924.4090409%40oddbird.net.
For more options, visit https

Re: Inconsistency when importing views and models

[Carl Meyer]

On 06/19/2015 11:41 PM, James Schneider wrote:
> This is strictly a Python question, nothing Django-specific, but I've
> found this site to be helpful in explaining the different ways to import
> things:
> 
> http://effbot.org/zone/import-confusion.htm

This is good as far as it goes, but it is also quite old, and thus
doesn't cover explicit relative imports at all (because they didn't
exist yet when it was written).

> In general, using 'from blah import *' is frowned upon except in very
> specific cases. While it is easier up front, it makes the code hard to
> read later on, since you aren't explicitly dating where the functions or
> classes you are using are coming from.
> 
> In general, you'll want to be as specific as possible when importing
> things to keep your code clean and easy to read, even if it does make it
> a bit longer.
> 
> For example:
> 
> from CatAppA import *
> from CatAppB import *
> 
> my_cat = get_cat('Katy Purry')
> 
> If we were inspecting this code, we wouldn't know whether or not
> get_cat() comes from CatApp A or B, which makes troubleshooting more
> difficult, especially if both apps (or modules) define a get_cat()
> function. It would be much better to say:
> 
> from CatAppA import get_cat
> from CatAppB import say_meow
> 
> my_cat = get_cat('Katy Purry')
> 
> Now we know that get_cat() comes from CatAppA.

Yes, that's a great description of the reason to avoid `import *`.

> The other aspect to consider is memory use. If a module defines 500
> functions, doing 'import modulename' will load all 500 in memory, rather
> than just picking out the one or two you might use.

This isn't true. When you first import a module (no matter what syntax
you use), its full code is executed, resulting in a module object with
all of its top-level objects and attributes defined, which is placed
into the `sys.modules` dictionary. So every function or class defined in
the module will be in memory no matter what.

The import syntax you use determines only which functions/objects you
get references to in the current module, it doesn't change memory usage
at all.

(Not to mention that, apart from extreme cases, memory usage of your
code objects themselves will be negligible compared to your data.)

> Aside from that, all of your references would be prefixed with
> 'modulename', which may or may not make the code more readable.

This isn't related to `import *`, it depends on the distinction between

   from somemodule import some_func

vs

   import somemodule

In the latter case, you would use `somemodule.some_func` in your code;
in the former you'd use `some_func`. Both of those are equally explicit;
the choice between them is really a matter of preference, conciseness,
and avoiding naming clashes. If I am using only one or two things from a
module (especially if I am using them many times), I'll usually use
`import from` to make the references shorter. If I am using many things
from the module, I'll prefer the latter form (and sometimes shorten the
module name with an alias, via `import somemodule as sm`).

> In general, I import everything explicitly, and I use relative import
> paths for imports in files that are in the same directory:
> 
> from .models import Cat
> 
> Yes, it makes my import lines much longer, but having them more explicit
> makes the code much easier to read on larger chunks of code.
> 
> There it's some contention as to whether or not to use relative imports,
> some devs prefer to use the more explicit appname.models version. Both
> do the same thing.
> 
> Check out the Django source code to see how they handle imports, and
> definitely look at PEP8:
> 
> https://www.python.org/dev/peps/pep-0008/#imports

Good advice; PEP 8 gives useful guidance on these questions.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55876B8F.70705%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Atomic block termination question

[Carl Meyer]

Hi Mike,

On 06/24/2015 02:16 AM, Mike Dewhirst wrote:
> On 24/06/2015 4:43 PM, Mike Dewhirst wrote:
>> When saving a model I'm getting a TransactionManagementError - You can't
>> execute queries until the end of the 'atomic' block
>>
>> Ticket #21540 seems fairly explicit at least where Postgres is
>> concerned. TransactionManagementError prevents what I want to do and I'm
>> not a nuclear expert.
>>
>> How do I terminate the save() method code in that atomic block and then
>> immediately execute my queries?

I'm afraid this description of what you're trying to do is too vague to
be useful. Maybe some sample code would help?

TransactionManagementError is a symptom, not a cause. It means that a
database error occurred inside a transaction, which leaves the
transaction in an unpredictable state, so Postgres wants you to roll
back the transaction (or roll back to a savepoint prior to the error)
before issuing any more database queries.

Possible solutions include:

a) Figuring out why there was a database error, and fixing it so it
doesn't occur.

b) Wrapping the code that might cause a database error in its own
`transaction.atomic` block, so on error that bit of code is rolled back
and later queries within the same transaction can go forward.

> I have implemented a workaround but not sure if it is the best way. Any
> comment appreciated ...
> 
> In the model's clean() method I test for self.pk and if true, execute
> the queries which previously caused the problem. This seems to work but
> hasn't had any testing in production.

Again it's hard to tell without seeing code or traceback, but it sounds
like probably what you've done here is fix the condition that was
causing the error in the first place (that is, solution (a) above). It
sounds like you were probably trying to do some related-model queries
with an unsaved model, and now you've guarded those queries to only
occur if the model is saved. If so, that's not a workaround, it's the
best solution.

>> I need the save() to complete so I can get_or_create some 1:1 records
>> belonging to the model being saved.

Again, sample code would really illuminate what you're trying to do.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/558ADC1E.6070709%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Atomic block termination question

[Carl Meyer]

Hi Mike,

On 06/25/2015 01:53 AM, Mike Dewhirst wrote:
> On 25/06/2015 2:34 AM, Carl Meyer wrote:
>> On 06/24/2015 02:16 AM, Mike Dewhirst wrote:
>>> On 24/06/2015 4:43 PM, Mike Dewhirst wrote:
>>>> When saving a model I'm getting a TransactionManagementError - You
>>>> can't
>>>> execute queries until the end of the 'atomic' block
>>>>
>>>> Ticket #21540 seems fairly explicit at least where Postgres is
>>>> concerned. TransactionManagementError prevents what I want to do and
>>>> I'm
>>>> not a nuclear expert.
>>>>
>>>> How do I terminate the save() method code in that atomic block and then
>>>> immediately execute my queries?
>>
>> I'm afraid this description of what you're trying to do is too vague to
>> be useful. Maybe some sample code would help?
>>
>> TransactionManagementError is a symptom, not a cause. It means that a
>> database error occurred inside a transaction, which leaves the
>> transaction in an unpredictable state, so Postgres wants you to roll
>> back the transaction (or roll back to a savepoint prior to the error)
>> before issuing any more database queries.
> 
> Ok. I thought from reading the ticket that I was trying to do something
> illegal in Postgres - that is issuing a query within a transaction which
> needed to be finalised or rolled back. I took it as a symptom or signal
> and think I understand that Postgres is somewhat more rigorous in this
> regard than MySQL.

Yes, that's right. (Well, the transaction needs to be rolled back. Once
there's been an error, there is no other "finalizing" possible besides a
rollback.) My point was that the transaction gets into this state
because some other query causes an error, so your efforts should first
focus on figuring out what that error was and making it not happen, if
possible.

>>
>> Possible solutions include:
>>
>> a) Figuring out why there was a database error, and fixing it so it
>> doesn't occur.
> 
> I separated out all the pre and post-save stuff without the offending
> queries and put them into ...
> 
> def save(self, *args, **kwargs):
> self._pre_save() # nothing tricky here
> super(Substance, self).save(*args, **kwargs)
> self._post_save() # nothing tricky here
> 
> ... which stopped the TransactionManagementError and everything worked
> on existing substances which already had the necessary 1:1 relations AND
> it kinda "worked" when I [saved as new] except obviously the 1:1
> relations were not created.
> 
> ... then did a _post_post_save() with the offending queries ...
> 
> def _post_post_save(self):
> if self.physical_state == SOLID:
> Solid.objects.get_or_create(substance=self)
> elif self.physical_state == LIQUID:
> Liquid.objects.get_or_create(substance=self)
> elif self.physical_state == GAS:
> Gas.objects.get_or_create(substance=self)
> elif self.physical_state == AEROSOL:
> Aerosol.objects.get_or_create(substance=self)
> Aquatic.objects.get_or_create(substance=self)
> Tox.objects.get_or_create(substance=self)
> Terrestrial.objects.get_or_create(substance=self)
> if self.terrestrial:
> # We can't do this in terrestrial.save() and it needs
> # to be recomputed on every save
> self.terrestrial.set_soil_m_factor()
> self.terrestrial.set_vertebrate_m_factor()
> self.terrestrial.set_invertebrate_m_factor()
> 
> ... which as I said is now called from substance.clean(). I realise
> clean() is called before save() but that's all I can think of at the
> moment. Those m_factors are unlikely to change once the concentration
> values (EC50, LD50 etc) upon which they are based are set.

I don't understand why you want to call this from clean() instead of
from save(), but it should work OK as long as you wrap it in an `if
self.pk:` so it doesn't try to run on an unsaved object. Of course that
means it will never run at all when saving a new object.

>> b) Wrapping the code that might cause a database error in its own
>> `transaction.atomic` block, so on error that bit of code is rolled back
>> and later queries within the same transaction can go forward.
> 
> That sounds like nuclear physics to me. I could probably follow a recipe
> but might have trouble figuring out when to use it.

Here's the general recipe. This code is problematic:

@transaction.atomic
def do_something():
do_the_first_thing()
try:
do_a_t

Re: Django model for non web applications

[Carl Meyer]

On 07/02/2015 05:49 PM, Russell Keith-Magee wrote:
> 
> On Thu, Jul 2, 2015 at 12:50 AM, Jeff Fritz  > wrote:
> 
> I'm fairly new to Django and making my way through the tutorial.
> 
> Knowing what I've learned so far, I'd like to explore using
> django.db.models.Model in a non-web application. I'd like to take
> the entire model layer and use it for database access in a
> standalone, multithreaded python 3.4 application. It is not a web
> application. I would like to be able to take advantage of the
> migration capabilities in manage.py.
> 
> Questions:
> 
> 1. Is it as simple as adding `from djanago.db import models`? Would
> this bring in any unnecessary django packages/modules, considering I
> will not be developing a web application, not using views,
> templating, etc?
> 
> 
> It's *almost* this simple - you will also need to configure your Django
> environment before you start making database calls. If you're looking to
> do this in a *completely* "non-web" way, this probably means a call to
> settings.configure(); details here:
> 
> https://docs.djangoproject.com/en/1.8/topics/settings/

Don't forget that in Django 1.7+ you also need to call django.setup()
yourself, if using Django outside the context of a WSGI server or
management command:
https://docs.djangoproject.com/en/1.8/ref/applications/#django.setup

Carl

>  
> 
> 2. Is the django model layer thread safe? Specifically, when using
> sqlite3 as a back end?
> 
> 
> It should be - after all, web servers are frequently multi-threaded.
> SQLite3's performance under multithreading, however, might leave
> something to be desired.
>  
> 
> 3. Are there other clean ORM/database model layer
> frameworks/packages that I should consider for python 3.4?
> 
>  
> The other obvious candidate would be SQLAlchemy; it's a perfectly fine
> ORM - without any of the other web framework overhead. It's a lot more
> like a "SQL building toolkit" than Django's ORM - whether this is a good
> or bad thing depends on your own preferences and use case.
> 
> Yours,
> Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5595DC21.5060600%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django formset hidden id field

[Carl Meyer]

On 06/28/2015 03:00 PM, Peter of the Norse wrote:
> On May 27, 2015, at 7:47 AM, Cheng Guo  > wrote:
>>
>> Hello,
>>
>> I have a formset and when I render it, Django would include this line
>> in the HTML:
>>
>> ||
>>
>> I am curious what is the purpose of having an id field here. 
>>
>> I mean in what situation would you use it. I did look through
>> Django's documentation on formset
>> but
>> cannot find much documentation on this.
>>
>> One answer I got is that this id field is the value of the primary key
>> of the model bound to this form. It is there so that when the formset
>> updates, people can use it to retrieve the corresponding record from
>> the database.
>>
>> Is the above explaination correct?
>>
>> If this explaination is correct, then my next question is, wouldn't it
>> be dangerous to expose the primary key like that? I can make a post
>> call to your server with a modified pk which can mess up your database.
> 
> So what?  It’s quite likely that whoever is editing this row of the
> database, also has permissions to edit the other rows as well.  There’s
> no reason for them to go through the hassle of manually editing a hidden
> field when they can go to a different page and edit it there.

That's a bad answer. It's common in many systems for a user to have
access to edit some records in a table but not others (this is often
known as "object-level permissions"). If it was really possible to edit
any row in the table by just manually editing the PK hidden field, that
would be a serious security flaw in formsets.

But it's not. Whatever queryset you pass to the model formset limits the
available rows for editing. The end user can edit the PK to refer to any
item in that queryset, but not any item in the table.

> In general, primary keys are not security flaws.  While it’s a good idea
> to hide them from front-end pages, that’s mostly because they make URLs
> hard to read.  I have heard that you don’t want to use them publicly,
> because your competitors can use them to gauge your success, but that’s
> the kind of “Nice Problem to Have” that can wait until you’re bigger.

Exposing primary keys themselves (e.g. in URLs) is not necessarily a
security flaw. Exposing a primary key in a hidden field that can be
edited to change a form to edit any row in a table often would be a
serious security flaw.


Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5595E969.1090205%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django model for non web applications

[Carl Meyer]

On 07/03/2015 07:46 AM, Vernon D. Cole wrote:
> Jeff:
>I think that Russell's answer might be more appropriate for your use
> case than Carl's.  django.setup() calls settings.configure(), but also
> tries to pull in other application modules, which you might not want.

No, `settings.configure()` and `django.setup()` are not alternatives.
They are _both_ needed for Jeff's use case (depending on how he wants to
handle settings). `settings.configure()` is one option for settings;
`django.setup()` is required no matter what (since Django 1.7).

Settings can be handled either via the usual Python settings module, in
which case the DJANGO_SETTINGS_MODULE env var needs to be set to the
import path to this module, or they can be handled by calling
`settings.configure()` with a dict of the desired settings.

One of those two things needs to be done before calling
`django.setup()`. `setup()` does not "call settings.configure()" - it
loads settings, which requires either that `settings.configure()` has
already been called, or that DJANGO_SETTINGS_MODULE is set.

And yes, `django.setup()` also loads installed applications. Since
Django 1.7, this is required in order to use the ORM (which is the core
of Jeff's use case).

Carl

> 
> 
> On Thursday, July 2, 2015 at 6:50:20 PM UTC-6, Carl Meyer wrote:
> 
> On 07/02/2015 05:49 PM, Russell Keith-Magee wrote:
> >
> > On Thu, Jul 2, 2015 at 12:50 AM, Jeff Fritz  
> > <mailto:jeff...@gmail.com >> wrote:
> >
> > I'm fairly new to Django and making my way through the tutorial.
> >
> > Knowing what I've learned so far, I'd like to explore using
> > django.db.models.Model in a non-web application. I'd like to take
> > the entire model layer and use it for database access in a
> > standalone, multithreaded python 3.4 application. It is not a web
> > application. I would like to be able to take advantage of the
> > migration capabilities in manage.py.
> >
> > Questions:
> >
> > 1. Is it as simple as adding `from djanago.db import models`?
> Would
> > this bring in any unnecessary django packages/modules,
> considering I
> > will not be developing a web application, not using views,
> > templating, etc?
> >
> >
> > It's *almost* this simple - you will also need to configure your
> Django
> > environment before you start making database calls. If you're
> looking to
> > do this in a *completely* "non-web" way, this probably means a
> call to
> > settings.configure(); details here:
> >
> > https://docs.djangoproject.com/en/1.8/topics/settings/
> <https://docs.djangoproject.com/en/1.8/topics/settings/>
> 
> Don't forget that in Django 1.7+ you also need to call django.setup()
> yourself, if using Django outside the context of a WSGI server or
> management command:
> https://docs.djangoproject.com/en/1.8/ref/applications/#django.setup
> <https://docs.djangoproject.com/en/1.8/ref/applications/#django.setup>
> 
> Carl
> 
> >  
> >
> > 2. Is the django model layer thread safe? Specifically, when
> using
> > sqlite3 as a back end?
> >
> >
> > It should be - after all, web servers are frequently multi-threaded.
> > SQLite3's performance under multithreading, however, might leave
> > something to be desired.
> >  
> >
> > 3. Are there other clean ORM/database model layer
> > frameworks/packages that I should consider for python 3.4?
> >
> >  
> > The other obvious candidate would be SQLAlchemy; it's a perfectly
> fine
> > ORM - without any of the other web framework overhead. It's a lot
> more
> > like a "SQL building toolkit" than Django's ORM - whether this is
> a good
> > or bad thing depends on your own preferences and use case.
> >
> > Yours,
> > Russ Magee %-)
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users+unsubscr...@googlegroups.com
> <mailto:django-users+unsubscr...@googlegroups.com>.
> To post to this group, send email to django-users@googlegroups.com
> <mailto:django-users@googlegroups.com>.
> Visit this group at http://groups.google.com/group/dja

Re: Django formset hidden id field

[Carl Meyer]

Hi Peter,

On 07/04/2015 12:42 AM, Peter of the Norse wrote:
>> On Jul 2, 2015, at 7:46 PM, Carl Meyer  wrote:
>> 
>>> So what?  It’s quite likely that whoever is editing this row of
>>> the database, also has permissions to edit the other rows as
>>> well.  There’s no reason for them to go through the hassle of
>>> manually editing a hidden field when they can go to a different
>>> page and edit it there.
>> 
>> That's a bad answer. It's common in many systems for a user to
>> have access to edit some records in a table but not others (this is
>> often known as "object-level permissions"). If it was really
>> possible to edit any row in the table by just manually editing the
>> PK hidden field, that would be a serious security flaw in
>> formsets.
>> 
>> But it's not. Whatever queryset you pass to the model formset
>> limits the available rows for editing. The end user can edit the PK
>> to refer to any item in that queryset, but not any item in the
>> table.
>> 
>>> In general, primary keys are not security flaws.  While it’s a
>>> good idea to hide them from front-end pages, that’s mostly
>>> because they make URLs hard to read.  I have heard that you don’t
>>> want to use them publicly, because your competitors can use them
>>> to gauge your success, but that’s the kind of “Nice Problem to
>>> Have” that can wait until you’re bigger.
>> 
>> Exposing primary keys themselves (e.g. in URLs) is not necessarily
>> a security flaw. Exposing a primary key in a hidden field that can
>> be edited to change a form to edit any row in a table often would
>> be a serious security flaw.
> 
> You can’t have it both ways. Either exposing the PK is a security
> flaw or it isn’t. It’s just as easy for nefarious n’er-do-wells to
> edit the form’s URL as a hidden field. In either case, if you are
> using object-level permissions, more checks (not made automatic in
> Django) are necessary. Having the ID passed as a parameter doesn’t
> necessitate, hinder, or alleviate running these checks.

Yes, this is a good clarification, thanks. What I meant to say is that
simply exposing the ID information to the user isn't necessarily a
security issue. But any server-side use of a client-provided ID (whether
provided in the URL or in a form field) needs to be properly validated
on the server side (just like any other data received from the client).
Formsets do this validation; if they didn't, they would be insecure.

My main point is that it's not at all adequate to say that "if a user
has access to edit one object, they probably have access to edit all the
others anyway." They may or they may not; that's application-specific.
Formsets allow the developer to specify exactly what queryset of objects
the user should be allowed to edit via the formset, and they validate
that the user can only edit those objects, and no others (even if the
hidden ID field is manually modified.)

> If you can come up with a method that associates form data with a
> database row that doesn’t involve the PK, I would love to know. Keep
> in mind that most tables only have the one unique identifier.

I'm not saying that you shouldn't use unique IDs to associate form data
with rows in the database (though there are some valid reasons to prefer
e.g. a UUID or a unique slug to an auto-incrementing ID in some cases).
I'm just saying that you should never trust an ID value (or any data)
coming from the client; you should ensure on the server side that the
user actually should have access to the object they are trying to access.

Javier showed a good method for doing this in typical cases, by passing
a limited queryset rather than a model class to get_object_or_404.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/559ACC8C.8000104%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Testing related models without saving

[Carl Meyer]

Hi Roland,

On 07/16/2015 07:46 AM, Roland Swingler wrote:
>> i'll just say that Django tests don't favor too much into the "unit"
> test spectrum;
> 
> That is what I'm seeing - it is the same with rails out of the box as
> well. I guess I'm just curious to know whether:
> 
> a) the 'unit' test end of the spectrum is feasible if that's what you like
> b) what is available in the Django community (if anything, whether
> libraries, 'ways-of-doing-things' etc.) to support this if it is an
> approach one wants to take.

I also write model tests using unsaved models where possible. I don't
think it has significant test isolation benefits (your tests are still
integrating with most of the ORM), but it does have test-suite speed
benefits!

I understand why the change was made in 1.8 to disallow assigning an
unsaved object to a ForeignKey/OneToOneField attribute; in production
code that would almost always be a bug. Personally, though, I've never
been bitten by that bug, I'm confident I could easily find and fix such
a bug if I did write it, and I don't want to give up the ability to use
related unsaved models in tests. So I just use my own subclasses of
ForeignKey and OneToOneField with `allow_unsaved_instance_assignment =
True` set on them (essentially reverting the safety change in 1.8). I
haven't attempted to switch it on dynamically for testing; that should
be possible using a setting and a custom subclass, but I wouldn't choose
to do that; differences between test and production behavior should be
minimized.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55A7DDB4.9080506%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: All posts from same day django

[Carl Meyer]

Hi,

On 07/22/2015 12:54 PM, Nkansah Rexford wrote:
> I currently have:
> 
> |
> @classmethod
> defget_past_week(self):
> start_date =datetime.now().date()
> end_date =datetime.now().date()-timedelta(weeks=1)
>
> returnMyModel.objects.filter(pub_date__range=(end_date,start_date)).aggregate(Sum('off_hours'))
> |
> 
> which simply pulls all posts from the current date minus 7 days
> 
> I want to pull posts from within the same day factoring in the time at
> the moment. Thus if the time is 15:00 GMT now, I want all posts from
> 14:59:49 GMT back to 00:00:01 GMT of the same day.
> 
> How can I do something like that?

I'll assume `pub_date` is a `DateTimeField`. If it's a `DateField`, then
this query isn't possible unless you change it to a `DateTimeField`.

You want something like:

from datetime import timedelta
from django.utils import timezone

now = timezone.now()
one_day_ago = now - timedelta(days=1)

return MyModel.objects.filter(pub_date__range=(one_day_ago, now))

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55AFEF6C.7090600%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: All posts from same day django

[Carl Meyer]

On 07/22/2015 02:26 PM, Nkansah Rexford wrote:
> Thanks Carl
> 
> Also got these approaches too, might help someone
> too: 
> http://stackoverflow.com/questions/31571607/all-posts-from-same-day-only-django

I just re-read your original post, and my suggestion was wrong - it gets
you all posts from the last 24 hours. Your accepted answer on SO is
right, except that really you should use `django.timezone.now()` instead
of `datetime.now()`.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55AFFD8C.7040301%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature