Re: Error configuring Django to run customized comments framework

[Phil Gyford]

A bit late, so you may have solved this or given up on it, but
still... I just had the same error message and eventually solved it by
changing the order of my INSTALLED_APPS. So maybe you have slightly
different settings on your local and live servers, and the apps are
ordered differently on each?

For example if I did this:

INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.comments',
'django.contrib.admin',
'django.contrib.markup',
'django.contrib.flatpages',
'taggit',
'myproject.weblog',
'myproject.aggregator',
'myproject.comments', # My custom comment app
)

then I got the error. But if I did this:

INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.comments',
'django.contrib.admin',
'django.contrib.markup',
'django.contrib.flatpages',
'taggit',
'myproject.comments', # My custom comment app
'myproject.weblog',
'myproject.aggregator',
)

(ie, move 'myproject.comments', my custom comments app, further up)
then it worked fine. It just seems to need to be above
'myproject.weblog', so maybe it's some dependency in my code I don't
fully understand. Anyway, hope that helps.


On Thu, Aug 26, 2010 at 1:54 PM, Groady  wrote:
> I'm having an issue with setting up a Django website which uses the
> Django comments framework on my server. The site runs fine when run
> locally (using manage.py runserver) but when pushed live I'm getting
> the error:
>
> ImproperlyConfigured at /
> The COMMENTS_APP setting refers to a non-existing package.
>
> My server is an apache/mod_wsgi setup. My site contains 2 applications
> called weblog and weblog_comments. I've appended my site's path and
> it's parent directories to my django.wsgi file as per the guide
> located here: http://code.google.com/p/modwsgi/wiki/IntegrationWithDjango
> I can comment out the COMMENTS_APP line from my settings.py and the
> site runs fine so I know site is on the python path correctly.
>
> My custom comment model is called WeblogComment and extends the
> default Comment model. It only extends this to add methods to the
> model, it doesn't change Comment model fields thus It has proxy=True
> in it's Meta class.
>
> Any advice would be great.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To post to this group, send email to django-us...@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-users?hl=en.
>
>



-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Customising comment framework and keeping moderation working

[Phil Gyford]

Hi,

I can't get my extended version of django.contrib.comments to take
notice of moderation.

If I use the standard comments framework, then my subclass of
CommentModerator does the job - I've just used it to define
enable_field, and this allows or prohibits comments as expected.

But if I switch to using my custom version of the comments framework
(which adds an extra field) it seems like the moderator is ignored -
all comments are allowed through, no matter whether the enable_field
on the commented-on object is true or false.

I can't see what I'm missing. Here's the code for my moderator, in
case it helps: http://dpaste.com/253308/ It works fine like that, with
the standard framework, but if I switch the import lines over, add my
custom comments app to INSTALLED_APPS and COMMENTS_APP settings, the
enable_field has no effect.

What's the magical missing step? Thanks.

Phil

-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Customising comment framework and keeping moderation working

[Phil Gyford]

I've investigated further and... it's very strange.

I've got a version of my custom comments framework, with
enable_comments moderation, working fine. Here are the contents of my
working customcomments/forms.py and customcomments/models.py:
http://dpaste.com/253382/

However, if I add "from weblog.models import Blog" to either of those
files, my custom moderation has no effect.

This is a problem, because what I really want to do is not add a
'title' field to each custom comment, but add a ForeignKey field that
links to the Blog object (comments are posted on Entries, each of
which is associated with a Blog). So I will need to import the Blog
class.

In case it helps, here's the contents of weblog/models.py that
contains the Blog class: http://dpaste.com/253384/

This modification really isn't worth the two days I've now spent
puzzling over it, but it's got personal, and I just want to understand
the solution now! Any help very much appreciated.


On Tue, Oct 5, 2010 at 11:27 AM, Phil Gyford  wrote:
> Hi,
>
> I can't get my extended version of django.contrib.comments to take
> notice of moderation.
>
> If I use the standard comments framework, then my subclass of
> CommentModerator does the job - I've just used it to define
> enable_field, and this allows or prohibits comments as expected.
>
> But if I switch to using my custom version of the comments framework
> (which adds an extra field) it seems like the moderator is ignored -
> all comments are allowed through, no matter whether the enable_field
> on the commented-on object is true or false.
>
> I can't see what I'm missing. Here's the code for my moderator, in
> case it helps: http://dpaste.com/253308/ It works fine like that, with
> the standard framework, but if I switch the import lines over, add my
> custom comments app to INSTALLED_APPS and COMMENTS_APP settings, the
> enable_field has no effect.
>
> What's the magical missing step? Thanks.
>
> Phil
>
> --
> http://www.gyford.com/
>



-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Customising comment framework and keeping moderation working

[Phil Gyford]

I've made as minimal a complete demonstration of this as possible, to
try and work out where I'm going wrong:
http://github.com/philgyford/django-commentstest/

It works fine: comments can successfully be disabled on a per-Entry
basis. But, if you add "from weblog.models import Entry" to either
customcomments/forms.py or customcomments/models.py then comments are
*always* allowed through. Any idea why? Thanks.


On Tue, Oct 5, 2010 at 2:46 PM, Phil Gyford  wrote:
> I've investigated further and... it's very strange.
>
> I've got a version of my custom comments framework, with
> enable_comments moderation, working fine. Here are the contents of my
> working customcomments/forms.py and customcomments/models.py:
> http://dpaste.com/253382/
>
> However, if I add "from weblog.models import Blog" to either of those
> files, my custom moderation has no effect.
>
> This is a problem, because what I really want to do is not add a
> 'title' field to each custom comment, but add a ForeignKey field that
> links to the Blog object (comments are posted on Entries, each of
> which is associated with a Blog). So I will need to import the Blog
> class.
>
> In case it helps, here's the contents of weblog/models.py that
> contains the Blog class: http://dpaste.com/253384/
>
> This modification really isn't worth the two days I've now spent
> puzzling over it, but it's got personal, and I just want to understand
> the solution now! Any help very much appreciated.
>
>
> On Tue, Oct 5, 2010 at 11:27 AM, Phil Gyford  wrote:
>> Hi,
>>
>> I can't get my extended version of django.contrib.comments to take
>> notice of moderation.
>>
>> If I use the standard comments framework, then my subclass of
>> CommentModerator does the job - I've just used it to define
>> enable_field, and this allows or prohibits comments as expected.
>>
>> But if I switch to using my custom version of the comments framework
>> (which adds an extra field) it seems like the moderator is ignored -
>> all comments are allowed through, no matter whether the enable_field
>> on the commented-on object is true or false.
>>
>> I can't see what I'm missing. Here's the code for my moderator, in
>> case it helps: http://dpaste.com/253308/ It works fine like that, with
>> the standard framework, but if I switch the import lines over, add my
>> custom comments app to INSTALLED_APPS and COMMENTS_APP settings, the
>> enable_field has no effect.
>>
>> What's the magical missing step? Thanks.
>>
>> Phil
>>
>> --
>> http://www.gyford.com/
>>
>
>
>
> --
> http://www.gyford.com/
>



-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Customising comment framework and keeping moderation working

[Phil Gyford]

Thanks Klaas. That second one does seem related to my problem although
after a couple of days struggling with this issue I'm now at the
limits of my knowledge and don't know if it actually helps me :)

I found what seems like a related ticket that's been opened and closed
a couple of times and tentatively re-opened it with an explanation:
http://code.djangoproject.com/ticket/12812

I'm going to have to move on though and try and work around the
problem for now as this frustration is doing me no good at all :(

Phil


On Wed, Oct 6, 2010 at 9:06 AM, Klaas van Schelven
 wrote:
> Phil,
>
> A quick reply so I may be wrong on the details.
> I think you're running into a limitation on the standard way of doing
> things in Django.
>
> I've talked about this before here:
> http://groups.google.com/group/django-users/browse_thread/thread/22875fd287d0aa81/d6cf04a857424678?show_docid=d6cf04a857424678
>
> and here:
> http://groups.google.com/group/django-developers/browse_thread/thread/363fc7f3ca107f94/25a85be6cce875ed?hide_quotes=no
>
> no time for a full reply here but if it's gotten personal this might
> help you a bit,
>
> Klaas
>
> On Oct 5, 4:36 pm, Phil Gyford  wrote:
>> I've made as minimal a complete demonstration of this as possible, to
>> try and work out where I'm going 
>> wrong:http://github.com/philgyford/django-commentstest/
>>
>> It works fine: comments can successfully be disabled on a per-Entry
>> basis. But, if you add "from weblog.models import Entry" to either
>> customcomments/forms.py or customcomments/models.py then comments are
>> *always* allowed through. Any idea why? Thanks.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Tue, Oct 5, 2010 at 2:46 PM, Phil Gyford  wrote:
>> > I've investigated further and... it's very strange.
>>
>> > I've got a version of my custom comments framework, with
>> > enable_comments moderation, working fine. Here are the contents of my
>> > working customcomments/forms.py and customcomments/models.py:
>> >http://dpaste.com/253382/
>>
>> > However, if I add "from weblog.models import Blog" to either of those
>> > files, my custom moderation has no effect.
>>
>> > This is a problem, because what I really want to do is not add a
>> > 'title' field to each custom comment, but add a ForeignKey field that
>> > links to the Blog object (comments are posted on Entries, each of
>> > which is associated with a Blog). So I will need to import the Blog
>> > class.
>>
>> > In case it helps, here's the contents of weblog/models.py that
>> > contains the Blog class:http://dpaste.com/253384/
>>
>> > This modification really isn't worth the two days I've now spent
>> > puzzling over it, but it's got personal, and I just want to understand
>> > the solution now! Any help very much appreciated.
>>
>> > On Tue, Oct 5, 2010 at 11:27 AM, Phil Gyford  wrote:
>> >> Hi,
>>
>> >> I can't get my extended version of django.contrib.comments to take
>> >> notice of moderation.
>>
>> >> If I use the standard comments framework, then my subclass of
>> >> CommentModerator does the job - I've just used it to define
>> >> enable_field, and this allows or prohibits comments as expected.
>>
>> >> But if I switch to using my custom version of the comments framework
>> >> (which adds an extra field) it seems like the moderator is ignored -
>> >> all comments are allowed through, no matter whether the enable_field
>> >> on the commented-on object is true or false.
>>
>> >> I can't see what I'm missing. Here's the code for my moderator, in
>> >> case it helps:http://dpaste.com/253308/It works fine like that, with
>> >> the standard framework, but if I switch the import lines over, add my
>> >> custom comments app to INSTALLED_APPS and COMMENTS_APP settings, the
>> >> enable_field has no effect.
>>
>> >> What's the magical missing step? Thanks.
>>
>> >> Phil
>>
>> >> --
>> >>http://www.gyford.com/
>>
>> > --
>> >http://www.gyford.com/
>>
>> --http://www.gyford.com/
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To post to this group, send email to django-us...@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-users?hl=en.
>
>



-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: What does an ideal django workflow setup look like?

[Phil Gyford]

After reading a few articles on this kind of thing, I tried to figure
out a good, repeatable way to do some of this, and wrote up the whole
process here: 
http://www.gyford.com/phil/writing/2010/09/29/django-environment.php

It doesn't (yet) cover the best ways of integrating dev/production
servers. I'm only really getting started myself, so would welcome any
further suggestions for improvement.

Phil


On Fri, Oct 22, 2010 at 4:02 PM, Ken  wrote:
> *I've Googled, but have not found any really organized and layman-
> friendly overview, which is why I came here*
>
> I know how to get python/django working on my computer (PC or Mac) and
> have developed simple test apps using django's built-in dev server and
> a mysql back-end. So I can get things to work and don't need help on
> installing stuff.
>
> But as a coding hobbyist, I don't know how to set up a pro workflow. I
> don't know best practices on how to setup and maintain a dev and
> production environment, how to efficiently publish to production, how
> to integrate all that into a version control system (i.e., git),
> basically anything that a pro coder at a startup would take for
> granted as "the way to do things".
>
> I understand there are many different ways and products to use to
> setup a great workflow for developing in django, but would like to
> hear how you or your startup team (or corporate dev group) does it.
> Specifics would be amazing, as I need a little hand holding, i.e.
> please cover anything and everything that I should know in order to
> develop efficiently, robustly, and eventually collaboratively.
>
> Basically, please explain it in a way that a layman can follow the
> steps and setup a workflow without pulling his hair out. =P
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To post to this group, send email to django-us...@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-users?hl=en.
>
>



-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Unit test failing when testing post of a comment

[Phil Gyford]

Hi,

I have a unit test that tests that a comment can be posted on an entry
in a Django blog. Posting a comment myself in the browser works fine,
but the test always fails with this error:

"TemplateSyntaxError: Caught VariableDoesNotExist while rendering:
Failed lookup for key [request] in u'[{}]'"

Which suggests the use of the 'request' variable in a template isn't
working. I guess it's a ContextProcessor problem, but I'm stuck as to
why it fails in the test. Here's the test:


from django.test.client import Client
from django.test import TestCase
import datetime, time
from weblog.models import Entry
from django.contrib.comments.forms import CommentSecurityForm

class WeblogBaseTestCase(TestCase):
fixtures = ['../../aggregator/fixtures/test_data.json', ]

class CommentTestCase(WeblogBaseTestCase):

def post_comment(self, entry_id=1):
form = CommentSecurityForm( Entry.live.get(pk=entry_id) )
timestamp = int(time.time())
security_hash = form.initial_security_hash(timestamp)
c = Client()
response = c.post('/cmnt/post/', {
'name': 'Bobby',
'email': 'bo...@example.com',
'url': '',
'comment': 'Hello',
'content_type': 'weblog.entry',
'timestamp': timestamp,
'object_pk': entry_id,
'security_hash': security_hash,
},follow=True)

self.failUnlessEqual(response.status_code, 200)


I'm stumped as to why this error is generated in the test, but not on
the site, and I'm not sure how to poke around to see where it's going
wrong. Any thoughts?

Thanks.

-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Unit test failing when testing post of a comment

[Phil Gyford]

Answering myself, in case this is useful to anyone in the future...

I eventually discovered the problem lay in the method which checked
submitted comments for spam (a method along these lines:
http://sciyoshi.com/blog/2008/aug/27/using-akismet-djangos-new-comments-framework/
). It expected the request object to have META['HTTP_REFERER'] and
META['HTTP_USER_AGENT']. When the comment was submitted from a unit
test, these weren't present and this was causing things to fall over.

I've no idea why that generated what looked like a template error, but
checking for the presence of these, and using empty strings if they're
not present, has got things working again.


On Thu, Nov 11, 2010 at 5:30 PM, Phil Gyford  wrote:
> Hi,
>
> I have a unit test that tests that a comment can be posted on an entry
> in a Django blog. Posting a comment myself in the browser works fine,
> but the test always fails with this error:
>
> "TemplateSyntaxError: Caught VariableDoesNotExist while rendering:
> Failed lookup for key [request] in u'[{}]'"
>
> Which suggests the use of the 'request' variable in a template isn't
> working. I guess it's a ContextProcessor problem, but I'm stuck as to
> why it fails in the test. Here's the test:
>
> 
> from django.test.client import Client
> from django.test import TestCase
> import datetime, time
> from weblog.models import Entry
> from django.contrib.comments.forms import CommentSecurityForm
>
> class WeblogBaseTestCase(TestCase):
>    fixtures = ['../../aggregator/fixtures/test_data.json', ]
>
> class CommentTestCase(WeblogBaseTestCase):
>
>    def post_comment(self, entry_id=1):
>        form = CommentSecurityForm( Entry.live.get(pk=entry_id) )
>        timestamp = int(time.time())
>        security_hash = form.initial_security_hash(timestamp)
>        c = Client()
>        response = c.post('/cmnt/post/', {
>                                'name': 'Bobby',
>                                'email': 'bo...@example.com',
>                                'url': '',
>                                'comment': 'Hello',
>                                'content_type': 'weblog.entry',
>                                'timestamp': timestamp,
>                                'object_pk': entry_id,
>                                'security_hash': security_hash,
>                            },follow=True)
>
>        self.failUnlessEqual(response.status_code, 200)
> 
>
> I'm stumped as to why this error is generated in the test, but not on
> the site, and I'm not sure how to poke around to see where it's going
> wrong. Any thoughts?
>
> Thanks.
>
> --
> http://www.gyford.com/
>



-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Having LiveServerTestCase use same database as tests

[Phil Gyford]

I'm writing integration tests using LiveServerTestCase/Selenium in
v1.4.1. By default the process that's started by LiveServerTestCase
uses a different database to the test database used by the tests. This
is a pain because I can't use factories (I'm using FactoryBoy at the
moment) to create data to then test in the browser.

Is there a way I can get LiveServerTestCase using the same database as
the tests themselves?

Thanks,
Phil

-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

django.contrib.markup deprecated in 1.5 - what's the alternative?

[Phil Gyford]

I noticed that django.contrib.markup is marked as deprecated in Django
1.5 . If
I'm starting a new project now, what alternative should I be using, to
avoid running into problems with future versions of Django? I could
write my own Markdown (or whatever) filter, but I wondered if there
was going to be a standard replacement built-in?

For bonus points - why are those filters being deprecated?

-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: django.contrib.markup deprecated in 1.5 - what's the alternative?

[Phil Gyford]

Thanks for the pointer Jirka - I hadn't managed to find that ticket.
Makes sense and, like you, I only have a few trusted users entering
text that will be filtered.

On 15 September 2012 19:37, Jirka Vejrazka  wrote:
> Hi Phil,
>
>   incidentally, I was looking at this just recently. The
> contrib.markup was deprecated mainly due to security issues with 3rd
> party libraries that could not be fixed in Django itself and were
> compromising its security. For more, read
> https://code.djangoproject.com/ticket/18054
>
>   Can't tell you what the replacement is. I rolled up my own markup
> filter, but I only have a handful of trusted users for my web app so I
> don't have to be too concerned with trusting their inputs.
>
>   You can copy the markup filter from 1.4 - just be aware of the
> security consequences.
>
>   HTH
>
> Jirka
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-users?hl=en.
>



-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Having LiveServerTestCase use same database as tests

[Phil Gyford]

No, I'm afraid not. I didn't get any replies (nor to a StackOverflow
question http://stackoverflow.com/q/12041315/250962 ) and haven't gone
back to try again. I'd tried everything I could think of already. Good
luck, and let us know if you get anywhere!


-- Forwarded message --
From: Дмитрий Белавенцев 
Date: 21 October 2012 09:43
Subject: Re: Having LiveServerTestCase use same database as tests
To: django-users@googlegroups.com
Cc: p...@gyford.com


Hi, Phil! I faced the same problem. Do you find the solution?

вторник, 21 августа 2012 г., 22:24:59 UTC+7 пользователь Phil Gyford написал:
>
> I'm writing integration tests using LiveServerTestCase/Selenium in
> v1.4.1. By default the process that's started by LiveServerTestCase
> uses a different database to the test database used by the tests. This
> is a pain because I can't use factories (I'm using FactoryBoy at the
> moment) to create data to then test in the browser.
>
> Is there a way I can get LiveServerTestCase using the same database as
> the tests themselves?
>
> Thanks,
> Phil
>
> --
> http://www.gyford.com/

--
You received this message because you are subscribed to the Google
Groups "Django users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/django-users/-/cgi4pq5P9nUJ.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.


-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

How best to cache a queryset of Comments?

[Phil Gyford]

I have a site that uses the standard Django Comments app (well, a subclass
of it, with not many changes) for comments on different content types.

I'd like to be able to cache a list of comments, as fetching the comments
is one of the most frequent and slowest queries that's happening at the
moment.

But I can't see the simplest way to perform this caching. (I'm already
using the per-site cache for logged-out users and per-view cache on a few
views, using memcache.)

At the moment the queryset comes from RenderCommentListNode's render()
method <
https://github.com/django/django/blob/stable/1.5.x/django/contrib/comments/templatetags/comments.py#L200>
So I could subclass RenderCommentListNode, copy the render() method, and
add caching there... but it feels a bit clunky and hacky compared to most
customising I end up doing with Django these days... so is this the
best/only way?

Phil

-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Displaying single-line progress while a management command runs

[Phil Gyford]

I have a custom management command which calls a method in another class,
which fetches lots of data from a third-party API. Fetching the data could
take a few seconds or it could take over an hour, depending on the quantity.

I'd like to display progress on the command line, so the user knows the
command's still going but am struggling with a good way to do this.
Criteria:

a) A single line that updates as the task progresses.
b) That line doesn't display while running unit tests (or is very easy to
disable for tests).
c) That line doesn't display while running the command automatically (via
cron or whatever).

Things I've tried so far:

*1) Using print()*, e.g.:

print('Fetched %d of %d' % (n, total), end='\r')

In a loop, this nicely shows a single line that constantly updates with
progress. But print() is nasty and when I run my unit tests, this output is
displayed among the testing output. I assume it'll also be a pain to have
that output when running the commands scheduled with cron (or whatever).

*2) Using Django logging.* This is "better" than print(), and doesn't mess
up test output, but as far as I can tell there's no way to display a
single, constantly updated, line showing progress. It's only going to show
one line after another:

Fetched 1 of 3000
Fetched 2 of 3000
Fetched 3 of 3000

*3) Using tqdm* . Makes it easy to
display a command line progress bar but, again, I end up with loads of
progress bars displaying in my test output, and I assume it'll do the same
when scheduling the task to run.


Have I missed a standard way to do this? Or is there a way I haven't found
to easily suppress tqdm's output during tests and when running the task
scheduled? I can't be the first to have wanted this...

Thanks,
Phil

-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAAU%3D2qRS8_HBadL5Oqga8h8k7yRnuu2NHyH%2B60PT_-Pi7RyskQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Displaying single-line progress while a management command runs

[Phil Gyford]

Belated thanks for this Erik - that does work nicely. It gets
complicated/annoying trying to untangle other kinds of logging too,
including logging from third-party modules, but that's a separate problem :)

On 9 May 2016 at 21:33, Erik Cederstrand  wrote:

>
> > Den 9. maj 2016 kl. 14.23 skrev Phil Gyford :
> >
> > I have a custom management command which calls a method in another
> class, which fetches lots of data from a third-party API. Fetching the data
> could take a few seconds or it could take over an hour, depending on the
> quantity.
> >
> > [...]
> > Things I've tried so far:
> >
> > 1) Using print(), e.g.:
> >
> > print('Fetched %d of %d' % (n, total), end='\r')
> >
> > In a loop, this nicely shows a single line that constantly updates with
> progress. But print() is nasty and when I run my unit tests, this output is
> displayed among the testing output. I assume it'll also be a pain to have
> that output when running the commands scheduled with cron (or whatever).
>
> I do this kind of progress reporting a lot. Usually, I get around the
> test/cron output pollution by adding a 'silent' argument to the management
> command which determines if the commend should print progress reports or
> not. See below.
>
> > 2) Using Django logging. This is "better" than print(), and doesn't mess
> up test output, but as far as I can tell there's no way to display a
> single, constantly updated, line showing progress. It's only going to show
> one line after another:
> >
> > Fetched 1 of 3000
> > Fetched 2 of 3000
> > Fetched 3 of 3000
>
> It's actually quite simple. You need to create a custom handler like so:
>
>   import logging
>   import time
>   from django.core.management.base import BaseCommand
>
>   class OverwriteHandler(logging.StreamHandler):
>   # The extra spaces wipe previous output in case your messages are
> wariable-width
>   terminator = ' '*80 + '\r'
>
>   log = logging.getLogger('')
>   h = OverwriteHandler()
>   log.addHandler(h)
>
>   class Command(BaseCommand):
> def handle(self, silent=False, **options):
>   log.setLevel(logging.DEBUG if silent else logging.INFO)
>   log.info('1 of 2')
>   time.sleep(1)
>   log.info('2 of 2')
>   time.sleep(1)
>
> If you want to mix normal and progress logging in your management command,
> you need to use two loggers with different handlers.
>
> Erik
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/463A7786-888C-4CB0-9C68-43F855401924%40cederstrand.dk
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
http://www.gyford.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAAU%3D2qRf6c%3DU%3DRtTAa8838NvXzxgn_aA1AF7DBA62jHuB9MaPQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.