Re: concurrency and threading question
On Oct 21, 2009, at 11:55 AM, Daniel Roseman wrote: > > On Oct 21, 9:28 am, Mike Thon wrote: >> I'm new to web programming and I have a basic question about the >> design of my Django application. my application will do some number >> crunching on data files uploaded by users. The data processing will >> take from minutes to hours for each job. I don't expect to ever >> get a >> large number of concurrent users but I'd still like to set it up so >> that I can control the maximum number of data processing jobs that >> are >> run in parallel. I was planning to write a simple FIFO queue manager >> (in fact I think there is a python package for this) and then run the >> data processing in separate threads. I'm also planning to use the >> Django data model for storing the data so I would have multiple >> threads writing to the data store. What is not clear to me is what >> happens when I have more than one visitor to the site. Are multiple >> instances of my Django app launched, one per visitor? I need to >> ensure that I only have one queue manager running on the server, not >> one per visitor. I would be using Apache and either mySQL or sqlite3 >> as the database, in case that matters. >> >> thanks for any help >> Mike > > Take a look at the Celery project[1]. This is a great distributed task > queue for Django that I think will do exactly what you need - each job > request is sent to the queue and managed there, so you don't need to > worry about multiple instances. > > [1]:http://ask.github.com/celery/introduction.html > Thanks for pointing me towards celery. Its probably overkill for what I want to do right now but I'm going to try to set it up anyway. Mike --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Table with 4 Milions of rows
On Jan 12, 2010, at 8:25 PM, nameless wrote: > My table with 4 milions of rows is queried often by ajax. > So I think a performance problems ( I am using also index ). > Ok now take a look at the contenttypes :) I don't know what overhead django will put on the queries but 4 million rows might not be a problem. I suggest you test it before you code a more complex solution. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
following relationships forward in queries
I have an Model with a field that is a oneToOne relationship with a model in another app: class SensePost (models.Model): feedjackpost = models.OneToOneField(Post, blank=True, null=True) ... I'm trying to to write a filter query that uses fields in the related model: senseposts = SensePost.objects.filter(processed=True, feedjackpost__date_created__exact=date) but that filter throws this error: Exception Type: FieldError Exception Value: Cannot resolve keyword 'date_created' into field. Choices are: feedjackpost, id, processed, raw_text, topic So I guess there is something basic about the query syntax that I don't understand? Thanks -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: following relationships forward in queries
Here is the Post model, from the feedjack app
class Post(models.Model):
feed = models.ForeignKey(Feed, verbose_name=_('feed'), null=False,
blank=False)
title = models.CharField(_('title'), max_length=255)
link = models.URLField(_('link'), )
content = models.TextField(_('content'), blank=True)
date_modified = models.DateTimeField(_('date modified'), null=True,
blank=True)
guid = models.CharField(_('guid'), max_length=200, db_index=True)
author = models.CharField(_('author'), max_length=50, blank=True)
author_email = models.EmailField(_('author email'), blank=True)
comments = models.URLField(_('comments'), blank=True)
tags = models.ManyToManyField(Tag, verbose_name=_('tags'))
date_created = models.DateField(_('date created'), auto_now_add=True)
Here is my SensePost model in my app:
class SensePost (models.Model):
feedjackpost = models.OneToOneField(Post, blank=True, null=True)
raw_text = models.TextField(blank=True, null=True)
processed = models.BooleanField(default=False)
For each calendar day, I need a list of SensePost objects. So, this is my code:
dates = Post.objects.dates('date_created', 'day')
for date in dates:
sensepost_list = SensePost.objects.filter(processed=True,
feedjackpost__date_created__exact=date)
and that last line is where I get the FieldError exception. I can think of
other ways to do this - like making a list of Posts rather than SensePosts, and
I should still be able to access the SensePost properties. The way I wrote it
seems the most appropriate...but then it doesn't work...
Thanks
Mike
On Jan 15, 2010, at 10:00 AM, Daniel Roseman wrote:
> On Jan 15, 5:43 am, Michael Thon wrote:
>> I have an Model with a field that is a oneToOne relationship with a model in
>> another app:
>>
>> class SensePost (models.Model):
>> feedjackpost = models.OneToOneField(Post, blank=True, null=True)
>> ...
>>
>> I'm trying to to write a filter query that uses fields in the related model:
>>
>> senseposts = SensePost.objects.filter(processed=True,
>> feedjackpost__date_created__exact=date)
>>
>> but that filter throws this error:
>>
>> Exception Type: FieldError
>> Exception Value:
>> Cannot resolve keyword 'date_created' into field. Choices are: feedjackpost,
>> id, processed, raw_text, topic
>>
>> So I guess there is something basic about the query syntax that I don't
>> understand?
>> Thanks
>
> Can you show the Post model?
> --
> DR.
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: following relationships forward in queries
On Jan 15, 2010, at 11:26 AM, Daniel Roseman wrote:
> On Jan 15, 9:40 am, Michael Thon wrote:
>> Here is the Post model, from the feedjack app
>>
>> class Post(models.Model):
>> feed = models.ForeignKey(Feed, verbose_name=_('feed'), null=False,
>> blank=False)
>> title = models.CharField(_('title'), max_length=255)
>> link = models.URLField(_('link'), )
>> content = models.TextField(_('content'), blank=True)
>> date_modified = models.DateTimeField(_('date modified'), null=True,
>> blank=True)
>> guid = models.CharField(_('guid'), max_length=200, db_index=True)
>> author = models.CharField(_('author'), max_length=50, blank=True)
>> author_email = models.EmailField(_('author email'), blank=True)
>> comments = models.URLField(_('comments'), blank=True)
>> tags = models.ManyToManyField(Tag, verbose_name=_('tags'))
>> date_created = models.DateField(_('date created'), auto_now_add=True)
>>
>> Here is my SensePost model in my app:
>>
>> class SensePost (models.Model):
>> feedjackpost = models.OneToOneField(Post, blank=True, null=True)
>> raw_text = models.TextField(blank=True, null=True)
>> processed = models.BooleanField(default=False)
>>
>> For each calendar day, I need a list of SensePost objects. So, this is my
>> code:
>>
>> dates = Post.objects.dates('date_created', 'day')
>> for date in dates:
>> sensepost_list = SensePost.objects.filter(processed=True,
>> feedjackpost__date_created__exact=date)
>>
>> and that last line is where I get the FieldError exception. I can think of
>> other ways to do this - like making a list of Posts rather than SensePosts,
>> and I should still be able to access the SensePost properties. The way I
>> wrote it seems the most appropriate...but then it doesn't work...
>>
>> Thanks
>> Mike
>
> Hmm, that should work. What happens if you drop the '__exact' from the
> end of the filter? It's the default anyway, so shouldn't make a
> difference to the result.
> --
dropping the __exact results in the same exception
This code works as expected:
post_list = SensePost.objects.filter(feedjackpost__isnull=False,
processed=True)
for post in post_list:
feedjackpost = post.feedjackpost
print feedjackpost.date_created
I remove the reference to the OneToOne field in the filter but then fetch the
OneToOne object through its field directly from the main object. I interpret
this as meaning that my db is populated correctly. --
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
fork: Resource temporarily unavailable running django on fastcgi
I set up django to run on FCGI in a shared hosting account using a guide that I found in my web host's forums. The setup is working except that after accessing some pages I get an error on my shell session: -jailshell: fork: Resource temporarily unavailable I was able to kill the python processes and then I kept a close eye on the number of processes that were launched as I accessed pages. I saw 5 instances of my dispatch.fcgi before I started getting the fork error. I have this in my dispatch.fcgi: runfastcgi(method="threaded", daemonize="false", maxchildren=3, minspare=0, maxspare=1) Also, one of my django views calls 2 web APIs over http and it does this for several dozen rows in the database. I guess that my django site is spawning more threads than my web host allows, but I'm not sure if the problem is in my code or in the way I'm running FCGI. Any ideas? Thanks Mike-- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: coding urls in templates
the root directlry of my domain is hosting wordpress, which I'm using to
develop the landing pages:
www.zetawrite.com (I know, it still needs a lot of work).
The app itself will only be available to logged in users so I thought that the
easiest way to deploy it would be to run it in a subdirectory of the main
website. Now I'm trying to do my first deployment to a staging server.
here's the project level urls.py:
urlpatterns = patterns('',
# Uncomment the admin/doc line below to enable admin documentation:
url(r'^admin/doc/', include('django.contrib.admindocs.urls')),
# Uncomment the next line to enable the admin:
url(r'^zetawrite/admin/', include(admin.site.urls)),
url(r'^zetawrite/accounts/', include('registration.backends.default.urls')),
url(r'zetawrite/', include('sieveapp.urls')),
)
Eventually, If the app gets traction, I'll want to have the wordpress landing
pages and the django project running on separate servers, so I should figure
out how I can move it without changing any urls...
On Aug 31, 2012, at 7:45 AM, Amyth Arora wrote:
> could you post the directory structure and your urls.py file. Thanks.
>
> On Fri, Aug 31, 2012 at 11:00 AM, Mike wrote:
> Quick question: I changed my urls.py so I can run my project in a
> subdirectory and I broke all the urls in my templates. Should I be using {%
> url path.to.some_view v1 v2 %} in all my templates instead of hard coding the
> path?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-users/-/SV-wL8jMdCUJ.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
>
>
>
> --
> Thanks & Regards
>
>
> Amyth [Admin - Techstricks]
> Email - aroras.offic...@gmail.com, ad...@techstricks.com
> Twitter - @a_myth_
> http://techstricks.com/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
Re: coding urls in templates
Hi Amyth- That is exactly what I needed. Thanks. I'm now updating all my
views. - Mike
On Aug 31, 2012, at 8:44 AM, Amyth Arora wrote:
> Hey Mike,
>
> I think i understand what you are trying to achieve now. You can basically
> name your urls and then call the url template tag in the templates with the
> url name as follows, this way you will not break any urls in your template if
> you simply move your app to a new location.
>
> Example:
>
> #zetawrite account urls
>
> urlpatterns += patterns('zetawrite.views',
> url(r'^zetawrite/account/signup/$', 'account_signup',
> name='account_signup'),
> url(r'^zetawrite/account/profile/$', 'account_profile',
> name='account_profile'),
> url(r'^zetawrite/account/settings/$', 'account_settings',
> name='account_settings'),
> )
>
> Then use the url template tag as follows:
>
> for an anchor tag like Profile
> use Profile
>
> Hope this helps.
>
> On Fri, Aug 31, 2012 at 11:33 AM, Michael Thon wrote:
> >
> > the root directlry of my domain is hosting wordpress, which I'm using to
> > develop the landing pages:
> > www.zetawrite.com (I know, it still needs a lot of work).
> >
> > The app itself will only be available to logged in users so I thought that
> > the easiest way to deploy it would be to run it in a subdirectory of the
> > main website. Now I'm trying to do my first deployment to a staging server.
> >
> > here's the project level urls.py:
> >
> > urlpatterns = patterns('',
> > # Uncomment the admin/doc line below to enable admin documentation:
> > url(r'^admin/doc/', include('django.contrib.admindocs.urls')),
> >
> > # Uncomment the next line to enable the admin:
> > url(r'^zetawrite/admin/', include(admin.site.urls)),
> > url(r'^zetawrite/accounts/',
> > include('registration.backends.default.urls')),
> > url(r'zetawrite/', include('sieveapp.urls')),
> >
> > )
> >
> > Eventually, If the app gets traction, I'll want to have the wordpress
> > landing pages and the django project running on separate servers, so I
> > should figure out how I can move it without changing any urls...
> >
> >
> > On Aug 31, 2012, at 7:45 AM, Amyth Arora wrote:
> >
> > could you post the directory structure and your urls.py file. Thanks.
> >
> > On Fri, Aug 31, 2012 at 11:00 AM, Mike wrote:
> >>
> >> Quick question: I changed my urls.py so I can run my project in a
> >> subdirectory and I broke all the urls in my templates. Should I be using
> >> {% url path.to.some_view v1 v2 %} in all my templates instead of hard
> >> coding the path?
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Django users" group.
> >> To view this discussion on the web visit
> >> https://groups.google.com/d/msg/django-users/-/SV-wL8jMdCUJ.
> >> To post to this group, send email to django-users@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> django-users+unsubscr...@googlegroups.com.
> >> For more options, visit this group at
> >> http://groups.google.com/group/django-users?hl=en.
> >
> >
> >
> >
> > --
> > Thanks & Regards
> >
> >
> > Amyth [Admin - Techstricks]
> > Email - aroras.offic...@gmail.com, ad...@techstricks.com
> > Twitter - @a_myth_
> > http://techstricks.com/
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django users" group.
> > To post to this group, send email to django-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-users+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/django-users?hl=en.
> >
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django users" group.
> > To post to this group, send email to django-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-users+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/django-users?hl=en.
>
>
>
>
> --
> Thanks & Rega
Re: Is there any good tutorial on how to daemonize celery?
> CELERY_RESULT_BACKEND = "database"
> CELERY_RESULT_DBURI =
> "postgresql://"+get_env_setting('DB_USER')+":"+get_env_setting('DB_PASSWD')+"@localhost/"+get_env_setting('DB_NAME')
>
> Now I tried to run it with this manage.py command
>
> python manage.py celery worker --app=project.settings.production
>
> It's starting up succesfully but I'm seeing several KeyErrors that celery
> received an unregistered task. I tried inserting the solutions I found when I
> researched. For example, adding a setting CELERY_IMPORTS hasn't solved it.
> Restarting celery also didn't help. Any idea why this could be happening?
>
What happens if you don't set CELERY_RESULT_DBURI and CELERY_RESULT_BACKEND ?
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
