Model validation across relationships
Given these models, would UserProfile.clean() make sense as written?
class PhoneNumber(models.Model):
user = models.ForeignKey(User, related_name='phone_numbers')
phone_number = models.CharField(max_length=20)
class UserProfile(models.Model):
user = models.OneToOneField(User)
sms_notifications_enabled = models.BooleanField(default=False)
# Given these models / fields, does this implementation make sense?
def clean(self):
if self.sms_notifications_enabled:
if not self.user.phone_numbers.exists():
raise ValidationError("SMS notifications cannot be enabled
because this user has no phone number")
I *think* it’s OK, but the documentation for Model.clean() seems somewhat
vague about what sorts of checks one may implement. Specifically, it says,
“This method should be used to provide custom model validation, and to
modify attributes on your model if desired. For instance, you could use it
to automatically provide a value for a field, or to do validation that
requires access to more than a single field.”
Is the above code in line with best practices?
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/084d8f27-0014-47d0-83bd-29c11b50c645%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Model validation across relationships
Thanks for the reply. I had a (perhaps unreasonable) malaise that
validation should be limited to the fields on a single model.
On Sunday, March 23, 2014 4:19:29 PM UTC-4, Simon Charette wrote:
>
> I'd say that's exactly what you should use `Model.clean()` for. In this
> case you're *doing validation that requires access to more than a single
> field.*
>
> What sounds vague to you in the documentation?
>
> Le vendredi 21 mars 2014 19:43:07 UTC-4, Joshua Pokotilow a écrit :
>>
>> Given these models, would UserProfile.clean() make sense as written?
>>
>> class PhoneNumber(models.Model):
>> user = models.ForeignKey(User, related_name='phone_numbers')
>> phone_number = models.CharField(max_length=20)
>>
>> class UserProfile(models.Model):
>> user = models.OneToOneField(User)
>> sms_notifications_enabled = models.BooleanField(default=False)
>>
>> # Given these models / fields, does this implementation make sense?
>> def clean(self):
>> if self.sms_notifications_enabled:
>> if not self.user.phone_numbers.exists():
>> raise ValidationError("SMS notifications cannot be enabled
>> because this user has no phone number")
>>
>> I *think* it’s OK, but the documentation for Model.clean() seems
>> somewhat vague about what sorts of checks one may implement. Specifically,
>> it says, “This method should be used to provide custom model validation,
>> and to modify attributes on your model if desired. For instance, you could
>> use it to automatically provide a value for a field, or to do validation
>> that requires access to more than a single field.”
>>
>> Is the above code in line with best practices?
>>
>
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/0b3f9b10-f20a-4684-81d7-f04a62bf3790%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Django admin suitable for external users?
Hello! I just had a fairly lengthy conversation with my colleagues about whether or not Django admin is well-suited to external users outside our company. I took the position that for certain use-cases, exposing Django admin to third parties makes a lot of sense, given that the admin application has all kinds of features baked in that are well-suited to certain admin tasks (ACL, customizable templates, dynamically built CRUD forms, etc.). Unfortunately, I met with a lot of resistance on account of fears over ease of customizability, security, and technology lock-in. Furthermore, there was some concern that exposing Django admin to third-parties might send us off the beaten path, and that doing so could be an antipattern. I would appreciate knowing how other developers feel on this subject, and would love to hear about how some larger companies that use Django (Instagram, Disqus) think things through. Thanks. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/59231ea7-4bd1-41c2-97ef-f294a380bcb4%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Django admin suitable for external users?
> > How technical are your users? Not very technical, necessarily. They're normal end-users. What are your security constraints? We're in the medical industry, so security is a key consideration for us in general. I think the main concerns about Django admin specifically were that it encompasses so much functionality, we might wind up shooting ourselves in the foot by failing to lock something down properly. Also, there might be discoverable exploits since it's an open source product. How much work can you do to make it "pretty"? (Believe me, someone will > ask.) > I believe the hope is that we wouldn't spend anymore time making it pretty than we would spend if we wrote our own custom HTML. We don't have strict style guidelines we would need to adhere to, except maybe for the header / footer. Are there fields that you want to administer internally but don't want to > expose to the users? > Yes, but I think(?) this can be tailored to the logged-in user. Will your users object if you decide to move to a newer Django version and > the interface changes/ I think it's possible they would, but perhaps not too much. Thank you for the insights! On Wednesday, September 23, 2015 at 4:50:28 PM UTC-4, ke1g wrote: > > How technical are your users? > What are your security constraints? > How much work can you do to make it "pretty"? (Believe me, someone will > ask.) > Are there fields that you want to administer internally but don't want to > expose to the users? > Will your users object if you decide to move to a newer Django version and > the interface changes/ > > You can make it work, but in many instances it will save you less effort > than you though (might even be harder). > > A few custom views are pretty easy to roll out. > > On Wed, Sep 23, 2015 at 4:00 PM, Joshua Pokotilow > wrote: > >> Hello! I just had a fairly lengthy conversation with my colleagues about >> whether or not Django admin is well-suited to external users outside our >> company. I took the position that for certain use-cases, exposing Django >> admin to third parties makes a lot of sense, given that the admin >> application has all kinds of features baked in that are well-suited to >> certain admin tasks (ACL, customizable templates, dynamically built CRUD >> forms, etc.). Unfortunately, I met with a lot of resistance on account of >> fears over ease of customizability, security, and technology lock-in. >> Furthermore, there was some concern that exposing Django admin to >> third-parties might send us off the beaten path, and that doing so could be >> an antipattern. >> >> I would appreciate knowing how other developers feel on this subject, and >> would love to hear about how some larger companies that use Django >> (Instagram, Disqus) think things through. >> >> Thanks. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-users...@googlegroups.com . >> To post to this group, send email to django...@googlegroups.com >> . >> Visit this group at http://groups.google.com/group/django-users. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/59231ea7-4bd1-41c2-97ef-f294a380bcb4%40googlegroups.com >> >> <https://groups.google.com/d/msgid/django-users/59231ea7-4bd1-41c2-97ef-f294a380bcb4%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/b1e73edd-a4b5-440b-ac27-05e2ccfa4bae%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Django admin suitable for external users?
That's an interesting idea. Thanks. I suppose it's possible to register models with two different admin sites, although I've never tried doing that myself. On Wednesday, September 23, 2015 at 5:21:04 PM UTC-4, luisza14 wrote: > > I suggest you to create a custom admin site for your external user where > you expose only the models that interact with the external user and create > a middleware for forbidden access to the other admin site. > > 2015-09-23 14:49 GMT-06:00 Bill Freeman >: > >> How technical are your users? >> What are your security constraints? >> How much work can you do to make it "pretty"? (Believe me, someone will >> ask.) >> Are there fields that you want to administer internally but don't want to >> expose to the users? >> Will your users object if you decide to move to a newer Django version >> and the interface changes/ >> >> You can make it work, but in many instances it will save you less effort >> than you though (might even be harder). >> >> A few custom views are pretty easy to roll out. >> >> On Wed, Sep 23, 2015 at 4:00 PM, Joshua Pokotilow > > wrote: >> >>> Hello! I just had a fairly lengthy conversation with my colleagues about >>> whether or not Django admin is well-suited to external users outside our >>> company. I took the position that for certain use-cases, exposing Django >>> admin to third parties makes a lot of sense, given that the admin >>> application has all kinds of features baked in that are well-suited to >>> certain admin tasks (ACL, customizable templates, dynamically built CRUD >>> forms, etc.). Unfortunately, I met with a lot of resistance on account of >>> fears over ease of customizability, security, and technology lock-in. >>> Furthermore, there was some concern that exposing Django admin to >>> third-parties might send us off the beaten path, and that doing so could be >>> an antipattern. >>> >>> I would appreciate knowing how other developers feel on this subject, >>> and would love to hear about how some larger companies that use Django >>> (Instagram, Disqus) think things through. >>> >>> Thanks. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Django users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to django-users...@googlegroups.com . >>> To post to this group, send email to django...@googlegroups.com >>> . >>> Visit this group at http://groups.google.com/group/django-users. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-users/59231ea7-4bd1-41c2-97ef-f294a380bcb4%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/django-users/59231ea7-4bd1-41c2-97ef-f294a380bcb4%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-users...@googlegroups.com . >> To post to this group, send email to django...@googlegroups.com >> . >> Visit this group at http://groups.google.com/group/django-users. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/CAB%2BAj0tme%3DvZdzKBf5ygJRqtACozy2ugi5qyKjG5WHDWENkUWA%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/django-users/CAB%2BAj0tme%3DvZdzKBf5ygJRqtACozy2ugi5qyKjG5WHDWENkUWA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > "La utopía sirve para caminar" Fernando Birri > > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/8f4edbde-28b5-4002-99e1-d08ea7c30f3b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Django admin suitable for external users?
Thanks Ryan. That's an interesting additional consideration. I don't think we're particularly married to Django admin as an open interface to the database where I work, although I can see how that wouldn't always be the case. Also, I think Luis's suggestion could help to quell the resistance you cited. On Thursday, September 24, 2015 at 4:15:59 PM UTC-4, Ryan Allen wrote: > > It really depends on the project you're working on. The resistance comes > mostly from developers who utilize the admin as an open interface to the > database with minimal restrictions. I like to use the admin this way for > large projects that are used by many people, building out a custom "staff" > portal for non-technical staff users (more data validation, form helpers, > general improved UX) and using the Django admin strictly for developer use. > > I have built smaller projects, generally standard brochure style sites > (things most people would use Wordpress for) and used the admin as the CMS > for the client. I only ever customized the nav bar with client brand > name/colors <http://ryangallen.com/wall/4/tidy-django-admin/> and > utilized the options on the ModelAdmin object > <https://docs.djangoproject.com/en/1.8/ref/contrib/admin/#modeladmin-options> > to make it as convenient as possible. Never had any complaints, it's pretty > straightforward and still allows you to upgrade Django gracefully. > > On Wednesday, September 23, 2015 at 4:00:01 PM UTC-4, Joshua Pokotilow > wrote: >> >> Hello! I just had a fairly lengthy conversation with my colleagues about >> whether or not Django admin is well-suited to external users outside our >> company. I took the position that for certain use-cases, exposing Django >> admin to third parties makes a lot of sense, given that the admin >> application has all kinds of features baked in that are well-suited to >> certain admin tasks (ACL, customizable templates, dynamically built CRUD >> forms, etc.). Unfortunately, I met with a lot of resistance on account of >> fears over ease of customizability, security, and technology lock-in. >> Furthermore, there was some concern that exposing Django admin to >> third-parties might send us off the beaten path, and that doing so could be >> an antipattern. >> >> I would appreciate knowing how other developers feel on this subject, and >> would love to hear about how some larger companies that use Django >> (Instagram, Disqus) think things through. >> >> Thanks. >> > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/7541cddd-3d1a-4ca1-baa6-cfb459428e92%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Scaling Django
At the startup where I work, we've written a lot of our server code in Django. So far, we've adopted a "build it fast" mentality, so we invested very little time in optimizing our code. A small amount of load testing has revealed our codebase / infrastructure as it stands today needs to run faster and support more users. We recently hired some new engineers who are extremely skeptical that we should optimize our existing code. Their main concerns are: - We need to move to a service-oriented infrastructure because Django is too monolithic (monolithic = technology lock-in & difficult to troubleshoot) - It's too easy to write slow queries using the Django ORM - It's hard to hire Django engineers - While Instagram and DISQUS use Django to service large numbers of people, they don't use it for any serious backend work After having worked with Django for the last 3 years, I'm a big believer in it, and I believe it would scale. To defend my position, I've pointed out to my colleagues that it's easy to identify bottlenecks with tools like the Django Debug Toolbar and Yet Another Django Profiler. With my colleagues present, I've isolated and fixed significant speed problems inside of a few hours. I don't believe the Django ORM is inherently bad, although I do think that coders who use it should Know What They're Doing. Finally, I've referenced blog entries that talk about how Instagram and Disqus use Django on the backend for backend-y tasks. Despite my best efforts, my colleagues are still pushing to have us rewrite large portions of our infrastructure as separate services before we try to fix them. For example, we have one slow REST endpoint that returns a boatload of user data, and so there's talk about using a new microservice for users in lieu of our existing Django models. Even if we are able to fix bottlenecks we encounter in a timely fashion, my colleagues fear that Django won't scale with the business. I'm writing this post to garner additional evidence that Django will scale. Anything compelling (and preferably not obvious) that would help shed some light on Django's ability to scale would be *greatly* appreciated, as it's very difficult for me to defend my position that Django is a viable long-term solution without solid evidence to back up my claims. It certainly doesn't help that I don't have any experience scaling Django myself! Thank you. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/83968c41-d415-4189-b33b-9f99b10b1c41%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Scaling Django
The service uses the Django REST Framework and takes multiple seconds to return a response. The response is a JSON array with thousands of dictionaries. We haven't yet investigated why it's slow, nor have we tried to cache / memoize anything to speed it up. On Wednesday, February 3, 2016 at 10:46:25 AM UTC-5, Avraham Serour wrote: > > what do you mean by slow? can you measure in ms? > > On Wed, Feb 3, 2016 at 5:30 PM, Joshua Pokotilow > wrote: > >> At the startup where I work, we've written a lot of our server code in >> Django. So far, we've adopted a "build it fast" mentality, so we invested >> very little time in optimizing our code. A small amount of load testing has >> revealed our codebase / infrastructure as it stands today needs to run >> faster and support more users. >> >> We recently hired some new engineers who are extremely skeptical that we >> should optimize our existing code. Their main concerns are: >> >> - We need to move to a service-oriented infrastructure because Django is >> too monolithic (monolithic = technology lock-in & difficult to troubleshoot) >> - It's too easy to write slow queries using the Django ORM >> - It's hard to hire Django engineers >> - While Instagram and DISQUS use Django to service large numbers of >> people, they don't use it for any serious backend work >> >> After having worked with Django for the last 3 years, I'm a big believer >> in it, and I believe it would scale. To defend my position, I've pointed >> out to my colleagues that it's easy to identify bottlenecks with tools like >> the Django Debug Toolbar and Yet Another Django Profiler. With my >> colleagues present, I've isolated and fixed significant speed problems >> inside of a few hours. I don't believe the Django ORM is inherently bad, >> although I do think that coders who use it should Know What They're Doing. >> Finally, I've referenced blog entries that talk about how Instagram and >> Disqus use Django on the backend for backend-y tasks. >> >> Despite my best efforts, my colleagues are still pushing to have us >> rewrite large portions of our infrastructure as separate services before we >> try to fix them. For example, we have one slow REST endpoint that returns a >> boatload of user data, and so there's talk about using a new microservice >> for users in lieu of our existing Django models. Even if we are able to fix >> bottlenecks we encounter in a timely fashion, my colleagues fear that >> Django won't scale with the business. >> >> I'm writing this post to garner additional evidence that Django will >> scale. Anything compelling (and preferably not obvious) that would help >> shed some light on Django's ability to scale would be *greatly* >> appreciated, as it's very difficult for me to defend my position that >> Django is a viable long-term solution without solid evidence to back up my >> claims. It certainly doesn't help that I don't have any experience scaling >> Django myself! >> >> Thank you. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-users...@googlegroups.com . >> To post to this group, send email to django...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/django-users. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/83968c41-d415-4189-b33b-9f99b10b1c41%40googlegroups.com >> >> <https://groups.google.com/d/msgid/django-users/83968c41-d415-4189-b33b-9f99b10b1c41%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/38b96b56-064a-4933-abb5-e6085cd24425%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Scaling Django
Thanks Bill. This was helpful. I understand that it's difficult to offer advice without too many specifics, so I'm hoping to get some high-level advice / examples of Django at scale, which you provided. Thank you! On Wednesday, February 3, 2016 at 10:49:51 AM UTC-5, Bill Blanchard wrote: > > Let's try to adress some of their concerns: > > - We need to move to a service-oriented infrastructure because Django is >> too monolithic > > > It depends on what your application does and what you're planning to do > with it in the future. People are quick prescribe SOA as the end all way > to scale, but they tend to ignore the added complexity that comes with > building out and integrating smaller services. > > - It's too easy to write slow queries using the Django ORM > > > It's just as easy (arguably easier) to write slow queries using pure SQL > or any other ORM. The ORM makes a lot of good decisions for mediocre > programmers (I'd put myself in that category). If you're a great > programmer and have great programmers who really understand SQL, then > you're just as likely to get your ORM queries right as you are straight > SQL. > >> >> - It's hard to hire Django engineers > > > Compared to what? .NET or Java engineers? Probably. Harder than the > newest shiny javascript framework engineers? Probably not. Django has > about as robust an engineering population as Ruby/Rails does. I don't > know what you'd convert to in order to make hiring easier. All engineers > (especially good ones) are really hard to come by these days. If you're > looking at outsourcing to Southwest Asia, then yes, the Django population > isn't as high as .NET/Java/PHP. However, hiring challenges are most > typically defined by your location and your ability and/or willingness to > explore remote workers. > > While Instagram and DISQUS use Django to service large numbers of people, >> they don't use it for any serious backend work > > > Reddit is also a large Django user. All engineering decisions should be > made around what your particular needs are and what skills your team > possesses or is able to acquire. Needs of an organization evolve over time > and the organizations adjust as they need to. > > Many organizations start with a Python/Django or Ruby/Rails application to > build a product *quickly *which is what those stacks excel at. A mantra > typically heard in the community is "don't optimize prematurely". If > you're saying "man, we're going to hit a wall at 100,000 users", well you > need to get to 90,000 users first before worrying about 100,000. Getting > the 90k users is the real hard part. > > All this being said, your colleagues could be right to want to move off > Django. We don't know much about your particular circumstances. > > For more information on optimizing Django for scale, check out this book. > https://highperformancedjango.com/ > > Best of luck. > > Bill > > > > > On Wed, Feb 3, 2016 at 10:30 AM, Joshua Pokotilow > wrote: > >> At the startup where I work, we've written a lot of our server code in >> Django. So far, we've adopted a "build it fast" mentality, so we invested >> very little time in optimizing our code. A small amount of load testing has >> revealed our codebase / infrastructure as it stands today needs to run >> faster and support more users. >> >> We recently hired some new engineers who are extremely skeptical that we >> should optimize our existing code. Their main concerns are: >> >> - We need to move to a service-oriented infrastructure because Django is >> too monolithic (monolithic = technology lock-in & difficult to troubleshoot) >> - It's too easy to write slow queries using the Django ORM >> - It's hard to hire Django engineers >> - While Instagram and DISQUS use Django to service large numbers of >> people, they don't use it for any serious backend work >> >> After having worked with Django for the last 3 years, I'm a big believer >> in it, and I believe it would scale. To defend my position, I've pointed >> out to my colleagues that it's easy to identify bottlenecks with tools like >> the Django Debug Toolbar and Yet Another Django Profiler. With my >> colleagues present, I've isolated and fixed significant speed problems >> inside of a few hours. I don't believe the Django ORM is inherently bad, >> although I do think that coders who use it should Know What They're Doing. >> Finally, I&
Re: Scaling Django
Thank you Sergiy! I agree that the code needs to be fixed. We don't have a Tomcat endpoint to compare with, although I did scare my coworkers a bit when we profiled a Django endpoint that took 300 - 400ms to return a response due (ostensibly) in large part to form object instantiation. Specifically, the bottleneck seemed to be that forms deep-copy their fields on instantiation. On Wednesday, February 3, 2016 at 11:01:43 AM UTC-5, Sergiy Khohlov wrote: > > Hello, > Your first words have a answer. Swift coding always produces performance > problem. This is expected. Looks like few new engineers use another one > technology and would not like to use django. This a reason of his > criticism. Mostly low performance is related to the DB performance. I'm > preferring avoid using ManyToMany ability by due to high res usage at the > DB level. Writing correct models and DB function helps in the most case. > I have no idea about proposed solution but I definitely sure that code > produces bottleneck not programming language. RubyOnRails has a same > problem such a Django for example.. Have you got good performance at > JAva+Tomkat+Apache ? I'm ready to see this high performance ASP. > Half year ago I've rewritten GTS service from C# to python. As result CPU > usage dropped from 45% to 7-9% and memory usage from 1.5Gb to 300kb. > Wrong solution, mistakes at the building project requirement stage > produces more problem that selecting programming language. > > Many thanks, > > Serge > > > +380 636150445 > skype: skhohlov > > On Wed, Feb 3, 2016 at 5:30 PM, Joshua Pokotilow > wrote: > >> At the startup where I work, we've written a lot of our server code in >> Django. So far, we've adopted a "build it fast" mentality, so we invested >> very little time in optimizing our code. A small amount of load testing has >> revealed our codebase / infrastructure as it stands today needs to run >> faster and support more users. >> >> We recently hired some new engineers who are extremely skeptical that we >> should optimize our existing code. Their main concerns are: >> >> - We need to move to a service-oriented infrastructure because Django is >> too monolithic (monolithic = technology lock-in & difficult to troubleshoot) >> - It's too easy to write slow queries using the Django ORM >> - It's hard to hire Django engineers >> - While Instagram and DISQUS use Django to service large numbers of >> people, they don't use it for any serious backend work >> >> After having worked with Django for the last 3 years, I'm a big believer >> in it, and I believe it would scale. To defend my position, I've pointed >> out to my colleagues that it's easy to identify bottlenecks with tools like >> the Django Debug Toolbar and Yet Another Django Profiler. With my >> colleagues present, I've isolated and fixed significant speed problems >> inside of a few hours. I don't believe the Django ORM is inherently bad, >> although I do think that coders who use it should Know What They're Doing. >> Finally, I've referenced blog entries that talk about how Instagram and >> Disqus use Django on the backend for backend-y tasks. >> >> Despite my best efforts, my colleagues are still pushing to have us >> rewrite large portions of our infrastructure as separate services before we >> try to fix them. For example, we have one slow REST endpoint that returns a >> boatload of user data, and so there's talk about using a new microservice >> for users in lieu of our existing Django models. Even if we are able to fix >> bottlenecks we encounter in a timely fashion, my colleagues fear that >> Django won't scale with the business. >> >> I'm writing this post to garner additional evidence that Django will >> scale. Anything compelling (and preferably not obvious) that would help >> shed some light on Django's ability to scale would be *greatly* >> appreciated, as it's very difficult for me to defend my position that >> Django is a viable long-term solution without solid evidence to back up my >> claims. It certainly doesn't help that I don't have any experience scaling >> Django myself! >> >> Thank you. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-users...@googlegroups.com . >> To post to this group, send email to django...@
Re: Scaling Django
Thanks Remco. I'll look at High Performance Django. On Wednesday, February 3, 2016 at 11:05:11 AM UTC-5, Remco Gerlich wrote: > > There is a book (ebook) named "High Performance Django" that has many > useful tips. > > Also, new software developers are _always_ skeptical when they start a new > job. They have an old way of doing things from their previous job, "that's > not how we work here!" reflexes also from that old job, and they take a > while to adjust to the new tools. > > First let them get productive with what exists, _then_ let them gather > real statistics about real problems when they want to change something. > > You can have scalibility issues and solve them with whatever framework, > Django included. > > Remco Gerlich > > > On Wed, Feb 3, 2016 at 4:30 PM, Joshua Pokotilow > wrote: > >> At the startup where I work, we've written a lot of our server code in >> Django. So far, we've adopted a "build it fast" mentality, so we invested >> very little time in optimizing our code. A small amount of load testing has >> revealed our codebase / infrastructure as it stands today needs to run >> faster and support more users. >> >> We recently hired some new engineers who are extremely skeptical that we >> should optimize our existing code. Their main concerns are: >> >> - We need to move to a service-oriented infrastructure because Django is >> too monolithic (monolithic = technology lock-in & difficult to troubleshoot) >> - It's too easy to write slow queries using the Django ORM >> - It's hard to hire Django engineers >> - While Instagram and DISQUS use Django to service large numbers of >> people, they don't use it for any serious backend work >> >> After having worked with Django for the last 3 years, I'm a big believer >> in it, and I believe it would scale. To defend my position, I've pointed >> out to my colleagues that it's easy to identify bottlenecks with tools like >> the Django Debug Toolbar and Yet Another Django Profiler. With my >> colleagues present, I've isolated and fixed significant speed problems >> inside of a few hours. I don't believe the Django ORM is inherently bad, >> although I do think that coders who use it should Know What They're Doing. >> Finally, I've referenced blog entries that talk about how Instagram and >> Disqus use Django on the backend for backend-y tasks. >> >> Despite my best efforts, my colleagues are still pushing to have us >> rewrite large portions of our infrastructure as separate services before we >> try to fix them. For example, we have one slow REST endpoint that returns a >> boatload of user data, and so there's talk about using a new microservice >> for users in lieu of our existing Django models. Even if we are able to fix >> bottlenecks we encounter in a timely fashion, my colleagues fear that >> Django won't scale with the business. >> >> I'm writing this post to garner additional evidence that Django will >> scale. Anything compelling (and preferably not obvious) that would help >> shed some light on Django's ability to scale would be *greatly* >> appreciated, as it's very difficult for me to defend my position that >> Django is a viable long-term solution without solid evidence to back up my >> claims. It certainly doesn't help that I don't have any experience scaling >> Django myself! >> >> Thank you. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-users...@googlegroups.com . >> To post to this group, send email to django...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/django-users. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/83968c41-d415-4189-b33b-9f99b10b1c41%40googlegroups.com >> >> <https://groups.google.com/d/msgid/django-users/83968c41-d415-4189-b33b-9f99b10b1c41%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/eff1cd01-58f0-48d8-b58d-ef4d71a6ab29%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Scaling Django
Thank you! I agree that we need to investigate before coming up with a solution. On Wednesday, February 3, 2016 at 11:11:04 AM UTC-5, Avraham Serour wrote: > > if your problem is the DB or network or small processor it won't help > rewriting the application. > The first step is to investigate the problem, then you can have solution, > sometimes people have a solution and then look for a problem, in your case > they want to leave python and django and are looking for problems with it. > > more than which language, library and tech stack you use the system design > impacts a lot in the overall speed among other things, I've worked with > systems with too many indirections for example, the performance was not > impacted so much by the libraries we were using or not. > > sometimes rewriting can be a good because of this, it is a way to give you > a chance to make a new design. > > Avraham > > On Wed, Feb 3, 2016 at 6:02 PM, Joshua Pokotilow > wrote: > >> The service uses the Django REST Framework and takes multiple seconds to >> return a response. The response is a JSON array with thousands of >> dictionaries. We haven't yet investigated why it's slow, nor have we tried >> to cache / memoize anything to speed it up. >> >> On Wednesday, February 3, 2016 at 10:46:25 AM UTC-5, Avraham Serour wrote: >>> >>> what do you mean by slow? can you measure in ms? >>> >>> On Wed, Feb 3, 2016 at 5:30 PM, Joshua Pokotilow >>> wrote: >>> >>>> At the startup where I work, we've written a lot of our server code in >>>> Django. So far, we've adopted a "build it fast" mentality, so we invested >>>> very little time in optimizing our code. A small amount of load testing >>>> has >>>> revealed our codebase / infrastructure as it stands today needs to run >>>> faster and support more users. >>>> >>>> We recently hired some new engineers who are extremely skeptical that >>>> we should optimize our existing code. Their main concerns are: >>>> >>>> - We need to move to a service-oriented infrastructure because Django >>>> is too monolithic (monolithic = technology lock-in & difficult to >>>> troubleshoot) >>>> - It's too easy to write slow queries using the Django ORM >>>> - It's hard to hire Django engineers >>>> - While Instagram and DISQUS use Django to service large numbers of >>>> people, they don't use it for any serious backend work >>>> >>>> After having worked with Django for the last 3 years, I'm a big >>>> believer in it, and I believe it would scale. To defend my position, I've >>>> pointed out to my colleagues that it's easy to identify bottlenecks with >>>> tools like the Django Debug Toolbar and Yet Another Django Profiler. With >>>> my colleagues present, I've isolated and fixed significant speed problems >>>> inside of a few hours. I don't believe the Django ORM is inherently bad, >>>> although I do think that coders who use it should Know What They're Doing. >>>> Finally, I've referenced blog entries that talk about how Instagram and >>>> Disqus use Django on the backend for backend-y tasks. >>>> >>>> Despite my best efforts, my colleagues are still pushing to have us >>>> rewrite large portions of our infrastructure as separate services before >>>> we >>>> try to fix them. For example, we have one slow REST endpoint that returns >>>> a >>>> boatload of user data, and so there's talk about using a new microservice >>>> for users in lieu of our existing Django models. Even if we are able to >>>> fix >>>> bottlenecks we encounter in a timely fashion, my colleagues fear that >>>> Django won't scale with the business. >>>> >>>> I'm writing this post to garner additional evidence that Django will >>>> scale. Anything compelling (and preferably not obvious) that would help >>>> shed some light on Django's ability to scale would be *greatly* >>>> appreciated, as it's very difficult for me to defend my position that >>>> Django is a viable long-term solution without solid evidence to back up my >>>> claims. It certainly doesn't help that I don't have any experience scaling >>>> Django myself! >>>> >>>> Thank you. >>>> >>
