Re: [Twisted-Python] Twisted webserver performance issues

2009-11-07 Thread Tom Leys
Neither of the below issues will be a problem Jean-Paul Logged in users have a cookie set (I imagine). by default Varnish will not present cached results to users with a cookie, nor will it cache their results for others to see. So the 10% of hits (totally my guess) that are for authenticated u

Re: [Twisted-Python] Daemon processes on windows

2009-11-07 Thread Brian Granger
Fantastic, thanks for sharing this code. I will definitely have a look. If it turns out to work for us, would you mind releasing this under a BSD license so we can include it in IPython (BSD open source project)? In either case, the ideas will be very helpful to us. Cheers, Brian It is possib

[Twisted-Python] Weekly Bug Summary

2009-11-07 Thread exarkun
Bug summary __ Summary for 2009-11-01 through 2009-11-08 Bugs opened: 4Bugs closed: 3 Total open bugs: 1187 (+1) |== Type Changes |== Priority Changes |== Component Changes |Defect: +1 |Normal: +2 |Core:+0 |Enhancement: +0

Re: [Twisted-Python] Twisted webserver performance issues

2009-11-07 Thread James Y Knight
When I last looked into the performance issues, I found that sometimes trac appears to block for long periods of time without releasing the GIL. That seems to be the core of the performance issues, currently. When it's responding normally, it's perfectly snappy. But, sometimes, it blocks fo

Re: [Twisted-Python] Twisted webserver performance issues

2009-11-07 Thread exarkun
On 02:55 am, ley...@ihug.co.nz wrote: >Thanks for the link info. > >I literally just set up Varnish for the first time on my web host, so I >am no pro. However, varnish uses a c like scripting language that can >run regexps on the incoming urls and apply rules to them. > >What this means is that

Re: [Twisted-Python] Twisted webserver performance issues

2009-11-07 Thread Tom Leys
Thanks for the link info. I literally just set up Varnish for the first time on my web host, so I am no pro. However, varnish uses a c like scripting language that can run regexps on the incoming urls and apply rules to them. What this means is that we can (as a starting point) choose caching

Re: [Twisted-Python] Twisted webserver performance issues (was volunteer...)

2009-11-07 Thread exarkun
On 7 Nov, 11:05 pm, exar...@twistedmatrix.com wrote: >On 10:41 pm, ley...@ihug.co.nz wrote: >> >>Isn't the simplest option to place a decent reverse proxy between the >>webserver and our clients? > >Possibly so! I gave this a half-hearted attempted a few weeks ago but >quickly gave up. If someone

Re: [Twisted-Python] Daemon processes on windows

2009-11-07 Thread Žiga Seilnacht
Brian Granger wrote: > Hi, > > I have a server-like process that uses twisted. I need it to daemonize > itself and on linux/os x I am simply > using the daemonize function from twistd. This works fine. What about > Windows thoughI saw that the > win32 version of twistd doesn't have (unless

Re: [Twisted-Python] Daemon processes on windows

2009-11-07 Thread exarkun
On 01:18 am, ellisonbg@gmail.com wrote: >Hi, > >I have a server-like process that uses twisted. I need it to daemonize >itself and on linux/os x I am simply >using the daemonize function from twistd. This works fine. What about >Windows thoughI saw that the >win32 version of twistd doesn

Re: [Twisted-Python] Twisted webserver performance issues (was volunteer...)

2009-11-07 Thread exarkun
On 7 Nov, 11:56 pm, sstein...@gmail.com wrote: > >On Nov 7, 2009, at 6:05 PM, exar...@twistedmatrix.com wrote: >>Looking at this, I'm reminded that we should try moving the sponsor >>logos to somewhere else > >1> Move any images to CDN of any description (Cloud Files @ rackspace >does this for free

[Twisted-Python] Daemon processes on windows

2009-11-07 Thread Brian Granger
Hi, I have a server-like process that uses twisted. I need it to daemonize itself and on linux/os x I am simply using the daemonize function from twistd. This works fine. What about Windows thoughI saw that the win32 version of twistd doesn't have (unless I am missing it) the ability to dae

Re: [Twisted-Python] Twisted webserver performance issues (was volunteer...)

2009-11-07 Thread sstein...@gmail.com
On Nov 7, 2009, at 6:05 PM, exar...@twistedmatrix.com wrote: > Looking at this, I'm reminded that we should try moving the sponsor > logos to somewhere else 1> Move any images to CDN of any description (Cloud Files @ rackspace does this for free, if I remember correctly, included in the cost of

Re: [Twisted-Python] Twisted webserver performance issues (was volunteer...)

2009-11-07 Thread exarkun
On 10:41 pm, ley...@ihug.co.nz wrote: > >Isn't the simplest option to place a decent reverse proxy between the >webserver and our clients? Possibly so! I gave this a half-hearted attempted a few weeks ago but quickly gave up. If someone who is familiar with configuring such a proxy would like

[Twisted-Python] Twisted webserver performance issues (was volunteer...)

2009-11-07 Thread Tom Leys
Isn't the simplest option to place a decent reverse proxy between the webserver and our clients? For example, varnish will cache and proxy simple GETs, so simple views of the front page and docs (which I imagine is a large chunk of the traffic) will be offloaded from the webserver: http://v

Re: [Twisted-Python] Volunteer for twisted code Reviews and Commits

2009-11-07 Thread sstein...@gmail.com
On Nov 7, 2009, at 8:51 AM, David Lyon wrote: > Hi Jean-Paul, > > On Sat, 07 Nov 2009 13:32:28 -, exar...@twistedmatrix.com wrote: > >> A server farm doesn't buy us anything, as trac can't be scaled up >> across >> multiple hosts (or even multiple CPUs on a single host). > > I'm surprised t

Re: [Twisted-Python] Volunteer for twisted code Reviews and Commits

2009-11-07 Thread sstein...@gmail.com
On Nov 6, 2009, at 11:43 PM, David Lyon wrote: > On Sat, 07 Nov 2009 00:54:38 -, exar...@twistedmatrix.com wrote: >> On 12:48 am, sstein...@gmail.com wrote: >> As far as I can tell, these problems are due solely to the quality of >> trac. There is no indication that they will disappear throu

Re: [Twisted-Python] Volunteer for twisted code Reviews and Commits

2009-11-07 Thread sstein...@gmail.com
On Nov 6, 2009, at 9:59 PM, exar...@twistedmatrix.com wrote: > On 01:11 am, sstein...@gmail.com wrote: >> >> On Nov 6, 2009, at 7:54 PM, exar...@twistedmatrix.com wrote: >>> As far as I can tell, these problems are due solely to the quality >>> of >>> trac. There is no indication that they wil

Re: [Twisted-Python] Volunteer for twisted code Reviews and Commits

2009-11-07 Thread exarkun
On 01:51 pm, david.l...@preisshare.net wrote: >Hi Jean-Paul, > >On Sat, 07 Nov 2009 13:32:28 -, exar...@twistedmatrix.com wrote: >>A server farm doesn't buy us anything, as trac can't be scaled up >>across >>multiple hosts (or even multiple CPUs on a single host). > >I'm surprised to hear that

Re: [Twisted-Python] Volunteer for twisted code Reviews and Commits

2009-11-07 Thread David Lyon
Hi Jean-Paul, On Sat, 07 Nov 2009 13:32:28 -, exar...@twistedmatrix.com wrote: > A server farm doesn't buy us anything, as trac can't be scaled up across > multiple hosts (or even multiple CPUs on a single host). I'm surprised to hear that. Usually replication of servers for read requests

Re: [Twisted-Python] Volunteer for twisted code Reviews and Commits

2009-11-07 Thread exarkun
On 04:43 am, david.l...@preisshare.net wrote: >On Sat, 07 Nov 2009 00:54:38 -, exar...@twistedmatrix.com wrote: >>On 12:48 am, sstein...@gmail.com wrote: >>As far as I can tell, these problems are due solely to the quality of >>trac. There is no indication that they will disappear through only