On 10 May 2016 at 10:54, John Dickinson <m...@not.mn> wrote: > On 9 May 2016, at 13:16, Gregory Haynes wrote: >> >> This is a bit of an aside but I am sure others are wondering the same >> thing - Is there some info (specs/etherpad/ML thread/etc) that has more >> details on the bottleneck you're running in to? Given that the only >> clients of your service are the public facing DNS servers I am now even >> more surprised that you're hitting a python-inherent bottleneck. > > In Swift's case, the summary is that it's hard[0] to write a network > service in Python that shuffles data between the network and a block > device (hard drive) and effectively utilizes all of the hardware > available. So far, we've done very well by fork()'ing child processes, ... > Initial results from a golang reimplementation of the object server in > Python are very positive[1]. We're not proposing to rewrite Swift > entirely in Golang. Specifically, we're looking at improving object > replication time in Swift. This service must discover what data is on > a drive, talk to other servers in the cluster about what they have, > and coordinate any data sync process that's needed. > > [0] Hard, not impossible. Of course, given enough time, we can do > anything in a Turing-complete language, right? But we're not talking > about possible, we're talking about efficient tools for the job at > hand. ...
I'm glad you're finding you can get good results in (presumably) clean, understandable code. Given go's historically poor perfornance with multiple cores (https://golang.org/doc/faq#Why_GOMAXPROCS) I'm going to presume the major advantage is in the CSP programming model - something that Twisted does very well: and frustratingly we've had numerous discussions from folk in the Twisted world who see the pain we have and want to help, but as a community we've consistently stayed with eventlet, which has a threaded programming model - and threaded models are poorly suited for the case here. All of which to say, I disagree quite strongly that its hard to drive a server's hardware at full capacity using Python. Using Python like *we have been* - yes, certainly. Thats not a reason to avoid golang - but if minimising the number of languages in play is a thing we want to do, we should make sure we're actually using the best facilities of our existing languages first! On the Designate side, PyPy got a 40% performance improvement over CPython for a benchmark of Twisted's DNS component 6 years ago - and Twisted and PyPy have both moved forward a great deal since then.... http://morepypy.blogspot.co.nz/2010/03/hello.html -Rob -- Robert Collins <rbtcoll...@hpe.com> Distinguished Technologist HP Converged Cloud __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
