On 9 May 2016, at 13:16, Gregory Haynes wrote: > > This is a bit of an aside but I am sure others are wondering the same > thing - Is there some info (specs/etherpad/ML thread/etc) that has more > details on the bottleneck you're running in to? Given that the only > clients of your service are the public facing DNS servers I am now even > more surprised that you're hitting a python-inherent bottleneck.
In Swift's case, the summary is that it's hard[0] to write a network service in Python that shuffles data between the network and a block device (hard drive) and effectively utilizes all of the hardware available. So far, we've done very well by fork()'ing child processes, using cooperative concurrency via eventlet, and basic "write more efficient code" optimizations. However, when it comes down to it, managing all of the async operations across many cores and many drives is really hard, and there just isn't a good, efficient interface for that in Python. Initial results from a golang reimplementation of the object server in Python are very positive[1]. We're not proposing to rewrite Swift entirely in Golang. Specifically, we're looking at improving object replication time in Swift. This service must discover what data is on a drive, talk to other servers in the cluster about what they have, and coordinate any data sync process that's needed. [0] Hard, not impossible. Of course, given enough time, we can do anything in a Turing-complete language, right? But we're not talking about possible, we're talking about efficient tools for the job at hand. [1] http://d.not.mn/python_vs_golang_gets.png and http://d.not.mn/python_vs_golang_puts.png --John
signature.asc
Description: OpenPGP digital signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
