Samuel Merritt wrote:
On 6/7/16 12:00 PM, Monty Taylor wrote:
[snip]
>
I'd rather see us focus energy on Python3, asyncio and its pluggable
event loops. The work in:
http://magic.io/blog/uvloop-blazing-fast-python-networking/
is a great indication in an actual apples-to-apples comparison of what
can be accomplished in python doing IO-bound activities by using modern
Python techniques. I think that comparing python2+eventlet to a fresh
rewrite in Go isn't 100% of the story. A TON of work has gone in to
Python that we're not taking advantage of because we're still supporting
Python2. So what I've love to see in the realm of comparative
experimentation is to see if the existing Python we already have can be
leveraged as we adopt newer and more modern things.
Asyncio, eventlet, and other similar libraries are all very good for
performing asynchronous IO on sockets and pipes. However, none of them
help for filesystem IO. That's why Swift needs a golang object server:
the go runtime will keep some goroutines running even though some other
goroutines are performing filesystem IO, whereas filesystem IO in Python
blocks the entire process, asyncio or no asyncio.
As you probably know by now, there was a majority of TC members to deny
the addition of Golang as another generally available tool in our
toolbelt. The resolution (which was made to follow the guidelines the TC
itself set as the process to follow in that specific case) was therefore
rejected.
What made this decision pretty hard for everyone involved is that the
Swift team built a compelling argument (see just above) on why they
can't really solve a real user problem they have (scale and perform
better) using the tools they are allowed to use in "OpenStack". There
was probably also a majority of TC members agreeing that Swift needs to
use another language to solve the specific filesystem IO problem they
are running into. So we have been looking at various trade-off solutions
to get the projects the tools they really need without making
cross-project work significantly more complex, and without triggering
the gigantic distraction of superfluous rewrites across OpenStack projects.
None of those middle-ground solutions is optimal. But I've been asked
to list them for clarity.
One option is to develop generally-useful libraries or services outside
of OpenStack that solve the specific issue you're running into.
OpenStack relies on a lot of dependencies in the wider open source
community, we don't have to develop *everything* as official "openstack
projects" under TC governance. We even have a number of them that are
using OpenStack infrastructure (git, gerrit, gate...). The benefit of
this approach is that it lets you pick whatever language or framework is
most appropriate, and forces clean contracts between the components,
which makes replacing the external piece easier in the future. The
drawback of this approach is that it creates local fragmentation at the
project team level, as they have to artificially split the project into
"OpenStack" and non-"OpenStack" bits. And it is also not optimal for
Swift due to the way it's currently designed, you can't really slice it
in this way.
Another option (raised by dims) is to find a way to allow usage of
golang (or another language) in a more granular way: selectively allow
projects which really need another tool to use it. The benefit is that
it lets project teams make a case and use the best tool for the job,
while limiting the cross-project impact and without opening the
distraction floodgates of useless rewrites. The drawback is that
depending on how it's done, it may place the TC in the role of granting
"you're tall enough to use Go" badges, creating new endless discussions
and more "you're special" exceptions. That said, I'm still interested in
exploring that option, for one reason. I think that whenever a project
team considers adding a component or a rewrite in another language, they
are running into an interesting problem with Python, on which they
really could use advice from the rest of the OpenStack community. I'd
really like to see a cross-project team of Python performance experts to
look at the problem this specific team has that makes them want to use
another language. That seems like a great way to drive more practice
sharing and reduce fragmentation in "OpenStack" in general. We might
just need to put the bar pretty high so that we are not flooded by silly
rewrite requests.
For completeness I'll also list the nuclear option: to reject the
Technical Committee oversight altogether and decide that your project is
better off as an autonomous project in the OpenStack ecosystem.
--
Thierry Carrez (ttx)
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
