On Thu, Aug 21, 2014 at 2:37 AM, Zane Bitter <[email protected]> wrote:
> On 11/08/14 05:24, Thierry Carrez wrote: > >> So the idea that being (and remaining) in the integrated release should >> also be judged on technical merit is a slightly different effort. It's >> always been a factor in our choices, but like Devananda says, it's more >> difficult than just checking a number of QA/integration checkboxes. In >> some cases, blessing one project in a problem space stifles competition, >> innovation and alternate approaches. In some other cases, we reinvent >> domain-specific solutions rather than standing on the shoulders of >> domain-specific giants in neighboring open source projects. >> > > I totally agree that these are the things we need to be vigilant about. > > Stifling competition is a big worry, but it appears to me that a lot of > the stifling is happening even before incubation. Everyone's time is > limited, so if you happen to notice a new project on the incubation > trajectory doing things in what you think is the Wrong Way, you're most > likely to either leave some drive-by feedback or to just ignore it and > carry on with your life. What you're most likely *not* to do is to start a > competing project to prove them wrong, or to jump in full time to the > existing project and show them the light. It's really hard to argue against > the domain experts too - when you're acutely aware of how shallow your > knowledge is in a particular area it's very hard to know how hard to push. > (Perhaps ironically, since becoming a PTL I feel I have to be much more > cautious in what I say too, because people are inclined to read too much > into my opinion - I wonder if TC members feel the same pressure.) I speak > from first-hand instances of guilt here - for example, I gave some feedback > to the Mistral folks just before the last design summit[1], but I haven't > had time to follow it up at all. I wouldn't be a bit surprised if they > showed up with an incubation request, a largely-unchanged user interface > and an expectation that I would support it. > > The result is that projects often don't hear the feedback they need until > far too late - often when they get to the incubation review (maybe not even > their first incubation review). In the particularly unfortunate case of > Marconi, it wasn't until the graduation review. (More about that in a > second.) My best advice to new projects here is that you must be like a > ferret up the pant-leg of any negative feedback. Grab hold of any criticism > and don't let go until you have either converted the person giving it into > your biggest supporter, been converted by them, or provoked them to start a > competing project. (Any of those is a win as far as the community is > concerned.) > > Perhaps we could consider a space like a separate mailing list > (openstack-future?) reserved just for announcements of Related projects, > their architectural principles, and discussions of the same? They > certainly tend to get drowned out amidst the noise of openstack-dev. > (Project management, meeting announcements, and internal project discussion > would all be out of scope for this list.) > > As for reinventing domain-specific solutions, I'm not sure that happens as > often as is being made out. IMO the defining feature of IaaS that makes the > cloud the cloud is on-demand (i.e. real-time) self-service. Everything else > more or less falls out of that requirement, but the very first thing to > fall out is multi-tenancy and there just aren't that many multi-tenant > services floating around out there. There are a couple of obvious > strategies to deal with that: one is to run existing software within a > tenant-local resource provisioned by OpenStack (Trove and Sahara are > examples of this), and the other is to wrap a multi-tenancy framework > around an existing piece of software (Nova and Cinder are examples of > this). (BTW the former is usually inherently less satisfying, because it > scales at a much coarser granularity.) The answer to a question of the form: > > "Why do we need OpenStack project $X, when open source project $Y already > exists?" > > is almost always: > > "Because $Y is not multi-tenant aware; we need to wrap it with a > multi-tenancy layer with OpenStack-native authentication, metering and > quota management. That even allows us to set up an abstraction layer so > that you can substitute $Z as the back end too." > > This is completely uncontroversial when you substitute X, Y, Z = Nova, > libvirt, Xen. However, when you instead substitute X, Y, Z = Zaqar/Marconi, > Qpid, MongoDB it suddenly becomes *highly* controversial. I'm all in favour > of a healthy scepticism, but I think we've passed that point now. (How > would *you* make an AMQP bus multi-tenant?) > > To be clear, Marconi did made a mistake. The Marconi API presented > semantics to the user that excluded many otherwise-obvious choices of > back-end plugin (i.e. Qpid/RabbitMQ). It seems to be a common thing (see > also: Mistral) to want to design for every feature an existing Enterprisey > application might use, which IMHO kind of ignores the fact that > applications need to be rewritten to use these new APIs anyway, and we > would be better off presenting _simple_ interfaces that attract developers, > lead to good design for new applications and provide flexibility on the > back-end side. I digress though, because that wasn't the mistake. The > mistake was failing to educate the entire community (but in particular the > TC) on the relative merits of this feature enough get either buy-in or > rejection on this critical detail much earlier in the process. > > By the way, I claimed without justification earlier that "there just > aren't that many multi-tenant services floating around out there", but > there is actually a very good reason for that. Before there were open > source IaaS platforms out there, there wasn't any reason to build such a > service. And now that there are, _the obvious place to build such a service > in in OpenStack_. We have the users, we have the authentication API that > already ties in to their existing directories (Keystone is well-named) and > we have the community. > > This all has created a world where you need to be*in* OpenStack to >> matter, or to justify the investment. This has created a world where >> everything and everyone wants to be in the "OpenStack" integrated >> release. This has created more pressure to add new projects, and less >> pressure to fix and make the existing projects perfect. 4 years in, we >> might want to inflect that trajectory and take steps to fix this world. >> > > We should certainly consider this possibility, that we've set up perverse > incentives leading to failure. But what if it's just because we haven't yet > come even close to satisfying all of our users' needs? I mean, AWS has more > than 30 services that could be considered equivalent in scope to an > OpenStack project... if anything our scope is increasing more _slowly_ than > the industry at large. I'm slightly shocked that nobody in this thread > appears to have even entertained the idea that *this is what success looks > like*. > > The world is not going to stop because we want to get off, take a > breather, do a "consolidation cycle". > +1 Well said, we need figure out (and quickly) how to scale our systems and *encourage* innovation. If we can't people will move on to other things. -Angus > I think we can all agree that any cross-project function program that aims > to _be_ that function for all of OpenStack, as opposed to enabling that > function in the individual projects, will not scale. It won't scale even if > we never accept another new project. It won't scale even if we de-integrate > some projects pour encourager les autres. I think that the work that Doug > has been doing with Oslo points the way here. Every cross-project function > (including release management - get rid of PTLs) should have a designated > co-ordinator/liason in each project. > > The move of functional tests out of Tempest and into the individual > projects' repositories is a great step in the right direction. Hopefully > that will also address the growth of CI testing capacity requirements - > while projects like Nova, Cinder and Neutron all depend on and benefit from > being gated against each other, that is not at all true for many projects, > and in particular the kinds of new projects that are now appearing. Let's > take actual dependencies into account when we choose what to gate on - the > resulting growth might not be sublinear, but it needn't be n^2 either. > > Simply put, I don't think we're anywhere near so close to running out of > ideas that we have to stop and freeze the world. We have scaling > challenges, and we'll have to make changes, and we'll have to do it - > sometimes under pressure - even as the world changes around us because > that's just how life works. > > The question that is not being asked enough IMO is how exactly does it > benefit our *users* (it is still all about the users, right?) to be telling > developers "you arrived too late, so you're not real OpenStack developers, > with free conference tickets and ATC badges and stuff". Because if it's > just that the support functions within OpenStack can't handle them yet, > that's on us not them. If it's that we don't have a clear way of > communicating to users the information they need to make the right decision > (for them) on whether to deploy some code, then we need to communicate that > information more clearly. And if we believe that somehow companies will > magically divert all of the resources they would otherwise have committed > to new projects into bug fixing on existing projects, I think we're being > incredibly naive. If companies are not seeing the value in strategically > contributing to the core of projects then it's because we have failed to > demonstrate the value, and simply blocking off other avenues for them to > create value (or blocking recognition of those avenues) will absolutely not > solve it. > > cheers, > Zane. > > > [1] http://lists.openstack.org/pipermail/openstack-dev/2014- > May/034641.html > > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
