Re: [openstack-dev] What's happening with stable release management?

2014-08-08 Thread Thierry Carrez
Thomas Goirand wrote:
> Hi,
> 
> For updating keystone from 2014.1.1 to 2014.1.2, I had to:
> 
> - Upgrade oslo-config from 1.2.1 to 1.4.0.0~a3
> - Upgrade oslo.messaging from 1.3.0~a9 to 1.4.0.0~a3
> - Upgrade python-six from 1.6 to 1.7
> - Upgrade python-pycadf from 0.4 to 0.5.1
> - Add python-ldappool
> - Add python-oslo.db
> - Add python-oslo.i18n
> - Add python-keystonemiddleware, which needs python-crypto >= 2.6
> (previously, 2.5 was enough)
> 
> So, we have 5 major Python module upgrades, and 4 completely new
> libraries which were not there in 2014.1.1. Some of the changes aren't
> small at all.
> 
> I'm sure that there's very valid reasons for each of the upgrades or
> library addition, but I don't think that it is overall reasonable. If
> this was to happen during the freeze of Debian, or worse, after a
> release, upgrading all of this would be a nightmare, and I'm sure that
> the Debian release team would simply refuse.
> 
> Should I assign myself to program a robot which will vote -1 on all
> change on the stable/Icehouse global-requirements.txt file? Or is sanity
> still possible in OpenStack? :)
> 
> It is my opinion that we need to review our release process for the
> stable releases, policy for requirement changes, and need to adopt a way
> more conservative attitude.

No, actually this is because the 2014.1.2 tarball is still completely
wrong. The tag is now OK, but due to some stale workspaces in our CI the
tarball was still generated from the wrong (Juno) tag.

I'll upload a new tarball ASAP. I took down the wrong one. Sorry for the
inconvenience... the issues here are not a policy problem, they are just
human error in the original tag, complicated by CI staleness that made
us think we fixed it while we didn't.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] What's happening with stable release management?

2014-08-08 Thread Thierry Carrez
Thierry Carrez wrote:
> I'll upload a new tarball ASAP. I took down the wrong one. Sorry for the
> inconvenience... the issues here are not a policy problem, they are just
> human error in the original tag, complicated by CI staleness that made
> us think we fixed it while we didn't.

keystone 2014.1.2.1 was just uploaded to fix that. I'm sorry for the
inconvenience this has caused.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Neutron] Simple proposal for stabilizing new features in-tree

2014-08-11 Thread Thierry Carrez
gustavo panizzo (gfa) wrote:
> only one think i didn't like it
> 
> why all url,api, etc has to include the word 'preview'?
> i imagine that i would be consuming the new feature using heat, puppet,
> local scripts, custom horizon, whatever. Why do you make me to change
> all them when the feature moves out of preview? it could be a lot of
> rework (for consumers) without gain. I would totally support other big
> fat warnings everywhere (logs, documentation, startup log of
> neutron-server) but don't change the API if is not necessary

I see two issues with this proposal: the first one is what Gustavo just
said: the use of the "preview" package/configoption/API creates friction
when the feature needs to go mainstream (package renaming, change in
configuration for deployers, change in API calls for users...).

My understanding is that the goal is to make it easy for people to "try"
the preview feature, and keeping the experimental feature in-tree is
seen as simpler to experiment with. But the pain from this friction imho
outweighs the pain of deploying an out-of-tree plugin for deployers.

The second issue is that once the feature is in "preview" in tree, it
moves the responsibility/burden of making it official (or removed) to
the core developers (as Salvatore mentioned). I kind of like the
approach where experimental features are developed in faster iterations
out-of-tree and when they are celebrated by experimenters and are ready
to be stable-supported, they are moved in tree.

Regards,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] The future of the integrated release

2014-08-11 Thread Thierry Carrez
Jeremy Stanley wrote:
> On 2014-08-08 09:43:54 -0700 (-0700), Devananda van der Veen wrote:
> [...]
>> this sounds like it will also solve the current problem when a
>> core reviewer has gone on vacation after blocking something for
>> procedural reasons.
> 
> I don't really think so, unless I'm misunderstanding which vacation
> problem you're talking about. We can either make a vote carry over
> on subsequent patchsets (in which case you need someone to reverse
> or delete the -2 before you can merge it) or not (in which case you
> will lose that -2 on the next uploaded patchset and have to keep
> reapplying it manually).
> 
> Unless perhaps you're suggesting that we should allow approved
> changes to merge even with a workflow -2 from another reviewer
> (keeping in mind that we don't currently allow changes with a
> workflow -1 to merge).

Hmm..

Ideally we'd want a blocking vote that carries over patchsets, but which
can be set or unset by any core/driver/ptl (to be determined). So if the
PTL sets the "procedural" Workflow-2 at feature freeze, I can clear it
when RC1 is published.

Not sure that's supported in Gerrit though :)

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] The future of the integrated release

2014-08-11 Thread Thierry Carrez
Eoghan Glynn wrote:
>> The TC has begun to scrutinize new projects more carefully on
>> technical grounds, particularly since some recently-integrated
>> projects have run into scaling limitations that have hampered their
>> adoption. I believe this sort of technical guidance (or curation, if
>> you will) is an essential function of the TC. We've learned that
>> integration bestows legitimacy, as well as assumptions of stability,
>> performance, and scalability, upon projects which are integrated.
>> While that wasn't the intent, I think we need to accept that that is
>> where we stand. We will be faced with some hard decisions regarding
>> projects, both incubated and already integrated, which are clearly not
>> meeting those expectations today.
> 
> How does this relate to the recent gap analysis undertaken by the
> TC for already integrated projects, in order to measure their status
> against the steadily rising bar for integration?

It is almost orthogonal. During the Icehouse cycle the TC created clear
requirements for projects wishing to incubate or graduate for
incubation. But it was unclear how well the *currently-incubated*
projects fulfilled those requirements. Since it was a bit unfair to hold
new entrants to different standards, during the Juno cycle we completed
a gap analysis of currently-integrated projects, created gap coverage
plans to address any identified gap, and reviewed progress on those
after each milestone.

So the idea that being (and remaining) in the integrated release should
also be judged on technical merit is a slightly different effort. It's
always been a factor in our choices, but like Devananda says, it's more
difficult than just checking a number of QA/integration checkboxes. In
some cases, blessing one project in a problem space stifles competition,
innovation and alternate approaches. In some other cases, we reinvent
domain-specific solutions rather than standing on the shoulders of
domain-specific giants in neighboring open source projects.

This all has created a world where you need to be *in* OpenStack to
matter, or to justify the investment. This has created a world where
everything and everyone wants to be in the "OpenStack" integrated
release. This has created more pressure to add new projects, and less
pressure to fix and make the existing projects perfect. 4 years in, we
might want to inflect that trajectory and take steps to fix this world.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Retrospective veto revert policy

2014-08-12 Thread Thierry Carrez
Dan Smith wrote:
>> Looks reasonable to me.
> 
> +1

+1

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] The future of the integrated release

2014-08-13 Thread Thierry Carrez
Nikola Đipanov wrote:
> While I agree with motivation for this - setting the expectations, I
> fail to see how this is different to what the Swift guys seem to be
> doing apart from more red tape.

It's not different imho. It's just that nova as significantly more
features being thrown at it, so the job of "selecting priority features"
is significantly harder, and the backlog is a lot bigger. The slot
system allows to visualize that backlog.

Currently we target all features to juno-3, everyone expects their stuff
to get review attention, nothing gets merged until the end of the
milestone period, and and in the end we merge almost nothing. The
blueprint "priorities" don't cut it, what you want is a ranked list. See
how likely you are to be considered for a release. Communicate that the
feature will actually be a Kilo feature earlier. Set downstream
expectations right. Merge earlier.

That ties into the discussions we are having for StoryBoard to support
task lists[1], which are arbitrary ranked lists of tasks. Those are much
more flexible than mono-dimensional priorities that fail to express the
complexity of priority in a complex ecosystem like OpenStack development.

[1] https://wiki.openstack.org/wiki/StoryBoard/Task_Lists

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] The future of the integrated release

2014-08-13 Thread Thierry Carrez
Rochelle.RochelleGrober wrote:
> [...]
> So, with all that prologue, here is what I propose (and please consider 
> proposing your improvements/changes to it).  I would like to see for Kilo:
> 
> - IRC meetings and mailing list meetings beginning with Juno release and 
> continuing through the summit that focus on core project needs (what Thierry 
> call "strategic") that as a set would be considered the primary focus of the 
> Kilo release for each project.  This could include high priority bugs, 
> refactoring projects, small improvement projects, high interest extensions 
> and new features, specs that didn't make it into Juno, etc.
> - Develop the list and prioritize it into "Needs" and "Wants." Consider these 
> the feeder projects for the two runways if you like.  
> - Discuss the lists.  Maybe have a community vote? The vote will "freeze" the 
> list, but as in most development project freezes, it can be a soft freeze 
> that the core, or drivers or TC can amend (or throw out for that matter).
> [...]

One thing we've been unable to do so far is to set "release goals" at
the beginning of a release cycle and stick to those. It used to be
because we were so fast moving that new awesome stuff was proposed
mid-cycle and ends up being a key feature (sometimes THE key feature)
for the project. Now it's because there is so much proposed noone knows
what will actually get completed.

So while I agree that what you propose is the ultimate solution (and the
workflow I've pushed PTLs to follow every single OpenStack release so
far), we have struggled to have the visibility, long-term thinking and
discipline to stick to it in the past. If you look at the post-summit
plans and compare to what we end up in a release, you'll see quite a lot
of differences :)

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] The future of the integrated release

2014-08-14 Thread Thierry Carrez
Russell Bryant wrote:
> I think perhaps some middle ground makes sense.
> 
> 1) Start doing a better job of generating a priority list, and
> identifying the highest priority items based on group will.
> 
> 2) Expect that reviewers use the priority list to influence their
> general review time.

2b) Discuss those reviews at the weekly team meeting to give them extra
exposure

> 3) Don't actually block other things, should small groups self-organize
> and decide it's important enough to them, even if not to the group as a
> whole.
> 
> That sort of approach still sounds like an improvement to what we have
> today, which is alack of good priority communication to direct general
> review time.

+1

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic] Feature proposal freeze

2014-08-18 Thread Thierry Carrez
Devananda van der Veen wrote:
> As previously announced, ironic is now entering feature proposal freeze
> for juno. This means that all open spec reviews will be blocked, and
> will need to be resubmitted when kilo opens. This will be when the first
> juno RC is tagged, or sooner, and will be announced on this list.

Technically, that's a specs proposal deadline [1], not a feature
proposal freeze [2]. Feature proposal freeze means all code reviews
needs to have been posted for review (and is generally held two weeks
before feature freeze on busy-review projects, August 21 for Juno).

[1] https://wiki.openstack.org/wiki/SpecProposalDeadline
[2] https://wiki.openstack.org/wiki/FeatureProposalFreeze

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] stable branches & failure to handle review backlog

2014-08-18 Thread Thierry Carrez
Mark McLoughlin wrote:
> [...]
> I don't see how any self-respecting open-source project can throw a
> release over the wall and have no ability to address critical bugs with
> that release until the next release 6 months later which will also
> include a bunch of new feature work with new bugs. That's not a distro
> maintainer point of view.

I agree that the job changed a bit since those early days, and now apart
from a very small group of stable specialists (mostly the stable release
managers), everyone else on stable-core is actually specialized in a
given project. It would make sense for each project to have a set of
dedicated "stable liaisons" that would work together with stable release
managers in getting critical bugfixes to stable branch point releases.
Relying on the same small group of people now that we have 10+ projects
to cover is unreasonable.

There are two issues to solve before we do that, though. The projects
have to be OK with taking on that extra burden (it becomes their
responsibility to dedicate resources to stable branches). And we need to
make sure the stable branch guidelines are well communicated.

> [...]
> That's quite a leap to say that -core team members will be so incapable
> of the appropriate level of conservatism that the branch will be killed.
> 
> The idea behind not automatically granting +2 on stable/* to -core
> members was simply we didn't want people diving in and approving
> unsuitable stuff out of ignorance.
> 
> I could totally see an argument now that everyone is a lot more familiar
> with gerrit, the concept of stable releases is well established and we
> should be able to trust -core team members to learn how to make the risk
> vs benefit tradeoffs needed for stable reviews.

The question here is whether every -core member on a project should
automatically be on stable-core (and we can reuse the same group) or do
we have to maintain two groups.

>From my experience reviewing stable branch changes, I see two types of
issues with regular core doing stable reviews. There is the accidental
stable/* review, where the person thinks he is reviewing a master
review. Gerrit makes it look extremely similar, so more often than not,
we have -core members wondering why they don't have +2 on review X, or
core members doing a code review (criticizing the code again) rather
than a stable review (checking the type of change and the backport).
Maybe we could solve that by making gerrit visually different on stable
reviews ?

And then there is the opportunistic review, where a core member bends
the stable rules because he wants a specific fix backported. So we get
database migrations, new configuration options, or default behavior
changes +1ed or +2ed in stable. When we discuss those on the review, the
-core person generally disagrees with the stable rules and would like
them relaxed.

This is why I tend to prefer an opt-in system where the core member
signs up to do stable review. He clearly says he agrees with the stable
rules and will follow them. He also signs up to do enough of them to
limit the opportunistic and accidental issues.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][core] Expectations of core reviewers

2014-08-18 Thread Thierry Carrez
Doug Hellmann wrote:
> On Aug 13, 2014, at 4:42 PM, Russell Bryant  wrote:
>> Let me try to say it another way.  You seemed to say that it wasn't much
>> to ask given the rate at which things happen in OpenStack.  I would
>> argue that given the rate, we should not try to ask more of individuals
>> (like this proposal) and risk burnout.  Instead, we should be doing our
>> best to be more open an inclusive to give the project the best chance to
>> grow, as that's the best way to get more done.
>>
>> I think an increased travel expectation is a raised bar that will hinder
>> team growth, not help it.
> 
> +1, well said.

Sorry, I was away for a few days. This is a topic I have a few strong
opinions on :)

There is no denial that the meetup format is working well, comparatively
better than the design summit format. There is also no denial that that
requiring 4 travels per year for a "core" dev is unreasonable. Where is
the limit ? Wouldn't we be more productive and aligned if we did one per
month ? No, the question is how to reach a sufficient level of focus and
alignment while keeping the number of "mandatory" travel at 2 per year.

I don't think our issue comes from not having enough F2F time. Our issue
is that the design summit no longer reaches its objectives of aligning
key contributors on a common plan, and we need to fix it.

We established the design summit as the once-per-cycle opportunity to
have face-to-face time and get alignment across the main contributors to
a project. That used to be completely sufficient, but now it doesn't
work as well... which resulted in alignment and team discussions to be
discussed at mid-cycle meetups instead. Why ? And what could we change
to have those alignment discussions at the design summit again ?

Why are design summits less productive that mid-cycle meetups those days
? Is it because there are too many non-contributors in the design summit
rooms ? Is it the 40-min format ? Is it the distractions (having talks
to give somewhere else, booths to attend, parties and dinners to be at)
? Is it that beginning of cycle is not the best moment ? Once we know
WHY the design summit fails its main objective, maybe we can fix it.

My gut feeling is that having a restricted audience and a smaller group
lets people get to the bottom of an issue and reach consensus. And that
you need at least half a day or a full day of open discussion to reach
such alignment. And that it's not particularly great to get such
alignment in the middle of the cycle, getting it at the start is still
the right way to align with the release cycle.

Nothing prevents us from changing part of the design summit format (even
the Paris one!), and restrict attendance to some of the sessions. And if
the main issue is the distraction from the conference colocation, we
might have to discuss the future of co-location again. In that "2 events
per year" objective, we could make the conference the optional cycle
thing, and a developer-oriented specific event the mandatory one.

If we manage to have alignment at the "design summit", then it doesn't
spell the end of the mid-cycle things. But then, ideally the extra
mid-cycle gatherings should be focused on getting specific stuff done,
rather than general team alignment. Think workshop/hackathon rather than
private gathering. The goal of the workshop would be published in
advance, and people could opt to join that. It would be totally optional.

Cheers,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] The future of the integrated release

2014-08-18 Thread Thierry Carrez
Clint Byrum wrote:
> Here's why folk are questioning Ceilometer:
> 
> Nova is a set of tools to abstract virtualization implementations.
> Neutron is a set of tools to abstract SDN/NFV implementations.
> Cinder is a set of tools to abstract block-device implementations.
> Trove is a set of tools to simplify consumption of existing databases.
> Sahara is a set of tools to simplify Hadoop consumption.
> Swift is a feature-complete implementation of object storage, none of
> which existed when it was started.
> Keystone supports all of the above, unifying their auth.
> Horizon supports all of the above, unifying their GUI.
> 
> Ceilometer is a complete implementation of data collection and alerting.
> There is no shortage of implementations that exist already.
> 
> I'm also core on two projects that are getting some push back these
> days:
> 
> Heat is a complete implementation of orchestration. There are at least a
> few of these already in existence, though not as many as their are data
> collection and alerting systems.
> 
> TripleO is an attempt to deploy OpenStack using tools that OpenStack
> provides. There are already quite a few other tools that _can_ deploy
> OpenStack, so it stands to reason that people will question why we
> don't just use those. It is my hope we'll push more into the "unifying
> the implementations" space and withdraw a bit from the "implementing
> stuff" space.
> 
> So, you see, people are happy to unify around a single abstraction, but
> not so much around a brand new implementation of things that already
> exist.

Right, most projects focus on providing abstraction above
implementations, and that abstraction is where the real "domain
expertise" of OpenStack should be (because no one else is going to do it
for us). Every time we reinvent something, we are at larger risk because
we are out of our common specialty, and we just may not be as good as
the domain specialists. That doesn't mean we should never reinvent
something, but we need to be damn sure it's a good idea before we do.
It's sometimes less fun to piggyback on existing implementations, but if
they exist that's probably what we should do.

While Ceilometer is far from alone in that space, what sets it apart is
that even after it was blessed by the TC as "the one we should all
converge on", we keep on seeing competing implementations for some (if
not all) of its scope. Convergence did not happen, and without
convergence we struggle in adoption. We need to understand why, and if
this is fixable.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all][oslo] official recommendations to handle oslo-incubator sync requests

2014-08-20 Thread Thierry Carrez
Ihar Hrachyshka wrote:
> [...]
> I hope I've described the existing oral tradition correctly. Please
> comment on that, and if we're ok with the way it's written above, I'd
> like to update our wiki pages ([1] and [2]) with that.

That matches my version of the oral tradition.

One point to note is that we should refrain from doing gratuitous
oslo-incubator syncs, since the process is so painful :) Once oslo
libraries graduate, they are handled through usual dependency version
updates, which are not really encouraged in stable either. So this
process should IMHO really be limited to critical bugs and security issues.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] The future of the integrated release

2014-08-20 Thread Thierry Carrez
Eoghan Glynn wrote:
> [...] 
>> And which cross-project concern do you think is most strained by the
>> current set of projects in the integrated release? Is it:
>>
>> * QA
>> * infra
>> * release management
>> * oslo
>> * documentation
>> * stable-maint
>>
>> or something else?
>>
>>
>> Good question.
>>
>> IMHO QA, Infra and release management are probably the most strained.
> 
> OK, well let's brain-storm on how some of those efforts could
> potentially be made more scalable.
> 
> Should we for example start to look at release management as a
> program onto itself, with a PTL *and* a group of cores to divide
> and conquer the load?
> 
> (the hands-on rel mgmt for the juno-2 milestone, for example, was
>  delegated - is there a good reason why such delegation wouldn't
>  work as a matter of course?)

For the record, I wouldn't say release management (as a role) is
strained. I'm strained, but that's because I do more than just release
management. We are taking steps to grow the team (both at release
management program level and at foundation development coordination
levels) that should help in that area. Oslo has some growth issues but I
think they are under control. Stable maint (which belongs to the release
management program, btw) needs more a restructuration that a resource
injection.

I think the most strained function is keeping on top of test failures
(which is most case is just about investigating, reproducing and fixing
rare issues bugs). It's a complex task, it falls somewhere between QA
and Infra right now, and the very few resources that have the unique
combination of knowledge and will/time to spend on those is quickly
dying of burnout.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] The future of the integrated release

2014-08-20 Thread Thierry Carrez
Jay Pipes wrote:
> [...]
> If either of the above answers is NO, then I believe the Technical
> Committee should recommend that the integrated project be removed from
> the integrated release.
> 
> HOWEVER, I *also* believe that the previously-integrated project should
> not just be cast away back to Stackforge. I think the project should
> remain in its designated Program and should remain in the openstack/
> code namespace. Furthermore, active, competing visions and
> implementations of projects that address the Thing the
> previously-integrated project addressed should be able to apply to join
> the same Program, and *also* live in the openstack/ namespace.
> 
> All of these projects should be able to live in the Program, in the
> openstack/ code namespace, for as long as the project is actively
> developed, and let the contributor communities in these competing
> projects *naturally* work to do any of the following:
> 
>  * Pick a best-of-breed implementation from the projects that address
> the same Thing
>  * Combine code and efforts to merge the good bits of multiple projects
> into one
>  * Let multiple valid choices of implementation live in the same Program
> with none of them being "blessed" by the TC to be part of the integrated
> release

That would work if an OpenStack Program was just like a category under
which you can file projects. However, OpenStack programs are not a
competition category where we could let multiple competing
implementations fight it out for becoming "the" solution; they are
essentially just a team of people working toward a common goal, having
meetings and sharing/electing the same technical lead.

I'm not convinced you would set competing solutions for a fair
competition by growing them inside the same team (and under the same
PTL!) as the current mainstream/blessed option. How likely is the
Orchestration PTL to make the decision to drop Heat in favor of a new
contender ?

I'm also concerned with making a program a collection of competing
teams, rather than a single team sharing the same meetings and electing
the same leadership, working all together. I don't want the teams
competing to get a number of contributors that would let them game the
elections and take over the program leadership. I think such a setup
would just increase the political tension inside programs, and we have
enough of it already.

If we want to follow your model, we probably would have to dissolve
programs as they stand right now, and have blessed categories on one
side, and teams on the other (with projects from some teams being
blessed as the current solution). That would leave the horizontal
programs like Docs, QA or Infra, where the team and the category are the
same thing, as outliers again (like they were before we did programs).

Finally, I'm slightly concerned with the brand aspect -- letting *any*
project call themselves "OpenStack something" (which is what living
under the openstack/* namespace gives you) just because they happen to
compete with an existing openstack project sounds like a recipe for
making sure "openstack" doesn't mean anything upstream anymore.

Regards,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Glance][Heat] Murano split dsicussion

2014-08-21 Thread Thierry Carrez
Georgy Okrokvertskhov wrote:
> During last Atlanta summit there were couple discussions about
> Application Catalog and Application space projects in OpenStack. These
> cross-project discussions occurred as a result of Murano incubation
> request [1] during Icehouse cycle.  On the TC meeting devoted to Murano
> incubation there was an idea about splitting the Murano into parts which
> might belong to different programs[2].
> 
> 
> Today, I would like to initiate a discussion about potential splitting
> of Murano between two or three programs.
> [...]

I think the proposed split makes a lot of sense. Let's wait for the
feedback of the affected programs to see if it's compatible with their
own plans.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] The future of the integrated release

2014-08-21 Thread Thierry Carrez
Zane Bitter wrote:
> On 11/08/14 05:24, Thierry Carrez wrote:
>> This all has created a world where you need to be*in*  OpenStack to
>> matter, or to justify the investment. This has created a world where
>> everything and everyone wants to be in the "OpenStack" integrated
>> release. This has created more pressure to add new projects, and less
>> pressure to fix and make the existing projects perfect. 4 years in, we
>> might want to inflect that trajectory and take steps to fix this world.
> 
> We should certainly consider this possibility, that we've set up
> perverse incentives leading to failure. But what if it's just because we
> haven't yet come even close to satisfying all of our users' needs? I
> mean, AWS has more than 30 services that could be considered equivalent
> in scope to an OpenStack project... if anything our scope is increasing
> more _slowly_ than the industry at large. I'm slightly shocked that
> nobody in this thread appears to have even entertained the idea that
> *this is what success looks like*.
> 
> The world is not going to stop because we want to get off, take a
> breather, do a "consolidation cycle".

That's an excellent counterpoint, thank you for voicing it so eloquently.

Our challenge is to improve our structures so that we can follow the
rhythm the world imposes on us. It's a complex challenge, especially in
an open collaboration experiment where you can't rely that much on past
experiences or traditional methods. So it's always tempting to slow
things down, to rate-limit our "success" to make that challenge easier.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][neutron] Migration from nova-network to Neutron for large production clouds

2014-08-21 Thread Thierry Carrez
Tim Bell wrote:
> Michael has been posting very informative blogs on the summary of the
> mid-cycle meetups for Nova. The one on the Nova Network to Neutron
> migration was of particular interest to me as it raises a number of
> potential impacts for the CERN production cloud. The blog itself is at
> http://www.stillhq.com/openstack/juno/14.html
> 
> I would welcome suggestions from the community on the approach to take
> and areas that the nova/neutron team could review to limit the impact on
> the cloud users.
> 
> For some background, CERN has been running nova-network in flat DHCP
> mode since our first Diablo deployment. We moved to production for our
> users in July last year and are currently supporting around 70,000
> cores, 6 cells, 100s of projects and thousands of VMs. Upgrades
> generally involve disabling the API layer while allowing running VMs to
> carry on without disruption. Within the time scale of the migration to
> Neutron (M release at the latest), these numbers are expected to double.

Thanks for bringing your concerns here. To start this discussion, it's
worth adding some context on the currently-proposed "cold" migration
path. During the Icehouse and Juno cycles the TC reviewed the gaps
between the integration requirements we now place on new entrants and
the currently-integrated projects. That resulted in a number of
identified gaps that we asked projects to address ASAP, ideally within
the Juno cycle.

Most of the Neutron gaps revolved around its failure to be a full
nova-network replacement -- some gaps around supporting basic modes of
operation, and a gap in providing a basic migration path. Neutron devs
promised to close that in Juno, but after a bit of discussion we
considered that a cold migration path was all we'd require them to
provide in Juno.

That doesn't mean a "hot" or "warm" migration path can't be worked on.
There are two questions to solve: how can we technically perform that
migration with a minimal amount of downtime, and is it reasonable to
mark nova-network deprecated until we've solved that issue.

On the first question, migration is typically an operational problem,
and operators could really help to design one that would be acceptable
to them. They may require developers to add features in the code to
support that process, but we seem to not even be at this stage. Ideally
I would like ops and devs to join to solve that technical challenge.

The answer to the second question lies in the multiple dimensions of
"deprecated".

On one side it means "is no longer in our future plans, new usage is now
discouraged, new development is stopped, explore your options to migrate
out of it". I think it's extremely important that we do that as early as
possible, to reduce duplication of effort and set expectations correctly.

On the other side it means "will be removed in release X" (not
necessarily the next release, but you set a countdown). To do that, you
need to be pretty confident that you'll have your ducks in a row at
removal date, and don't set up operators for a nightmare migration.

> For us, the concerns we have with the ‘cold’ approach would be on the
> user impact and operational risk of such a change. Specifically,
> 
> 1.  A big bang approach of shutting down the cloud, upgrade and the
> resuming the cloud would cause significant user disruption
> 
> 2.  The risks involved with a cloud of this size and the open source
> network drivers would be difficult to mitigate through testing and could
> lead to site wide downtime
> 
> 3.  Rebooting VMs may be possible to schedule in batches but would
> need to be staggered to keep availability levels

What minimal level of "hot" would be acceptable to you ?

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] The future of the integrated release

2014-08-21 Thread Thierry Carrez
Jay Pipes wrote:
> I don't believe the Programs are needed, as they are currently
> structured. I don't really believe they serve any good purposes, and
> actually serve to solidify positions of power, slanted towards existing
> power centers, which is antithetical to a meritocratic community.

Let me translate that, considering programs are just teams of people...
You're still OK with the concept of teams of people working toward a
common goal, but you don't think "blessing" some teams serves any good
purpose. Is that right? (if yes, see below for more on what that
actually means).

> [...]
>> If we want to follow your model, we probably would have to dissolve
>> programs as they stand right now, and have blessed categories on one
>> side, and teams on the other (with projects from some teams being
>> blessed as the current solution).
> 
> Why do we have to have "blessed" categories at all? I'd like to think of
> a day when the TC isn't picking winners or losers at all. Level the
> playing field and let the quality of the projects themselves determine
> the winner in the space. Stop the incubation and graduation madness and
> change the role of the TC to instead play an advisory role to upcoming
> (and existing!) projects on the best ways to integrate with other
> OpenStack projects, if integration is something that is natural for the
> project to work towards.

I'm still trying to wrap my head around what you actually propose here.
Do you just want to get rid of incubation ? Or do you want to get rid of
the whole "integrated release" concept ? The idea that we collectively
apply effort around a limited set of projects to make sure they are
delivered in an acceptable fashion (on a predictable schedule, following
roughly the same rules, with some amount of integrated feature, some
amount of test coverage, some amount of documentation...)

Because I still think there is a whole lot of value in that. I don't
think our mission is to be the sourceforge of cloud projects. Our
mission is to *produce* the ubiquitous Open Source Cloud Computing
platform. There must be some amount of opinionated choices there.

Everything else in our structure derives from that. If we have an
integrated release, we need to bless a set of projects that will be part
of it (graduation). We need to single out promising projects so that we
mentor them on the common rules they will have to follow there (incubation).

Now there are bad side-effects we need to solve, like the idea that
incubation and integration are steps on a openstack ecosystem holy
ladder that every project should aspire to climb.

>> That would leave the horizontal programs like Docs, QA or Infra,
>> where the team and the category are the same thing, as outliers again
>> (like they were before we did programs).
> 
> What is the purpose of having these programs, though? If it's just to
> have a PTL, then I think we need to reconsider the whole concept of
> Programs. [...]

The main purpose of programs (or "official teams") is that being part of
one of them gives you the right to participate in electing the Technical
Committee, and as a result places you under its authority. Both parties
have to agree to be placed under that contract, which is why teams have
to apply (we can't force them), and the TC has to accept (they can't
force us).

Programs have *nothing* to do with PTLs, which are just a convenient way
to solve potential decision deadlocks in teams (insert your favorite
dysfunctional free software project example here). We could get rid of
the PTL concept (to replace them for example with a set of designated
liaisons) and we would still have programs (teams) and projects (the
code repos that team is working on).

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [oslo] Launchpad tracking of oslo projects

2014-08-22 Thread Thierry Carrez
TL;DR:
Let's create an Oslo projectgroup in Launchpad to track work across all
Oslo libraries. In library projects, let's use milestones connected to
published versions rather than the common milestones.

Long version:
As we graduate more Oslo libraries (which is awesome), tracking Oslo
work in Launchpad (bugs, milestones...) has become more difficult.

There used to be only one Launchpad project ("oslo", which covered the
oslo incubator). It would loosely follow the integrated milestones
(juno-1, juno-2...), get blueprints and bugs targeted to those, get tags
pushed around those development milestones: same as the integrated
projects, just with no source code tarball uploads.

When oslo.messaging graduated, a specific Launchpad project was created
to track work around it. It still had integrated development milestones
-- only at the end it would publish a 1.4.0 release instead of a 2014.2
one. That approach creates two problems. First, it's difficult to keep
track of "oslo" work since it now spans two projects. Second, the
release management logic of marking bugs "Fix released" at development
milestones doesn't really apply (bugs should rather be marked released
when a published version of the lib carries the fix). Git tags and
Launchpad milestones no longer align, which creates a lot of confusion.

Then as more libraries appeared, some of them piggybacked on the general
"oslo" Launchpad project (generally adding tags to point to the specific
library), and some others created their own project. More confusion ensues.

Here is a proposal that we could use to solve that, until StoryBoard
gets proper milestone support and Oslo is just migrated to it:

1. Ask for an "oslo" project group in Launchpad

This would solve the first issue, by allowing to see all oslo work on
single pages (see examples at [1] or [2]). The trade-off here is that
Launchpad projects can't be part of multiple project groups (and project
groups can't belong to project groups). That means that Oslo projects
will no longer be in the "openstack" Launchpad projectgroup. I think the
benefits outweigh the drawbacks here: the openstack projectgroup is not
very strict anyway so I don't think it's used in people workflows that much.

2. Create one project per library, adopt tag-based milestones

Each graduated library should get its own project (part of the oslo
projectgroup). It should use the same series/cycles as everyone else
("juno"), but it should have milestones that match the alpha release
tags, so that you can target work to it and mark them "fix released"
when that means the fix is released. That would solve the issue of
misaligned tags/milestones. The trade-off here is that you lose the
common milestone rhythm (although I guess you can still align some
alphas to the common development milestones). That sounds a small price
to pay to better communicate which version has which fix.

3. Rename "oslo" project to "oslo-incubator"

Keep the Launchpad "oslo" project as-is, part of the same projectgroup,
to cover oslo-incubator work. This can keep the common development
milestones, since the incubator doesn't do "releases" anyway. However,
it has to be renamed to "oslo-incubator" so that it doesn't conflict
with the projectgroup namespace. Once it no longer contains graduated
libs, that name makes much more sense anyway.


This plan requires Launchpad admin assistance to create a projectgroup
and rename a project, so I'd like to get approval on it before moving to
ask them for help.

Comments, thoughts ?

[1] https://blueprints.launchpad.net/openstack
[2] https://bugs.launchpad.net/openstack

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-22 Thread Thierry Carrez
Hi everyone,

We all know being a project PTL is an extremely busy job. That's because
in our structure the PTL is responsible for almost everything in a project:

- Release management contact
- Work prioritization
- Keeping bugs under control
- Communicate about work being planned or done
- Make sure the gate is not broken
- Team logistics (run meetings, organize sprints)
- ...

They end up being completely drowned in those day-to-day operational
duties, miss the big picture, can't help in development that much
anymore, get burnt out. Since you're either "the PTL" or "not the PTL",
you're very alone and succession planning is not working that great either.

There have been a number of experiments to solve that problem. John
Garbutt has done an incredible job at helping successive Nova PTLs
handling the release management aspect. Tracy Jones took over Nova bug
management. Doug Hellmann successfully introduced the concept of Oslo
liaisons to get clear point of contacts for Oslo library adoption in
projects. It may be time to generalize that solution.

The issue is one of responsibility: the PTL is ultimately responsible
for everything in a project. If we can more formally delegate that
responsibility, we can avoid getting up to the PTL for everything, we
can rely on a team of people rather than just one person.

Enter the Czar system: each project should have a number of liaisons /
official contacts / delegates that are fully responsible to cover one
aspect of the project. We need to have Bugs czars, which are responsible
for getting bugs under control. We need to have Oslo czars, which serve
as liaisons for the Oslo program but also as active project-local oslo
advocates. We need Security czars, which the VMT can go to to progress
quickly on plugging vulnerabilities. We need release management czars,
to handle the communication and process with that painful OpenStack
release manager. We need Gate czars to serve as first-line-of-contact
getting gate issues fixed... You get the idea.

Some people can be czars of multiple areas. PTLs can retain some czar
activity if they wish. Czars can collaborate with their equivalents in
other projects to share best practices. We just need a clear list of
areas/duties and make sure each project has a name assigned to each.

Now, why czars ? Why not rely on informal activity ? Well, for that
system to work we'll need a lot of people to step up and sign up for
more responsibility. Making them "czars" makes sure that effort is
recognized and gives them something back. Also if we don't formally
designate people, we can't really delegate and the PTL will still be
directly held responsible. The Release management czar should be able to
sign off release SHAs without asking the PTL. The czars and the PTL
should collectively be the new "project drivers".

At that point, why not also get rid of the PTL ? And replace him with a
team of czars ? If the czar system is successful, the PTL should be
freed from the day-to-day operational duties and will be able to focus
on the project health again. We still need someone to keep an eye on the
project-wide picture and coordinate the work of the czars. We need
someone to pick czars, in the event multiple candidates sign up. We also
still need someone to have the final say in case of deadlocked issues.

People say we don't have that many deadlocks in OpenStack for which the
PTL ultimate power is needed, so we could get rid of them. I'd argue
that the main reason we don't have that many deadlocks in OpenStack is
precisely *because* we have a system to break them if they arise. That
encourages everyone to find a lazy consensus. That part of the PTL job
works. Let's fix the part that doesn't work (scaling/burnout).

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-22 Thread Thierry Carrez
Russell Bryant wrote:
> On 08/22/2014 09:40 AM, Russell Bryant wrote:
>> Another area worth calling out is a gate czar.  Having someone who
>> understands infra and QA quite well and is regularly on top of the
>> status of the project in the gate is helpful and quite important.
> 
> Oops, you said this one, too.  Anyway, +1.

The one I forgot in my example list would be the Docs czar. I'm pretty
sure Anne would appreciate a clear point contact for docs in every
integrated project.

Another interesting side-effect of having clear positions is that if
nobody fills them (which is a possibility), it's clear and public that
there is a gap. Today the gap just ends up on the PTL's list of other
things to also do.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-22 Thread Thierry Carrez
Daniel P. Berrange wrote:
> On Fri, Aug 22, 2014 at 02:33:27PM +0200, Thierry Carrez wrote:
>> Hi everyone,
>>
>> We all know being a project PTL is an extremely busy job. That's because
>> in our structure the PTL is responsible for almost everything in a project:
>>
>> - Release management contact
>> - Work prioritization
>> - Keeping bugs under control
>> - Communicate about work being planned or done
>> - Make sure the gate is not broken
>> - Team logistics (run meetings, organize sprints)
>> - ...
> 
> This is a good list of responsbilities, but I feel like we're missing
> something that is possibly a little too fuzzy to describe a bullet
> point. In terms of our governance process, PTL is an elected role, so
> in that sense the PTL holds an implicit duty to represent the interests
> of the project constituents. I'd characterise this as setting the overall
> big picture direction, so that the core team (and/or czars) are focusing
> their efforts in the right directioon and generally ensuring that the
> project is operating in a healthy manner. Perhaps you could class that
> as 'team logistics' but I feel that the idea of 'representation the
> voters' is is worth calling out explicitly.

Indeed. I touch on the "keep an eye on the big picture" aspect of the
job later in the email, but I didn't call it out in that list.

> Is there any formal writeup of the PTL role's responsibilities ?
> With Google so far I only found one paragraph of governance
> 
>   https://wiki.openstack.org/wiki/Governance/Foundation/Structure
> 
>   "Project Technical Leads (PTLs) lead individual projects. A PTL
>is ultimately responsible for the direction for each project,
>makes tough calls when needed, organizes the work and teams in
>the project and determines if other forms of project leadership
>are needed. The PTL for a project is elected by the body of
>contributors to that particular project."

The reference would be:
https://wiki.openstack.org/wiki/PTLguide

> Anyway, with the idea of elections & representation in mind
> 
>> Enter the Czar system: each project should have a number of liaisons /
>> official contacts / delegates that are fully responsible to cover one
>> aspect of the project. We need to have Bugs czars, which are responsible
>> for getting bugs under control. We need to have Oslo czars, which serve
>> as liaisons for the Oslo program but also as active project-local oslo
>> advocates. We need Security czars, which the VMT can go to to progress
>> quickly on plugging vulnerabilities. We need release management czars,
>> to handle the communication and process with that painful OpenStack
>> release manager. We need Gate czars to serve as first-line-of-contact
>> getting gate issues fixed... You get the idea.
>>
>> Some people can be czars of multiple areas. PTLs can retain some czar
>> activity if they wish. Czars can collaborate with their equivalents in
>> other projects to share best practices. We just need a clear list of
>> areas/duties and make sure each project has a name assigned to each.
>>
>> Now, why czars ? Why not rely on informal activity ? Well, for that
>> system to work we'll need a lot of people to step up and sign up for
>> more responsibility. Making them "czars" makes sure that effort is
>> recognized and gives them something back. Also if we don't formally
>> designate people, we can't really delegate and the PTL will still be
>> directly held responsible. The Release management czar should be able to
>> sign off release SHAs without asking the PTL. The czars and the PTL
>> should collectively be the new "project drivers".
>>
>> At that point, why not also get rid of the PTL ? And replace him with a
>> team of czars ? If the czar system is successful, the PTL should be
>> freed from the day-to-day operational duties and will be able to focus
>> on the project health again. We still need someone to keep an eye on the
>> project-wide picture and coordinate the work of the czars. We need
>> someone to pick czars, in the event multiple candidates sign up. We also
>> still need someone to have the final say in case of deadlocked issues.
> 
>  I'm wondering how people come to be czars ? You don't say explicitly,
> but reading this it feels like the team of czars would be more or less
> self-selecting amongst the project contributors or nominated by the PTL ?

I think you would have volunteers (and volunteered) people. In the
unlikely case people fight to become the czar, I guess the PTL could
have final 

Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-22 Thread Thierry Carrez
Zane Bitter wrote:
> On 22/08/14 08:33, Thierry Carrez wrote:
>> We also
>> still need someone to have the final say in case of deadlocked issues.
> 
> -1 we really don't.

I know we disagree on that :)

>> People say we don't have that many deadlocks in OpenStack for which the
>> PTL ultimate power is needed, so we could get rid of them. I'd argue
>> that the main reason we don't have that many deadlocks in OpenStack is
>> precisely *because* we have a system to break them if they arise.
> 
> s/that many/any/ IME and I think that threatening to break a deadlock by
> fiat is just as bad as actually doing it. And by 'bad' I mean
> community-poisoningly, trust-destroyingly bad.

I guess I've been active in too many dysfunctional free and open source
software projects -- I put a very high value on the ability to make a
final decision. Not being able to make a decision is about as
community-poisoning, and also results in inability to make any
significant change or decision.

>> That
>> encourages everyone to find a lazy consensus. That part of the PTL job
>> works. Let's fix the part that doesn't work (scaling/burnout).
> 
> Let's allow projects to decide for themselves what works. Not every
> project is the same.

The net effect of not having a PTL having the final call means the final
call would reside at the Technical Committee level. I don't feel like
the Technical Committee should have final say on a project-specific
matter. It's way better that the local leader, chosen by all the
contributors of THAT project every 6 months, makes that final decision.
Or do you also want to get rid of the Technical Committee ?

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Heat] Heat Juno Mid-cycle Meetup report

2014-08-25 Thread Thierry Carrez
Zane Bitter wrote:
> [...]
> Here are a few of the conclusions:
> 
> * Everyone wishes the Design Summit worked like this.
> The meetup seemed a lot more productive than the design summit ever is.
> It's really nice to be in a room small enough that you can talk normally
> and hear everyone, instead of in a room designed for 150 people. It's
> really nice to be able to discuss stuff that isn't related to a
> particular feature - we had a long discussion about how to get through
> the review backlog, for example. It's really nice to not have fixed time
> slots for discussions - because everyone was in the room the whole time,
> we could dip in and out of different topics at will. Often we came back
> to one that we'd previously discussed because we had discovered new
> information. Finally, it's critical to be in a room covered in
> full-sized whiteboards that everyone can see. A single tiny flip chart
> doesn't cut it.

That's good feedback, thanks. The current discussion on design summit
format changes is a bit lost under a Nova thread, so I should revive it
as a separate thread very soon. The idea being to implement whatever
changes we can to the summit to make it more productive (in the limited
remaining time and options we have for that).

> [...]
> * Marc^W Zaqar is critical to pretty much every major non-Convergence
> feature on the roadmap.
> We knew that we wanted to use it for notifications, but we also want to
> make those a replacement for events, and a conduit for warnings and
> debugging information to the user. This is becoming so important that
> we're going to push ahead with an implementation now without waiting to
> see when Zaqar will graduate. Zaqar would also be a good candidate for
> pushing metadata changes to servers, to resolve the performance issues
> currently caused by polling.

Could you expand on that ? Do you need some kind of user-facing queue
service, or is there something in the marc^WZaqar approach that makes it
especially appealing ?

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-25 Thread Thierry Carrez
Anne Gentle wrote:
> Rochelle.RochelleGrober wrote:
>>/flame-on
>> Let's call spades, spades here.  Czar is not only overkill, but the
>> wrong metaphor.
>> /flame-off
> 
> I'm with Rocky on the anti-czar-as-a-word camp. We all like clever names
> to shed the "corporate" stigma but this word ain't it. Liaison or lead?

Sure! I certainly didn't want to offend anyone with my choice of terms.
Google lists two meanings for the word:

1. an emperor of Russia before 1917 ("Tsar Nicholas II")
2. a person appointed by government to advise on and coordinate policy
in a particular area ("the former British drugs czar")

I was referring to the latter meaning (like the US presidency has had
"cybersecurity czars" in the past), but I'm fine with using "liaison"
instead -- it's just slightly more boring.

> Also wanted to point to https://wiki.openstack.org/wiki/PTLguide as it's
> quite nice.
> 
> I think PTLs tend to find help when they absolutely are ready to fall
> over, and I'm all for a plan that helps us not fall over. I've had
> people step up for bug triaging, gate work, tests, and oslo, sometimes
> one person did three or four at once. I certainly can't do it all for
> cross-project. Based on what I've seen, I doubt that we can add this
> much formality to this across 20+ programs. It's the "bigger more
> integrated project" vs. "smaller more focused project" difference that
> won't let us do a pattern here. We can certainly document the
> responsibilities and let programs know there are some best practices.

In smaller programs, I would expect the PTL to keep filling most, if not
all, of those roles. As long as the PTL is not overworked, the roles are
well-known and everyone knows who to contact, it works.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-25 Thread Thierry Carrez
Zane Bitter wrote:
> On 22/08/14 12:45, Dolph Mathews wrote:
>>> >I'm all for getting a final decision, but a 'final' decision that
>>> has been
>>> >imposed from outside rather than internalised by the participants is...
>>> >rarely final.
>>> >
>> The expectation of a PTL isn't to stomp around and make "final"
>> decisions,
>> it's to step in when necessary and help both sides find the best
>> solution.
>> To moderate.
> 
> Oh sure, but that's not just the PTL's job. That's everyone's job. Don't
> you think?
> 
> I did that before I was the PTL and will continue to do it after I'm no
> longer the PTL. And if anyone in the (especially) the core or wider Heat
> team sees an opportunity to step in and moderate a disagreement I
> certainly expect them to take it and not wait for me to step in.
> 
> I'm not calling for no leadership here - I'm calling for leadership from
> _everyone_, not just from one person who holds a particular role.

I guess the difference between you and me is that I don't see having a
PTL as preventing that moderation and leadership from everyone. I really
see it as a safety valve in case things ever go badly wrong. I prefer
that safety valve to be built into the project leadership, rather than
at the TC level.

Could you explain how having a PTL is preventing that "leadership from
everyone" ? Did it prevent it in Heat ? Did having the PTL safety valve
hurt you ?

I'm open to the alternative solution (which would be for programs which
are not interested in having a PTL to just not have one). But then if
things go badly wrong, you require the TC to step in with threats of
removal of OpenStack and/or to force an election/vote in the middle of
the crisis. I'm really failing to see how that would result, in those
hypothetical crisis scenarios, in a better outcome.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-25 Thread Thierry Carrez
Tim Bell wrote:
> As part of the user feedback loop, we've found the PTL role extremely useful 
> to channel feedback.  The operator PTL discussions during the Atlanta summit 
> helped to clarify a number of areas where the PTL can then take the points 
> back to the design summit. It is not clear how czars would address the 
> outward facing part of the PTL role which is clearly needed in view of the 
> various discussions around program management and priorities.
> 
> If we have lots of czars or major differences in how each project is 
> structured, it is not clear how we channel user feedback to the project 
> teams. Would there be a user czar on each project ?

I agree that this external/user communication role is essential, and is
likely to stay with the PTL, which is why I didn't have a "User czar" in
my list.

> I have no problem with lots of czars to delegate activities across the 
> projects but having a single accountable and elected PTL who can choose the 
> level of delegation (and be assessed on that) seems to be a very good feature.

Indeed, I see it more as a clear list of the duties that usually fall
onto the PTL lap, but which could be delegated. If some programs want to
keep it the way it is (with the PTL being responsible for all those
duties), it's totally fine.

It's a framework for clear delegation within the current system, not a
whole new system :)

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tc] Procedure for adding official projects

2014-08-25 Thread Thierry Carrez
Zane Bitter wrote:
> Over the past couple of release cycles, the TC has put together a fairly
> comprehensive checklist for projects entering into incubation with a
> view to being included in the integrated release. However, I'm not aware
> of anything equivalent for projects that are becoming official (i.e.
> moving to the openstack/ namespace) but that are not targeting eventual
> integration - or, indeed, that _are_ targeting eventual integration but
> have not yet applied for incubation.
> 
> The current procedure afaict is to submit a review to the governance
> repo listing the repository under a particular program. It seems like at
> a minimum we should be checking for basic due diligence stuff like "Is
> it Apache licensed?", "Did everyone sign the CLA?" (may it diaf) and
> "Are there any trademark issues?". And maybe there are other things the
> TC should be looking at too.

I agree... currently we only check that the program is fine with adding
that new repository (by checking with the PTL :P) and that the project
intent seems to fit in the program mission scope. Extra basic due
diligence can't hurt, and we could turn that into a new requirements list.

Would you be interested in proposing a basic "new projects in existing
program requirements" list as a governance repo change ? Then we could
iterate on it.

Cheers,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Launchpad tracking of oslo projects

2014-08-26 Thread Thierry Carrez
Doug Hellmann wrote:
> 
> On Aug 22, 2014, at 5:59 AM, Thierry Carrez  wrote:
> 
>> TL;DR:
>> Let's create an Oslo projectgroup in Launchpad to track work across all
>> Oslo libraries. In library projects, let's use milestones connected to
>> published versions rather than the common milestones.
>>
>> Long version:
>> As we graduate more Oslo libraries (which is awesome), tracking Oslo
>> work in Launchpad (bugs, milestones...) has become more difficult.
>>
>> There used to be only one Launchpad project ("oslo", which covered the
>> oslo incubator). It would loosely follow the integrated milestones
>> (juno-1, juno-2...), get blueprints and bugs targeted to those, get tags
>> pushed around those development milestones: same as the integrated
>> projects, just with no source code tarball uploads.
>>
>> When oslo.messaging graduated, a specific Launchpad project was created
>> to track work around it. It still had integrated development milestones
>> -- only at the end it would publish a 1.4.0 release instead of a 2014.2
>> one. That approach creates two problems. First, it's difficult to keep
>> track of "oslo" work since it now spans two projects. Second, the
>> release management logic of marking bugs "Fix released" at development
>> milestones doesn't really apply (bugs should rather be marked released
>> when a published version of the lib carries the fix). Git tags and
>> Launchpad milestones no longer align, which creates a lot of confusion.
>>
>> Then as more libraries appeared, some of them piggybacked on the general
>> "oslo" Launchpad project (generally adding tags to point to the specific
>> library), and some others created their own project. More confusion ensues.
>>
>> Here is a proposal that we could use to solve that, until StoryBoard
>> gets proper milestone support and Oslo is just migrated to it:
>>
>> 1. Ask for an "oslo" project group in Launchpad
>>
>> This would solve the first issue, by allowing to see all oslo work on
>> single pages (see examples at [1] or [2]). The trade-off here is that
>> Launchpad projects can't be part of multiple project groups (and project
>> groups can't belong to project groups). That means that Oslo projects
>> will no longer be in the "openstack" Launchpad projectgroup. I think the
>> benefits outweigh the drawbacks here: the openstack projectgroup is not
>> very strict anyway so I don't think it's used in people workflows that much.
>>
>> 2. Create one project per library, adopt tag-based milestones
>>
>> Each graduated library should get its own project (part of the oslo
>> projectgroup). It should use the same series/cycles as everyone else
>> ("juno"), but it should have milestones that match the alpha release
>> tags, so that you can target work to it and mark them "fix released"
>> when that means the fix is released. That would solve the issue of
>> misaligned tags/milestones. The trade-off here is that you lose the
>> common milestone rhythm (although I guess you can still align some
>> alphas to the common development milestones). That sounds a small price
>> to pay to better communicate which version has which fix.
> 
> We don’t necessarily decide the version numbers for all of the libraries in 
> advance. I think we talked about this on IRC, and your suggestion was to use 
> a “next” milestone and then rename it at the point of release. Am I 
> remembering correctly?

Yes, using "next" and renaming it once you know is the way to go.

>> 3. Rename "oslo" project to "oslo-incubator"
>>
>> Keep the Launchpad "oslo" project as-is, part of the same projectgroup,
>> to cover oslo-incubator work. This can keep the common development
>> milestones, since the incubator doesn't do "releases" anyway. However,
>> it has to be renamed to "oslo-incubator" so that it doesn't conflict
>> with the projectgroup namespace. Once it no longer contains graduated
>> libs, that name makes much more sense anyway.
>>
>>
>> This plan requires Launchpad admin assistance to create a projectgroup
>> and rename a project, so I'd like to get approval on it before moving to
>> ask them for help.
>>
>> Comments, thoughts ?
>>
>> [1] https://blueprints.launchpad.net/openstack
>> [2] https://bugs.launchpad.net/openstack
>>
> 
> This makes sense to me, so let’s move ahead with your plan.

I'd like to wait for Mark McLoughl

Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-26 Thread Thierry Carrez
OK, now that we have evacuated the terminology issue (we'll use liaison
or janitor or secretary, not czar), and side-stepped the offtopic
development (this is not about suppressing PTLs, just a framework to let
them delegate along predetermined lines if they want to)... which of
those unnamed roles do we need ?

In the thread were mentioned:
- Bugs janitor (keep reported bugs under control)
- Oslo liaison (already in place)
- Security mule (VMT first point of contact)
- Release secretary (communication with integrated release management)
- Infrastructure contact (for gate and other infra issues)
- Docs lieutenant (docs point of contact)

Anita mentioned the "3rd party space" person, but I wonder if it would
not be specific to some projects. Would it actually be separate from the
"Infra contact" role ?

Do we need someone to cover the QA space ? Anything else missing ?

At first glance I don't think we need a role for logistics (chairing
meetings and organizing meetups), design summit planning, roadmapping,
user point of contact, or spokesperson -- as I expect the PTL to retain
those roles anyway...

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [infra] [keystone] pysaml2/xmlsec1 dep blocking keystone-to-keystone federation

2014-08-26 Thread Thierry Carrez
Hi keystone/infra,

One key upcoming Juno feature (Keystone to keystone federation) is
currently blocked on adding pysaml2 to requirements:

https://review.openstack.org/#/c/113294/

It was -1ed by Doug after the discussion at the release meeting last
week, where the xmlsec1 dependency was raised as a potential infra issue.

There doesn't seem to be so many good alternatives though. Steve
mentioned saml, but it's a bit alpha, and I have no idea how much work
would be required to use that instead of pysaml2 at this point.

How blocking is the xmlsec1 dependency from an Infra perspective ? How
doable would a migration to saml at this point be ? I'm trying to find a
solution so that we can ship this feature :)

Regards,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [all] Design Summit reloaded

2014-08-27 Thread Thierry Carrez
Hi everyone,

I've been thinking about what changes we can bring to the Design Summit
format to make it more productive. I've heard the feedback from the
mid-cycle meetups and would like to apply some of those ideas for Paris,
within the constraints we have (already booked space and time). Here is
something we could do:

Day 1. Cross-project sessions / incubated projects / other projects

I think that worked well last time. 3 parallel rooms where we can
address top cross-project questions, discuss the results of the various
experiments we conducted during juno. Don't hesitate to schedule 2 slots
for discussions, so that we have time to come to the bottom of those
issues. Incubated projects (and maybe "other" projects, if space allows)
occupy the remaining space on day 1, and could occupy "pods" on the
other days.

Day 2 and Day 3. Scheduled sessions for various programs

That's our traditional scheduled space. We'll have a 33% less slots
available. So, rather than trying to cover all the scope, the idea would
be to focus those sessions on specific issues which really require
face-to-face discussion (which can't be solved on the ML or using spec
discussion) *or* require a lot of user feedback. That way, appearing in
the general schedule is very helpful. This will require us to be a lot
stricter on what we accept there and what we don't -- we won't have
space for courtesy sessions anymore, and traditional/unnecessary
sessions (like my traditional "release schedule" one) should just move
to the mailing-list.

Day 4. Contributors meetups

On the last day, we could try to split the space so that we can conduct
parallel midcycle-meetup-like contributors gatherings, with no time
boundaries and an open agenda. Large projects could get a full day,
smaller projects would get half a day (but could continue the discussion
in a local bar). Ideally that meetup would end with some alignment on
release goals, but the idea is to make the best of that time together to
solve the issues you have. Friday would finish with the design summit
feedback session, for those who are still around.


I think this proposal makes the best use of our setup: discuss clear
cross-project issues, address key specific topics which need
face-to-face time and broader attendance, then try to replicate the
success of midcycle meetup-like open unscheduled time to discuss
whatever is hot at this point.

There are still details to work out (is it possible split the space,
should we use the usual design summit CFP website to organize the
"scheduled" time...), but I would first like to have your feedback on
this format. Also if you have alternative proposals that would make a
better use of our 4 days, let me know.

Cheers,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Design Summit reloaded

2014-08-27 Thread Thierry Carrez
Daniel P. Berrange wrote:
> On Wed, Aug 27, 2014 at 02:51:55PM +0200, Thierry Carrez wrote:
>> [...]
>> I think this proposal makes the best use of our setup: discuss clear
>> cross-project issues, address key specific topics which need
>> face-to-face time and broader attendance, then try to replicate the
>> success of midcycle meetup-like open unscheduled time to discuss
>> whatever is hot at this point.
>>
>> There are still details to work out (is it possible split the space,
>> should we use the usual design summit CFP website to organize the
>> "scheduled" time...), but I would first like to have your feedback on
>> this format. Also if you have alternative proposals that would make a
>> better use of our 4 days, let me know.
> 
> +1, I think what you've proposed is a pretty attractive evolution of
> our previous design summit formats. I figure it is safer trying such
> an evolutionary approach for Paris, rather than trying to make too
> much of a "big bang" revolution at one time. 

We have too many fixed constraints at this time for a "big bang" anyway.

What I like in the format is that the nature of the 4th day can change
completely based on the result of the 3 previous days. If a major topic
emerged, you can address it. If a continuation discussion is needed, you
can have it. If you are completely drained of any energy, you can spend
a quiet time together with a lighter agenda, or no agenda   at all.

It would still be open for everyone, but the placement (Friday) and
title in the schedule ("X contributors gathering") should feel
unattractive enough so that attendance is generally smaller and more
on-topic. We'll likely have to split rooms, so people who have been
complaining about giant rooms being detrimental should be happy too.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Launchpad tracking of oslo projects

2014-08-28 Thread Thierry Carrez
Thierry Carrez wrote:
> Doug Hellmann wrote:
>> This makes sense to me, so let’s move ahead with your plan.

OK, this is now done:

Project group @ https://launchpad.net/oslo
Oslo incubator: https://launchpad.net/oslo-incubator
oslo.messaging: https://launchpad.net/oslo.messaging

General blueprint view: https://blueprints.launchpad.net/oslo
General bug view: https://bugs.launchpad.net/oslo

>> We do have launchpad projects for some of the other oslo libraries, we just 
>> haven’t been using them for release tracking:
>>
>> https://launchpad.net/python-stevedore
>> https://launchpad.net/python-cliff
>> https://launchpad.net/taskflow
>> https://launchpad.net/pbr
>> https://launchpad.net/oslo.vmware
> 
> Cool, good to know. I'll include them in the oslo group if we create it.

I added pbr, but I don't have the rights to move the other ones. It
would generally be good to have oslo-drivers added as maintainer or
driver for those projects so that we can fix them, if they are part of oslo.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] gate debugging

2014-08-28 Thread Thierry Carrez
David Kranz wrote:
> On 08/27/2014 03:43 PM, Sean Dague wrote:
>> On 08/27/2014 03:33 PM, David Kranz wrote:
>>> Race conditions are what makes debugging very hard. I think we are in
>>> the process of experimenting with such an idea: asymetric gating by
>>> moving functional tests to projects, making them deeper and more
>>> extensive, and gating against their own projects. The result should be
>>> that when a code change is made, we will spend much more time running
>>> tests of code that is most likely to be growing a race bug from the
>>> change. Of course there is a risk that we will impair integration
>>> testing and we will have to be vigilant about that. One mitigating
>>> factor is that if cross-project interaction uses apis (official or not)
>>> that are well tested by the functional tests, there is less risk that a
>>> bug will only show up only when those apis are used by another project.
>>
>> So, sorry, this is really not about systemic changes (we're running
>> those in parallel), but more about skills transfer in people getting
>> engaged. Because we need both. I guess that's the danger of breaking the
>> thread is apparently I lost part of the context.
>>
> I agree we need both. I made the comment because if we can make gate
> debugging less daunting
> then less skill will be needed and I think that is key. Honestly, I am
> not sure the full skill you have can be transferred. It was gained
> partly through learning in simpler times.

I think we could develop tools and visualizations that would help the
debugging tasks. We could make those tasks more visible, and therefore
more appealing to the brave souls that step up to tackle them. Sean and
Joe did a ton of work improving the raw data, deriving graphs from it,
highlighting log syntax or adding helpful Apache footers. But those days
they spend so much time fixing the issues themselves, they can't
continue on improving those tools.

And that's part of where the gate burnout comes from: spending so much
time on the issues themselves that you can no longer work on preventing
them from happening, or making the job of handling the issues easier, or
documenting/mentoring other people so that they can do it in your place.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-28 Thread Thierry Carrez
acknowledging that code repositories should be
organized in the way that makes the most sense technically. They should
not be artificially organized to match our governance structure.

Before programs existed, it was difficult for teams to organize their
code the way they wanted, because there was only one code repository
("The Project"), so everything had to be in it. Then we added an
exception for the Python client projects, so the Nova team could work on
the Nova project *and* the Python client for it. But then it made sense
to organize the code differently, so rather than continuing to add
exceptions (which you can see traces of at stale page [1]), the easiest
way to organize that was to just say that a given team could rule a set
of code repositories, and organize them as they preferred.

So teams, organized around a clear mission statement, could decide which
code repositories they wanted to organize their code in. We call those
teams "programs".

[1] https://wiki.openstack.org/wiki/ProjectTypes

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Is the BP approval process broken?

2014-08-29 Thread Thierry Carrez
Joe Gordon wrote:
> On Thu, Aug 28, 2014 at 2:43 PM, Alan Kavanagh
> mailto:alan.kavan...@ericsson.com>> wrote:
> 
>> I share Donald's points here, I believe what would help is to
>> clearly describe in the Wiki the process and workflow for the BP
>> approval process and build in this process how to deal with
>> discrepancies/disagreements and build timeframes for each stage and
>> process of appeal etc.
>> The current process would benefit from some fine tuning and helping
>> to build safe guards and time limits/deadlines so folks can expect
>> responses within a reasonable time and not be left waiting in the cold.
> 
> This is a resource problem, the nova team simply does not have enough
> people doing enough reviews to make this possible. 

I think Nova lacks core reviewers more than it lacks reviewers, though.
Just looking at the ratio of core developers vs. patchsets proposed,
it's pretty clear that the core team is too small:

Nova: 750 patchsets/month for 21 core = 36
Heat: 230/14 = 16
Swift: 50/16 = 3

Neutron has the same issue (550/14 = 39). I think above 20, you have a
dysfunctional setup. No amount of process, spec, or runway will solve
that fundamental issue.

The problem is, you can't just add core reviewers, they have to actually
understand enough of the code base to be trusted with that +2 power. All
potential candidates are probably already in. In Nova, the code base is
so big it's difficult to find people that know enough of it. In Neutron,
the contributors are often focused on subsections of the code base so
they are not really interested in learning enough of the rest. That
makes the pool of core candidates quite dry.

I fear the only solution is smaller groups being experts on smaller
codebases. There is less to review, and more candidates that are likely
to be experts in this limited area.

Applied to Nova, that means modularization -- having strong internal
interfaces and trusting subteams to +2 the code they are experts on.
Maybe VMWare driver people should just +2 VMware-related code. We've had
that discussion before, and I know there is a dangerous potential
quality slope there -- I just fail to see any other solution to bring
that 750/21=36 figure down to a bearable level, before we burn out all
of the Nova core team.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Design Summit reloaded

2014-08-29 Thread Thierry Carrez
Sean Dague wrote:
> On 08/28/2014 03:06 PM, Jay Pipes wrote:
>> On 08/28/2014 02:21 PM, Sean Dague wrote:
>>> On 08/28/2014 01:58 PM, Jay Pipes wrote:
>>>> On 08/27/2014 11:34 AM, Doug Hellmann wrote:
>>>>> On Aug 27, 2014, at 8:51 AM, Thierry Carrez 
>>>>> wrote:
>>>>>> Day 1. Cross-project sessions / incubated projects / other
>>>>>> projects
>>>>>>
>>>>>> I think that worked well last time. 3 parallel rooms where we can
>>>>>> address top cross-project questions, discuss the results of the
>>>>>> various experiments we conducted during juno. Don't hesitate to
>>>>>> schedule 2 slots for discussions, so that we have time to come to
>>>>>> the bottom of those issues. Incubated projects (and maybe "other"
>>>>>> projects, if space allows) occupy the remaining space on day 1, and
>>>>>> could occupy "pods" on the other days.
>>>>>
>>>>> If anything, I’d like to have fewer cross-project tracks running
>>>>> simultaneously. Depending on which are proposed, maybe we can make
>>>>> that happen. On the other hand, cross-project issues is a big theme
>>>>> right now so maybe we should consider devoting more than a day to
>>>>> dealing with them.
>>>>
>>>> I agree with Doug here. I'd almost say having a single cross-project
>>>> room, with serialized content would be better than 3 separate
>>>> cross-project tracks. By nature, the cross-project sessions will attract
>>>> developers that work or are interested in a set of projects that looks
>>>> like a big Venn diagram. By having 3 separate cross-project tracks, we
>>>> would increase the likelihood that developers would once more have to
>>>> choose among simultaneous sessions that they have equal interest in. For
>>>> Infra and QA folks, this likelihood is even greater...
>>>>
>>>> I think I'd prefer a single cross-project track on the first day.
>>>
>>> So the fallout of that is there will be 6 or 7 cross-project slots for
>>> the design summit. Maybe that's the right mix if the TC does a good job
>>> picking the top 5 things we want accomplished from a cross project
>>> standpoint during the cycle. But it's going to have to be a pretty
>>> directed pick. I think last time we had 21 slots, and with a couple of
>>> doubling up that gave 19 sessions. (about 30 - 35 proposals for that
>>> slot set).
>>
>> I'm not sure that would be a bad thing :)
>>
>> I think one of the reasons the mid-cycles have been successful is that
>> they have adequately limited the scope of discussions and I think by
>> doing our homework by fully vetting and voting on cross-project sessions
>> and being OK with saying "No, not this time.", we will be more
>> productive than if we had 20+ cross-project sessions.
>>
>> Just my two cents, though..
> 
> I'm not sure it would be a bad thing either. I just wanted to be
> explicit about what we are saying the cross projects sessions are for in
> this case: the 5 key cross project activities the TC believes should be
> worked on this next cycle.

There is a trade-off here. Parallel cross-project tracks let us address
more issues in the limited time we have, and they also let us split the
audience so that we don't end up at 500 in the same room and nothing
gets done in 40min. It's true that sometimes you wish you could be in
two different places at the same time, but we generally prevent the most
blatant collisions during scheduling, and sometimes forcing people to
choose what they really care about is not that bad.

The feedback I got from Atlanta was that the 3-parallel-room setup went
well, and there weren't that many conflicts.

Maybe having *2* cross-project topics running at the same time (instead
of 3 or 1) would be the right trade-off. We would still need to be more
picky in selecting which issues we want to address, we would split the
audience into two rooms, and we would reduce the likelihood of conflict
significantly.

> The other question is if we did that what's running in competition to
> cross project day? Is it another free form pod day for people not
> working on those things?

The 3 or 4 other rooms would give incubated projects (and "other"
projects) some scheduled time. It also runs at the same time as the
conference.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Design Summit reloaded

2014-08-29 Thread Thierry Carrez
Anne Gentle wrote:
> On Wed, Aug 27, 2014 at 7:51 AM, Thierry Carrez  <mailto:thie...@openstack.org>> wrote:
> 
> Hi everyone,
> 
> I've been thinking about what changes we can bring to the Design Summit
> format to make it more productive. I've heard the feedback from the
> mid-cycle meetups and would like to apply some of those ideas for Paris,
> within the constraints we have (already booked space and time). Here is
> something we could do:
> 
> Day 1. Cross-project sessions / incubated projects / other projects
> 
> I think that worked well last time. 3 parallel rooms where we can
> address top cross-project questions, discuss the results of the various
> experiments we conducted during juno. Don't hesitate to schedule 2 slots
> for discussions, so that we have time to come to the bottom of those
> issues. Incubated projects (and maybe "other" projects, if space allows)
> occupy the remaining space on day 1, and could occupy "pods" on the
> other days.
> 
> Yep, I think this works in theory, the tough part will be when all the
> incubating projects realize they're sending people for a single day?
> Maybe it'll work out differently than I think though. It means fitting
> ironic, barbican, designate, manila, marconi in a day? 

Actually those projects would get pod space for the rest of the week, so
they should stay! Also some of them might have graduated by then :)

> Also since QA, Infra, and Docs are cross-project AND Programs, where do
> they land?

I think those teams work on different issues. Some issues require a lot
of communication and input because they are cross-project problems that
those teams are tasked with solving -- in which case that belongs to the
cross-project day. Other issues are more implementation details and
require mostly the team members but not so much external input -- those
belong to the specific slots or the "contributors meetup". Obviously
some things will be a bit borderline and we'll have to pick one or the
other based on available slots.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-29 Thread Thierry Carrez
James Polley wrote:
> 
> > However, Thierry pointed
> > to https://wiki.openstack.org/wiki/Governance/Foundation/Structure
> which
> > still refers to Project Technical Leads and says explicitly that they
> > lead individual projects, not programs. I actually have edit access to
> > that page, so I could at least update that with a simple
> > "s/Project/Program/", if I was sure that was the right thing to do.
> 
> Don't underestimate how stale wiki pages can become! Yes, fix it.
> 
> I don't know if I've fixed it, but I've certainly replaced all users of
> the word Project with Program.
> 
> Whether or not it now matches reality, I'm not sure.
> 
> I alsp removed (what I assume is) a stale reference to the PPB and added
> a new heading for the TC.

It looks correct to me, thanks!

> > http://www.openstack.org/ has a link in the bottom nav that says
> > "Projects"; it points to http://www.openstack.org/projects/ which
> > redirects to http://www.openstack.org/software/ which has a list of
> > things like "Compute" and "Storage" - which as far as I know are
> > Programs, not Projects. I don't know how to update that link in
> the nav
> > panel.
> 
> That's because the same word ("compute") is used for two different
> things: a program name ("Compute") and an "official OpenStack name" for
> a project ("OpenStack Compute a.k.a. Nova"). Basically official
> OpenStack names reduce confusion for newcomers ("What is Nova ?"), but
> they confuse old-timers at some point ("so the Compute program produces
> Nova a.k.a. OpenStack Compute ?").
> 
> 
> That's confusing to me. I had thought that part of the reason for the
> separation was to enable a level of indirection - if the Compute program
> team decide that a new project called (for example) SuperNova should be
> the main project, that just means that Openstack Compute is now a
> pointer to a different project, supported by the same program team.
> 
> It sounds like that isn't the intent though?

That's more of a side-effect than the intent, IMHO. The indirection we
created is between teams and code repositories.

> > I wasn't around when the original Programs/Projects discussion was
> > happening - which, I suspect, has a lot to do with why I'm confused
> > today - it seems as though people who were around at the time
> understand
> > the difference, but people who have joined since then are relying on
> > multiple conflicting verbal definitions. I believe, though,
> > that
> http://lists.openstack.org/pipermail/openstack-dev/2013-June/010821.html
> > was one of the earliest starting points of the discussion. That page
> > points at https://wiki.openstack.org/wiki/Projects, which today
> contains
> > a list of Programs. That page does have a definition of what a Program
> > is, but doesn't explain what a Project is or how they relate to
> > Programs. This page seems to be locked down, so I can't edit it.
> 
> https://wiki.openstack.org/wiki/Projects was renamed to
> https://wiki.openstack.org/wiki/Programs with the wiki helpfully leaving
> a redirect behind. So the content you are seeing here is the "Programs"
> wiki page, which is why it doesn't define "projects".
> 
> We don't really use the word "project" that much anymore, we prefer to
> talk about code repositories. Programs are teams working on a set of
> code repositories. Some of those code repositories may appear in the
> integrated release.
> 
> This explanation of the difference between projects and programs sounds
> like it would be useful to add to /Programs - but I can't edit that page. 

This page reflects the official list of programs, which is why it's
protected. it's supposed to be replaced by an automatic publication from
http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml
which is the ultimate source of truth on that topic.

> [1] https://wiki.openstack.org/wiki/ProjectTypes
> 
> I *can* edit that page; I'd like to bring it up-to-date. It seems like a
> good basis for explaining the difference between Programs and Projects
> and the historical reasons for the split. I'll aim to take a stab at
> this next week.

Please feel free to do so, however that page is really an artifact of
the old way we were structured, and is therefore useful as an historic
leftover :) It's not linked from anywhere those days. Maybe you should
create a new page, like
https://wiki.openstack.org/wiki/Projects_vs_Programs ? What you want to
talk about is not really about "Project Types" anyway.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] change to deprecation policy in the incubator

2014-08-29 Thread Thierry Carrez
That all makes sense to me.

Doug Hellmann wrote:
> Before Juno we set a deprecation policy for graduating libraries that said 
> the incubated versions of the modules would stay in the incubator repository 
> for one full cycle after graduation. This gives projects time to adopt the 
> libraries and still receive bug fixes to the incubated version (see 
> https://wiki.openstack.org/wiki/Oslo#Graduation).
> 
> That policy worked well early on, but has recently introduced some challenges 
> with the low level modules. Other modules in the incubator are still 
> importing the incubated versions of, for example, timeutils, and so tests 
> that rely on mocking out or modifying the behavior of timeutils do not work 
> as expected when different parts of the application code end up calling 
> different versions of timeutils. We had similar issues with the notifiers and 
> RPC code, and I expect to find other cases as we continue with the 
> graduations.
> 
> To deal with this problem, I propose that for Kilo we delete graduating 
> modules as soon as the new library is released, rather than waiting to the 
> end of the cycle. We can update the other incubated modules at the same time, 
> so that the incubator will always use the new libraries and be consistent.
> 
> We have not had a lot of patches where backports were necessary, but there 
> have been a few important ones, so we need to retain the ability to handle 
> them and allow projects to adopt libraries at a reasonable pace. To handle 
> backports cleanly, we can “freeze” all changes to the master branch version 
> of modules slated for graduation during Kilo (we would need to make a good 
> list very early in the cycle), and use the stable/juno branch for backports.
> 
> The new process would be:
> 
> 1. Declare which modules we expect to graduate during Kilo.
> 2. Changes to those pre-graduation modules could be made in the master branch 
> before their library is released, as long as the change is also backported to 
> the stable/juno branch at the same time (we should enforce this by having 
> both patches submitted before accepting either).
> 3. When graduation for a library starts, freeze those modules in all branches 
> until the library is released.
> 4. Remove modules from the incubator’s master branch after the library is 
> released.
> 5. Land changes in the library first.
> 6. Backport changes, as needed, to stable/juno instead of master.
> 
> It would be better to begin the export/import process as early as possible in 
> Kilo to keep the window where point 2 applies very short.
> 
> If there are objections to using stable/juno, we could introduce a new branch 
> with a name like backports/kilo, but I am afraid having the extra branch to 
> manage would just cause confusion.
> 
> I would like to move ahead with this plan by creating the stable/juno branch 
> and starting to update the incubator as soon as the oslo.log repository is 
> imported (https://review.openstack.org/116934).
> 
> Thoughts?
> 
> Doug
> 
> 
> _______
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Design Summit reloaded

2014-08-29 Thread Thierry Carrez
Hayes, Graham wrote:
>>> Yep, I think this works in theory, the tough part will be when all the
>>> incubating projects realize they're sending people for a single day?
>>> Maybe it'll work out differently than I think though. It means fitting
>>> ironic, barbican, designate, manila, marconi in a day? 
>>
>> Actually those projects would get pod space for the rest of the week, so
>> they should stay! Also some of them might have graduated by then :)
> 
> Would the programs for those projects not get design summit time? I
> thought the Programs got Design summit time, not projects... If not, can
> the Programs get design summit time? 

Sure, that's what Anne probably meant. Time for the program behind every
incubated project.

Regards,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-09-01 Thread Thierry Carrez
James Polley wrote:
> I'm fairly certain the buzzing sound I can hear is a bee in my bonnet...
> so I suspect that I'm starting to sound like someone chasing a bee that
> only they can hear. I'm not sure if it's helpful to keep this discussion
> on this list - would there be a better forum somewhere else?

Not really, feel free to send to me personally if that works better for you.

>> This page reflects the official list of programs, which is why it's
>> protected. it's supposed to be replaced by an automatic publication from
>> 
>> http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml
>> which is the ultimate source of truth on that topic.
> 
> 
> I was going to ask about the reference to "The process new projects can
> follow to become an Integrated project" - is that intended to refer to a
> project or a program?
> 
> But then I read https://review.openstack.org/#/c/116727/ and
> and 
> http://git.openstack.org/cgit/openstack/governance/tree/reference/incubation-integration-requirements.rst,
> seem to make it clear that it's entirely possible that the Kitty program
> might have a mix of Integrated and non-Integrated projects.
> 
> Is it safe to assume that the Governance repo is canonical and
> up-to-date, and rework the wiki pages based on the information in the
> Governance repo?

Yes, the governance repository reflects the current governance. The wiki
pages are derived from it.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [all] Juno Feature freeze is coming - Last day for feature reviews !

2014-09-02 Thread Thierry Carrez
Hi everyone,

As you probably know, the Juno-3 milestone should be published Thursday
this week, and with it comes the Juno feature freeze. The general
schedule is as follows:

Tuesday:
Defer/-2 blueprints that will obviously not get the required approvals
in the next 20 hours. Review and approve as much as you can.

Wednesday:
Everything that is not approved and in-flight in the gate should be
deferred/-2.

Thursday:
Wait for the last approved stuff to make it through the gate and publish
the milestone.

Friday:
Start considering feature freeze exceptions for critical Juno features.

That really means *today is the last day for feature reviews*.
So please plan that extra review effort today, rather than tomorrow !

Thanks everyone for helping us do a successful Juno release.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Design Summit reloaded

2014-09-02 Thread Thierry Carrez
Hayes, Graham wrote:
> On Fri, 2014-08-29 at 17:56 +0200, Thierry Carrez wrote:
>> Hayes, Graham wrote:
>>> Would the programs for those projects not get design summit time? I
>>> thought the Programs got Design summit time, not projects... If not, can
>>> the Programs get design summit time? 
>>
>> Sure, that's what Anne probably meant. Time for the program behind every
>> incubated project.
> 
> Sure,
> I was referring to the the 2 main days - (days 2 and 3)
> 
> I thought that was a benefit of having a Program? The PTL chooses the
> sessions, and the PTL is over a program, so I assumed that programs
> would get both Pods and some design summit time (not 1/2 a day on the
> Tuesday)
> 
> I know we (designate) got some great work done last year, but most of it
> was in isolation, as we had one 40 min session, and one 1/2 day session,
> but the rest of the sessions were unofficial ones, which meant that
> people in the community who were not as engaged missed out on the
> discussions.
> 
> Would there be space for programs with incubated projects at the
> 'Contributors meetups' ?

We have limited space in Paris, so there won't be pods for everyone like
in Atlanta. I'm still waiting for venue maps to see how we can make the
best use of the space we have.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [releases] pbr, postversioning, integrated release workflow

2014-09-03 Thread Thierry Carrez
Robert Collins wrote:
> [...]
> Here's what happens.
> 
> The two interacting rules are this:
>  - pbr will now error if it tries to create a version number lower
> than the last release (and releases are found via the git tags in the
> branch).
>  - pbr treats preversion version numbers as a hard target it has to use
> 
> When we make a release (say 2014.1.0), we tag it, and we now have
> setup.cfg containing that version, and that version tagged.
> 
> The very next patch will be one patch *after* that version, so it has
> to be a patch for the next release. That makes the minimum legitimate
> release version 2014.1.1, and the local version number will be a dev
> build so 2014.1.1.dev1.g$shahere.
> 
> But the target it is required to use is 2014.1.0 - so we get a dev
> build of that (2014.1.0.dev1.g$shahere) - and thats lower than the
> last release (2014.1.0) and thus we trigger the error.
> 
> So, if we tag an API server branch with the same version it has in the
> branch, any patch that attempts to change that branch will fail,
> unless that patch is updating the version number in setup.cfg.

I think the current release process already enforces that (that the
patch after the tag is the setup.cfg version bump). That was the only
way to avoid building erroneous versions (2014.1.0.dev1 after 2014.1.0).
Here is what we do currently (say for Icehouse release):

At RC1 time on master branch, a patch is pushed to bump setup.cfg to
2014.2 (a.k.a. "open Juno development"). On master, future tags will be
2014.2.b1 etc.

A release branch (proposed/icehouse) is created from the last commit
before that version bump. That branch still has 2014.1 in setup.cfg, and
we control what lands on it. At release time, we tag 2014.1.0 on
proposed/icehouse. The very next commit on that branch is a version bump
on setup.cfg to go to 2014.1.1. The future tag on that branch will be
2014.1.1.

If I understand the issue correctly, that process will just continue to
work.

For details, see:
https://wiki.openstack.org/wiki/ReleaseTeam/How_To_Release#Final_release
in particular the "Push new version to master" and "Push .1 version to
stable/$SERIES branch" sections.

> [...]
> Going forward:
> 
> * We could just do the above - tag and submit a version fix
> 
> * We could submit the version fix as soon as the release sha is
> chosen, before the tag
>   - we could also wait for the version fixes to land before tagging
> 
> * We could change pbr to not enforce this check again
>   - or we could add an option to say 'I don't care'
> 
> * We could remove the version numbers from setup.cfg entirely
> 
> * We could change pbr to treat preversion versions as a *minimum*
> rather than a *must-reach*.
> 
> I'm in favour of the last of those options. Its quite a conceptual
> change from the current definition, which is why we didn't do it
> initially. The way it would work is that when pbr calculates the
> minimum acceptable version based on the tags and sem-ver: headers in
> git history, it would compare the result to the preversion version,
> and if the preversion version is higher, take that. The impact would
> be that if someone lands an ABI break on a stable-release branch, the
> major version would have to be bumped - and for API servers we don't
> want that. *but* thats something we should improve in pbr anyway -
> teach it how we map semver onto the API server projects [e.g. that
> major is the first two components of 2014.1.0, and minor and patch are
> bundled together into the third component.
> 
> The reason I prefer the last option over the second last, is that we
> don't have any mechanism to skip versions other than tagging today -
> and doing an alpha-0 tag at the opening of a new cycle just feels a
> little odd to me.

If my analysis above is right, I don't think we need to change anything:
the issue in pbr is only triggered if you try to do something you should
not do (i.e. have setup.cfg <= tag on non-tagged commits).

Let me know what you think,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [all] Juno Feature freeze - stop approving random changes

2014-09-03 Thread Thierry Carrez
Hi everyone,

Feature freeze is upon us, and with it, its inevitable 20-hour deep gate
queue. At this point the goal is to complete as many features as
possible before we tag juno-3 (ideally on Thursday). Given the queue
depth, anything that's not already in-flight has little chances of
making it by juno-3.

In order to preserve the gate for those last feature patches and reduce
the amount of disruptive feature freeze exceptions, I would like to ask
all core reviewers to stop approving random changes to the gate until
juno-3 milestone is completed.

That means at this point, only approve critical bug fixes, regressions,
security bugfixes or patches that directly complete a targeted feature.
Do *not* approve random bugfixes, or the 3rd patch in a series of 10
that you clearly know won't make it all in time for feature freeze.

Later today I'll be in touch with PTLs (or their release monkey) to
untarget all features that are not already gating: only features with
approved patches stuck in the gate will be kept on the list.

Thanks for helping us do a successful Juno release!

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] znc as a service (was Re: [nova] Is the BP approval process broken?)

2014-09-03 Thread Thierry Carrez
Stefano Maffulli wrote:
> On 08/29/2014 11:17 AM, John Garbutt wrote:
>> After moving to use ZNC, I find IRC works much better for me now, but
>> I am still learning really.
> 
> There! this sentence has two very important points worth highlighting:
> 
> 1- when people say IRC they mean IRC + a hack to overcome its limitation
> 2- IRC+znc is complex, not many people are used to it

Note that ZNC is not the only IRC proxy out there. Bip is also working
quite well.

> I never used znc, refused to install, secure and maintain yet another
> public facing service. For me IRC is: be there when it happens or read
> the logs on eavesdrop, if needed.
> 
> Recently I found out that there are znc services out there that could
> make things simpler but they're not easy to join (at least the couple I
> looked at).
> 
> Would it make sense to offer znc as a service within the openstack project?

We could at least document the steps required to set up a proxy. Or
propose pre-configured images/containers for individuals to run "in the
cloud". I agree with Ryan that running an IRC proxy for someone else
creates... interesting privacy issues that may just hinder adoption of
said solution.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] Meeting time update

2014-09-03 Thread Thierry Carrez
Tomas Sedovic wrote:
> As you all (hopefully) know, our meetings alternate between Tuesdays
> 19:00 UTC and Wednesdays 7:00 UTC.
> 
> Because of the whining^W weekly-expressed preferences[1] of the
> Europe-based folks, the latter meetings are going to be moved by +1 hour.
> 
> So the new meeting times are:
> 
> * Tuesdays at 19:00 UTC (unchanged)
> * Wednesdays at 8:00 UTC (1 hour later)
> 
> The first new EU-friendly meeting will take place on Wednesday 17th
> September.
> 
> The wiki page has been updated accordingly:
> 
> https://wiki.openstack.org/wiki/Meetings/TripleO
> 
> but I don't know how to reflect the change in the iCal feed. Anyone
> willing to do that, please?

Time updated. For some reason I don't get notified on that page change
anymore. Sigh.

Also:
http://lists.openstack.org/pipermail/openstack-infra/2013-December/000517.html

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] New meeting rotation starting next week

2014-09-04 Thread Thierry Carrez
Kevin Benton wrote:
> How is the master list compiled into a calendar? Is it possible to use
> that same system to filter by project?

It's manual. I susbscribe to the wikipage and reflect the change in the
Google Cal. It's painful and error-prone. If anyone wants to do it, I'm
happy to give the keys and delegate the responsibility. But frankly,
what we really need is this:

http://lists.openstack.org/pipermail/openstack-infra/2013-December/000517.html

There was a group of students working on it:

http://git.openstack.org/cgit/openstack-infra/gerrit-powered-agenda/

Lance, any news on that? Should we reboot the project?

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers

2014-09-04 Thread Thierry Carrez
Like I mentioned before, I think the only way out of the Nova death
spiral is to split code and give control over it to smaller dedicated
review teams. This is one way to do it. Thanks Dan for pulling this
together :)

A couple comments inline:

Daniel P. Berrange wrote:
> [...]
> This is a crisis. A large crisis. In fact, if you got a moment, it's
> a twelve-storey crisis with a magnificent entrance hall, carpeting
> throughout, 24-hour portage, and an enormous sign on the roof,
> saying 'This Is a Large Crisis'. A large crisis requires a large
> plan.
> [...]

I totally agree. We need a plan now, because we can't go through another
cycle without a solution in sight.

> [...]
> This has quite a few implications for the way development would
> operate.
> 
>  - The Nova core team at least, would be voluntarily giving up a big
>amount of responsibility over the evolution of virt drivers. Due
>to human nature, people are not good at giving up power, so this
>may be painful to swallow. Realistically current nova core are
>not experts in most of the virt drivers to start with, and more
>important we clearly do not have sufficient time to do a good job
>of review with everything submitted. Much of the current need
>for core review of virt drivers is to prevent the mis-use of a
>poorly defined virt driver API...which can be mitigated - See
>later point(s)
> 
>  - Nova core would/should not have automatic +2 over the virt driver
>repositories since it is unreasonable to assume they have the
>suitable domain knowledge for all virt drivers out there. People
>would of course be able to be members of multiple core teams. For
>example John G would naturally be nova-core and nova-xen-core. I
>would aim for nova-core and nova-libvirt-core, and so on. I do not
>want any +2 responsibility over VMWare/HyperV/Docker drivers since
>they're not my area of expertize - I only look at them today because
>they have no other nova-core representation.
> 
>  - Not sure if it implies the Nova PTL would be solely focused on
>Nova common. eg would there continue to be one PTL over all virt
>driver implementation projects, or would each project have its
>own PTL. Maybe this is irrelevant if a Czars approach is chosen
>by virt driver projects for their work. I'd be inclined to say
>that a single PTL should stay as a figurehead to represent all
>the virt driver projects, acting as a point of contact to ensure
>we keep communication / co-operation between the drivers in sync.
> [...]

At this point it may look like our current structure (programs, one PTL,
single core teams...) prevents us from implementing that solution. I
just want to say that in OpenStack, organizational structure reflects
how we work, not the other way around. If we need to reorganize
"official" project structure to work in smarter and long-term healthy
ways, that's a really small price to pay.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Thierry Carrez
Sean Dague wrote:
> [...]
> So, honestly, I'll probably remain -1 on the final integration vote, not
> because Zaqar is bad, but because I'm feeling more firmly that for
> OpenStack to not leave the small deployers behind we need to redefine
> the tightly integrated piece of OpenStack to basically the Layer 1 & 2
> parts of my diagram, and consider the rest of the layers exciting parts
> of our ecosystem that more advanced users may choose to deploy to meet
> their needs. Smaller tent, big ecosystem, easier on ramp.
> 
> I realize that largely means Zaqar would be caught up in a definition
> discussion outside of it's control, and that's kind of unfortunate, as
> Flavio and team have been doing a bang up job of late. But we need to
> stop considering "integration" as the end game of all interesting
> software in the OpenStack ecosystem, and I think it's better to have
> that conversation sooner rather than later.

I think it's pretty clear at this point that:

(1) we need to have a discussion about layers (base nucleus, optional
extra services at the very least) and the level of support we grant to
each -- the current binary approach is not working very well

(2) If we accept Zaqar next week, it's pretty clear it would not fall in
the base nucleus layer but more in an optional extra services layer,
together with at the very least Trove and Sahara

There are two ways of doing this: follow Sean's approach and -1
integration (and have zaqar apply to that "optional layer" when we
create it), or +1 integration now (and have zaqar follow whichever other
integrated projects we place in that layer when we create it).

I'm still hesitating on the best approach. I think they yield the same
end result, but the -1 approach seems to be a bit more unfair, since it
would be purely for reasons we don't (yet) apply to currently-integrated
projects...

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Design Summit reloaded

2014-09-04 Thread Thierry Carrez
Eoghan Glynn wrote:
> [...]
> Am I missing some compelling advantage of moving all these emergent
> project-specific meetups to the Friday?

One is that due to space limitations, we won't have nearly as many
"pods" as in Atlanta (more like half or a third of them). Without one
pod per program, the model breaks a bit.

The Friday setup also allows for more room (rather than a small
roundtable) since we can reuse regular rooms (and maybe split them up).

It appears on the schedule as a specific set of hours where contributors
on a given program gather, so it's easier to reach critical mass.

Finally for projects like Nova, which had regular sessions all the days.
I also like having them all on the last day so that you can react to
previous sessions and discuss useful stuff.

If that makes you feel more comfortable, you can think of it as a
pod-only day, with a bit more publicity, larger pods and where we use
all the summit space available for pods :)

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Design Summit reloaded

2014-09-05 Thread Thierry Carrez
Eoghan Glynn wrote:
>>> Am I missing some compelling advantage of moving all these emergent
>>> project-specific meetups to the Friday?
>>
>> One is that due to space limitations, we won't have nearly as many
>> "pods" as in Atlanta (more like half or a third of them). Without one
>> pod per program, the model breaks a bit.
> 
> A-ha, OK.
> 
> Will the subset of projects allocated a pod be fixed, or will the
> pod space float between projects as the week progresses?
> 
> (for example, it's unlikely that a project will be using its pod
> space when its design session track is in-progress, so the pod could
> be passed on to another project)

We'll have to design some novel pod-switching algorithm, but I kinda
want to know how many pods we can have before we start designing. I'm
visiting the space again on Monday.

Regards,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Brainstorming summit sessions

2014-09-05 Thread Thierry Carrez
Michael Still wrote:
> On Thu, Sep 4, 2014 at 11:40 AM, Steve Gordon  wrote:
> 
>> Did you have a specific goal/date in mind for when you might start to 
>> finalize this list? I am guessing at least after the dust settles on J-3 and 
>> possibly even the first RCs but just curious.
> 
> Good question. Looking at the release calendar, I think we need this
> done by mid-October, but I'm not sure when ttx wants the schedule for
> the summit done by. So we have at least a few weeks, but I'll be more
> concrete when I know more details of summit scheduling.

We have until the week after release to finalize the summit schedule, so
plenty of time. Also there will be Kilo PTLs elections in the middle of
this (end of September), and the Kilo themes are actually a Kilo PTL thing.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Thierry Carrez
Tim Bell wrote:
>> -Original Message-
>> From: Thierry Carrez [mailto:thie...@openstack.org]
>> Sent: 04 September 2014 16:59
>> To: openstack-dev@lists.openstack.org
>> Subject: Re: [openstack-dev] [Zaqar] Comments on the concerns arose during
>> the TC meeting
>>
>> Sean Dague wrote:
>>> [...]
>>> So, honestly, I'll probably remain -1 on the final integration vote,
>>> not because Zaqar is bad, but because I'm feeling more firmly that for
>>> OpenStack to not leave the small deployers behind we need to redefine
>>> the tightly integrated piece of OpenStack to basically the Layer 1 & 2
>>> parts of my diagram, and consider the rest of the layers exciting
>>> parts of our ecosystem that more advanced users may choose to deploy
>>> to meet their needs. Smaller tent, big ecosystem, easier on ramp.
>>>
>>> I realize that largely means Zaqar would be caught up in a definition
>>> discussion outside of it's control, and that's kind of unfortunate, as
>>> Flavio and team have been doing a bang up job of late. But we need to
>>> stop considering "integration" as the end game of all interesting
>>> software in the OpenStack ecosystem, and I think it's better to have
>>> that conversation sooner rather than later.
>>
>> I think it's pretty clear at this point that:
>>
>> (1) we need to have a discussion about layers (base nucleus, optional extra
>> services at the very least) and the level of support we grant to each -- the
>> current binary approach is not working very well
>>
>> (2) If we accept Zaqar next week, it's pretty clear it would not fall in the 
>> base
>> nucleus layer but more in an optional extra services layer, together with at 
>> the
>> very least Trove and Sahara
>>
>> There are two ways of doing this: follow Sean's approach and -1 integration
>> (and have zaqar apply to that "optional layer" when we create it), or +1
>> integration now (and have zaqar follow whichever other integrated projects we
>> place in that layer when we create it).
>>
>> I'm still hesitating on the best approach. I think they yield the same end 
>> result,
>> but the -1 approach seems to be a bit more unfair, since it would be purely 
>> for
>> reasons we don't (yet) apply to currently-integrated projects...
>>
> 
> The one concern I have with a small core is that there is not an easy way to 
> assess the maturity of a project on stackforge. The stackforge projects may 
> be missing packaging, Red Hat testing, puppet modules, install/admin 
> documentation etc. Thus, I need to have some indication that a project is 
> deployable before looking at it with my user community to see if it meets a 
> need that is sustainable.
> 
> Do you see the "optional layer" services being blessed / validated in some 
> way and therefore being easy to identify ?

Yes, I think whatever exact shape this takes, it should convey some
assertion of stability to be able to distinguish itself from random
projects. Some way of saying "this is good and mature, even if it's not
in the inner circle".

Being in The "integrated release" has been seen as a sign of stability
forever, while it was only ensuring "integration" with other projects
and OpenStack processes. We are getting better at requiring "maturity"
there, but if we set up layers, we'll have to get even better at that.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Feature freeze + Juno-3 milestone candidates available

2014-09-05 Thread Thierry Carrez
Hi everyone,

We just hit feature freeze[1], so please do not approve changes that add
features or new configuration options unless those have been granted a
feature freeze exception.

This is also string freeze[2], so you should avoid changing translatable
strings. If you have to modify a translatable string, you should give a
heads-up to the I18N team.

Finally, this is also DepFreeze[3], so you should avoid adding new
dependencies (bumping oslo or openstack client libraries is OK until
RC1). If you have a new dependency to add, raise a thread on
openstack-dev about it.

The juno-3 development milestone was tagged, it contains more than 135
features and 760 bugfixes added since the juno-2 milestone 6 weeks ago
(not even counting the Oslo libraries in the mix). You can find the full
list of new features and fixed bugs, as well as tarball downloads, at:

https://launchpad.net/keystone/juno/juno-3
https://launchpad.net/glance/juno/juno-3
https://launchpad.net/nova/juno/juno-3
https://launchpad.net/horizon/juno/juno-3
https://launchpad.net/neutron/juno/juno-3
https://launchpad.net/cinder/juno/juno-3
https://launchpad.net/ceilometer/juno/juno-3
https://launchpad.net/heat/juno/juno-3
https://launchpad.net/trove/juno/juno-3
https://launchpad.net/sahara/juno/juno-3

Many thanks to all the PTLs and release management liaisons who made us
reach this important milestone in the Juno development cycle. Thanks in
particular to John Garbutt, who keeps on doing an amazing job at the
impossible task of keeping the Nova ship straight in troubled waters
while we head toward the Juno release port.

Regards,

[1] https://wiki.openstack.org/wiki/FeatureFreeze
[2] https://wiki.openstack.org/wiki/StringFreeze
[3] https://wiki.openstack.org/wiki/DepFreeze

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers

2014-09-05 Thread Thierry Carrez
Daniel P. Berrange wrote:
> For a long time I've use the LKML 'subsystem maintainers' model as the
> reference point for ideas. In a more LKML like model, each virt team
> (or other subsystem team) would have their own separate GIT repo with
> a complete Nova codebase, where they did they day to day code submissions,
> reviews and merges. Periodically the primary subsystem maintainer would
> submit a large pull / merge requests to the overall Nova maintainer.
> The $1,000,000 question in such a model is what kind of code review
> happens during the big pull requests to integrate subsystem trees. 

Please note that the Kernel subsystem model is actually a trust tree
based on 20 years of trust building. OpenStack is only 4 years old, so
it's difficult to apply the same model as-is.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Feature Freeze Exception process for Juno

2014-09-05 Thread Thierry Carrez
Michael Still wrote:
> We're soon to hit feature freeze, as discussed in Thierry's recent
> email. I'd like to outline the process for requesting a freeze
> exception:
> 
> * your code must already be up for review
> * your blueprint must have an approved spec
> * you need three (3) sponsoring cores for an exception to be granted
> * exceptions must be granted before midnight, Friday this week
> (September 5) UTC
> * the exception is valid until midnight Friday next week
> (September 12) UTC when all exceptions expire
> 
> For reference, our rc1 drops on approximately 25 September, so the
> exception period needs to be short to maximise stabilization time.
> 
> John Garbutt and I will both be granting exceptions, to maximise our
> timezone coverage. We will grant exceptions as they come in and gather
> the required number of cores, although I have also carved some time
> out in the nova IRC meeting this week for people to discuss specific
> exception requests.

I'd like to add that every exception approved adds up to create moving
parts at a moment where we want to slow down to let QA and Docs and
other downstream stakeholders catch up.

Obviously, things that are already approved and working their way
through the gate should be in early enough to limit this disruption. But
in general, targeting more than 25% of your juno-3 velocity to -rc1 is a
bit unreasonable. For Nova, that means that more than 7 exceptions is
starting to be a stability issue.

Please keep that in mind every time you go to support a FFE.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [glance][feature freeze exception] Proposal for using Launcher/ProcessLauncher for launching services

2014-09-05 Thread Thierry Carrez
Kekane, Abhishek wrote:
> [...]
> If we have this feature in glance then we can able to use features like
> reload glance configuration file without restart, graceful shutdown etc.
> 
> Also it will use common code like other OpenStack projects nova,
> keystone, cinder does.

I think it makes a lot of sense but it also has a lot of documentation
consequences. If it was ready to merge and had the necessary reviews
piled up I would +1 this but the way it stands now (and given Glance's
current review velocity) I'm more leaning towards -1.

Cheers,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Glance][FFE] glance_store switch-over and random access to image data

2014-09-05 Thread Thierry Carrez
Flavio Percoco wrote:
> Greetings,
> 
> I'd like to request a FFE for 2 features I've been working on during
> Juno which, unfortunately, haven been delayed for different reasons
> during this time.
> [...]

I would be inclined to give both a chance, but they really need to merge
quickly, and the current Glance review velocity is not exactly feeding
my hopes. +0 as far as I'm concerned, and definitely -1 if it takes more
than one week.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Sahara][FFE] Requesting exception for Swift trust authentication blueprint

2014-09-05 Thread Thierry Carrez
Smaller review teams don't really need to line up core sponsors as much
as Nova does. As long as Sergey and myself are fine with it, you can go
for it. I'm +1 on this one becauise it's actually a security bug we need
to plug before release.

Trevor McKay wrote:
> Not sure how this is done, but I'm a core member for Sahara, and I
> hereby sponsor it.
> 
> On Fri, 2014-09-05 at 09:57 -0400, Michael McCune wrote:
>> hey folks,
>>
>> I am requesting an exception for the Swift trust authentication 
>> blueprint[1]. This blueprint addresses a security bug in Sahara and 
>> represents a significant move towards increased security for Sahara 
>> clusters. There are several reviews underway[2] with 1 or 2 more starting 
>> today or monday.
>>
>> This feature is initially implemented as optional and as such will have 
>> minimal impact on current user deployments. By default it is disabled and 
>> requires no additional configuration or management from the end user.
>>
>> My feeling is that there has been vigorous debate and discussion surrounding 
>> the implementation of this blueprint and there is consensus among the team 
>> that these changes are needed. The code reviews for the bulk of the work 
>> have been positive thus far and I have confidence these patches will be 
>> accepted within the next week.
>>
>> thanks for considering this exception,
>> mike
>>
>>
>> [1]: 
>> https://blueprints.launchpad.net/sahara/+spec/edp-swift-trust-authentication
>> [2]: 
>> https://review.openstack.org/#/q/status:open+topic:bp/edp-swift-trust-authentication,n,z
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] List of granted FFEs

2014-09-06 Thread Thierry Carrez
Michael Still wrote:
> I've built this handy dandy list of granted FFEs, because searching
> email to find out what is approved is horrible. It would be good if
> people with approved FFEs could check their thing is listed here:
> 
> https://etherpad.openstack.org/p/juno-nova-approved-ffes

It would also be great if the contents of that list matched the
Launchpad milestone page we use to track progress on them:

https://launchpad.net/nova/+milestone/juno-rc1

Cheers,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Feature freeze + Juno-3 milestone candidates available

2014-09-06 Thread Thierry Carrez
In that precise case, given how early it is in the freeze, I think
giving a quick heads-up to the -i18n team/list should be enough :) Also
/adding/ a string is not as disruptive to their work as modifying a
potentially-already-translated one.

Joe Cropper wrote:
> +1 to what Jay said.
> 
> I’m not sure whether the string freeze applies to bugs, but the defect
> that Matt mentioned (for which I authored the fix) adds a string, albeit
> to fix a bug.  Hoping it’s more desirable to have an untranslated
> correct message than a translated incorrect message.  :-)
> 
> - Joe
> On Sep 5, 2014, at 3:41 PM, Jay Bryant  <mailto:jsbry...@electronicjungle.net>> wrote:
> 
>> Matt,
>>
>> I don't think that is the right solution.
>>
>> If the string changes I think the only problem is it won't be
>> translated if it is thrown.   That is better than breaking the coding
>> standard imho.
>>
>> Jay
>>
>> On Sep 5, 2014 3:30 PM, "Matt Riedemann" > <mailto:mrie...@linux.vnet.ibm.com>> wrote:
>>
>>
>>
>> On 9/5/2014 5:10 AM, Thierry Carrez wrote:
>>
>> Hi everyone,
>>
>> We just hit feature freeze[1], so please do not approve
>> changes that add
>> features or new configuration options unless those have been
>> granted a
>> feature freeze exception.
>>
>> This is also string freeze[2], so you should avoid changing
>> translatable
>> strings. If you have to modify a translatable string, you
>> should give a
>> heads-up to the I18N team.
>>
>> Finally, this is also DepFreeze[3], so you should avoid adding new
>> dependencies (bumping oslo or openstack client libraries is OK
>> until
>> RC1). If you have a new dependency to add, raise a thread on
>> openstack-dev about it.
>>
>> The juno-3 development milestone was tagged, it contains more
>> than 135
>> features and 760 bugfixes added since the juno-2 milestone 6
>> weeks ago
>> (not even counting the Oslo libraries in the mix). You can
>> find the full
>> list of new features and fixed bugs, as well as tarball
>> downloads, at:
>>
>> https://launchpad.net/__keystone/juno/juno-3
>> <https://launchpad.net/keystone/juno/juno-3>
>> https://launchpad.net/glance/__juno/juno-3
>> <https://launchpad.net/glance/juno/juno-3>
>> https://launchpad.net/nova/__juno/juno-3
>> <https://launchpad.net/nova/juno/juno-3>
>> https://launchpad.net/horizon/__juno/juno-3
>> <https://launchpad.net/horizon/juno/juno-3>
>> https://launchpad.net/neutron/__juno/juno-3
>> <https://launchpad.net/neutron/juno/juno-3>
>> https://launchpad.net/cinder/__juno/juno-3
>> <https://launchpad.net/cinder/juno/juno-3>
>> https://launchpad.net/__ceilometer/juno/juno-3
>> <https://launchpad.net/ceilometer/juno/juno-3>
>> https://launchpad.net/heat/__juno/juno-3
>> <https://launchpad.net/heat/juno/juno-3>
>> https://launchpad.net/trove/__juno/juno-3
>> <https://launchpad.net/trove/juno/juno-3>
>> https://launchpad.net/sahara/__juno/juno-3
>> <https://launchpad.net/sahara/juno/juno-3>
>>
>> Many thanks to all the PTLs and release management liaisons
>> who made us
>> reach this important milestone in the Juno development cycle.
>> Thanks in
>> particular to John Garbutt, who keeps on doing an amazing job
>> at the
>> impossible task of keeping the Nova ship straight in troubled
>> waters
>> while we head toward the Juno release port.
>>
>> Regards,
>>
>> [1] https://wiki.openstack.org/__wiki/FeatureFreeze
>> <https://wiki.openstack.org/wiki/FeatureFreeze>
>> [2] https://wiki.openstack.org/__wiki/StringFreeze
>> <https://wiki.openstack.org/wiki/StringFreeze>
>> [3] https://wiki.openstack.org/__wiki/DepFreeze
>> <https://wiki.openstack.org/wiki/DepFreeze>
>>
>>
>> I should probably know this, but at least I'm asking first. :)
>>
>> Here is an example of a new translatable user-facing error message
>> [1].
>>
>> From the StringFreeze wiki,

Re: [openstack-dev] [neutron] non-deterministic gate failures due to unclosed eventlet Timeouts

2014-09-08 Thread Thierry Carrez
John Schwarz wrote:
> Long story short: for future reference, if you initialize an eventlet
> Timeout, make sure you close it (either with a context manager or simply
> timeout.close()), and be extra-careful when writing tests using
> eventlet Timeouts, because these timeouts don't implicitly expire and
> will cause unexpected behaviours (see [1]) like gate failures. In our
> case this caused non-deterministic failures on the dsvm-functional test
> suite.

Nice catch, John!

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [cinder] new cinderclient release this week?

2014-09-08 Thread Thierry Carrez
John Griffith wrote:
> Yes, now that RC1 is tagged I'm planning to tag a new cinderclient
> tomorrow.  I'll be sure to send an announcement out as soon as it's up.

You mean, now that juno-3 is tagged ;) RC1 is still a few weeks and a
few dozens bugfixes away.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Designate][Horizon][Tempest][DevStack] Supporting code for incubated projects

2014-09-09 Thread Thierry Carrez
Mac Innes, Kiall wrote:
> While requesting a openstack/designate-dashboard project from the TC/Infra
> – The topic of why Designate panels, as an incubated project, can’t
> be merged into openstack/horizon was raised.
> 
> In the openstack/governance review[1], Russell asked:
> 
>> Hm, I think we should discuss this with the horizon team, then. We are
>> telling projects that incubation is a key time for integrating with
>> other
>> projects. I would expect merging horizon integration into horizon itself
>> to be a part of that.

We are actually telling projects that they should work on their Horizon
panels while in incubation, and use their first "integrated" cycle (once
they graduate, before their first release), to get their panels into
Horizon mainline code.

That's what Sahara did over this cycle (they had a dashboard, they got
it merged in Horizon during juno, in time for final Juno release).

Now it's not a perfect setup: it put a lot of stress between Sahara and
Horizon teams -- it was essential for Sahara to get it merged, while no
horizon-core really signed up to review it. It took a bit of
cross-project coordination to get it in in time... I expect the same to
happen again.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Kilo Cycle Goals Exercise

2014-09-10 Thread Thierry Carrez
Joe Gordon wrote:
> To that end, I would like to propose an exercise as discussed in the TC
> meeting yesterday [1]:
> Have anyone interested (especially TC members) come up with a list of
> what they think the project wide Kilo cycle goals should be and post
> them on this thread by end of day Wednesday, September 10th. After which
> time we can begin discussing the results.
> The goal of this exercise is to help us see if our individual world
> views align with the greater community, and to get the ball rolling on a
> larger discussion of where as a project we should be focusing more time.

Thanks again to Joe for starting this important discussion.

My personal list of Kilo goals goes as follows:

#1: Fix our growth pains

We grew a lot, and our recipes were designed for a smaller group where
trust happens more naturally. With our community growing to a Dunbar
order of magnitude above, some of those recipes don't work so great, so
we need to revisit them... That includes the binary "integrated release"
(introduce layers?), cross-project functional testing (push it at
project level?), the "programs" concept, encouraging PTL delegation (the
czar/liaisons proposal ?), scaling core reviewing in larger projects
(Nova driver split ?), etc.

We already started the conversation on those important topics, but Kilo
is the timeframe for us to implement those changes, because I don't see
our community wait for more than one cycle to see the light at the end
of the tunnel.

#2: Fix the end user experience

Monty expressed it better than I can. We need consistent and
better-designed APIs, client SDKs that provide useful primitives and
actually abstract differences in implementations, etc.

#3: Fix what we have: bugfixes, consistency, scaling up, scaling down,
upgrading

Rather than adding more features for the mid-size private cloud, let's
make sure that what we have works well, provides a consistent experience
across projects, scales up beautifully, can be easily used at smaller
scale as well (simplicity), and allows seamless upgrades. This is
another way of looking at "paying our technical debt" that others have
mentioned as goals -- generally polishing what we have rather than
building out new things.


Regards,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Zaqar graduation (round 2) [was: Comments on the concerns arose during the TC meeting]

2014-09-10 Thread Thierry Carrez
Flavio Percoco wrote:
> [...]
> Based on the feedback from the meeting[3], the current main concern is:
> 
> - Do we need a messaging service with a feature-set akin to SQS+SNS?
> [...]

I think we do need, as Samuel puts it, "some sort of durable
message-broker/queue-server thing". It's a basic application building
block. Some claim it's THE basic application building block, more useful
than database provisioning. It's definitely a layer above pure IaaS, so
if we end up splitting OpenStack into layers this clearly won't be in
the inner one. But I think "IaaS+" basic application building blocks
belong in OpenStack one way or another. That's the reason I supported
Designate ("everyone needs DNS") and Trove ("everyone needs DBs").

With that said, I think yesterday there was a concern that Zaqar might
not fill the "some sort of durable message-broker/queue-server thing"
role well. The argument goes something like: if it was a queue-server
then it should actually be built on top of Rabbit; if it was a
message-broker it should be built on top of postfix/dovecot; the current
architecture is only justified because it's something in between, so
it's broken.

I guess I don't mind that much zaqar being "something in between":
unless I misunderstood, exposing extra primitives doesn't prevent the
"queue-server" use case from being filled. Even considering the
message-broker case, I'm also not convinced building it on top of
postfix/dovecot would be a net win compared to building it on top of
Redis, to be honest.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] webnumbr bug graphs appear dead for bug triage

2014-09-10 Thread Thierry Carrez
Sean Dague wrote:
> All the various bug triage graphs point out to webnumbr urls from our
> wiki - https://wiki.openstack.org/wiki/BugTriage
> 
> All of webnumbr appears to be dead and not returning any data.
> 
> Why this service was used predates me. Does anyone know why? Anyone know
> if it's going to come back? Or should we just purge those links?

It used to be a convenient way to graph bug stats from launchpad over
time. But I think the service died. So yes, links can probably be purged
now.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][neutron][cinder] Averting the Nova crisis by splitting out virt drivers

2014-09-11 Thread Thierry Carrez
Sean Dague wrote:
> [...]
> Why don't we start with "let's clean up the virt interface and make it
> more sane", as I don't think there is any disagreement there. If it's
> going to take a cycle, it's going to take a cycle anyway (it will
> probably take 2 cycles, realistically, we always underestimate these
> things, remember when no-db-compute was going to be 1 cycle?). I don't
> see the need to actually decide here and now that the split is clearly
> at least 7 - 12 months away. A lot happens in the intervening time.

Yes, that sounds like the logical next step. We can't split drivers
without first doing that anyway. I still think "people need smaller
areas of work", as Vish eloquently put it. I still hope that refactoring
our test architecture will let us reach the same level of quality with
only a fraction of the tests being run at the gate, which should address
most of the harm you see in adding additional repositories. But I agree
there is little point in discussing splitting virt drivers (or anything
else, really) until the internal interface below that potential split is
fully cleaned up and it becomes an option.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Cinder] Request for J3 FFE - add reset-state function for backups

2014-09-12 Thread Thierry Carrez
Jay Bryant wrote:
> It isn't a huge change.   I am ok with it if we can get the issues
> addressed.   Especially Duncan's concern.

Given the gate backlog, if it's not already in-flight, I fear that it
would push too much down into the stabilization period and delay RC1.

At this point, unless it's critical to the success of the release (like,
it completes a feature that is 99% there, or it increases consistency by
plugging a feature gap, or it fixes a potential security vulnerability),
I would rather avoid adding exceptions. Could you explain why adding
reset-state function for backups absolutely needs to be in Juno ? Feels
like a nice-to-have to me, and I fear we are past that point now.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Zaqar graduation (round 2) [was: Comments on the concerns arose during the TC meeting]

2014-09-12 Thread Thierry Carrez
Clint Byrum wrote:
> Excerpts from Flavio Percoco's message of 2014-09-11 04:14:30 -0700:
>> Is Zaqar being optimized as a *queuing* service? I'd say no. Our goal is
>> to optimize Zaqar for delivering messages and supporting different
>> messaging patterns.
> 
> Awesome! Just please don't expect people to get excited about it for
> the lighter weight queueing workloads that you've claimed as use cases.
> 
> I totally see Horizon using it to keep events for users. I see Heat
> using it for stack events as well. I would bet that Trove would benefit
> from being able to communicate messages to users.
> 
> But I think in between Zaqar and the backends will likely be a lighter
> weight queue-only service that the users can just subscribe to when they
> don't want an inbox. And I think that lighter weight queue service is
> far more important for OpenStack than the full blown random access
> inbox.
> 
> I think the reason such a thing has not appeared is because we were all
> sort of running into "but Zaqar is already incubated". Now that we've
> fleshed out the difference, I think those of us that need a lightweight
> multi-tenant queue service should add it to OpenStack.  Separately. I hope
> that doesn't offend you and the rest of the excellent Zaqar developers. It
> is just a different thing.
> 
>> Should we remove all the semantics that allow people to use Zaqar as a
>> queue service? I don't think so either. Again, the semantics are there
>> because Zaqar is using them to do its job. Whether other folks may/may
>> not use Zaqar as a queue service is out of our control.
>>
>> This doesn't mean the project is broken.
> 
> No, definitely not broken. It just isn't actually necessary for many of
> the stated use cases.

Clint,

If I read you correctly, you're basically saying the Zaqar is overkill
for a lot of people who only want a multi-tenant queue service. It's
doing A+B. Why does that prevent people who only need A from using it ?

Is it that it's actually not doing A well, from a user perspective ?
Like the performance sucks, or it's missing a key primitive ?

Is it that it's unnecessarily complex to deploy, from a deployer
perspective, and that something only doing A would be simpler, while
covering most of the use cases?

Is it something else ?

I want to make sure I understand your objection. In the "user
perspective" it might make sense to pursue both options as separate
projects. In the "deployer perspective" case, having a project doing A+B
and a project doing A doesn't solve anything. So this affects the
decision we have to take next Tuesday...

Cheers,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-12 Thread Thierry Carrez
Flavio Percoco wrote:
> On 09/12/2014 12:14 AM, Zane Bitter wrote:
>> The final question is the one of arbitrary access to messages in the
>> queue (or "queue" if you prefer). Flavio indicated that this effectively
>> came for free with their implementation of Pub-Sub. IMHO it is
>> unnecessary and limits the choice of potential back ends in the future.
>> I would personally be +1 on removing it from the v2 API, and also +1 on
>> the v2 API shipping in Kilo so that as few new adopters as possible get
>> stuck with the limited choices of back-end. I hope that would resolve
>> Clint's concerns that we need a separate, light-weight queue system; I
>> personally don't believe we need two projects, even though I agree that
>> all of the use cases I personally care about could probably be satisfied
>> without Pub-Sub.
> 
> Right, being able to support other backends is one of the reasons we're
> looking forward to remove the support for arbitrary access to messages.
> As of now, the plan is to remove that endpoint unless a very good use
> case comes up that makes supporting other backends not worth it, which I
> doubt. The feedback from Zaqar's early adopters is that the endpoint is
> indeed not useful.

Thanks Zane, that was indeed useful. I agree with you it would be better
to avoid needing 2 separate projects for such close use cases.

Let's assume we remove arbitrary access to messages in v2. When you say
it would remove limits on the choice of potential backends, does that
mean we could have a pure queue backend (like RabbitMQ), at least in
theory ? Would a ZaqarV2 address all of Clint and Devananda's concerns
about queue semantics ? If yes, then the graduation question becomes,
how likely is that work to be completed early enough in Kilo.

If it's a no-brainer and takes a week to sort out, I think we could
approve Zaqar's Kilo graduation, even if that stretches the "no major
API rewrite planned" requirement.

But if we think this needs careful discussion so that the v2 API design
(and backend support) satisfies the widest set of users, then incubating
for another cycle while v2 is implemented seems like the right course of
action. We shouldn't graduate if there is any risk we would end up with
ZaqarV1 in Kilo, and then have to deprecate it for n cycles just because
it was shipped in the official release and therefore inherits its API
deprecation rules.

Regards,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Cinder] Request for J3 FFE - add reset-state function for backups

2014-09-12 Thread Thierry Carrez
Duncan Thomas wrote:
> On 12 September 2014 09:54, Thierry Carrez  wrote:
>> At this point, unless it's critical to the success of the release (like,
>> it completes a feature that is 99% there, or it increases consistency by
>> plugging a feature gap, or it fixes a potential security vulnerability),
>> I would rather avoid adding exceptions. Could you explain why adding
>> reset-state function for backups absolutely needs to be in Juno ? Feels
>> like a nice-to-have to me, and I fear we are past that point now.
> 
> 1. It is 99% done, we've been reviewing the patch and fixing niggles
> for a while now
> 
> 2. We have equivalent features for volumes and snapshots (the other
> two entities in cinder with state) and they are heavily used in
> production
> 
> 3. The alternative is getting admins to go editing the DB directly
> (which is what we do now) and the logic for doing so is extremely hard
> to get right
> 
> I'm a strong supporter of this feature, and I just gave the patch its first +2

OK, it feels like a good consistency/usability thing to get in-release
rather than past-release. If it can get all the +2s required today (and
John's approval), I won't object to it.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [all] Design Summit planning

2014-09-12 Thread Thierry Carrez
Hi everyone,

I visited the Paris Design Summit space on Monday and confirmed that it
should be possible to split it in a way that would allow to have
per-program "contributors meetups" on the Friday. The schedule would go
as follows:

Tuesday: cross-project workshops
Wednesday, Thursday: traditional "scheduled" slots
Friday: contributors meetups

We'll also have "pods" available all 4 days for more ad-hoc small meetings.

In the mean time, we need to discuss how we want to handle the selection
of session topics.

In past summits we used a Design-Summit-specific "session suggestion"
website, and PTLs would approve/deny them. This setup grew less and less
useful: session topics were selected collaboratively on etherpads,
discussed in meetings, and finally filed/reorganized/merged on the
website just before scheduling. Furthermore, with even less "scheduled"
slots, we would have to reject most of the suggestions, which is more
frustrating for submitters than the positive experience of joining team
meetings to discuss which topics are the most important. Finally, topics
will need to be split between "scheduled" sessions and the "contributors
meetup" agenda, and that's easier to do on an Etherpad anyway.

This is why I'd like to suggest that all programs use etherpads to
collect important topics, select which ones would get in the very few
"scheduled" slots we'll have left, which will get discussed in the
"contributors meetup", and which are better left for a "pod" discussion.
I suggest we all use IRC team meetings to collaboratively discuss that
content between interested contributors.

To simplify the communication around this, I tried to collect the
already-announced etherpads on a single page at:

https://wiki.openstack.org/wiki/Summit/Planning

Please add any that I missed !

If you think this is wrong and think the "design summit suggestion"
website is a better way to do it, let me know why! If some programs
really can't stand the 'etherpad/IRC' approach I'll see how we can spin
up a limited instance.

Regards,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack-dev][Cinder] FFE request for adding Huawei SDSHypervisor driver and connector

2014-09-12 Thread Thierry Carrez
Zhangni wrote:
> I'd like to request an Juno feature freeze exception for this blueprint
> and Spec:
> 
> https://blueprints.launchpad.net/cinder/+spec/huawei-sdshypervisor-driver
> 
> https://review.openstack.org/#/c/101688/
> 
> as implemented by the following patch:
> 
> https://review.openstack.org/#/c/108609

I would say it's way too late at this point for a new driver in Juno. At
this point we should be focused on fixing what we already have, not add
more surface.

Cheers,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Design Summit planning

2014-09-12 Thread Thierry Carrez
Eoghan Glynn wrote:
>> If you think this is wrong and think the "design summit suggestion"
>> website is a better way to do it, let me know why! If some programs
>> really can't stand the 'etherpad/IRC' approach I'll see how we can spin
>> up a limited instance.
> 
> +1 on a collaborative scheduling process within each project.
> 
> That's pretty much what we did within the ceilometer core group for
> the Juno summit, except that we used a googledocs spreadsheet instead
> of an etherpad.
> 
> So I don't think we need to necessarily mandate usage of an etherpad,
> just let every project decide whatever shared document format they
> want to use.
> 
> FTR the benefit of a googledocs spreadsheet in my view would include
> the ease of totalling votes & sessions slots, color-coding candidate
> sessions for merging etc.

Good point. I've replaced the wording in the wiki page -- just use
whatever suits you best, as long as it's a public document and you can
link to it.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Design Summit planning

2014-09-12 Thread Thierry Carrez
Russell Bryant wrote:
> On 09/12/2014 07:37 AM, Thierry Carrez wrote:
>> If you think this is wrong and think the "design summit suggestion"
>> website is a better way to do it, let me know why! If some programs
>> really can't stand the 'etherpad/IRC' approach I'll see how we can spin
>> up a limited instance.
> 
> I think this is fine, especially if it's a better reflection of reality
> and lets the teams work more efficiently.
> 
> However, one of the benefits of the old submission system was the
> clarity of the process and openness to submissions from anyone.  We
> don't want to be in a situation where non-core folks feel like they have
> a harder time submitting a session.
> 
> Once this is settled, as long as the wiki pages [1][2] reflect the
> process and is publicized, it should be fine.
> 
> [1] https://wiki.openstack.org/wiki/Summit
> [2] https://wiki.openstack.org/wiki/Summit/Planning

Yes, I'll document the new process and heavily publicize it, once I'm
sure that's the way forward :)

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Design Summit planning

2014-09-12 Thread Thierry Carrez
Anita Kuno wrote:
> My question involves third party discussions. Now I know at least
> Neutron is going to have a chat about drivers which involves third party
> ci accounts as a supportive aspect of that discussion, but I am
> wondering about the framework for a discussion which I can recommend
> attendees of the third party meetings to attend. They are shaping up to
> be an attentive, forward thinking group and are supporting each other
> which I was hoping for from the beginning so I am very heartened by our
> progress. I am feeling that as a group folks have questions and concerns
> they would appreciate the opportunity to air in a mutually constructive
> venue.
> 
> What day and where would be the mutually constructive venue?
> 
> I held off on Joe's thread since third party ci affects 4 or 5 programs,
> not enough to qualify in my mind as a topic that is OpenStack wide, but
> the programs it affects are quite affected, so I do feel it is time to
> mention it.

I think those discussions could happen in a cross-project workshop.
We'll run 2 or 3 of those in parallel all day Tuesday, so there is
definitely room there.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Cinder] Request for J3 FFE - NetApp: storage pools for scheduler

2014-09-15 Thread Thierry Carrez
Mike Perez wrote:
> On 14:24 Fri 05 Sep , Alex Meade wrote:
>> Hi Cinder Folks,
>>
>> I would like to request a FFE for cinder pools support with the NetApp
>> drivers[1][2].
> 
> Looks like this is being reviewed now.

Looks like it merged, so I retroactively added it to RC1.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][neutron][cinder] Averting the Nova crisis by splitting out virt drivers

2014-09-15 Thread Thierry Carrez
Chris Friesen wrote:
> On 09/12/2014 04:59 PM, Joe Gordon wrote:
>> [...]
>> Can't you replace the word 'libvirt code' with 'nova code' and this
>> would still be true? Do you think landing virt driver code is harder
>> then landing non virt driver code? If so do you have any numbers to back
>> this up?
>>
>> If the issue here is 'landing code in nova is too painful', then we
>> should discuss solving that more generalized issue first, and maybe we
>> conclude that pulling out the virt drivers gets us the most bang for our
>> buck.  But unless we have that more general discussion, saying the right
>> fix for that is to spend a large amount of time  working specifically on
>> virt driver related issues seems premature.
> 
> I agree that this is a nova issue in general, though I suspect that the
> virt drivers have quite separate developer communities so maybe they
> feel the pain more clearly.  But I think the solution is the same in
> both cases:
> 
> 1) Allow people to be responsible for a subset of the nova code
> (scheduler, virt, conductor, compute, or even just a single driver).
> They would have significant responsibility for that area of the code.
> This would serve several purposes--people with deep domain-specific
> knowledge would be able to review code that touches that domain, and it
> would free up the nova core team to look at the higher-level picture.
> For changes that cross domains, the people from the relevant domains
> would need to be involved.
> 
> 2) Modify the gate tests such that changes that are wholly contained
> within a single area of code are not blocked by gate-blocking-bugs in
> unrelated areas of the code.

I agree... Landing code in Nova is generally too painful, but the pain
is most apparent in areas which require specific domain expertise (like
a virt driver, where not so many -core are familiar enough with the
domain to review, while the code proposer generally is).

IMHO, like I said before, the solution to making Nova (or any other
project, actually) more fluid is to create separate and smaller areas of
expertise, and allow new people to step up and own things. Splitting
virt drivers (once the driver interface is cleaned up) is just one way
of doing it -- that just seems like a natural separation line to use if
we do split. But that would just be a first step: as more internal
interfaces are cleaned up we could (and should) split more. Smaller
groups responsible for smaller areas of code is the way to go.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] Ipset, merge refactor for J?

2014-09-16 Thread Thierry Carrez
Miguel Angel Ajo Pelayo wrote:
> During the ipset implementatio, we designed a refactor [1] to cleanup 
> the firewall driver a bit, and move all the ipset low-level knowledge 
> down into the  IpsetManager.
> 
> I'd like to see this merged for J, and, it's a bit of an urgent matter 
> to decide, because we keep adding small changes [2] [3] fruit of the
> early testing which break the refactor, and will add extra work which
> needs to be refactored too.
> 
> The advantage of merging now, vs in J, is having K & J share a more common 
> code base, which would help us during bug backports/etc in the future.
> 
> Shihanzhang and I, are happy to see this merge during K, as it doesn't 
> incur in functional changes, just code blocks are moved from the iptables
> firewall driver to IpsetManager, and the corresponding tests are moved too.
> [...]

IMHO code refactoring should be considered a superfluous change at this
point in the cycle. The risk/benefit ratio is too high, and focus should
be on bugfixing at this point.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] What's holding nova development back?

2014-09-16 Thread Thierry Carrez
Michael Still wrote:
> Yes, that was my point. I don't mind us debating how to rearrange
> hypervisor drivers. However, if we think that will solve all our
> problems we are confused.
> 
> So, how do we get people to start taking bugs / gate failures more seriously?

I think we need to build a cross-project team working on that. Having
gate liaisons designated in every project should help bootstrap that
team -- it doesn't mean it's a one-person-per-project job, but at least
you have a contact person when you need an expert in some project that
is also versed in the arts of the gate.

I also think we need to do a slightly better job at visualizing issues.
Like Dims said, even with tabs opened to the right places, it's
non-trivial to determine which is the killer bug from which isn't. And
without carefully checking IRC backlog in 4 different channels, it's
also hard to find out that a bug is already taken care of. I woke up one
morning with gate being obviously stuck on some issue, investigated it,
only to realize after 30 minutes that the fix was already in the gate
queue. That's a bit of a frustrating experience.

Finally, it's not completely crazy to use a specific channel
(#openstack-gate ?) for that. Yes, there is a lot of overlap with -qa
and -infra channels, but those channels aren't dedicated to that
problem, so 25% of the issues are discussed on one, 25% on the other,
25% on the project-specific channel, and the remaining 25% on some
random channel the right people happen to be in. Having a clear channel
where all the gate liaisons hang out and all issues are discussed may go
a long way into establishing a team to work on that (rather than
continue to rely on the same set of willing individuals).

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Design Summit planning

2014-09-18 Thread Thierry Carrez
Maish Saidel-Keesing wrote:
> On 17/09/2014 23:12, Anita Kuno wrote:
>> On 09/17/2014 04:01 PM, Maish Saidel-Keesing wrote:
>>> This looks great - but I am afraid that something might be missing.
>>>
>>> As part of the Design summit in Atlanta there was an Ops Meetup track.
>>> [1] I do not see where this fits into the current planning process that
>>> has been posted.
>>> I would like to assume that part of the purpose of the summit is to also
>>> collect feedback from Enterprise Operators and also from smaller ones as
>>> well.
>>>
>>> If that is so then I would kindly request that there be some other way
>>> of allowing that part of the community to voice their concerns, and
>>> provide feedback.
>>>
>>> Perhaps a track that is not only Operator centric - but also an End-user
>>> focused one as well (mixing the two would be fine as well)
>>>
>>> Most of them are not on the openstack-dev list and they do not
>>> participate in the IRC team meetings, simply because they have no idea
>>> that these exist or maybe do not feel comfortable there. So they will
>>> not have any exposure to the process.
>>>
>>> My 0.02 Shekels.
>>>
>>> [1] - http://junodesignsummit.sched.org/overview/type/ops+meetup
>>>
>> Hi Maish:
>>
>> This thread is about the Design Summit, the Operators Track is a
>> different thing.
>>
>> In Atlanta the Operators Track was organized by Tom Fifield and I have
>> every confidence he is working hard to ensure the operators have a voice
>> in Paris and that those interested can participate.
>>
>> Last summit the Operators Track ran on the Monday and the Friday giving
>> folks who usually spend most of the time at the Design summit to
>> participate and hear the operator's voices. I know I did and I found it
>> highly educational.
>>
>> Thanks,
>> Anita.
> Thanks for the clarification Anita :)

I think the plan is to have the Ops summit run on Monday, with an extra
day on Thursday to continue the discussions. I CCed Tom for more input
on that.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [requirements][horizon] Dependency freeze exceptions

2014-09-19 Thread Thierry Carrez
Radomir Dopieralski wrote:
> I would like to request dependency freeze exceptions for the following
> patches for Horizon:
> 
> https://review.openstack.org/#/c/121509/
> https://review.openstack.org/#/c/122410/
> 
> and
> 
> https://review.openstack.org/#/c/113184/
> 
> They are all fixing high priority bugs. The first two are related to
> bugs with parsing Bootstrap 3.2.0 scss files that have been fixed
> upstream. The third one makes the life of packagers a little eaiser,
> by using versions that both Debian and Fedora, and possibly many
> other distros, can ship.
> 
> I am not sure what the formal process for such an exception should be,
> so I'm writing here. Please let me know if I should have done it
> differently.

No, that's the right way of doing it.

These look like valid requests, but this is mostly affecting the
distributions, so let's wait for the packagers to chime in on how much
of a disruption that would cause to them at this point.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Oslo final releases ready

2014-09-19 Thread Thierry Carrez
Thomas Goirand wrote:
> However, I made a small mistake. I used 1.4.0.0~a5 instead of 1.4.0~a5.
> As a consequence, I may upload 1.4.0.0 instead of 1.4.0, so that it is
> greater than 1.4.0.0~a5 (to avoid adding an EPOC, which is ugly and
> annoying to maintain). It's going to be the same version though, I just
> need to add a new tag, which isn't much of a problem for me (since I use
> a git based workflow).
> 
> The 1.4.0.0a5 is confusing... :(

According to Donald Stufft PEP 440 now allows 1.4.0.a5 now so we may not
need that extra 0 anymore.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Zaqar and SQS Properties of Distributed Queues

2014-09-19 Thread Thierry Carrez
Joe Gordon wrote:
> On Thu, Sep 18, 2014 at 9:02 AM, Devananda van der Veen
> mailto:devananda@gmail.com>> wrote:
>> - guaranteed message order
>> - not distributing work across a configurable number of back ends
>> 
>> These are scale-limiting design choices which are reflected in the
>> API's characteristics.
> 
> I agree with Clint and Devananda

The underlying question being... can Zaqar evolve to ultimately reach
the massive scale use case Joe, Clint and Devananda want it to reach, or
are those design choices so deeply rooted in the code and architecture
that Zaqar won't naturally mutate to support that use case.

The Zaqar team has shown great willingness to adapt in order to support
more use cases, but I guess there may be architectural design choices
that would just mean starting over ?

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model

2014-09-19 Thread Thierry Carrez
one before
end of cycle)
- the "Client libraries" model (release as-needed)

If possible, I would like to avoid the "Swift" model, which is the most
costly from a release management standpoint. All projects following the
ring 0 model are easy to keep track of, using common freezes etc. So
it's easy to make sure they will be ready in time for the coordinated
release. Each project following the Swift model would be a special case,
and that adds up to a lot of load on the release management team.
Keeping track of one project doing that is OK. 5 or 20, not so much. So
I'd advise we only keep ring0, Oslo and client lib models as options.
Release management would just care about ring 0, and provide tools and
advice for all the others, but not being responsible for them.

What about the development cycle ? I think it's part of our "identity":
being "OpenStack" is also about having the same notion of passing time,
the same reference points. So I think it would probably be a good idea,
even for projects that purely release "as-needed", to organize their
development at least vaguely around the notion of a common cycle.

## The design summit

I'm not a big fan of your #11 suggestion. I guess I just don't want to
defer all in-project alignment and work to mid-cycle meetups, and force
all contributors to travel all the time. I think the right time to reach
alignment on the goals is at the start of a development cycle, not in
the middle of it. The middle of a cycle is a good time to get things
done. So I would still like teams to be able to discuss their goals at
the Design Summit, at the start of a cycle.

Now I realize this creates a pretty significant issue, since Design
Summit space is one resource which can't really support the "infinite
tent" model. My proposal to solve it would be to give ample space to
cross-project issues, ring 0 projects, and 'theme' programs. Everyone
else gets limited space, which may boil down to a pod. Then we work on
options, so that whoever who wants to organize a large meetup in
parallel with the summit may find space to do so in neighboring hotels.

The main value in that proposal is that it makes the "Design Summit"
space needs more predictable. I've been visiting spaces for the 2016 and
2018 summits recently -- it's really difficult to make plans when you
don't know how many projects you'll have to host. And it's really
difficult to get good locations and times (and prices) if you don't book
at least 2 years in advance.

## The Foundation bylaws

There are a number of terms used in the bylaws, and we may need to map
our new structure to it, make sure it doesn't require a bylaws change.
I didn't spot any showstopper yet. The "integrated release" could be the
ring 0 (that would mean that defcore would only get to pick between
Nova/Neutron/Cinder/Keystone/Designate for the trademark programs). The
"OpenStack project" could be the big tent. We keep the TC, the ATCs
concepts the same.


That was a long answer, but then so was your original post :P

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [publiccloud-wg] Serving vendor json from RFC 5785 well-known dir

2018-11-05 Thread Thierry Carrez

Monty Taylor wrote:

[...]
What if we added support for serving vendor data files from the root of 
a primary URL as-per RFC 5785. Specifically, support deployers adding a 
json file to .well-known/openstack/client that would contain what we 
currently store in the openstacksdk repo and were just discussing 
splitting out.

[...]
What do people think?


I love the idea of public clouds serving that file directly, and the 
user experience you get from it. The only two drawbacks on top of my 
head would be:


- it's harder to discover available compliant openstack clouds from the 
client.


- there is no vetting process, so there may be failures with weird 
clouds serving half-baked files that people may blame the client tooling 
for.


I still think it's a good idea, as in theory it aligns the incentive of 
maintaining the file with the most interested stakeholder. It just might 
need some extra communication to work seamlessly.


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Asking for suggestion of video conference tool for team and webinar

2018-11-07 Thread Thierry Carrez

Trinh Nguyen wrote:
I tried several free tools for team meetings, vPTG, and webinars but 
there are always drawbacks ( because it's free, of course). For example:


  * Google Hangout: only allow a maximum of 10 people to do the video calls
  * Zoom: limits about 45m per meeting. So for a webinar or conference
call takes more than 45m we have to splits into 2 sessions or so.

If anyone in the community can suggest some better video conferencing 
tools, that would be great. So we can organize better communication for 
our team and the community's webinars.


Jitsi meet is an open source + free as in beer solution based on WebRTC:

https://meet.jit.si/

No account needed, no participant limit.

--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] We're combining the lists!

2018-11-10 Thread Thierry Carrez
Robert Collins wrote:
> There don't seem to be any topics defined for -discuss yet, I hope
> there will be, as I'm certainly not in a position of enough bandwidth
> to handle everything *stack related.
> 
> I'd suggest one for each previously list, at minimum.

As we are ultimately planning to move lists to mailman3 (which decided
to drop the "topics" concept altogether), I don't think we planned to
add serverside mailman topics to the new list.

We'll still have standardized subject line topics. The current list
lives at:

https://etherpad.openstack.org/p/common-openstack-ml-topics

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] PTL election

2013-11-19 Thread Thierry Carrez
Thierry Carrez wrote:
> Thierry Carrez wrote:
>> The two candidates who nominated themselves in time for this election are:
>>
>> * David Lyle
>> * Matthias Runge
>>
>> The election will be set up tomorrow, and will stay open for voting for
>> a week.

The poll is now closed, and the winner is David Lyle !

You can see results at:

http://www.cs.cornell.edu/w8/~andru/cgi-perl/civs/results.pl?id=E_cd16dd051e519ef2

Congrats, David!

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] When is a blueprint unnecessary?

2013-11-20 Thread Thierry Carrez
Russell Bryant wrote:
> One of the bits of feedback that came from the "Nova Project Structure
> and Process" session at the design summit was that it would be nice to
> skip having blueprints for smaller items.
> 
> In an effort to capture this, I updated the blueprint review criteria
> [1] with the following:
> 
>   Some blueprints are closed as unnecessary. Blueprints are used for
>   tracking significant development efforts. In general, small and/or
>   straight forward cleanups do not need blueprints. A blueprint should
>   be filed if:
> 
>- it impacts the release notes
>- it covers significant internal development or refactoring efforts
> [...]

While I agree we should not *require* blueprints for minor
features/efforts, should we actively prevent people from filing them (or
close them if they are filed ?)

Personally (I know I'm odd) I like to have my work (usually small stuff)
covered by a blueprint so that I can track and communicate its current
completion status -- helps me keep track of where I am.

So the question is... is there a cost associated with tolerating "small"
blueprints ? Once they are set to "Low" priority they mostly disappear
from release management tracking so it's not really a burden there.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Search Project - summit follow up

2013-11-20 Thread Thierry Carrez
Dmitri Zimin(e) | StackStorm wrote:
> Hi Stackers,
> 
> The project Search is a service providing fast full-text search for
> resources across OpenStack services.
> [...]

At first glance this looks slightly scary from a security / tenant
isolation perspective. Most search results would be extremely
user-specific (and leaking data from one user to another would be
catastrophic), so the benefits of indexing (vs. querying DB) would be
very limited ?

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [UX] Automatically post new threads from AskBot to the list

2013-11-20 Thread Thierry Carrez
Julie Pichon wrote:
> Hi folks,
> 
> I've been thinking about the AskBot UX website [0] and its lack of
> visibility, particularly for new community members.
> 
> I think it would be valuable to have the first post from new
> conversation threads be posted to the -dev list with the appropriate
> [UX] tag, with a link to AskBot and a request to continue the
> conversation over there. This would help newcomers realising there is a
> space for UX conversations, and existing community members to pop by
> when a topic comes up around an area of interest. The traffic has been
> low so far, and I wouldn't expect it to become more overwhelming than
> any of the current projects currently communicating via the developers
> list.
> 
> Would there be any concern or objections?

Frankly, automatically duplicating information (or cross-posting) should
never be the solution.

I'd rather have an edited, weekly summary of UX discussions posted to
-dev: that would be both useful to people who do not follow that site
*and* regularly remember the existence of that site for UX-minded folks.

Yes, it's more work, but instead of something I'd ignore (by setting up
yet another filter), it would be something I would actually enjoy reading.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Propose "project story wiki" idea

2013-11-20 Thread Thierry Carrez
Boris Pavlovic wrote:
> The idea of this proposal is that every OpenStack project should have
> "story" wiki page. It means to publish every week one short message that
> contains most interesting updates for the last week, and high level road
> map for future week. So reading this for 10-15 minutes you can see what
> changed in project, and get better understanding of high level road map
> of the project.
> [...]

I like the idea, can be very short updates, I don't think it should be
automated (and it doesn't have to be every week if there is nothing to say).

Ideally we would have a single forum for all of those, rather than have
to fish for each appropriate wiki page. If everyone posted to planet.o.o
that would be a start... Some other aggregator or site (like the one
Flavio suggested) could also be used.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] FOSDEM 2014 devroom CFP

2013-11-20 Thread Thierry Carrez
Only 10 days left -- and there aren't that many OpenStack talks
submitted yet.

If you plan to be around and you have a topic to discuss that may be
interesting for a developers audience, please consider submitting a talk !

Thierry Carrez wrote:
> There will be a two-day "Virtualisation and IaaS" devroom at FOSDEM 2014
> (Brussels, February 1-2). See below for the CFP.
> 
> Note: For this edition we'll avoid high-level, generic project
> presentations and give priority to deep dives and developer-oriented
> content, so please take that into account before submitting anything.
> 
> --
> Call for Participation
> --
> 
> The scope for this devroom is open source, openly-developed projects in
> the areas of virtualisation and IaaS type clouds, ranging from low level
> to data center, up to cloud management platforms and cloud resource
> orchestration.
> 
> Sessions should always target a developer audience. Bonus points for
> collaborative sessions that would be appealing to developers from
> multiple projects.
> 
> We are particularly interested in the following themes:
> * low level virtualisation aspects
> * new features in classic and container-based virtualisation technologies
> * new use cases for virtualisation, such as virtualisation in mobile,
> automotive and embedded in general
> * other resource virtualisation technologies: networking, storage, …
> * deep technical dives into specific IaaS or virtualisation management
> projects features
> * relationship between IaaS projects and specific dependencies (not just
> virtualisation)
> * integration and development leveraging solutions from multiple projects
> 
> 
> Important dates
> ---
> 
> Submission deadline: Sunday, December 1st, 2013
> Acceptance notifications: Sunday, December 15th, 2013
> Final schedule announcement: Friday January 10th, 2014
> Devroom @ FOSDEM'14: February 1st & 2nd, 2014
> 
> 
> Practical
> -
> 
> Submissions should be 40 minutes, and consist of a 30 minute
> presentation with 10 minutes of Q&A or 40 minutes of discussions (e.g.,
> requests for feedback, open discussions, etc.). Interactivity is
> encouraged, but optional. Talks are in English only.
> 
> We do not provide travel assistance or reimbursement of travel expenses
> for accepted speakers.
> 
> Submissions should be made via the FOSDEM submission page at
> https://penta.fosdem.org/submission/FOSDEM14 :
> 
> * If necessary, create a Pentabarf account and activate it
> * In the “Person” section, provide First name, Last name (in the
> “General” tab), Email (in the “Contact” tab) and Bio (“Abstract” field
> in the “Description” tab)
> * Submit a proposal by clicking on “Create event"
> * Important! Select the "Virtualisation and IaaS" track (on the
> “General” tab)
> * Provide the title of your talk (“Event title” in the “General” tab)
> * Provide a 250-word description of the subject of the talk and the
> intended audience (in the “Abstract” field of the “Description” tab)
> * Provide a rough outline of the talk or goals of the session (a short
> list of bullet points covering topics that will be discussed) in the
> “Full description” field in the “Description” tab
> 
> 
> Contact
> ---
> 
> For questions w.r.t. the Virtualisation and IaaS DevRoom at FOSDEM'14,
> please contact the organizers via
> fosdem14-virt-and-iaas-devr...@googlegroups.com (or via
> https://groups.google.com/forum/#!forum/fosdem14-virt-and-iaas-devroom).
> 
> 
> This CFP is also visible at:
> https://groups.google.com/forum/#!topic/fosdem14-virt-and-iaas-devroom/04y5YkyqzIo
> 


-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


  1   2   3   4   5   6   7   8   9   10   >