On Thu, Aug 28, 2014 at 5:17 AM, Thierry Carrez <thie...@openstack.org> wrote:
> David Kranz wrote: > > On 08/27/2014 03:43 PM, Sean Dague wrote: > >> On 08/27/2014 03:33 PM, David Kranz wrote: > >>> Race conditions are what makes debugging very hard. I think we are in > >>> the process of experimenting with such an idea: asymetric gating by > >>> moving functional tests to projects, making them deeper and more > >>> extensive, and gating against their own projects. The result should be > >>> that when a code change is made, we will spend much more time running > >>> tests of code that is most likely to be growing a race bug from the > >>> change. Of course there is a risk that we will impair integration > >>> testing and we will have to be vigilant about that. One mitigating > >>> factor is that if cross-project interaction uses apis (official or not) > >>> that are well tested by the functional tests, there is less risk that a > >>> bug will only show up only when those apis are used by another project. > >> > >> So, sorry, this is really not about systemic changes (we're running > >> those in parallel), but more about skills transfer in people getting > >> engaged. Because we need both. I guess that's the danger of breaking the > >> thread is apparently I lost part of the context. > >> > > I agree we need both. I made the comment because if we can make gate > > debugging less daunting > > then less skill will be needed and I think that is key. Honestly, I am > > not sure the full skill you have can be transferred. It was gained > > partly through learning in simpler times. > > I think we could develop tools and visualizations that would help the > debugging tasks. We could make those tasks more visible, and therefore > more appealing to the brave souls that step up to tackle them. Sean and > Joe did a ton of work improving the raw data, deriving graphs from it, > highlighting log syntax or adding helpful Apache footers. But those days > they spend so much time fixing the issues themselves, they can't > continue on improving those tools. > Some tooling improvements I would like to do but probably don't have the time for: * Add the ability to filter http://status.openstack.org/elastic-recheck/ by project. So a neutron dev can see the list of bugs that are neutron related * Make the list of open reviews on http://status.openstack.org/elastic-recheck/ easier to find * Create an up to date diagram of what OpenStack looks like when running, how services interact etc. http://docs.openstack.org/training-guides/content/figures/5/figures/image31.jpg and http://docs.openstack.org/admin-guide-cloud/content/figures/2/figures/openstack-arch-havana-logical-v1.jpg are out of date * Make http://jogo.github.io/gate easier to understand. This is what I check to see the health of the gate. * Build a request-id tracker for logs. Make it easier to find the logs for a given request-id across multiple services (nova-api,nova-scheduler etc.) > > And that's part of where the gate burnout comes from: spending so much > time on the issues themselves that you can no longer work on preventing > them from happening, or making the job of handling the issues easier, or > documenting/mentoring other people so that they can do it in your place. > > -- > Thierry Carrez (ttx) > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
