Hi Akihiro, That would be nice like we are in the process of setting CI test setup and will be useful.
Regards, Balaji.P > -----Original Message----- > From: Akihiro Motoki [mailto:mot...@da.jp.nec.com] > Sent: Monday, February 10, 2014 3:53 PM > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [neutron][ml2] Maintaining support for the > Tail-f NCS mech driver in Icehouse > > I think testing with sandbox repo is really useful during setting up > third party CI. Not so many people know sandbox repo. > When neutron folks were setting up third party CI, we created test > reviews in openstack/neutron, but if we knew it it would be great. > > It is worth documented in "Third Party Testing" web page [1]. > I will add it if nobody adds it. > > [1] http://ci.openstack.org/third_party.html > > Akihiro > > (2014/02/10 12:13), Anita Kuno wrote: > > On 02/06/2014 09:52 AM, Luke Gorrie wrote: > >> Howdy! > >> > >> My name is Luke and I'm helping my friends at Tail-f Systems to > >> support Neutron with their NCS [1] product. This went really smoothly > >> for us on the Havana cycle, but lately we're having a harder time > >> with Icehouse. In particular, our attempt to fulfill the 3rd party > >> testing requirements has caused a lot of frustration for the > >> #openstack-infra team around New Year. So I'm writing now to look for > a good solution. > >> > >> Our goal for Icehouse is to keep our mechanism driver officially > >> supported. The code already works, and has unit tests to keep it > >> working. The risk we want to avoid is going on the dreaded > >> "deprecated" list for some other reason, which would confuse our > >> customers. > >> > >> For background, our mechanism driver is about 150 lines of code. It > >> translates each network/port/subnet API call into a REST/JSON > >> notification to an external system. That external system returns HTTP > >> "200 OK". That's about all. It's a pretty trivial program. > >> > >> In December I sat down with Tobbe Tornqvist and we tried to setup > >> Jenkins 3rd party testing. We created a Vagrantfile that spins up an > >> Ubuntu VM, installs Neutron and NCS, and performs integration tests > >> with them connected together. We hooked this into Jenkins with a > >> service account. > >> > >> This went fine to start with, but after Christmas our tests started > >> failing due to random setup issues with 'tox', and the script started > >> making -1 votes. Those -1's created major headaches for the > >> infrastructure team and others upstream, I am sorry to say, and ended > >> up requiring a messy process of manual cleanup, and a public flogging > >> for us on IRC. Bad feeling all around ... > > > > The part to keep in mind is that random issues crop up all the time. > > For us in -infra they are a weekly event, and some weeks they are a > > daily event. The part that was the most difficult was the lack of > > responsiveness to our efforts to communicate, to get an > > acknowledgement from the maintainers that this account was experiencing > some issues. > > Then to get the issues addressed. In the end we failed and had to > > resort to revoking verify voting permissions for this account. > > > > It has become evident through an IRC conversation with Tobbe Tornqvist > > [0] that the human resources required to stay available to maintain > > this testing system is an issue. > > > > While understanding of the need of different sized companies to make > > decisions to provide value for their customers, in -infra, we can not > > lose sight of the fact that our customers are our developers and we > > need to be responsive to their requirements. > > > > We are moving to be able to merge 200 patches a day with our system, > > by making a commitment to that rate of development as our target, we > > have to ensure any dependencies for testing also are able to move at a > > similar pace or at least be responsive to ours. > > > > Third-party CI [1] service accounts can still place comments on open > > reviews even if they lack the ability to set a verify vote (-1/+1), > > and this can be used to show whether the backend is reliably > > representing the changes it tests in an effort to build community > > trust in its results. Your account, Tail-f, can vote on our sandbox > > repo [2] as a way of testing voting for the system. > > > > Thank you, > > Anita. > > > > [0] > > http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack > > -infra.2014-01-10.log > > timestamp ~ 2014-01-10T14:48:01 > > [1] https://review.openstack.org/#/admin/groups/270,members > > [2] http://git.openstack.org/cgit/openstack-dev/sandbox/ > > > > > >> > >> And that's where we are today. > >> > >> Now, reviewing the situation, I would love to find a solution that > >> doesn't make us deprecated _and_ doesn't require us to be so deeply > >> hooked into the OpenStack Jenkins infrastructure. > >> > >> Could we, for example, write an accurate emulator for the external > >> system so that the MD code can be tested on OpenStack's own servers? > >> That would suit us very well. It seems like a reasonable request to > >> make given the simplicity of our driver code. > >> > >> Hoping for a simple solution... > >> > >> Cheers, > >> -Luke & friends at Tail-f > >> > >> [1] > >> http://blog.ipspace.net/2013/05/tail-f-network-control-system-first.h > >> tml > >> > >> _______________________________________________ > >> OpenStack-dev mailing list > >> OpenStack-dev@lists.openstack.org > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > > > > _______________________________________________ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
