Guys, I guess what you're missing is that Bigtop isn't a testing framework for Hadoop. It is stack framework that verifies that components are dealing with each other nicely. Every single stack is different: Bigtop 0.5.0 differs from 0.6.0, and so on. Bigtop - as any other ASF project - has its releases that might or might not be aligned with particular version of Hadoop. Hence, an ethalon stack needs to be defined first and foremost.
Before we even start talking about running it nightly (another question is on what hardware, let's not get there for now) let's understand who will can help with triage'ing test failures? Downstreams, Hadoop or Bigtop? Judging by a number of other emails there's a number of people on this list who care plenty about integration issues. Any volunteers to help with integration testing in the open? > Is this a previously solved problem? Yes. The problem is solved by separating actively developed (aka unstable) release from more mature and less volatile ones. This is what has been concluded upon two days ago in this voting thread http://s.apache.org/MnU Cos On Wed, May 15, 2013 at 04:54PM, Matt Foley wrote: > Roman, what is your model for how test results from Bigtop should feed back > into Hadoop-2 development? > With the understanding that (a) software does have bugs, and (b) you're not > going to get an SLA on community-sponsored software, > what are your ideas for how to close the loop better? > > Would "CI" runs of Bigtop against branch-2 be feasible, as Arun suggests? > How should we accomodate changes in individual components (Hadoop Core, but > others as well) that may require changes in one or more other components? > How does Bigtop keep doing a viable nightly build in that chaotic > environment? > Is this a previously solved problem? > > Thanks, > --Matt > > > On Wed, May 15, 2013 at 4:47 PM, Arun C Murthy <a...@hortonworks.com> wrote: > > > > > On May 15, 2013, at 3:50 PM, Roman Shaposhnik wrote: > > > > > Arun, > > > > > > am I reading yours answer to my binary question correctly? It is a 'no'. > > > > No. > > > > > > > > My reading of your response is that while you appreciate the feedback > > > Bigtop is providing you're not of an opinion that investigating the level > > > of stability of Hadoop wrt. downstream any further than what is currently > > > happening would be a worthy investment of Hadoop's community > > > (or your personal for that matter) time? > > > > Everyone is welcome to contribute in any and all manner. I can't speak for > > everyone. > > It would be useful if Bigtop could run regressions on releases here > > consistently. We've also talked in the past about running Bigtop on > > branch-2, nightly. Is that something you could help with? You'd earn my > > personal gratitude. > > > > thanks, > > Arun > > > > > > > > Thanks, > > > Roman. > > > > > > On Wed, May 15, 2013 at 3:02 PM, Arun C Murthy <a...@hortonworks.com> > > wrote: > > >> Roman, > > >> > > >> Furthermore, before we rush into finding flaws and scaring kids at > > night it would be useful to remember one thing: > > >> Software has *bugs*. We can't block any release till the entire > > universe validates it, in fact they won't validate it if we don't release > > since are at the bottom of the stack. > > >> > > >> Any help prior to the release is welcome; I know people who work for > > the same employer as I do have plans to do further testing after we freeze > > apis via the beta release(s). I hope and pray others can join this effort - > > thanks to everyone who already has. > > >> > > >> Again, freezing APIs and protocols is the primary aim of 2.0.5-beta. > > There are no guarantees it's 100% bug-free, we can never make such > > guarantees anyway. > > >> > > >> If, and when, we find bugs with 2.0.5-beta I'm more than happy to > > quickly turn around and make more releases (2.0.6-beta, 2.0.7-beta). > > Obviously I'll make a call on which bugs are critical - feedback to help me > > decide is, as always, welcome. > > >> I've been clear, many times, that we might need more than one beta > > release to iron out bugs etc. > > >> > > >> None of this should be a surprise - this has happened many, many times > > in the lifetime of this and other projects. 2.0.3-alpha vis-a-vis > > 2.0.4-alpha is the most recent example - it won't be the last. > > >> > > >> So, I hope, concludes this meme. > > >> > > >> thanks, > > >> Arun > > >> > > >> On May 15, 2013, at 2:20 PM, Arun C Murthy wrote: > > >> > > >>> Great summary, thanks Vinod. > > >>> > > >>> On May 15, 2013, at 2:14 PM, Vinod Kumar Vavilapalli wrote: > > >>> > > >>>> > > >>>> Roman, I keep this same argument again and again. Should've refuted > > earlier. > > >>>> > > >>>> Please list down all the issues that BigTop ran into *because of* new > > features. You continue to argue that new features are destabilizing 2.0.*, > > which I don't agree with at all. 2.0.3-alpha was the last time major > > features got merged in, and we found blockers irrespective of those. > > >>>> > > >>>> MAPREDUCE-5240 specifically isn't due to any feature merge. This was > > a bug. I'd say this is a long standing bug in 2.0.x. You sure this passed > > in 2.0.3? Even so, this is mostly broken by another bug-fix and *not* > > because of any feature. > > >>>> > > >>>> I quickly checked other bugs you reported in 2.0.x: > > >>>> - MAPREDUCE-5088 was caused by the fix for HADOOP-9299 which was > > again a long standing issue in 2.0.x > > >>>> - MAPREDUCE-3728 is similar > > >>>> - MAPREDUCE-5117 is similar > > >>>> - MAPREDUCE-4219 was a security related feature request from you. > > >>>> - MAPREDUCE-3916 was because of new proxy-server added. > > >>>> > > >>>> I am not arguing that new features *may* destabilize the branch, but > > you've repeatedly stated this as if that were a fact. > > >>>> > > >>>> Really appreciate the testing done by BigTop, but please don't > > distort the facts. > > >>>> > > >>>> Thanks, > > >>>> +Vinod > > >>>> > > >>>> > > >>>> On May 15, 2013, at 1:29 PM, Roman Shaposhnik wrote: > > >>>> > > >>>>> Please tell me if my expectations are incorrect, but to me the -beta > > would > > >>>>> signify it being a 'safe' target for the downstream components. > > We're still > > >>>>> finding *very* basic and *very* disruptive issues (MAPREDUCE-5240 is > > >>>>> a good example) that essentially mean DOA for downstream that depends > > >>>>> on this functionality. > > >>>>> > > >>>>> Are we comfortable with delivering 2.0.5-beta and later on starting > > >>>>> to discover things like MAPREDUCE-5240 more or less accidentally? > > >>>> > > >>> > > >>> -- > > >>> Arun C. Murthy > > >>> Hortonworks Inc. > > >>> http://hortonworks.com/ > > >>> > > >>> > > >> > > >> -- > > >> Arun C. Murthy > > >> Hortonworks Inc. > > >> http://hortonworks.com/ > > >> > > >> > > > > -- > > Arun C. Murthy > > Hortonworks Inc. > > http://hortonworks.com/ > > > > > >
signature.asc
Description: Digital signature
