Brian, Just some thoughts on collaboration. Most of the contributors are local is definite something advantage we can take. Since it’s a framework, we probably need some initial meetings/phone calls to come up with a robust easy-to-use framework. After that, we can start to implement test cases individually.
Cynthia On 11/4/14, 10:54 AM, "Narayan Balasubramanian (நாராயண் )" <bv.nara...@gmail.com> wrote: >Brian, > >Agree with Josh. You have captured the requirements well. > >AFAICT, most of the contributors are local - LinkedIn, Apple and Yahoo. >Phone call, irc is all good. We can even meet to nail this. > >Thanks, >Narayan > > >On Tue, Nov 4, 2014 at 10:01 AM, Brian Geffon <bri...@apache.org> wrote: > >> Hi All, thanks for your patience I know many people are eager to start >> pooling resources to make this happen. (Thanks Susan for helping with >>notes >> during the summit) >> >> To briefly summarize what was discussed at the summit, we have an >>existing >> framework called TSQA which is based on bash, while this is a nice start >> it's not really what we need. Josh Blatt hacked together a prototype in >> NodeJS, but I think the consensus was that the tooling and existing code >> for Python would result it in being a better language choice, it seemed >>to >> be a more-or-less unanimous agreement (were there any objections to >>using >> Python?). >> >> With the language defined, we outlined the following high level >> requirements: >> - It must be very easy to write simple tests (ie. basic http get -> >>proxy >> -> simple http origin). >> - It must be expressive enough to handle complex test cases (ie >>testing >> an ESI plugin or advanced networking cases such as sending a FIN >>randomly). >> - Such a framework MUST allow for integration testing plugins. >> - We must be able to bootstrap trafficserver with relevant configs and >> plugins. >> - We must have a port manager that is shared between each component of >> the integration test. >> - We would punt on multiple OS support to start in favor of developing >> something more generally useful to start. However, at some point in the >> future it would be nice to have. Thus design decisions shouldn't be >>linux >> specific with the end goal of supporting multiple OSes. >> - This framework will run in our existing CI environment, thus >>existing >> testing frameworks for generating reports suitable for Jenkins is a >>must. >> >> Other issues raised during this session were: >> - Using such a framework for perf testing, most people agreed this >>should >> be considered separately from this testing framework. >> - We should investigate what other proxies are using before >>development >> begins to determine if we can reuse/share components if the goals of our >> framework aligns with an existing framework. >> - The following wiki page exists related to QA: >> https://cwiki.apache.org/confluence/display/TS/Quality+Assurance >> >> Please feel free to respond if I left anything out or if I in anyway >> misstated the discussions at the summit. >> >> Now moving along to the fun part. Several people have expressed >>interest in >> this project, which is awesome, but let's all coordinate. Let's work >> together and make something great that can really benefit the entire >> community. Does anyone have suggestions for how we can coordinate and/or >> deal with task distribution? Obviously Jira should be used for tasks, >>but >> should we do regular IRC check-ins and summarize those discussions on >>the >> mailing list? Ideas? >> >> Brian >>
