Hi Al, Thanks for the explanation, I am OK if this is the group consensus. Roni
> -----Original Message----- > From: Al Morton [mailto:[email protected]] > Sent: Friday, August 19, 2011 10:04 PM > To: Roni Even; draft-ietf-bmwg-igp-dataplane-conv- > [email protected] > Cc: [email protected] > Subject: RE: Gen-ART LC review of draft-ietf-bmwg-igp-dataplane-conv- > meth-23 > > At 06:12 AM 8/19/2011, Roni Even wrote: > >I had another question whether there is some experience from actually > >conducting the tests based on the long time this draft was under > >development. > > Roni, > > There were several labs conducting these tests before > the drafts were prepared, in fact, the state of the art > for convergence testing advanced while these drafts were in > development - which is one reason we had difficulty in the > first IESG review. So there is plenty of test experience. > > You actually asked if we could provide guidance on how many > times to repeat tests, but the WG has declined to do this. > I'm sure you understand that with a wide range of equipment > and testing circumstances only general guidance can be given, > such as: > > section 5.6 > ... It is RECOMMENDED to repeat a test multiple times with > different random ranges of the header fields such that convergence > time benchmarks are measured for different distributions of traffic > over the available paths. > > and > 5.8. Measurement Statistics > > The benchmark measurements may vary for each trial, due to the > statistical nature of timer expirations, cpu scheduling, etc. > Evaluation of the test data must be done with an understanding of > generally accepted testing practices regarding repeatability, > variance and statistical significance of a small number of trials. > > hope this helps, > Al > _______________________________________________ Gen-art mailing list [email protected] https://www.ietf.org/mailman/listinfo/gen-art
