At 06:12 AM 8/19/2011, Roni Even wrote:
I had another question whether there is some experience from actually conducting the tests based on the long time this draft was under development.
Roni, There were several labs conducting these tests before the drafts were prepared, in fact, the state of the art for convergence testing advanced while these drafts were in development - which is one reason we had difficulty in the first IESG review. So there is plenty of test experience. You actually asked if we could provide guidance on how many times to repeat tests, but the WG has declined to do this. I'm sure you understand that with a wide range of equipment and testing circumstances only general guidance can be given, such as: section 5.6 ... It is RECOMMENDED to repeat a test multiple times with different random ranges of the header fields such that convergence time benchmarks are measured for different distributions of traffic over the available paths. and 5.8. Measurement Statistics The benchmark measurements may vary for each trial, due to the statistical nature of timer expirations, cpu scheduling, etc. Evaluation of the test data must be done with an understanding of generally accepted testing practices regarding repeatability, variance and statistical significance of a small number of trials. hope this helps, Al _______________________________________________ Gen-art mailing list [email protected] https://www.ietf.org/mailman/listinfo/gen-art
