-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thanks guys.
joel On 11/3/14 3:30 PM, Vijay K. Gurbani wrote: > On Oct 2, 2014 Tom Taylor wrote: > >> Summary: Ready with minor issues and nits. I did not check out >> the correctness of the code in Appendix A. > > Tom: Thank you for your detailed review and your time on our > draft. > > Please see inline for resolutions. > >> Minor issues: ------------ >> >> (1) Sec. 4.10 paragraphs 3 and 4: I believe these are about >> setting the initial value for the attempt rate. The first >> paragraph offers the alternative of using a vendor-provided value >> R as a starting point or using r=R as a starting point. Doesn't >> seem like two alternatives. >> >> The second paragraph ignores all this and sets r=100 as the >> starting point. But then it repeats text about using a >> vendor-supplied value. >> >> The two paragraphs need cleaning up. > > This is a very good comment. Here is what I propose: > > OLD: > > If the DUT vendor provides a value for R, the tester can use this > value. Alternatively, in cases where the DUT vendor does not > provide a value for R, or in cases where the tester wants to > ascertain a vendor provided value using local media > characteristics, the algorithm could be run by setting "r = R" and > observing the value at convergence. > > The algorithm proceeds by initializing "r = 100"; "r" is the > session attempt rate. The algorithm dynamically increases and > decreases "r" as it converges to the a maximum sps value for R. The > dynamic increase and decrease rate is controlled by the weights "w" > and "d", respectively. If the DUT vendor provides a value for R, > the tester can use that value; however, because the requirements > and media characteristics are a function of the test environment, > it is best that the tester reflect these requirements during > testing and allow the algorithm to converge to R. > > NEW: > > If the DUT vendor provides a value for R, the tester can use this > value to initialize "r = R", and proceed with the algorithm. In > cases where the DUT vendor does not provide a value for R, or in > cases where the tester wants to ascertain a vendor provided value > using local media characteristics, it is best that the tester > reflect these requirements during testing and allow the algorithm > to converge to R. In such cases, the algorithm proceeds by > initializing "r = 100"; "r" is the session attempt rate. The > algorithm dynamically increases and decreases "r" as it converges > to the a maximum sps value for R. The dynamic increase and decrease > rate is controlled by the weights "w" and "d", respectively. > >> (2) The algorithm in 4.10 seems fairly unsophisticated, and >> amounts to ten iterations of constrained binary search. Was >> application of Newton's Method >> >> http://www.ima.umn.edu/%7Emiller/Newton-MVT.pdf >> >> to give you faster convergence and shorter test runs considered? > > Very pertinent feedback (thanks!). Pursuant to our offline > conversations, I believe that the algorithm as specified in > bmwg-sip- bench-meth-11 does not need to be modified. In the end, > the algorithm in the draft is a version of gradient descent, or in > our case, ascent. Such algorithms may not be fast, but they do find > the maxima (minima) on a bounded number of iterations. > > An important reason why the algorithm in bmwg-sip-bench-meth-11 > appears in the form that is does is for the simple reason that the > tester may not know R a-priori. Recall that R is a SIP session > attempt rate, which the DUT can process with zero errors over a > defined, extended period. > > Our draft and the algorithm takes the conservative approach that > the tester may not be given the value of R by the vendor a-priori, > or the tester may independently want to approximate R to > double-check vendor claim using locally pertinent media > characteristics. > > The binary search method depends on knowing the value of R, since > that will become a ceiling to the process. The Newton (N-R) method > also depends on knowing the value of R, since to get N-R going, we > would start the first run at the claimed steady-state (R) capacity. > I have no doubt that we can make binary search or N-R work such > that R is not known a-priori. But by doing so I suspect that we'd > be close to the gradient descent that we have now. > > Based on the simulation results for a binary search I shared with > you, I suggest adding a sentence in the revision to bench-meth as > follows: > > OLD: The pseudo-code corresponding to the description above > follows, and a simulation written in the R statistical language is > provided in Appendix A. > > NEW: The pseudo-code corresponding to the description above > follows, and a simulation written in the R statistical language is > provided in Appendix A. We note that it is possible to have other > algorithms that converge faster to R than the one shown below > (e.g., an algorithm based on a binary search), however, such > algorithms may not produce an optimal value for R, that is, they > may sacrifice some accuracy for speed. Such algorithms are for > further study. > >> Nits/editorial comments: ----------------------- >> >> IDNits requests that the code in Appendix A be preceded by a >> <CODE BEGINS> line and terminated by a <CODE ENDS> line. >> >> Sec. 3 para 2 last line: s/arrangements/arrangement/ > > Got it. Thanks! > >> Next para: spell out EA as presumably "Emulated Agents" on first >> use?Term is from the terminology document. > > Done. > >> Add <CODE BEGINS> and <CODE ENDS> around the pseudo-code in >> 4.10. >> >> Reference [I-D.sip-bench-term] is now dated July, 2014 (version >> -11). > > Good catch. Updated. > > I will skip the <CODE BEGINS> and <CODE ENDS> nits, if it is okay > with you. > > The results will show up in -12 that will be released shortly. > Thanks a lot, Tom, for your insights and feedback on this work. > > Cheers, > > - vijay -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - http://gpgtools.org iEUEARECAAYFAlRYLBEACgkQ8AA1q7Z/VrLluQCTBGVerQquk2OaoEmu8hyZA6uR 1wCeIwN+edezGlMu1quQ8j8jS2obnEI= =bZ0f -----END PGP SIGNATURE----- _______________________________________________ Gen-art mailing list [email protected] https://www.ietf.org/mailman/listinfo/gen-art
