Am 21.05.2014 18:38, schrieb Michael Greenwald:
> On 2014-05-20 15:19, Detlef Bosau wrote:
>> Am 19.05.2014 21:36, schrieb Bob Braden:
>>> Detlef,
>>>
>>> Yes, any network researcher who wants to call him/herself a computer
>>> scientist should take seriously the experimentalist's task of fully
>>> understanding the assumptions and implementations of their test
>>> environment. That includes NS-2 simulations of TCP.
>>
>> Myself, I have to admit that I trusted too much in the NS-2. And quite
>> some papers I've read rely heavily on the NS-2.
>
> NS-2 is a simulator. As such it is, at best, an approximation of a
> real network --- including approximate implementation of various
> protocols.
No discussion about that.
> One of the first rules about using simulation results is that you
> must always validate your results in the real system. (Not necessarily
> *every* result, but compare enough runs to know when the simulation
> becomes
> inaccurate). I am surprised to hear you say (or at least imply) that a
> reasonable fraction of people studying networks believe that NS-2 is or
> was "truth". (In fact, I am skeptical of this claim.)
One of the oldest questions of mankind, I think it is even mentioned in
the bible, is "What is truth?"
However, the particular problem is that at least myself learned quite a
lot about TCP from the NS-2 code.
Hence, perhaps the NS-2 may have some influence on practical protocol
implemenation.
Even that is no problem.
But it IS a problem, when RFC and some implementations, one of which is
the NS-2, diverge. Especially when this is not said appropriately.
>
>>
>> That doesn't mean, simulations were worthless. But I think, we should
>> treat them with a certain professional distance.
>
> I think it is already the case that people treat simulation results
> with "a certain professional distance."
At least I know several colleagues who do so.
Unfortunately, I know some colleagues as well who mix up simulations
with reality.
> Simulators *model* the real world. Our abstract models (whether when
> making back of the envelope estimates, or simulating real phenomena)
> commonly trade off accuracy for ease of modeling (whether the "ease"
> is because of performance, obtaining closed form solutions, or
> generality,
> or...). I think it is/was well understood that NS-2 simplified and
> abstracted things, and was therefore inaccurate. The question is always
> whether the accuracy is good enough to support any claims you make based
> on the simulation results.
>
> I completely agree that researchers need to experiment, analyze, and
> understand
> TCP and other network protocols more carefully; I just don't think
> people were
> as confused by NS-2 (or other simulations) as much as you seem to think.
>
Admittedly, I was extremely surprised when I noticed this very point
with the RTO ;-)
It is my fault.
However, I still don't know for sure, whether VJs original code for
TCP/Tahoe did GBN or not....
--
------------------------------------------------------------------
Detlef Bosau
Galileistraße 30
70565 Stuttgart Tel.: +49 711 5208031
mobile: +49 172 6819937
skype: detlef.bosau
ICQ: 566129673
detlef.bo...@web.de http://www.detlef-bosau.de
