Problems with read_eager and Telnet
Hi everybody, I'm totally new to Python but well motivated :-) I'm fooling around with Python in order to interface with FlightGear using a telnet connection. I can do what I had in mind (send some commands and read output from Flightgear using the telnetlib) with a read_until() object to catch every output line I need, but it proved to be very slow (it takes 1/10 of a sec for every read_until(). I tried using the read_eager() object and it's w faster (it does the job in 1/100 of a sec, maybe more, I didn't tested) but it gives me problems, it gets back strange strings, repeated ones, partially broken ones, well ... I don't know what's going on with it. You see, I don't know telnet (the protocol) very good, I'm very new to Python and Python's docs are not very specific about that read_eager(9 stuff. Could someone point me to some more documentation about that? or at least help me in getting a correct idea of what's going on with read_eager()? I'm going on investigating but a help from here would be very appreciated :-) Thanks in advance, Roberto -- http://mail.python.org/mailman/listinfo/python-list
Re: Problems with read_eager and Telnet
> Telnet sends two kinds of data over the same channel (a simple TCP > stream). It sends the bytes you actually see in your terminal and it > sends control commands that do things like turn echo on/off and > negotiate what terminal type to use. Each time telnetlib reads from > the socket it puts the control stuff in one bucket and stores the > plain text in a buffer to return from all the read_* commands. > > read_eager() returns the plain text that has already been read from > the socket. That might be a partial line. It won't try to read from > the socket to get a full line. That's why it is fast, because it > never does I/O. > > -Jack Ok, that's a start (I'm reading RFC 854 in the meanwhile). Still that doesn't help me much (sorry, I know it's me, not you). You mean read_eager() doesn't wait until it gets a complete reading of a line, instead it reads what's on the socket (even if it's to quick and there's till nothing) and let's the python script running anyway, right? Then with the subsequent read_eager() it will read (if there's something more on the socket in the meanwhile) the previous data bits and maybe the new ones too (a new line of data) into a single data chunk. Is that why I get sometimes repeated empty lines followed by many consequent lines all together out of a single read_eager() call? -- http://mail.python.org/mailman/listinfo/python-list
Re: Problems with read_eager and Telnet
Can you point me to a pratical usage example of read_eager()? Maybe that will help me in making all this clear. I'm still very fuzzy about the socket and the processing stuff. I'm still convinced I cannot use read_until() in my project and I'm determined in looking into the read_eager(), maybe that will be of any help if carefully used. -- http://mail.python.org/mailman/listinfo/python-list
Re: Problems with read_eager and Telnet
On 28 Feb, 18:35, Jack Diederich wrote: > On Mon, Feb 28, 2011 at 12:13 PM, Roberto Inzerillo > > wrote: > > Yes. read_eager() will never actually read from the socket, if it has > > >> any data it has already read & processed it will return those. If you > >> call it enough times it will just start returning empty strings > >> because it never asks the socket to read & wait for new data. > > > Can you point me to a pratical usage example of read_eager()? Maybe that > > will help me in making all this clear. I'm still very fuzzy about the socket > > and the processing stuff. > > > I'm still convinced I cannot use read_until() in my project and I'm > > determined in looking into the read_eager(), maybe that will be of any help > > if carefully used. > > If you always want to see the response you have to use read_until(). > All the other read_* methods don't guarantee you will get the text you > want because they don't wait for the other end of the connection to > have actually sent the data. It's a bummer, but if the server hasn't > returned the data there is nothing magic that telnetlib can do to make > it true. Well, you see, I could prefetch some usefull data in a spare cycle, maybe this way I could "fake" almost-realtime ... I don't know, I'm just guessing here. Fact is, in my case, when I do a 100 write/read cycles in a second with telnet.write() followed by telnet.read-eager(), I see I get all those 100 things done properly, that means, both client and server are doing it right, there's no lag on the processing and network sides. But when I use telnet.read_until() I get those feedbacks in the right order (good) but it's way too slow (it's at least 10 times slower, t bad!!! and, in my case, unusable). I'm trying to find a way out, and I know in these cases knowledge is the only way out. I've seen a huge speed improvement using telnet.read_eager(), I want to investigate more. Who knows, maybe I can stitch my code around it's appearant unusability. -- http://mail.python.org/mailman/listinfo/python-list
Re: Problems with read_eager and Telnet
> Given that FlightGear is a graphical flight
> simulatorhttp://www.flightgear.org/https://secure.wikimedia.org/wikipedia/en/wiki/FlightGear
> using a text terminal connection seems a bit odd,
> unless using it just to get/set configuration,
> in which case, speed should hardly seem an issue.
Right, I'm using it that way, I get/set properties changing them in
real time (I whish!).
> I presume you are using read_until(b'\n').
> read_until cannot return any faster than the server sends complete lines
Actually it's read_until('\n') (because it's python 2.6, anyway I know
what you meant with b'\n').
And yes, after today I've come to the conclusion that even read_eager
is of no use to me. I have to settle down for that poor telnet speed,
I can't get more than that :-( In fact, the telnet connection speed is
not the issue, it's the time FGFS spends in reading its internals and
answearing back to the telnet client.
I've made some tests, it looks like it doesn't matter how many fgfs's
properties I try and read with telnet at the same time with python, I
always have the same lag, I can even run multiple python instances of
the same script, in order to get/set those same amount of properties
in parallel, well ... result is I always get the same lag, it doesn't
even become slower.
In my test I run a simple loop, in each loop I do a telnet.write() and
a telnet.read_until('\n'). I get back results at 20Hz (that means I
get 120 write/read cycles in 6 seconds).
I repeated the same test doing three telnet.write() followed by three
telnet.read_until('\n') in each loop. Result is always 20Hz (but now
that means I get 120 loop cycles in 6 seconds, which is now 120x3
write/read cycles in 6 seconds).
I even run three instances of the same script at the same time (trying
to saturate fgfs's capabilities and find some limits). No luck (or
yes?!?), I always get a 20hz frequency.
My conclusion being, fgfs cannot answer back quicker than this: 20Hz.
This is quite poor for a quasi-realtime hardware interface (which is
in my intention).
Maybe it's enough for something ... I'll give it a try anyway and find
out what can be a usable scenario for such values.
I guees even telnet is not the best apporach to use for my project,
I'll keep experiencing.
Thank you all anyway, it's always a pleasure to share some
considerations and get back usefull hints.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Problems with read_eager and Telnet
> > My conclusion being, fgfs cannot answer back quicker than this: 20Hz. > > I suspect that is by design, so as to not interfere with the simulation > itself. Actually it's not quite like that. I talked about it in flightgear-devel mailing list; I was told FGFS default telnet polling frequency is 5Hz (but it's nowhere documented than in the source code). I had extremely low interaction at first, slower than what I get now. Increasing this frequency to 120Hz gave me those results I was talking about in the previous posts; I got increasing speed but with my (quite crude) benchmark it never goes over a real 20Hz as stated, it doesn't reach those 120Hz set at hand by me. But there is no cap at 20Hz set by design in FlightGear, not that I'm aware of. I'm going to dig into some other communication media to talk to FlightGear and leave Telnet for those cases where such a lag is acceptable. -- http://mail.python.org/mailman/listinfo/python-list
