On 02/25/2009 04:46:46 PM, Paolo Lucente wrote: > Hi Karl, > > I would be myself interested into it. Currently the situation is not > too bad in the sense that who wants to separate client and server can > still use ssh/rsh as a glue.
<snip> > The downside of it is inefficiency: aside from encryption, it is a > "shell-based" solution. It's still OK for spawining macro-operations > like retrieving the full memory table or erasing it: connect remotely > once, then parse locally the results and feed them into the 3rd party > application. It's definitely bad for micro-operations, say for > example, > to retrieve on a 5-minutes basis traffic statistics for tens or > hundreds > of individual IP addresses, as for each of them it should pay the ssh/ > rsh toll. > > The toll would indeed be still there with socat but greatly mitigated, > granted that i understand correctly that socat also comes as a library > you can link against (otherwise the point would be the same). As far as I know socat is not a library, just a program. Socat is more efficient than the ssh/rsh solution. I'd guess _much_ more efficient. First, there is not repeated login/process startup overhead. Socat runs continuously as a daemon (or as a child of my shell program which is running as a daemon). Second, if there is encryption (which there isn't in my code at the moment) there is no repeated startup overhhead of key exchange and so forth. It only happens once when the socat deamon starts. The remaining inefficiencies are context switching overhead and memory use. I don't believe these are really worth worrying about. I wrote the socat code so I could get pmacctd statistics every 1 to 10 seconds, or maybe more. Given the hardware I'm working with this would be prohibitive using rsh/ssh. > I agree > on the fact that adding a (even optional) dependency is not optimal, > expecially if the piece in the middle is not immediatly installable by > the end-user because it's not packaged for his OS (might not be the > case > for socat, is it?). I've no idea. Socat is packaged for openbsd and debian. Aside from that I've no clue. My guess would be that it's in fedora, but not in fringe distros like openwrt. It runs on lots of Un*xs: http://www.dest-unreach.org/socat/ > > All this said, let me just shoot a proposal: as the in-memory table > client/server communication is already based on request/reply headers, > operation codes, etc. (so let's say the protocol is there) what about > keeping it simple by encoding all of this in either UDP or TCP packets > with a bit of socket programming? MTU is the only real thing to watch > out. (I'd say you only care about MTU if you're using UDP, depending on the protocol you've defined.) For encryption (or perhaps just message digesting would > suffice?) > OpenSSL is almost everywhere out there and ready to be linked. That's the alternative. The one bit of functionality you get by coding sockets in pmacct(d) is that you can use a single port on the server side and fork to allow multiple data paths to the client side. This would require the client pass some sort of token indicating what data stream was desired. You can do the same thing in socat/shell, but seems to me that the overhead is a lot higher. The shell scripting solution gets you a lot of bang for a little coding. The only line of code that does real work is the one that invokes socat, the rest is argument parsing and writing/removing pids in /var/run/ and such, mostly. Check out the code I had attached. What I want is port assignment and bind address(es) on the pmacctd side. I also don't want to have to worry about weather the client or the server side has started first. (Apparently this is not as trivial as it seems. It took a while for me to debug this in my code.) Beyond that there's a whole lot of frobbing that could be done, whether it's mtu, tcp vs. udp, various security measures or whatnot. Socat makes all the extras really easy to add-in. I would guess this is especially true when it comes to security; I hear it's easy to make serious mistakes. It seems to me that if you wanted to add network client/server support directly into pmacct/pmacctd there should be a single daemon that listens on a single port. Using more than one port is butt-ugly from a network standpoint. It may as well be the master pmacctd daemon that spawns the in-memory children. (That is how it works now, right?) It's not clear to me how that daemon would communicate with the processes that accumulate in-memory tables. Going through the Unix socket seems the wrong way to do it, in my very limited experience. Getting the listening process communicating with the in-memory data collecting processes sounds like a lot more work than adding some network sockets. Maybe. How is it usually done? I'm thinking that each child gets pointer to a network socket from the parent. The child waits on a signal from the parent. The parent listens, and opens a network socket upon client request. The parent gets a token from the client to indicate the desired data stream and uses that to decide which child to wake up with a signal that a network socket is ready. Eventually if encryption is desired there will have to be other credentials passed, but I guess that can be done in the children. Anyhow, sounds non-trivial. It occurs to me that if you're going to support encryption, and probably otherwise, you want persistent network connections between the client and server boxes. Otherwise your startup overhead for key exchange and such is too big for fast sampling. At which point you may as well have your network layer create Unix sockets on the client box and use a plain old pmacct client to talk with them. (Or have some sort of client library for real efficiency.) Sounds like a daemon on the client is needed to feed data from the server to the client-side Unix sockets. This brings me around a full circle because the whole point of my socat code is to re-create the pmacctd Unix sockets on a box elsewhere on the network and be able to use pmacct to talk through them. Ultimately, you might also want to put timestamps into the pmacct/pmacctd protocol so that the client knows for sure when the data was collected off the server. That way you can use satellite or other high latency links, and have slow equipment that takes a long time to for the occasional key-regeneration, and so forth. The other question that would come up with the network socket approach is what happens if 2 clients try to connect to the same server data stream? What happens now if 2 clients read from the same unix socket? As much as I would like to I don't think I've the time to put the code directly into pmacct. At least not if it's to be done the right way where the daemon listens on a single port. I might be able to help, but I need something that works pretty quickly. Sorry for the rambling email. I'm working things out in my head as I go. I always figure you're better off if you have the best design at the beginning, and then make tradoffs when it comes to implementation. What are your thoughts? > On Mon, Feb 23, 2009 at 08:53:05PM -0600, Karl O. Pinc wrote: > > Hi, > > > > I'm working on code that uses socat to re-create > > the pmacctd socket on a machine elsewhere on the > > network. The purpose is so that pmacctd can > > be run with memory tables and be very lightweight, > > and all the other work can be done on a > > bigger box elsewhere on the network. > > > > I would like this code to eventually be > > distributed with pmacct. > > > > What I've got now works, for some value of > > "works". There's bugs to fix and features > > to add. This is pre-alpha code. > > > > The big feature would be a pmacct-netd-spawn > > program. I'm thinking it will talk directly > > over the network to a daemon spawned by pmacct-netd-spwan > > (or maybe that program will run as a deamon itself, > > in which case I should probably change the name.) > > Anyhow, pmacctd-netd-spawn will "publish" a list > > of socketpath/port pairs, so that pmacct-netd-spawn > > can spawn pmacct-netd daemons for every socket > > on the pmacctd machine so the socket structure > > can be re-created on the pmacct machine. > > > > I'd also like to (eventually) write some docbook to generate > > man pages. I suppose this means a Makefile.in > > as well. Stuff that's not documented is awfully > > hard to use. > > > > The alternative is to build networking into > > the pmacct/pmacctd client/server code. > > > > The downside of the "socat solution" is that > > it adds an additional dependency (socat) > > and an additional layer of complication > > and a certain amount of inefficiency. > > > > The upside is that it's modular. People > > who don't need network client/server don't > > have to think about it. Socat can also > > support OpenSSL for serious encryption, > > logs, supports socks proxies, IPv6, UDP, > > and so forth. (I've no particular plans > > for these features, but I suppose if nothing > > else there could be an option that > > passes arguments through to socat so > > the end-user can do what he wants.) > > > > I thought this would be a good time to post > > what I've got so I can get feedback > > earlier rather than later. Ultimately, > > socat is glue, albeit versatile glue. > > > > Is the pmacct project interested in > > this approach? > > > > What would be a good default port > > number? (You need one port number > > per exported Unix socket, so the > > port numbers used would count up, > > like xdm does per remote X display.) > > > > Regards, > > > > Karl <[email protected]> > > Free Software: "You don't pay back, you pay forward." > > -- Robert A. Heinlein > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists > > Karl <[email protected]> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
