Hi Chris, Chris Wilson, 07.11.2006 14:43: >> Yes, traffic shaping between interfaces should be better done in kernel. >> And i fully agree with you: doing the job twice is not great idea. So, >> if you can see a way to, say, get the flows from libpcap and >> classification infos from the kernel, just let me/us know as it sounds a >> good idea! > > OK, I have some idea of how this might work. Harald Welte, one of the > Netfilter developers, has proposed a system for accounting flows in the > kernel as part of Netfilter's Conntrack code. He presented a paper on this > at LinuxTag 2005, which unfortunately is not available online in PDF form > (since LinuxTag apparently charges for access to conference papers). I > generated an HTML version and attached it here: > > [http://bmo.aidworld.org/attach/Chris/paper.html]
He gave the same talk on the Linux Symposium 2005, you can find the paper in the proceedings: http://www.linuxsymposium.org/2005/linuxsymposium_procv2.pdf One year ago I had a short mail conversation with Harald about exactly that subject. - First, he clearly pointed out, that flow accounting in the conntrack module makes sense _only_ if you use conntrack anyway (like firewall, NAT, ...). To use conntrack just for flow accounting would be just "overkill", he wrote. - Second, you are strictly bound to the classical flow keys which are kept in the conntrack table anyway, that is source and destination IP and port. So the usage of the flow-accounting module in conntrack is quite restricted, but as long as these restrictions don't bother, it's a good alternative of course. (At the moment pmacct also only has a fixed flow data structure, but with the propagation of IPFIX I hope we will move to a more flexible structure.) But for traffic-shaping based on application level analysis you have a problem already: You can classify packets, but you cannot store that information in the conntrack table as a flow key (AFAIK). Of course you could store that information in another place and map it to the flows in the conntrack table, but then the - let's call it - "L7ClassID" is not a real flow key, since it it possible that one flow (in the conntrack table) has several different L7ClassIDs over time, splitting it in different flows in fact. In general you have to ask yourself the question, if having both routing and monitoring on the same machine is a good idea. You will probably always end up in a situation, where both functionalities interfere with each other. That's why I think, having a dedicated metering-probe is in most cases the better choice. And then, as the machine is not doing anything else with the monitored packets, handling everything in user-space is the better approach. Under Linux you can even optimize the network-adapter->user-space transition with PF_RING by Luca Deri. Of course, you cannot use this set-up if you want to do traffic shaping or similar based on the monitoring. > I'm still concerned about the performance of the MySQL plugin with > threading, so I'm considering providing an option to disable the extra > threads, and run updates synchronously. Interesting. What about having also a switch to have "numbers-only" tables, that is IP addresses, timestamps, class_id, mac addresses and protocol are all stored as integers? While on the subject of changing everything: what about a different timestamp set? I would prefer three timestamps: one for the first and the last packet in a flow, and one for the time the flow got "closed" (or updated the last time) which would correspond to the "time-slot" the flow belongs to. The third one is probably not really necessary, as you can calculate it from the other timestamps and the configuration, but it would give you a good index-key for the time-slots. Cheers, Sven -- Sven Anderson Institute for Informatics - http://www.ifi.informatik.uni-goettingen.de Georg-August-Universitaet Goettingen Lotzestr. 16-18, 37083 Goettingen, Germany _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
