Hi,
In order to test pmacct, it can be very handy to feed it with various
flow-tools Netflow files. As I wrote in a previous letter, I failed
to do it with "flow-export -f1" - the result was a bad pcap file, that
cannot even read correctly with tcpdump (I tried an old version using
libcap 0.7.2, and one of the latest versions, compiled from sources,
using libcap 0.7.2).
Looking in the source of flow-export, in the section that tries
to export to pcap format, discouraged me from trying to fix it (one
problem is that I would like to have full export, and the code in
flow-export (format1) only does a partial export, and of UDP and TCP
only).
Flow-export has a "wire" format (format 4), which is a raw Netflow
stream. So I decided to change nfacctd in order that it will be able
to read Netflow streams as stdin, much like pmacctd with pcap files.
To that end I extended "nfacctd_ip" to get argument "-" which means
stdin (but not files since I didn't find it too useful) (it may be more
desirable to have a common (to pmacctd and nfacctd) input file key
directive like "input_file").
But, there was still a problem: nfacctd expects clear boundaries for
Netflow packets (it consider each UDP packet as a complete Netflow packet)
and a Netflow stream which are read as stdin doesn't have such boundaries.
It can be fixed in two ways:
1. The routines process_v1_packet() to process_v9_packet() can be changed
to return the packet length (instead of checking that they got the
exact packet length). However, it was not obvious to me how to change
process_v8_packet() and process_v8_packet() for that.
2. Modify flow-export to prefix each Netflow record by its length,
and modify nfacctd to read such modified Netflow stream when reading
from stdin is requested.
I selected way 2 because it was easy to implement. I added
format 6 to flow-export which converts to "length prefix" wire format.
The results seem to work very well.
If someone things way 1 is better (because it doesn't need a modified
flow-export), than I can do it this way, but I need help in making
process_v8_packet() and process_v8_packet() to return the actual length
of the packet.
If you finds it useful and can send diffs (to flow-export 0.67
and pmacctd 0.9.2 - but it looks like it is the same for 0.9.3 because
the modified section has not been changed there).
Amir
