On 03/04/2014 21:52, Brian Candler wrote:
Using some Perl to add up the raw data I get around 1.2GB in total:
$ nfdump -M /var/nfsen/profiles-data/live/wrn-asa1 -T -r
2014/04/03/nfcapd.201404031445 | perl -ne 'if (/\s+(\d+)\s+(\d+)$/) {
$in+=$1,$out+=$2; } END { print "in=$in,out=$out\n"; }'
in=35021687,out=1196947214
I forgot the "-N" flag. Now the total matches up:
$ nfdump -N -M /var/nfsen/profiles-data/live/wrn-asa1 -T -r
2014/04/03/nfcapd.201404031445 | perl -ne 'if (/\s+(\d+)\s+(\d+)$/) {
$in+=$1,$out+=$2; } END { print "in=$in,out=$out\n"; }'
in=118151324,out=11971950092
and note that 11971950092+118151324=12090101416
But then, there must be a lot of flows which aren't being counted in the
per-protocol grouping:
$ nfdump -N -M /var/nfsen/profiles-data/live/wrn-asa1 -T -r
2014/04/03/nfcapd.201404031445 -s proto/bytes
Top 10 Protocol ordered by bytes:
Date first seen Duration Proto Protocol Flows(%)
Packets(%) Bytes(%) pps bps bpp
2014-04-03 14:44:59.963 299.827 6 6 15491(35.0) 0( 0.0)
106062229( 0.9) 0 2829958 0
2014-04-03 14:45:16.740 278.051 50 50 37( 0.1) 0( 0.0)
9551428( 0.1) 0 274810 0
2014-04-03 14:45:00.203 299.597 17 17 28500(64.5) 0( 0.0)
2510273( 0.0) 0 67030 0
2014-04-03 14:45:08.151 244.776 105 105 15( 0.0) 0(
0.0) 16104( 0.0) 0 526 0
2014-04-03 14:45:00.353 299.197 1 1 172( 0.4) 0( 0.0)
11290( 0.0) 0 301 0
Summary: total flows: 44215, total bytes: 12090101416, total packets: 0,
avg bps: 322577971, avg pps: 0, avg bpp: 0
Time window: 2014-04-03 14:44:59 - 2014-04-03 14:49:59
Total flows processed: 44215, Blocks skipped: 0, Bytes read: 5723064
Sys: 0.040s flows/second: 1105319.7 Wall: 0.038s flows/second: 1156976.1
If I add up the bytes with a script I get much more for TCP, and some
more for UDP, but the ESP total is identical:
$ nfdump -N -M /var/nfsen/profiles-data/live/wrn-asa1 -T -r
2014/04/03/nfcapd.201404031445 | perl -ne 'if
(/^\S+\s+\S+\s+\S+\s+\S+\s+(\S+).*\s+(\d+)\s+(\d+)$/ && $1 == 6) {
$flows+=1,$bytes+=$2+$3; } END { print "flows=$flows,bytes=$bytes\n"; }'
flows=15370,bytes=12068762411
$ nfdump -N -M /var/nfsen/profiles-data/live/wrn-asa1 -T -r
2014/04/03/nfcapd.201404031445 | perl -ne 'if
(/^\S+\s+\S+\s+\S+\s+\S+\s+(\S+).*\s+(\d+)\s+(\d+)$/ && $1 == 50) {
$flows+=1,$bytes+=$2+$3; } END { print "flows=$flows,bytes=$bytes\n"; }'
flows=37,bytes=9551428
$ nfdump -N -M /var/nfsen/profiles-data/live/wrn-asa1 -T -r
2014/04/03/nfcapd.201404031445 | perl -ne 'if
(/^\S+\s+\S+\s+\S+\s+\S+\s+(\S+).*\s+(\d+)\s+(\d+)$/ && $1 == 17) {
$flows+=1,$bytes+=$2+$3; } END { print "flows=$flows,bytes=$bytes\n"; }'
flows=28500,bytes=11750463
Something very odd here. For TCP I guess it might be 2^32 wraparound,
but that doesn't explain UDP.
Here are the ten largest TCP flows:
$ nfdump -N -M /var/nfsen/profiles-data/live/wrn-asa1 -T -r
2014/04/03/nfcapd.201404031445 | perl -ne 'if
(/^\S+\s+\S+\s+\S+\s+\S+\s+(\S+).*\s+(\d+)\s+(\d+)$/ && $1 == 6) { print
$2+$3,"\n"; }' | sort -n | tail
271085786
282543210
282862359
287168871
291073520
302766693
328700743
398284017
403970269
425913479
One example large flow:
2014-04-03 14:45:01.342 DELETE 2027 6 192.168.5.66:54317 ->
xxx.xx.xx.xxx:443 yyy.yy.yy.y:54317 -> xxx.xx.xx.xxx:443 11808
425901671
Note that there is NAT going on here, so we have both the untranslated
and translated records. There's no NAT going on for the ESP. I wonder if
that could be something to do with it? No, that doesn't add up; if I sum
just the records with no NAT I still get a larger total (nearly 1GB)
$ nfdump -N -M /var/nfsen/profiles-data/live/wrn-asa1 -T -r
2014/04/03/nfcapd.201404031445 | perl -ne '@a = split(" ");
$bytes+=$a[15]+$a[16] if $a[6] eq $a[11] and $a[9] eq $a[14]; END {
print "$bytes\n"; }'
956821693
Looking in the records, I do see some have "Ignore" in the "Xevent"
column. That's not it either:
$ nfdump -N -M /var/nfsen/profiles-data/live/wrn-asa1 -T -r
2014/04/03/nfcapd.201404031445 | perl -ne '@a = split(" ");
$bytes+=$a[15]+$a[16] if $a[3] ne "Ignore"; END { print "$bytes\n"; }'
8606805610
Regards,
Brian.
------------------------------------------------------------------------------
_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
