My guess is this is the issue - support for variable length records in IPFIX:
/
From: https://lists.sei.cmu.edu/pipermail/netsa-tools-discuss/2014-June/000002.html

//"nfcap does not have support for variable-length elements in IPFIX.... In IPFIX the length of variable-length elements is encoded as 65535 in the template. Since nfcap in not aware of these elements, it uses that length, 65535, instead of reading the actual length of the data which is encoded in the IPFIX data record. I’m sure you have noticed that you sometimes get valid flow data, this is because nfcap can process the first data record in a flow set, but the invalid length is throwing off the offset calculations of the subsequent data records."/

If true it looks like I either fix it myself (unlikely) or look for a different collector.


On 10/29/2016 01:39 PM, James A. Klun wrote:

I am successfully running nfdump compiled via gcc/cygwin.

Basic functionality is there:


        > E:\netflow>nfdump -r 2055/nfcapd.201610281538 | more

    > Date first seen          Duration Proto      Src IP
    Addr:Port          Dst IP Addr:Port   Packets    Bytes Flows
> 1969-12-31 18:00:00.000 0.000 UDP xxx.xxx.xxx.xxx:xxxx -> xxx.xxx.xxx.xxx:49962 20 15642 1 > 1969-12-31 18:00:00.000 0.000 TCP xxx.xxx.xxx.xxx:7800 -> xxx.xxx.xxx.xxx:30488 2 104 1

     <continues - note the date oddity>


The netflow source is Cisco gear running V9 netflow

 SNMP interface numbers are important to me for analysis.

 What I am finding is that they are not captured correctly

        > E:\netflow>nfdump -r 2055/nfcapd.201610281538 -s if
        > Top 10 In/Out If ordered by -:
> Date first seen Duration Proto In/Out If Flows(%) Packets(%) Bytes(%) pps bps bpp > 1969-12-31 18:00:00.000 0.000 any 0 16214(50.3) 1.1 M(79.5) 476.2 M(92.5) 0 0 452 > 1969-12-31 18:00:00.000 0.000 any 5 16214(50.3) 1.1 M(79.5) 476.2 M(92.5) 0 0 452 > 1969-12-31 18:00:00.000 0.000 any 327680 16015(49.7) 272336(20.5) 38.7 M( 7.5) 0 0 142 > 1969-12-31 18:00:00.000 0.000 any 16777216 16015(49.7) 272336(20.5) 38.7 M( 7.5) 0 0 142

        > Summary: total flows: 32229, total bytes: 514879163, total
        packets: 1325511, avg bps: 0, avg pps: 0, avg bpp: 0
        > Time window: 2016-10-28 15:38:29 - 2016-10-28 15:43:29
        > Total flows processed: 32229, Blocks skipped: 0, Bytes read:
        2642986
        > Sys: 0.000s flows/second: 0.0        Wall: 0.031s
        flows/second: 1032980.8

    and....


        nfdump -r 2055/nfcapd.201610281538 -o csv |  cut -d, -f16,17 |
        sort  | uniq -c

        16 and 17 are the produces:

                     in,         out
          24243 327680 16777216
          80632 5         0

I know the actual snmp index values from the router in question from running: snmp mib ifmib ifindex They range from 1-95. A number of them have activity. In the above, 5 (and 0) are legitimate, 327680 and 16777216 are not. 9 - an active interface shown in the wireshark excerpt below - simply does not appear all. Most active interfaces are absent

I ran wireshark to capture netflow data directly......
I waited long enough for the V9 flow template to be delivered as discussed in
https://www.wireshark.org/lists/wireshark-users/200905/msg00119.html

Meaningful interface numbers ARE being delivered to nfcapd ( wireshark excerpt below ) See: ==>

>>No. Time Source Destination Protocol Length OutputInt InputInt Info
            >>  24949 2016-10-28 21:20:01.990125000
            xx.xx.xx.xx         xx.xx.xx.xx          CFLOW
            1340              64,66,68,70,72,74,11,13,15,17,18,76
            total: 13 (v9) records
            >>
            >>Frame 24949: 1340 bytes on wire (10720 bits), 1340 bytes
            captured (10720 bits)
            >>    Arrival Time: Oct 28, 2016 21:20:01.990125000 EDT
            >>    Epoch Time: 1477704001.990125000 seconds
            >>    [Time delta from previous captured frame:
            0.000021000 seconds]
            >>    [Time delta from previous displayed frame:
            0.000091000 seconds]
            >>    [Time since reference or first frame: 459.323921000
            seconds]
            >>    Frame Number: 24949
            >>    Frame Length: 1340 bytes (10720 bits)
            >>    Capture Length: 1340 bytes (10720 bits)
            >>    [Frame is marked: False]
            >>    [Frame is ignored: False]
            >>    [Protocols in frame: eth:ip:udp:cflow]
            >>    [Coloring Rule Name: UDP]
            >>    [Coloring Rule String: udp]
            >>Ethernet II, Src: Cisco_22: (), Dst: Vmware ()
            >>
            >>Cisco NetFlow/IPFIX   ==> Note
            >>    Version: 9              ==> Note
            >>    Count: 13
            >>    SysUptime: 1113116230
            >>    Timestamp: Oct 28, 2016 21:20:02.000000000 EDT
            >>        CurrentSecs: 1477704002
            >>    FlowSequence: 808
            >>    SourceId: 6
            >>    FlowSet 1
            >>        FlowSet Id: Options Template(V9) (1)
            >>        FlowSet Length: 26
            >>        Options Template (Id = 256) (Scope Count = 1;
            Data Count = 3)
            >>            Template Id: 256
            >>            Option Scope Length: 4
            >>            Option Length: 12
            >>            Field (1/1) [Scope]: System
            >>                Scope Type: System (1)
            >>                Length: 4
            >>            Field (1/3): INPUT_SNMP
            >>                Type: INPUT_SNMP (10)
            >>                Length: 4
            >>            Field (2/3): IF_NAME
            >>                Type: IF_NAME (82)
            >>                Length: 32
            >>            Field (3/3): IF_DESC
            >>                Type: IF_DESC (83)
            >>                Length: 64
            >>    FlowSet 2
            >>        FlowSet Id: (Data) (256)
            >>        FlowSet Length: 1252
            >>        Flow 1
            >>            ScopeSystem: 0a65fef0
            >>            InputInt: 64 ==> interface number is appearing
            >>            IfName: Se0/2/0/23:0           ==> correct
            association
            >>            IfDescr: Serial0/2/0/23:0
            >>        Flow 2
            >>            ScopeSystem: 0a65fef0
            >>            InputInt: 66
            >>            IfName: Se0/2/0/24:0
            >>            IfDescr: Serial0/2/0/24:0
            >>        Flow 3
            >>            ScopeSystem: 0a65fef0
            >>            InputInt: 68
            >>            IfName: Se0/2/0/25:0
            >>            IfDescr: Serial0/2/0/25:0

            and

            >>Cisco NetFlow/IPFIX
            >>    Version: 9
            >>    Count: 38
            >>    SysUptime: 261103507
            >>    Timestamp: Oct 28, 2016 21:12:22.000000000 EDT
            >>        CurrentSecs: 1477703542
            >>    FlowSequence: 159997
            >>    SourceId: 2304
            >>    FlowSet 1
            >>        FlowSet Id: (Data) (264)
            >>        FlowSet Length: 1336
            >>        Flow 1
            >>            SrcAddr: 122.x.x.x.(122.x.x.x)
            >>            DstAddr: 122.x.x.x (122.x.x.x)
            >>            IP ToS: 0x68
            >>            Protocol: 17
            >>            SrcPort: 20903
            >>            DstPort: 53
            >>            OutputInt: 9                  ===> interface
            number appears (and interface is in fact active )
            >>            Direction: Egress (1)
            >>            Octets: 79
            >>            Packets: 1




The interface number information is clearly being delivered, the interfaces have activity, and yet my nfdump reporting runs fail to reveal them.

Nfdump (1.6.13) has V9 has support ( my understanding ). I would expect the correct interface numbers to be there.

Any help appreciated ... assumption is there is something I am simply doing wrong.







--
James A. Klun           jk...@microsolved.com
Security Engineer                 (614) 351 - 1237
PGP Key Available by Request
MicroSolved is security expertise you can trust!

HoneyPoint Security Server
Attackers get stung, instead of you!
http://www.microsolved.com/honeypoint


--
James A. Klun                     jk...@microsolved.com
Security Engineer                 (614) 351 - 1237
PGP Key Available by Request
MicroSolved is security expertise you can trust!

HoneyPoint Security Server
Attackers get stung, instead of you!
http://www.microsolved.com/honeypoint

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers
Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive. 
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss

Reply via email to