Your NetFlow exporter outputs each bi-directional flow as two separate data records -- one for the client-to-server direction and the other for the server-to-client direction. Once the direction is given, one counter for the bytes (i.e., IN_BYTES) is necessary.
By default, nProbe automatically figures out and merges the two directions of collected flows, turning the server-to-client IN_BYTES into the flow OUT_BYTES. If you --disable-cache, you will prevent nProbe from doing any merge. In this case, nProbe will act as a transparent proxy and will immediately output each data record as-is, leaving the OUT_BYTES at zero. Contrarily, if you don't --disable-cache, nProbe will automatically merge the two data records into the same flow and then will only output one flow with non-zero OUT_BYTES and IN_BYTES. Simone > On 4 Jan 2018, at 20:33, Mark Petronic <[email protected]> wrote: > > I am running nprobe as a collector, received flows from a few routers and > forwarding to Kafka. I was doing some ingest performance testing and wanted > to increase the rate of flow records/sec so I added --disable-cache to my > configuration. This results in nearly 10X more records per second output from > nprobe into Kafka which I wanted for testing. However, while testing > throughput, I noticed that the OUT_BYTES field in the JSON records published > to Kafka where always zero values now. If I remove --disable-cache, the > OUT_BYTES values return to as expected (non-zero values)? Seems quite > strange. Can anyone explain what is happening here? > > > Welcome to nProbe v.8.2.171214 (r5982) for x86_64-unknown-linux-gnu > with native PF_RING acceleration. > Copyright 2002-17 ntop.org <http://ntop.org/> > > Build OS: CentOS Linux release 7.4.1708 (Core) > SystemID: 68A2B43E76056A7E > GIT rev: 8.2-stable:81448aa287d3968208054e69cdef2a10aa91758a:20171214 > > > --disable-cache << Adding this config makes OUT_BYTES become zero for all > flow records > --interface=none > --collector=none > --collector-port=2055 > --verbose=1 > --lifetime-timeout=120 > --idle-timeout=30 > --queue-timeout=30 > --flow-version=9 > --hash-size=256000 > --kafka-add-timestamp > --kafka="kafka01:9092;netflow-raw;1" > -T="%IPV4_SRC_ADDR %IPV4_DST_ADDR %L4_SRC_PORT %L4_DST_PORT %IPV4_SRC_MASK > %IPV4_DST_MASK %IPV4_NEXT_HOP %IN_PKTS %IN_BYTES %OUT_PKTS %OUT_BYTES > %FIRST_SWITCHED %LAST_SWITCHED %TCP_FLAGS %PROTOCOL %SRC_TOS %DIRECTION > %EXPORTER_IPV4_ADDRESS %L4_SRC_PORT_MAP %L4_DST_PORT_MAP %INPUT_SNMP > %OUTPUT_SNMP %FLOW_ID" > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
