We are in the process of adding netflow collection to libreqos. Any potential testers using any of these backends described below out there?
On Thu, Mar 28, 2024, 5:02 PM Brian Knight via NANOG <nanog@nanog.org> wrote: > Thanks to all who took the time to comment and make suggestions. > > To summarize the private messages, one respondent suggested Argus as a > collector. Another mentioned that they are still using AS-Stats. > > I'm drawn to Akvorado. I like the self-contained nature of the > application. NF collector, database, and modern web GUI are all bundled in > one docker container. The full-featured demo <https://demo.akvorado.net/> > is fantastic. That the app can enrich the Netflow data with BMP is an added > bonus. > > The best part is, the GUI has the report viz I need, and it is actually > the default visualization in the demo. It also has the graph types that I > didn't know I needed, like the Sankey graph. > > FlowViewer looks interesting as well. I suspect getting the reports right > may take some time, given the amount of GUI filtering options. > > pmacct and Argus seem to be capable tools that have been around for a long > time, but I haven't seen a concise stack building guide to get Netflow data > into a good GUI using these. Looks like there are some older Docker images > available for both. I could write my own SQL or roll my own stack, but I'd > much rather spend my time on other things. > > I appreciate the conversation around sFlow. I actually wasn't aware that > XR supported it. AS path probably doesn't add a whole lot of value given > that I'm focused on flows across our IP transit circuits. I'm able to > determine my next AS hop simply by looking at the flow's associated tuple > of (flow exporter, interface). I can use other tools like RouteViews or > RIPE's RIS to determine the destination AS's upstreams if needed. The rest > of the path is probably not too helpful for determining peering > opportunities. > > I think I'm going to get Akvorado running in my environment. If that > doesn't pan out, I'll likely go back to AS-Stats. > > Can those running Akvorado comment on their system specs? The only spec > I've seen is a mention in this blog post > <https://vincent.bernat.ch/en/blog/2022-akvorado-flow-collector>: > "Akvorado is performant enough to handle 100 000 flows per second with 64 > GB of RAM and 24 vCPU. With 2 TB of disk, you should expect to keep data > for a few years." > > Thanks again all, > > -Brian > > > On 2024-03-26 19:04, Brian Knight via NANOG wrote: > > What's presently the most commonly used open source toolset for monitoring > AS-to-AS traffic? > > I want to see with which ASes I am exchanging the most traffic across my > transits and IX links. I want to look for opportunities to peer so I can > better sell expansion of peering to upper management. > > Our routers are mostly $VENDOR_C_XR so Netflow support is key. > > In the past, I've used AS-Stats <https://github.com/manuelkasper/AS-Stats> > for this purpose. However, it is particularly CPU and disk IO intensive. > Also, it has not been actively maintained since 2017. > > InfluxDB wants to sell me > <https://www.influxdata.com/what-are-netflow-and-sflow/> on Telegraf + > InfluxDB + Chronograf + Kapacitor, but I can't find any clear guide on what > hardware I would need for that, never mind how to set up the software. It > does appear to have an open source option, however. > > pmacct seems to be good at gathering Netflow, but doesn't seem to analyze > data. I don't see any concise howto guides for setting this up for my > purpose, however. > > I'm aware Kentik does this very well, but I have no budget at the moment, > my testing window is longer than the 30 day trial, and we are not prepared > to share our Netflow data with a third party. > > Elastiflow <https://www.elastiflow.com/> appears to have been open source > <https://github.com/robcowart/elastiflow?tab=readme-ov-file> at one time > in the past, but no longer. Since it too appears to be hosted, I have the > same objections as I do with Kentik above. > > On-list and off-list replies are welcome. > > Thanks, > > -Brian > > > >
