I'm not sure I'd want more than 60 days of data unless required by law. You can't turn over data you don't have, and I don't want to be digging up 3 year old data on customers if I don't have to. When the data cops do their stop-and-frisk, the best thing may be to turn out your pockets and have them be empty. As I said, you can't turn over what you don't have. If there is a major criminal investigation, they should send you a court order telling you to preserve the data.
And I know GDPR doesn't technically apply to us unless we have customers in Europe, but I think globally that will drive a trend toward not keeping customer data longer than absolutely needed for legal or operational reasons. I'm sure if you do webhosting you've seen changes in software to anonymize logfiles by hiding IP addresses and to purge the data after relatively short times like 1 or 2 weeks. -----Original Message----- From: AF <[email protected]> On Behalf Of Steve Sent: Tuesday, October 16, 2018 9:28 AM To: AnimalFarm Microwave Users Group <[email protected]> Subject: Re: [AFMUG] Netflow Wow only 60 days. Since netflow is doing constant amount of writing probably best to use an SSD. 250GB does it compress the data? Like splunk you can have a ton of data come in daily but it compresses the heck out of it so you can fit years of data. ----- Original Message ----- From: "Josh Baird" <[email protected]> To: "AnimalFarm Microwave Users Group" <[email protected]> Sent: Tuesday, October 16, 2018 9:09:07 AM Subject: Re: [AFMUG] Netflow I keep 60 days worth at it uses about 250GB and very low compute resources. The VM has 4 cores and 4GB of memory. This is with nfsen+nfcapd - which - but it's free, doesn't use a pile of resources (no SQL) and is functional ENOUGH to provide data when I need it. On Tue, Oct 16, 2018 at 9:03 AM Steve < [ mailto:[email protected] | [email protected] ] > wrote: Elistiflow? Really. I'll need to look that up. See I've had to deal with law enforcement a few times where I've helped them find some bad people. Fast forward a few years later all of a sudden there is a court case and I get called into court. I had provided the legal evidence at the time and kept the correspondence but I generally don't keep all of the traffic data for that long as I'm not legally required to. I would have liked to have kept that information. I only keep it for a certain length of time for troubleshooting. But going back 3 years would require some serious space and hardware. So I'm looking for an ultra efficient collector capable of years of retention of mainly just metadata, but able to run on prem with minimal resources for privacy's sake. ----- Original Message ----- From: "Josh Baird" < [ mailto:[email protected] | [email protected] ] > To: "AnimalFarm Microwave Users Group" < [ mailto:[email protected] | [email protected] ] > Sent: Monday, October 15, 2018 8:14:20 PM Subject: Re: [AFMUG] Netflow Solarwinds is IPO'ing on Friday. Net loss of 87M in the past 6mos. Anyways - I have also been trying to find time to give ElastiFlow a whirl.. but I really don't want to spin up an ELK stack just for Netflow. On Mon, Oct 15, 2018 at 8:11 PM Ken Hohhof < [ mailto: [ mailto:[email protected] | [email protected] ] | [ mailto:[email protected] | [email protected] ] ] > wrote: I thought Solarwinds was like Tiffany’s (as in Breakfast at Tiffany’s, please don’t tell me you’ve never seen it). Fun to window shop, nothing you can afford. From: AF < [ mailto: [ mailto:[email protected] | [email protected] ] | [ mailto:[email protected] | [email protected] ] ] > On Behalf Of Erich Kaiser Sent: Monday, October 15, 2018 6:20 PM To: AnimalFarm Microwave Users Group < [ mailto: [ mailto:[email protected] | [email protected] ] | [ mailto:[email protected] | [email protected] ] ] > Subject: Re: [AFMUG] Netflow We use: solarwinds as-stats for Fusion We collect sflow data. These are also good options Fastnetmon Scrutinizer Kentik Ntop is junk. On Mon, Oct 15, 2018 at 2:32 PM Steve < [ mailto: [ mailto:[email protected] | [email protected] ] | [ mailto:[email protected] | [email protected] ] ] > wrote: What collectors do you use? I've used these the past Scrutinizer ManageEngine's Netflow Anaylzer Polygraph.io (cloud based now another company owns) NTOP Anyone have any other user friendly options that perform very well and are not too costly? For my purposes I'd be spitting out 1 x 10Gbps interface that would hold all the headers to the collector. Anyone know of any other on prem collectors or SSL secured cloud based? -- AF mailing list [ mailto: [ mailto:[email protected] | [email protected] ] | [ mailto:[email protected] | [email protected] ] ] [ [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] | [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] ] -- AF mailing list [ mailto: [ mailto:[email protected] | [email protected] ] | [ mailto:[email protected] | [email protected] ] ] [ [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] | [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] ] -- AF mailing list [ mailto:[email protected] | [email protected] ] [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] -- AF mailing list [ mailto:[email protected] | [email protected] ] [ http://af.afmug.com/mailman/listinfo/af_af.afmug.com | http://af.afmug.com/mailman/listinfo/af_af.afmug.com ] -- AF mailing list [email protected] http://af.afmug.com/mailman/listinfo/af_af.afmug.com -- AF mailing list [email protected] http://af.afmug.com/mailman/listinfo/af_af.afmug.com -- AF mailing list [email protected] http://af.afmug.com/mailman/listinfo/af_af.afmug.com
