Re: [DISCUSS] NiFi 2.0 Release Goals

2021-08-03 Thread Ryan Hendrickson
I'll preface with what I'm about to say is not rooted in any real understanding of the lift required or other customer experiences... We have used NiFi for a long time and are excited for what new features may lay ahead in the 2.x line. ---After having written this, I think our succinct pain poin

Re: Using nifi in separated networks

2021-08-03 Thread Marc
Hi Daniel, we tested two different diodes (two different manufacturer) for a longer time. Both diodes count the UDP packets and log any loss. Additionally we are using hashes to detect any form of content errors. We have tested the two diodes for over a year and have not seen any packet loss du

Re: [DISCUSS] NiFi 2.0 Release Goals

2021-08-03 Thread Joe Witt
Kevin, This may well be a very interesting approach which opens up a lot of options for us in how we tackle a 2.x... Thanks On Tue, Aug 3, 2021 at 8:45 AM Kevin Doran wrote: > > Outside of core NiFi, for MiNiFi, I think it would be beneficial to align > MiNiFi Java and MiNiFI C++ to use the sa

Re: [DISCUSS] NiFi 2.0 Release Goals

2021-08-03 Thread Kevin Doran
Outside of core NiFi, for MiNiFi, I think it would be beneficial to align MiNiFi Java and MiNiFI C++ to use the same method(s) for command and control / flow deployment. This was discussed when MiNiFi Java was merged into the NiFi codebase [1]. My proposal would be to create a Java implementati

Re: [DISCUSS] NiFi 2.0 Release Goals

2021-08-03 Thread David Handermann
Thanks for the continued discussion around what can or should be removed. Having a clear migration path from version 1 to version 2 will be very important for supporting current deployments. Following the example of some other projects, one way to consider the upgrade is from the point of view of

PutParquet - Compression Type: SNAPPY (NiFi 1.14.0)

2021-08-03 Thread Bilal Bektas
Hi Dev Team, SNAPPY compression feature of PutParquet Processor works correctly on NiFi 1.12.1 but NiFi 1.14.0 gives the following error. Could you help in order to resolve this situation? Thank you in advance, --Bilal PutParquet - Config: Hadoop Configuration Resources File locations Kerb

Re: Using nifi in separated networks

2021-08-03 Thread Daniel Chaffelson
This is a very interesting area of integration investigation Marc, thank you for sharing your work! I looked into this a little after conversations with folks in security applications, and I wonder if you investigated approaches to tracking and reporting/handling packet loss and error rates in thi