Hello Corentin, >From here https://nifi.apache.org/download/ Apache NiFi 1.28 is the last minor release of the version 1 series. The project management committee may consider critical bug fixes for essential framework features on an exceptional basis. Critical bug fixes do not include upgrading project dependencies. Multiple fundamental dependencies in NiFi 1 cannot be upgraded. These dependencies include Jetty 9.4, Spring Framework 5.3, and AngularJS 1.8. The Apache NiFi Team strongly encourages users to upgrade to NiFi 2.
The design of the HDFS processors are made to ensure nothing is lost. If you'd like to verify its behavior you might want to implement a ReportingTask to write to provenance information to your destination of choice. Any changes to MergeContent are generally discouraged. It is a rather complicated processor and extremely well used. Thanks On Mon, Jan 13, 2025 at 8:22 AM Corentin Régent <corentin.reg...@gmail.com> wrote: > Hello NiFi team, > > I am trying to monitor NiFi data ingestions from Kafka to HDFS, by tracing > the offsets of the messages to make sure none is lost. The idea I have > would be to aggregate Kafka provenance attributes as the messages are > merged into a single file, then atomically save them as HDFS extended > attributes when writing the output file. > > So I *may* (if I find enough time and if it is worth the effort) wish to > contribute and: > - Implement a new attribute strategy for the MergeContent processor, that > would allow the user to merge the attributes of the flowfiles. I only need > a "join-like" operation so I probably won't be creating a fully-fledged > aggregation expression language though. > - Add the ability to write extended file attributes in the PutHDFS file > (based on flowfile attributes). I saw that ticket NIFI-10524 > <https://issues.apache.org/jira/browse/NIFI-10524> discusses this subject > a > little, though does not really address it. > > I am unfamiliar with NiFi / Apache development processes. First of all, > would you be interested in me implementing such features? (I make no > promises though) > > And, at my company we are using NiFi 1.x through Cloudera Data Flow. So, I > was wondering whether you were still adding new features in NiFi 1.x? I > assume this would imply making one PR for NiFi 1, and one PR for NiFi 2, > for each feature? And, how long (approximately) could I expect to be > waiting for a new NiFi release after the (potential) merge? Also, if you > are still closely related with Cloudera, how long can it take to support a > new NiFi version in CDF? > > Thank you for your work and support, > > Corentin Régent >
