Hi Robert, Have you checked your minifi agent logs? Specifically the minifi-app.log. There should be an error describing the reason for the failed connection/communication with the remote NiFi instance... Is https://odc-nifi-test.nci.org.au:444/nifi the right URI of your remote NiFi instance where you are expecting flow files from the minifi agent?
Regards, Peter Robert Cohen <robert.co...@anu.edu.au.invalid> ezt írta (időpont: 2025. jan. 8., Sze, 5:20): > OK, I fixed the validation. I had an error in my bootstrap.conf. > > Its not ideal that the toolkit transform-nifi generates a invalid flow > file. > > > Now I can get minifi to start. But the code isnt working. > > This is a trivial "ping" . It just uses a GenFlowFile to make a flow. > And sends it to a remote process group. The remote process group lives > in another nifi instance, and just logs and discards the flows. > > > I had to move the code in the source nifi into sub group. And I exported > that. And ran it through the converter. > > Inside nifi, its working fine. > > But in the minifi, its generating the flows. I can see them piling up in > the WAL. But they arent being transmitted. Theres nothing being received > by the remote nifi. > > OK, I fixed the validation. I had an error in my bootstrap.conf. > > Its not ideal that the toolkit transform-nifi generates a invalid flow > file. > > > Now I can get minifi to start. But the code isnt working. > > This is a trivial "ping" . It just uses a GenFlowFile to make a flow. > And sends it to a remote process group. The remote process group lives > in another nifi instance, and just logs and discards the flows. > > > I had to move the code in the source nifi into sub group. And I exported > that. And ran it through the converter. > > Inside nifi, its working fine. > > But in the minifi, its generating the flows. I can see them piling up in > the WAL. But they arent being transmitted. Theres nothing being received > by the remote nifi. > > > > On 8/1/2025 2:33 pm, Robert Cohen wrote: > > I googled the error message. > > > > I found a thread here which seems the same > > > > > https://community.cloudera.com/t5/Support-Questions/Unable-to-start-MiNiFi-because-of-an-error-caused-due-to-the/m-p/393219 > > > > > > > > One thing it suggests is editing the flow file and changing the > > following setting from 0. > > > > |"maxTimerDrivenThreadCount":5 But when I do that minifi fails to > > start as it says the flow file fails validation Does the flow file > > contain a checksum? | > > > > > > On 8/1/2025 1:27 pm, Robert Cohen wrote: > >> Thanks for that. It clears some things up. > >> > >> Im not sure if the doco is supposed to be suitable for nifi 1/2 or > >> just nifi 2. > >> > >> But you might want to clarify in the doco whether config files are > >> just used by nifi 1. Just used by nifi 2. Or supported in nifi 2 for > >> legacy. But not recommended. > >> > >> > >> Anyway, Ive put my code from a nifi 2.1.0 into minifi 2.1.0. > >> > >> I get the following error > >> > >> 2025-01-08 13:16:09,065 ERROR [MiNiFi logging handler] > >> org.apache.nifi.minifi.StdErr Failed to start flow service: Unable to > >> load flow due to: > >> org.apache.nifi.controller.serialization.FlowSynchronizationException: > >> java.lang.IllegalArgumentException: Cannot set max number of threads > >> to less than 1 > >> > >> I attach the converted flow file > >> > >> I can't see any obvious parameter related to max number of threads. > >> > >> > >> > >> On 7/1/2025 6:56 pm, Kedvessy Péter wrote: > >>> [You don't often get email from pkedve...@gmail.com. Learn why this > >>> is important at https://aka.ms/LearnAboutSenderIdentification ] > >>> > >>> Hi Robert, > >>> > >>> Thanks for reporting the documentation issues. I filed a JIRA to fix > >>> them: > >>> https://issues.apache.org/jira/browse/NIFI-14093 > >>> > >>> > >>> In terms of security configuration, you can follow the following guide: > >>> > https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#security-properties-in-bootstrapconf > >>> > >>> > >>> About the different configuration files you mentioned: > >>> - bootstrap.conf is the main configuration file what we should update > >>> - minifi.properties is a generated properties file based on the > >>> bootstrap.conf > >>> - nifi.properties is the legacy configuration file of minifi > >>> - config.yaml is the legacy flow configuration, now we have the json > >>> one > >>> what you have tried to import previously > >>> > >>> I hope it helps to set up your minfi agent. > >>> > >>> Regards, > >>> Peter > >>> > >>> Robert Cohen <robert.co...@anu.edu.au.invalid> ezt írta (időpont: > 2025. > >>> jan. 7., K, 6:19): > >>> > >>>> I'm having a certain amount of trouble getting good info on > >>>> administration of minifi 2. > >>>> > >>>> The README file says thats documentation can be found at > >>>> > >>>> https://nifi.apache.org/minifi > >>>> and > >>>> https://cwiki.apache.org/confluence/display/NIFI/MiNiFi > >>>> > >>>> > >>>> neither of which URL works. > >>>> > >>>> > >>>> Im assuming the real documentation is at > >>>> > >>>> > >>>> > https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#config-file > >>>> > >>>> > >>>> > >>>> But Im having some Im finding it a bit confusing. > >>>> > >>>> I want to configure the keystore/truststore and sensitive prop > >>>> passwords. > >>>> > >>>> It appears I can configure them either in bootstrap.conf or > >>>> minifi.properties. Is either to be prefered over the other? > >>>> > >>>> The documention mentions nifi.properties and config.yml. But doesnt > >>>> mention minifi.properties. > >>>> > >>>> Im assuming the mentions of config.yml are leftover from minifi 1 > >>>> documentation. Or does minifi 2 use it for backwards compatability. > >>>> > >>>> And I assume nifi.properties has been renamed to minifi.properties and > >>>> the documentation hasnt caught up. > >>>> > >>>> > >>>> Actually dumping strings out of the jar files, I can find instances of > >>>> nifi.properties. But I cant find any references to minifi.properties. > >>>> > >>>> However the tarball contains a minifi.properties file but not a > >>>> nifi.properties file. > >>>> > >>>> So Im really not sure what is going on. > >>>> > >>>> > >>>> > >>>> > >>>> On 6/1/2025 4:40 pm, Robert Cohen wrote: > >>>>> Ahh, it appears the functionality is there. Its just that the README > >>>>> file hasnt been updated, > >>>>> > >>>>> Theres now a transform-nifi command in addition to the transform-yml > >>>>> command. > >>>>> > >>>>> > >>>>> On 6/1/2025 2:59 pm, Robert Cohen wrote: > >>>>>> [You don't often get email from robert.co...@anu.edu.au.invalid. > >>>>>> Learn why this is important at > >>>>>> https://aka.ms/LearnAboutSenderIdentification ] > >>>>>> > >>>>>> I had a look at the minifi toolkit. > >>>>>> > >>>>>> However, it only appears to contain functionality for converting > >>>>>> legacy > >>>>>> minifi config to minifi 2 config > >>>>>> > >>>>>> Heres an excerpt from the minifi-tookit 2.1.0 README file > >>>>>> > >>>>>> Valid commands include: > >>>>>> transform-yml: Transforms legacy MiNiFi flow config YAML into > >>>>>> MiNiFi flow config JSON > >>>>>> > >>>>>> ## Example > >>>>>> - You have an older version of MiNiFi located in > >>>>>> <legacy_minifi_directory>. > >>>>>> - You would like upgrade to the latest version of MiNiFi. You > >>>>>> downloaded > >>>>>> and extracted the latest MiNiFi into <latest_minifi_directory>. > >>>>>> - Run the following command to migrate the flow and the bootstrap > >>>>>> config > >>>>>> ``` > >>>>>> ./config.sh transform-yml <legacy_minifi_directory>/conf/config.yml > >>>>>> <legacy_minifi_directory>/conf/bootstrap.conf > >>>>>> <latest_minifi_directory>/conf/flow.json.raw > >>>>>> <latest_minifi_directory>/conf/bootstrap.conf > >>>>>> ``` > >>>>>> > >>>>>> > >>>>>> Is there functionality somewhere to convert a nifi 2.x flow file > >>>>>> to a > >>>>>> minifi 2 flow file, > >>>>>> > >>>>>> > >>>>>> On 12/12/2024 6:48 pm, Kedvessy Péter wrote: > >>>>>>> [You don't often get email from pkedve...@gmail.com. Learn why > this > >>>>>>> is important at https://aka.ms/LearnAboutSenderIdentification ] > >>>>>>> > >>>>>>> Hi Robert, > >>>>>>> > >>>>>>> Seems like the documentation has not been updated. I'm going to > >>>>>>> file > >>>>>>> a JIRA > >>>>>>> ticket to apply doc update about how we can import NiFi 2 > >>>>>>> exported flow > >>>>>>> files into MiNiFi. > >>>>>>> In a nutshell: before moving the json file into minifi, we should > >>>>>>> convert > >>>>>>> it with minifi toolkit. Related ticket for more details: > >>>>>>> https://issues.apache.org/jira/browse/NIFI-12644 > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> Regards, > >>>>>>> Peter > >>>>>>> > >>>>>>> Robert Cohen <robert.co...@anu.edu.au.invalid> ezt írta (időpont: > >>>> 2024. > >>>>>>> dec. 12., Cs, 2:11): > >>>>>>> > >>>>>>>> Im having trouble getting code into minifi 2 from nifi 2. > >>>>>>>> > >>>>>>>> The documentation here > >>>>>>>> > >>>>>>>> > >>>> > https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md > >>>> > >>>>>>>> > >>>>>>>> says > >>>>>>>> > >>>>>>>> # Export the dataflow in JSON format. > >>>>>>>> # Move your new .json file to |minifi/conf|. > >>>>>>>> # Rename your .json file /flow.json.raw/. > >>>>>>>> > >>>>>>>> > >>>>>>>> Im using the option in nifi to "download flow definition" > >>>>>>>> > >>>>>>>> However when I load that into minifi, I get an error message. > >>>>>>>> > >>>>>>>> java.lang.Exception: Unable to load flow due to: > >>>>>>>> > org.apache.nifi.controller.serialization.FlowSerializationException: > >>>>>>>> > >>>>>>>> Could not parse flow as a VersionedDataflow > >>>>>>>> > >>>>>>>> > >>>>>>>> When I compare the file to the dummy flow file that ships with > >>>>>>>> minifi. > >>>>>>>> > >>>>>>>> The file that ships with minifi starts with > >>>>>>>> > >>>>>>>> { > >>>>>>>> "encodingVersion": { > >>>>>>>> "majorVersion": 2, > >>>>>>>> "minorVersion": 0 > >>>>>>>> }, > >>>>>>>> > >>>>>>>> > >>>>>>>> But the export from nifi has > >>>>>>>> > >>>>>>>> > {"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec", > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> So the 2 files dont appear to be the same format. > >>>>>>>> > >>>>>>>> Is there some other method of exporting code from nifi that Im > >>>>>>>> missing? > >>>>>>>> > >>>>>>>> >
