OK, I fixed the validation. I had an error in my bootstrap.conf.
Its not ideal that the toolkit transform-nifi generates a invalid flow file.
Now I can get minifi to start. But the code isnt working.
This is a trivial "ping" . It just uses a GenFlowFile to make a flow.
And sends it to a remote process group. The remote process group lives
in another nifi instance, and just logs and discards the flows.
I had to move the code in the source nifi into sub group. And I exported
that. And ran it through the converter.
Inside nifi, its working fine.
But in the minifi, its generating the flows. I can see them piling up in
the WAL. But they arent being transmitted. Theres nothing being received
by the remote nifi.
OK, I fixed the validation. I had an error in my bootstrap.conf.
Its not ideal that the toolkit transform-nifi generates a invalid flow file.
Now I can get minifi to start. But the code isnt working.
This is a trivial "ping" . It just uses a GenFlowFile to make a flow.
And sends it to a remote process group. The remote process group lives
in another nifi instance, and just logs and discards the flows.
I had to move the code in the source nifi into sub group. And I exported
that. And ran it through the converter.
Inside nifi, its working fine.
But in the minifi, its generating the flows. I can see them piling up in
the WAL. But they arent being transmitted. Theres nothing being received
by the remote nifi.
On 8/1/2025 2:33 pm, Robert Cohen wrote:
I googled the error message.
I found a thread here which seems the same
https://community.cloudera.com/t5/Support-Questions/Unable-to-start-MiNiFi-because-of-an-error-caused-due-to-the/m-p/393219
One thing it suggests is editing the flow file and changing the
following setting from 0.
|"maxTimerDrivenThreadCount":5 But when I do that minifi fails to
start as it says the flow file fails validation Does the flow file
contain a checksum? |
On 8/1/2025 1:27 pm, Robert Cohen wrote:
Thanks for that. It clears some things up.
Im not sure if the doco is supposed to be suitable for nifi 1/2 or
just nifi 2.
But you might want to clarify in the doco whether config files are
just used by nifi 1. Just used by nifi 2. Or supported in nifi 2 for
legacy. But not recommended.
Anyway, Ive put my code from a nifi 2.1.0 into minifi 2.1.0.
I get the following error
2025-01-08 13:16:09,065 ERROR [MiNiFi logging handler]
org.apache.nifi.minifi.StdErr Failed to start flow service: Unable to
load flow due to:
org.apache.nifi.controller.serialization.FlowSynchronizationException:
java.lang.IllegalArgumentException: Cannot set max number of threads
to less than 1
I attach the converted flow file
I can't see any obvious parameter related to max number of threads.
On 7/1/2025 6:56 pm, Kedvessy Péter wrote:
[You don't often get email from pkedve...@gmail.com. Learn why this
is important at https://aka.ms/LearnAboutSenderIdentification ]
Hi Robert,
Thanks for reporting the documentation issues. I filed a JIRA to fix
them:
https://issues.apache.org/jira/browse/NIFI-14093
In terms of security configuration, you can follow the following guide:
https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#security-properties-in-bootstrapconf
About the different configuration files you mentioned:
- bootstrap.conf is the main configuration file what we should update
- minifi.properties is a generated properties file based on the
bootstrap.conf
- nifi.properties is the legacy configuration file of minifi
- config.yaml is the legacy flow configuration, now we have the json
one
what you have tried to import previously
I hope it helps to set up your minfi agent.
Regards,
Peter
Robert Cohen <robert.co...@anu.edu.au.invalid> ezt írta (időpont: 2025.
jan. 7., K, 6:19):
I'm having a certain amount of trouble getting good info on
administration of minifi 2.
The README file says thats documentation can be found at
https://nifi.apache.org/minifi
and
https://cwiki.apache.org/confluence/display/NIFI/MiNiFi
neither of which URL works.
Im assuming the real documentation is at
https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#config-file
But Im having some Im finding it a bit confusing.
I want to configure the keystore/truststore and sensitive prop
passwords.
It appears I can configure them either in bootstrap.conf or
minifi.properties. Is either to be prefered over the other?
The documention mentions nifi.properties and config.yml. But doesnt
mention minifi.properties.
Im assuming the mentions of config.yml are leftover from minifi 1
documentation. Or does minifi 2 use it for backwards compatability.
And I assume nifi.properties has been renamed to minifi.properties and
the documentation hasnt caught up.
Actually dumping strings out of the jar files, I can find instances of
nifi.properties. But I cant find any references to minifi.properties.
However the tarball contains a minifi.properties file but not a
nifi.properties file.
So Im really not sure what is going on.
On 6/1/2025 4:40 pm, Robert Cohen wrote:
Ahh, it appears the functionality is there. Its just that the README
file hasnt been updated,
Theres now a transform-nifi command in addition to the transform-yml
command.
On 6/1/2025 2:59 pm, Robert Cohen wrote:
[You don't often get email from robert.co...@anu.edu.au.invalid.
Learn why this is important at
https://aka.ms/LearnAboutSenderIdentification ]
I had a look at the minifi toolkit.
However, it only appears to contain functionality for converting
legacy
minifi config to minifi 2 config
Heres an excerpt from the minifi-tookit 2.1.0 README file
Valid commands include:
transform-yml: Transforms legacy MiNiFi flow config YAML into
MiNiFi flow config JSON
## Example
- You have an older version of MiNiFi located in
<legacy_minifi_directory>.
- You would like upgrade to the latest version of MiNiFi. You
downloaded
and extracted the latest MiNiFi into <latest_minifi_directory>.
- Run the following command to migrate the flow and the bootstrap
config
```
./config.sh transform-yml <legacy_minifi_directory>/conf/config.yml
<legacy_minifi_directory>/conf/bootstrap.conf
<latest_minifi_directory>/conf/flow.json.raw
<latest_minifi_directory>/conf/bootstrap.conf
```
Is there functionality somewhere to convert a nifi 2.x flow file
to a
minifi 2 flow file,
On 12/12/2024 6:48 pm, Kedvessy Péter wrote:
[You don't often get email from pkedve...@gmail.com. Learn why this
is important at https://aka.ms/LearnAboutSenderIdentification ]
Hi Robert,
Seems like the documentation has not been updated. I'm going to
file
a JIRA
ticket to apply doc update about how we can import NiFi 2
exported flow
files into MiNiFi.
In a nutshell: before moving the json file into minifi, we should
convert
it with minifi toolkit. Related ticket for more details:
https://issues.apache.org/jira/browse/NIFI-12644
Regards,
Peter
Robert Cohen <robert.co...@anu.edu.au.invalid> ezt írta (időpont:
2024.
dec. 12., Cs, 2:11):
Im having trouble getting code into minifi 2 from nifi 2.
The documentation here
https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
says
# Export the dataflow in JSON format.
# Move your new .json file to |minifi/conf|.
# Rename your .json file /flow.json.raw/.
Im using the option in nifi to "download flow definition"
However when I load that into minifi, I get an error message.
java.lang.Exception: Unable to load flow due to:
org.apache.nifi.controller.serialization.FlowSerializationException:
Could not parse flow as a VersionedDataflow
When I compare the file to the dummy flow file that ships with
minifi.
The file that ships with minifi starts with
{
"encodingVersion": {
"majorVersion": 2,
"minorVersion": 0
},
But the export from nifi has
{"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec",
So the 2 files dont appear to be the same format.
Is there some other method of exporting code from nifi that Im
missing?
