Russell, To echo Matt’s point, the NiFi UI uses the REST API on every command, and developer tools is a great way to observe this.
Another example using the REST API is Andrew Grande’s “nifi-deploy-api” project
on GitHub [1]. It is a groovy script that can automate the deployment of NiFi
instances and does this using the API. If you run it with the ‘--debug’ flag,
you can see the HTTP endpoints it invokes to accomplish this.
Finally, the REST API is extremely well documented here [2] (also locally in
the installation). It is extensive, but should be able to answer a number of
the questions you asked. For example, to get processor statistics, you would
invoke:
GET /controller/process-groups/{process-group-id}/status
Gets the status for a process group
The status for a process group includes status for all descendent components.
When invoked on the root group with recursive set to true, it will return the
current status of every component in the flow.
When I invoke that endpoint on a simple cluster I have running with a single
Base64EncodeContent processor named “Token Example” on the canvas, I get this
response:
https://ncm.nifi.apache.org:4567/nifi-api/controller/process-groups/root/status
<processGroupStatusEntity>
<revision>
<clientId>392b161b-d9cc-442a-9394-8db9cee33ba7</clientId>
</revision>
<processGroupStatus>
<activeThreadCount>0</activeThreadCount>
<id>d17d7a95-6301-46ff-b003-11c58698bd31</id>
<input>0 / 0 bytes</input>
<name>NiFi Flow</name>
<output>0 / 0 bytes</output>
<processorStatus>
<activeThreadCount>0</activeThreadCount>
<groupId>d17d7a95-6301-46ff-b003-11c58698bd31</groupId>
<id>b642a34d-c18c-4bb1-b836-6bbc9d32f31e</id>
<input>0 / 0 bytes</input>
<name>Token Example</name>
<output>0 / 0 bytes</output>
<read>0 bytes</read>
<runStatus>Invalid</runStatus>
<tasks>0</tasks>
<tasksDuration>00:00:00.000</tasksDuration>
<type>Base64EncodeContent</type>
<written>0 bytes</written>
</processorStatus>
<queued>0 / 0 bytes</queued>
<queuedCount>0</queuedCount>
<queuedSize>0 bytes</queuedSize>
<read>0 bytes</read>
<received>0 / 0 bytes</received>
<sent>0 / 0 bytes</sent>
<statsLastRefreshed>16:09:46 PDT</statsLastRefreshed>
<transferred>0 / 0 bytes</transferred>
<written>0 bytes</written>
</processGroupStatus>
</processGroupStatusEntity>
[1] https://github.com/aperepel/nifi-api-deploy
<https://github.com/aperepel/nifi-api-deploy>
[2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html
Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Jul 8, 2016, at 3:18 PM, Matt Gilman <matt.c.gil...@gmail.com> wrote:
>
> Russell,
>
> I can provide a more detailed response when I'm back in front of my computer
> but thought I'd offer this as a quick suggestion.
>
> All flows use process groups. The blank canvas when you load your nifi
> instance is the root level process group. If you don't know the actual ID,
> you can use the alias 'root' as the path parameter for the process group ID.
>
> Please check out your browser's Dev Tools to see these API's in action. The
> UI uses these API's exclusively.
>
> Also the API's have been completely refactored in the upcoming 1.0.0 release
> to align with the authorization policies of each resource. So the answers to
> your questions will depend on which version your running.
>
> Matt
>
> Sent from my iPhone
>
>> On Jul 8, 2016, at 5:35 PM, Russell Bateman
>> <russell.bate...@perfectsearchcorp.com> wrote:
>>
>> I'm trying to figure out how to use the ReST (nifi-api) interface to
>> accomplish a number of things. First, I've played successfully with it
>> doing simple things like getting configuration, a list of existing
>> processors, and the like. I've even discovered that in
>> controller/history/processors/{processorId} I can use the processor name
>> for {processorId} instead of an intangible or impossible number.
>>
>> What puzzles me is that I need a {process-group-id} to do so many things.
>> I'm not using a process group, my flow is very simple for now. Maybe, think
>> I, I have one process group and I just need its id, but how do I get that?
>> I see no list API for it.
>>
>> The long list of things I'm hoping to do down the road from a UI is to be
>> able to see things like:
>>
>> 1. How many times a processor processed anything?
>> 2. How many flowfiles were processed?
>> 3. How many flowfiles were produced?
>> 4. Can I get this information per processor?
>> 5. How can I tell different instances of the same processor apart?
>> 6. Why can't I see the name I gave a processor in configuration? (For
>> example, I named an instance of GetFile to "Get PDF files from test fodder".
>> 7. How can I get a list of items on the NiFi canvas like processors and
>> relationships.
>> 8. How many times processors or a processor failed?
>> 9. What is the profile of resource usage, like memory in use?
>> 10. What is the profile of processor latency?
>> - flowfiles backed up awaiting processing
>> - flowfiles in other states (?)
>>
>> Any nudge on any of this would be very welcome. NiFi is new enough that
>> there's precious few samples of people using the ReST interface.
>>
>> Thanks
signature.asc
Description: Message signed with OpenPGP using GPGMail
