Thanks Mark.
I had java.arg.15 setting in bootstrap.conf from NiFi 0.4.x to make putHDFS
processor work with my MapR cluster. If I remove it, putHDFS will fail.
Now with NiFi 0.5.1,
if I set nifi.state.management.embedded.zookeeper.start=false ,
can I point to dedicated external ZooKeeper in conf/state-management.xml
without interfering with java.arg.15 setting that is used by putHDFS?
Thanks
Sumo
> On Mar 14, 2016, at 10:10 AM, Mark Payne <marka...@hotmail.com> wrote:
>
> Sumo,
>
> If your intent is to use an external ZooKeeper, you should not be starting
> the embedded ZooKeeper.
> You will also not want to set the java.arg.15 parameter there to point to a
> login config file, as that is
> necessary only when enabling Kerberos - not for use when security disabled.
>
> So you would want to change the following in your config:
> - Remove the java.arg.15 parameter from bootstrap.conf
> - Set the nifi.state.management.embedded.zookeeper.start property to false
> - Change the conf/state-management.xml to point to the external ZooKeeper via
> the Connect String property.
>
> Does all of this make sense?
>
> Thanks
> -Mark
>
>
>> On Mar 14, 2016, at 12:59 PM, Sumanth Chinthagunta <xmlk...@gmail.com> wrote:
>>
>> I am getting following error when NiFi Cluster started with embedded
>> ZooKeeper enabled.
>> I would like to configure external ZooKeeper for NiFi State Management
>> without interfering with java.security.auth.login.config setting for MapR
>> HDFS. Can somebody provide me instructions how to configure external
>> ZooKeeper for NiFi cluster State Management without security enabled?
>>
>> Env :
>> NiFi 0.5.1 cluster (NCM, Node1, Node2)
>> I have following line in bootstrap.conf
>> # ***For MapR HDFS***
>> java.arg.15=-Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
>>
>> /opt/mapr/conf/mapr.login.conf has
>> /**
>> * Used by Zookeeper
>> */
>> Server {
>> com.mapr.security.maprsasl.MaprSecurityLoginModule required
>> checkUGI=false
>> cldbkeylocation="/opt/mapr/conf/cldb.key"
>> debug=true;
>> };
>>
>> I have following in zookeeper.properties
>> server.2=myhost2:2888:3888
>>
>>
>> 2016-03-12 16:39:07,518 INFO [Framework Task Thread Thread-1]
>> o.a.zookeeper.server.ZooKeeperServer Server environment:java.io.tmpdir=/tmp
>> 2016-03-12 16:39:07,518 INFO [Framework Task Thread Thread-1]
>> o.a.zookeeper.server.ZooKeeperServer Server environment:java.compiler=<NA>
>> 2016-03-12 16:39:07,518 INFO [Framework Task Thread Thread-1]
>> o.a.zookeeper.server.ZooKeeperServer Server environment:os.name=Linux
>> 2016-03-12 16:39:07,518 INFO [Framework Task Thread Thread-1]
>> o.a.zookeeper.server.ZooKeeperServer Server environment:os.arch=amd64
>> 2016-03-12 16:39:07,518 INFO [Framework Task Thread Thread-1]
>> o.a.zookeeper.server.ZooKeeperServer Server
>> environment:os.version=2.6.32-573.3.1.el6.x86_64
>> 2016-03-12 16:39:07,518 INFO [Framework Task Thread Thread-1]
>> o.a.zookeeper.server.ZooKeeperServer Server environment:user.name=sumo
>> 2016-03-12 16:39:07,518 INFO [Framework Task Thread Thread-1]
>> o.a.zookeeper.server.ZooKeeperServer Server environment:user.home=/home/ sumo
>> 2016-03-12 16:39:07,518 INFO [Framework Task Thread Thread-1]
>> o.a.zookeeper.server.ZooKeeperServer Server
>> environment:user.dir=/app/runtime/nifi-node2
>> 2016-03-12 16:39:07,519 INFO [Framework Task Thread Thread-1]
>> o.a.zookeeper.server.ZooKeeperServer tickTime set to 2000
>> 2016-03-12 16:39:07,519 INFO [Framework Task Thread Thread-1]
>> o.a.zookeeper.server.ZooKeeperServer minSessionTimeout set to -1
>> 2016-03-12 16:39:07,519 INFO [Framework Task Thread Thread-1]
>> o.a.zookeeper.server.ZooKeeperServer maxSessionTimeout set to -1
>> 2016-03-12 16:39:07,539 ERROR [Framework Task Thread Thread-1]
>> o.apache.nifi.controller.FlowController NiFi was connected to the cluster
>> but failed to start embedded ZooKeeper Server
>> java.io.IOException: Failed to start embedded ZooKeeper Server
>> at
>> org.apache.nifi.controller.state.server.ZooKeeperStateServer.startStandalone(ZooKeeperStateServer.java:87)
>> ~[na:na]
>> at
>> org.apache.nifi.controller.state.server.ZooKeeperStateServer.start(ZooKeeperStateServer.java:60)
>> ~[na:na]
>> at
>> org.apache.nifi.controller.FlowController$5.run(FlowController.java:3145)
>> ~[na:na]
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> [na:1.8.0_65]
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> [na:1.8.0_65]
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>> [na:1.8.0_65]
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>> [na:1.8.0_65]
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> [na:1.8.0_65]
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> [na:1.8.0_65]
>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
>> Caused by: java.io.IOException: Could not configure server because SASL
>> configuration did not allow the ZooKeeper server to authenticate itself
>> properly: javax.security.auth.login.LoginException: unable to find
>> LoginModule class: com.mapr.security.maprsasl.MaprSecurityLoginModule
>> at
>> org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:207)
>> ~[na:na]
>> at
>> org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:87)
>> ~[na:na]
>> at
>> org.apache.nifi.controller.state.server.ZooKeeperStateServer.startStandalone(ZooKeeperStateServer.java:81)
>> ~[na:na]
>> ... 9 common frames omitted
>>
>> Thanks
>> Sumo
>
