Jagdish, I appreciate your help.
It worked though netcat gave "0' output I still changed the grid script to "https" [root@csadatlak-wc-a1t hello-samza]# nc github.com 9418 < /dev/null; echo $? 0 Before I build my Samza job I wanted to confirm if I am able to access my YARN When I start YARN it fails to start, Below are the last few lines from Yarn log LOG : 2017-09-20 19:45:50,586 INFO org.mortbay.log: jetty-6.1.26 2017-09-20 19:45:50,625 INFO org.mortbay.log: Extract jar:file:/home/sanant003c/hello-samza/deploy/yarn/share/hadoop/yarn/hadoop-yarn-common-2.6.1.jar!/webapps/cluster to /tmp/Jetty_127_0_0_1_8088_cluster____wfih6e/webapp 2017-09-20 19:45:50,922 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 2017-09-20 19:45:50,923 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2017-09-20 19:45:50,923 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 2017-09-20 19:45:50,970 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 2017-09-20 19:45:50,971 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2017-09-20 19:45:50,971 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 2017-09-20 19:45:50,972 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 2017-09-20 19:45:50,972 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2017-09-20 19:45:50,972 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 2017-09-20 19:45:50,983 INFO org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@127.0.0.1:8088 2017-09-20 19:45:50,983 INFO org.apache.hadoop.yarn.webapp.WebApps: Web app /cluster started at 8088 2017-09-20 19:45:51,629 INFO org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules 2017-09-20 19:45:51,665 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue 2017-09-20 19:45:51,667 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8033 2017-09-20 19:45:51,670 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2017-09-20 19:45:51,671 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2017-09-20 19:45:51,672 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8033: starting 2017-09-20 19:55:50,040 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler: Release request cache is cleaned up Command line output of Zookeepr and Yarn process Start : [root@csadatlak-wc-a1t hello-samza]# bin/grid start zookeeper EXECUTING: start zookeeper JMX enabled by default Using config: /home/sanant003c/hello-samza/deploy/zookeeper/bin/../conf/zoo.cfg Starting zookeeper ... already running as process 11824. Waiting for zookeeper to start... ^C [root@csadatlak-wc-a1t hello-samza]# bin/grid start yarn EXECUTING: start yarn starting resourcemanager, logging to /home/sanant003c/hello-samza/deploy/yarn/logs/yarn-root-resourcemanager-csadatlak-wc-a1t.sys.comcast.net.out Waiting for resourcemanager to start... Ncat: Connection refused. .Ncat: Connection refused. . Regards, ~Sri -----Original Message----- From: Jagadish Venkatraman [mailto:jagadish1...@gmail.com] Sent: Wednesday, September 20, 2017 1:36 PM To: dev@samza.apache.org Subject: Re: Connection timed out error while installing "Hello Samza" Hi Anantharaman, It's likely that your server blocks the GIT protocol. If so, I'd recommend cloning via HTTPS instead of git. *First, determine if the git protocol is blocked on the server as follows:* Please run: # connects to github default port ➜ ~ nc github.com 9418 < /dev/null; echo $? You should see the output as "0". Any other exit code means that the GIT protocol is blocked. *If the above command gets you a non-zero exit code, please clone using HTTPS as follows:* git clone https://github.com/apache/samza.git Alternately, you can edit the line in the bin/grid script replacing git clone git://git.apache.org/samza.git with git clone https://github.com/apache/samza.git and re-run the same command. Please let us know how it goes! :-) Best, Jagdish On Wed, Sep 20, 2017 at 9:04 AM, Anantharaman, Srinatha (Contractor) < srinatha_ananthara...@comcast.com> wrote: > Yi, > > As you suggested this time I tried to install Samza on clean server > (there is nothing installed on this server) Still I am getting > connection timeout error, Please find below command line outputs. > My System Admin/Dev Ops confirmed that this server does not have any > firewall restrictions > > > [root@csadatlak-wc-a1t ~]# echo $JAVA_HOME > /usr/lib/jvm/java-1.8.0-openjdk [root@csadatlak-wc-a1t ~]# echo $PATH > /usr/lib/jvm/java-1.8.0-openjdk/bin:/usr/lib/jvm/java- > 1.8.0-openjdk/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/ > usr/bin:/usr/local/sbin:/usr/sbin:/root/bin > [root@csadatlak-wc-a1t ~]# clear > [root@csadatlak-wc-a1t ~]# pwd > /root > [root@csadatlak-wc-a1t ~]# cd > [root@csadatlak-wc-a1t ~]# set -o vi > [root@csadatlak-wc-a1t ~]# ^Ct -o vi > [root@csadatlak-wc-a1t ~]# > [root@csadatlak-wc-a1t ~]# > [root@csadatlak-wc-a1t ~]# > [root@csadatlak-wc-a1t ~]# > [root@csadatlak-wc-a1t ~]# cd /home/sanant003c/ [root@csadatlak-wc-a1t > sanant003c]# ll total 4 drwxr-xr-x 7 root root 4096 Sep 20 15:32 > hello-samza [root@csadatlak-wc-a1t sanant003c]# cd hello-samza/ > [root@csadatlak-wc-a1t hello-samza]# ll total 312 > drwxr-xr-x 2 root root 119 Sep 20 15:32 bin > -rw-r--r-- 1 root root 6946 Sep 20 15:32 build.gradle > drwxr-xr-x 2 root root 26 Sep 20 15:32 conf > drwxr-xr-x 3 root root 20 Sep 20 15:32 gradle > -rw-r--r-- 1 root root 898 Sep 20 15:32 gradle.properties > -rwxr-xr-x 1 root root 5080 Sep 20 15:32 gradlew > -rw-r--r-- 1 root root 2404 Sep 20 15:32 gradlew.bat > -rw-r--r-- 1 root root 30449 Sep 20 15:32 LICENSE > -rw-r--r-- 1 root root 9343 Sep 20 15:32 pom.xml > -rw-r--r-- 1 root root 1095 Sep 20 15:32 README-gradle.md > -rw-r--r-- 1 root root 1118 Sep 20 15:32 README.md > drwxr-xr-x 3 root root 17 Sep 20 15:32 src > -rw-r--r-- 1 root root 240217 Sep 20 15:32 wikipedia-raw.json > [root@csadatlak-wc-a1t hello-samza]# bin/grid bootstrap Bootstrapping > the system... > EXECUTING: stop kafka > Kafka is not installed. Run: bin/grid install kafka > EXECUTING: stop yarn > YARN is not installed. Run: bin/grid install yarn > EXECUTING: stop zookeeper > Zookeeper is not installed. Run: bin/grid install zookeeper > EXECUTING: install samza > ~/.samza/download /home/sanant003c/hello-samza Cloning into 'samza'... > fatal: unable to connect to git.apache.org: > git.apache.org[0: 54.84.58.65]: errno=Connection timed out > > [root@csadatlak-wc-a1t hello-samza]# bin/grid install kafka > EXECUTING: install kafka > Downloading kafka_2.11-0.10.1.1.tgz... > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > 100 32.8M 100 32.8M 0 0 614k 0 0:00:54 0:00:54 --:--:-- > 620k > [root@csadatlak-wc-a1t hello-samza]# bin/grid install yarn > EXECUTING: install yarn > Downloading hadoop-2.6.1.tar.gz... > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > 100 187M 100 187M 0 0 717k 0 0:04:28 0:04:28 --:--:-- > 734k > [root@csadatlak-wc-a1t hello-samza]# bin/grid install zookeeper > EXECUTING: install zookeeper > Downloading zookeeper-3.4.3.tar.gz... > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > 100 15.4M 100 15.4M 0 0 727k 0 0:00:21 0:00:21 --:--:-- > 747k > [root@csadatlak-wc-a1t hello-samza]# bin/grid install samza > EXECUTING: install samza > ~/.samza/download /home/sanant003c/hello-samza Cloning into 'samza'... > fatal: unable to connect to git.apache.org: > git.apache.org[0: 54.84.58.65]: errno=Connection timed out > > Kindly help me to install Samza on my UNIX server > > Regards, > ~Sri > > -----Original Message----- > From: Yi Pan [mailto:nickpa...@gmail.com] > Sent: Thursday, September 14, 2017 6:42 PM > To: dev@samza.apache.org > Subject: Re: Connection timed out error while installing "Hello Samza" > > P.S., the attachment you post w/ the previous email does not show up > (Apache mailing list disallow attachments). If would be better if you > send the command line outputs in pure text or html format. > > -Yi > > On Thu, Sep 14, 2017 at 3:34 PM, Yi Pan <nickpa...@gmail.com> wrote: > > > Hi, Anantharaman, > > > > A lot of firewall setup prevents "incoming connections", not > > "outgoing connections" from a box. I.e. if you are downloading > > files, you are establishing outgoing connections and it is fine. > > When you are hosting a server (i.e. ZooKeeper or Kafka), you are > > making an "incoming > connections" > > to the box and will trigger the firewall rules. > > > > Just trying to pin down the problem, could you try on a Linux box > > w/o any firewall rules setup and see whether that works? At least, > > verify that localhost resolves to 127.0.0.1 and try telnet locahost > > 9092 or > 2181? > > > > NOTE: Success in ping simply means that you can access via ICMP, not > > guarantee you can establish TCP connection to the ports via firewall > rules. > > > > -Yi > > > > On Thu, Sep 14, 2017 at 12:33 PM, Anantharaman, Srinatha > > (Contractor) < srinatha_ananthara...@comcast.com> wrote: > > > >> That is a good catch Naveena the server which I am trying to > >> install is an Hadoop edge node, it has kafka broker and Zookeeper > >> client already installed for my Hadoop Does it matter If I am > >> installing it on a separate folder?. My guess Even if there are > >> conflicts on earlier installations it should throw some other error > >> but not the connection time out > >> > >> Coming to your question " it could either related resolving "localhost" > >> or firewall that prevents communication between ports" - how to > >> prove it is because of Firewall issue. I am able to clone Samza > >> files from > Apache git. > >> I can download any external files on this server > >> > >> ~Sri > >> > >> -----Original Message----- > >> From: Navina Ramesh [mailto:nram...@linkedin.com] > >> Sent: Thursday, September 14, 2017 2:51 PM > >> To: dev@samza.apache.org > >> Subject: Re: Connection timed out error while installing "Hello Samza" > >> > >> I wonder if this has anything to do with previous kafka / zookeeper > >> installed on your box. Just for sanity, try clearing > >> /tmp/zookeeeper* and > >> /tmp/kafka* before re-trying those steps. > >> > >> > >> Same as Yi, I strongly suspect issues with your local laptop setup > >> - it could either related resolving "localhost" or firewall that > >> prevents communication between ports. > >> > >> > >> Navina > >> > >> ________________________________ > >> From: Yi Pan <nickpa...@gmail.com> > >> Sent: Thursday, September 14, 2017 11:37:46 AM > >> To: dev@samza.apache.org > >> Subject: Re: Connection timed out error while installing "Hello Samza" > >> > >> Hi, Anantharaman, > >> > >> I just did the same steps as you described in your email and all > >> passed on my box. Hence, I strongly suspect that it is related to > >> your local laptop network setup. > >> > >> Could you post all the command line output when you ran the > >> sequence of commands? > >> > >> -Yi > >> > >> On Thu, Sep 14, 2017 at 11:21 AM, Yi Pan <nickpa...@gmail.com> wrote: > >> > >> > Hi, Ananarath, > >> > > >> > It is very strange that you are seeing this timeout exception > >> > that we do not see. I am trying to follow the exact steps you did > >> > to see whether there is anything broken. I will update you this > >> > afternoon. > >> > > >> > Meanwhile, could you check your hostname setup and firewall > >> > configuration to see whether your local laptop has blocked access > >> > via the public IP address to your laptop? Could you verify that > >> > your localhost is resolved to > >> > 127.0.0.1 and is accessible? > >> > > >> > -Yi > >> > > >> > On Thu, Sep 14, 2017 at 11:18 AM, Anantharaman, Srinatha > >> > (Contractor) < srinatha_ananthara...@comcast.com> wrote: > >> > > >> >> Yi, > >> >> > >> >> Is there any alternate way to install Samza Or solution to the > >> >> connection time out error? > >> >> > >> >> Regards, > >> >> ~Sri > >> >> > >> >> From: Anantharaman, Srinatha (Contractor) > >> >> Sent: Wednesday, September 13, 2017 11:37 AM > >> >> To: dev@samza.apache.org > >> >> Subject: RE: Connection timed out error while installing "Hello > Samza" > >> >> > >> >> > >> >> Yi, > >> >> > >> >> > >> >> > >> >> I am trying to build Samza locally by following the steps > >> >> provided by Navina. > >> >> > >> >> As per those steps kafka will be installed after Zookeeper, I am > >> >> getting Error while starting Zookeeper after it is installed > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> Steps Followed : > >> >> > >> >> > >> >> > >> >> Yes. You can clone apache/samza locally and build it with: > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> cd <samza-workspace> > >> >> > >> >> > >> >> > >> >> gradle -b bootstrap.gradle > >> >> > >> >> > >> >> > >> >> ./gradlew clean build -x test > >> >> > >> >> > >> >> > >> >> ./gradlew publishToMavenLocal ## This publishes a snapshot version > >> of > >> >> the latest apache/samza into your local maven repo > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> Then, head to hello-samza workspace and build again: > >> >> > >> >> > >> >> > >> >> cd <hello-samza-workspace> > >> >> > >> >> > >> >> > >> >> mvn clean package ## This should create a build target > >> >> > >> >> > >> >> > >> >> ./bin/grid install zookeeper > >> >> > >> >> > >> >> > >> >> ./bin/grid start zookeeper > >> >> > >> >> > >> >> > >> >> ./bin/grid install kafka > >> >> > >> >> > >> >> > >> >> ./bin/grid start kafka > >> >> > >> >> > >> >> > >> >> ./bin/grid install yarn > >> >> > >> >> > >> >> > >> >> ./bin/grid start yarn > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> mkdir -p deploy/samza > >> >> > >> >> > >> >> > >> >> tar -xvf ./target/hello-samza-*-SNAPSHOT-dist.tar.gz -C > >> >> deploy/samza > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> NOTE : BTW From the above steps I could not execute " gradle -b > >> >> bootstrap.gradle" since that command does not exists > >> >> > >> >> > >> >> > >> >> Regards, > >> >> > >> >> ~Sri > >> >> > >> >> > >> >> > >> >> -----Original Message----- > >> >> From: Yi Pan [mailto:nickpa...@gmail.com] > >> >> Sent: Tuesday, September 12, 2017 7:13 PM > >> >> To: dev@samza.apache.org<mailto:dev@samza.apache.org> > >> >> Subject: Re: Connection timed out error while installing "Hello > Samza" > >> >> > >> >> > >> >> > >> >> Hi, Anantharam, > >> >> > >> >> > >> >> > >> >> Could you confirm at which step your setup failed? It seems that > >> >> your zookeeper server is running. Could you check to see whether > >> >> your Kafka broker is running? You can either do a telnet > >> >> localhost > >> >> 9092 or do a ps auxww | grep kafka to see whether you got any > >> >> broker > running. > >> >> > >> >> > >> >> > >> >> Sometimes, the Kafka service takes time to start on a single laptop. > >> >> You can just try to run ./bin/grid start kafka > >> >> > >> >> > >> >> > >> >> again to see whether the service is up. > >> >> > >> >> > >> >> > >> >> Thanks! > >> >> > >> >> > >> >> > >> >> -Yi > >> >> > >> >> > >> >> > >> >> On Tue, Sep 12, 2017 at 11:38 AM, Anantharaman, Srinatha > >> >> (Contractor) < > >> >> srinatha_ananthara...@comcast.com<mailto:Srinatha_Anantharam > >> >> a...@comcast.com>> wrote: > >> >> > >> >> > >> >> > >> >> > Find below Zookeeper logs: > >> >> > >> >> > > >> >> > >> >> > 2017-09-12 17:05:19,124 [myid:] - INFO > >> >> > [main:QuorumPeerConfig@101] > >> >> > - > >> >> > >> >> > Reading configuration from: /app/home/eventsvc/samza-git/ > >> >> > >> >> > hello-samza/deploy/zookeeper/bin/../conf/zoo.cfg > >> >> > >> >> > 2017-09-12 17:05:19,133 [myid:] - INFO > >> >> > >> >> > [main:DatadirCleanupManager@78] - autopurge.snapRetainCount > >> >> > set to > >> >> > 3 > >> >> > >> >> > 2017-09-12 17:05:19,134 [myid:] - INFO > >> >> > >> >> > [main:DatadirCleanupManager@79] - autopurge.purgeInterval set > >> >> > to > >> >> > 0 > >> >> > >> >> > 2017-09-12 17:05:19,134 [myid:] - INFO > >> >> > >> >> > [main:DatadirCleanupManager@101] > >> >> > >> >> > - Purge task is not scheduled. > >> >> > >> >> > 2017-09-12 17:05:19,135 [myid:] - WARN > >> >> > [main:QuorumPeerMain@113] - > >> >> > >> >> > Either no config or no quorum defined in config, running in > >> >> > >> >> > standalone mode > >> >> > >> >> > 2017-09-12 17:05:19,161 [myid:] - INFO > >> >> > [main:QuorumPeerConfig@101] > >> >> > - > >> >> > >> >> > Reading configuration from: /app/home/eventsvc/samza-git/ > >> >> > >> >> > hello-samza/deploy/zookeeper/bin/../conf/zoo.cfg > >> >> > >> >> > 2017-09-12 17:05:19,162 [myid:] - INFO > >> >> > [main:ZooKeeperServerMain@95] > >> >> > >> >> > - Starting server > >> >> > >> >> > 2017-09-12 17:05:19,175 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:zookeeper.version=3.4.3-1240972, built on > >> >> > 02/06/2012 > >> >> > 10:48 > >> >> > >> >> > GMT > >> >> > >> >> > 2017-09-12 17:05:19,176 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:host.name=codehdplak-po-r19p.sys.comcast.net > >> >> > >> >> > 2017-09-12 17:05:19,176 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:java.version=1.8.0_31 > >> >> > >> >> > 2017-09-12 17:05:19,177 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:java.vendor=Oracle Corporation > >> >> > >> >> > 2017-09-12 17:05:19,177 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1. > >> >> > >> >> > 8.0.31-2.b13.el7.x86_64/jre > >> >> > >> >> > 2017-09-12 17:05:19,178 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:java.class.path=/app/home/eventsvc/samza-git/ > >> >> > >> >> > hello-samza/deploy/zookeeper/bin/../build/classes:/app/ > >> >> > >> >> > home/eventsvc/samza-git/hello-samza/deploy/zookeeper/bin/../ > >> >> > >> >> > build/lib/*.jar:/app/home/eventsvc/samza-git/hello- > >> >> > >> >> > samza/deploy/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/ > >> >> > >> >> > app/home/eventsvc/samza-git/hello-samza/deploy/zookeeper/ > >> >> > >> >> > bin/../lib/slf4j-api-1.6.1.jar:/app/home/eventsvc/samza- > >> >> > >> >> > git/hello-samza/deploy/zookeeper/bin/../lib/netty-3. > >> >> > >> >> > 2.2.Final.jar:/app/home/eventsvc/samza-git/hello- > >> >> > >> >> > samza/deploy/zookeeper/bin/../lib/log4j-1.2.15.jar:/app/ > >> >> > >> >> > home/eventsvc/samza-git/hello-samza/deploy/zookeeper/bin/../ > >> >> > >> >> > lib/jline-0.9.94.jar:/app/home/eventsvc/samza-git/hello- > >> >> > >> >> > samza/deploy/zookeeper/bin/../zookeeper-3.4.3.jar:/app/home/ > >> >> > >> >> > eventsvc/samza-git/hello-samza/deploy/zookeeper/bin/../ > >> >> > >> >> > src/java/lib/*.jar:/app/home/eventsvc/samza-git/hello- > >> >> > >> >> > samza/deploy/zookeeper/bin/../conf: > >> >> > >> >> > 2017-09-12 17:05:19,178 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:java.library.path=/usr/java/packages/lib/amd64:/ > >> >> > >> >> > usr/lib64:/lib64:/lib:/usr/lib > >> >> > >> >> > 2017-09-12 17:05:19,179 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:java.io.tmpdir=/tmp > >> >> > >> >> > 2017-09-12 17:05:19,179 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:java.compiler=<NA> > >> >> > >> >> > 2017-09-12 17:05:19,180 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:os.name=Linux > >> >> > >> >> > 2017-09-12 17:05:19,180 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:os.arch=amd64 > >> >> > >> >> > 2017-09-12 17:05:19,181 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:os.version=3.10.0-123.20.1.el7.x86_64 > >> >> > >> >> > 2017-09-12 17:05:19,181 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:user.name=root > >> >> > >> >> > 2017-09-12 17:05:19,182 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:user.home=/root > >> >> > >> >> > 2017-09-12 17:05:19,182 [myid:] - INFO [main:Environment@98] > >> >> > - Server > >> >> > >> >> > environment:user.dir=/app/home/eventsvc/samza-git/hello- > >> >> > >> >> > samza/deploy/zookeeper > >> >> > >> >> > 2017-09-12 17:05:19,195 [myid:] - INFO > >> >> > [main:ZooKeeperServer@733] > >> >> > - > >> >> > >> >> > tickTime set to 2000 > >> >> > >> >> > 2017-09-12 17:05:19,195 [myid:] - INFO > >> >> > [main:ZooKeeperServer@742] > >> >> > - > >> >> > >> >> > minSessionTimeout set to -1 > >> >> > >> >> > 2017-09-12 17:05:19,195 [myid:] - INFO > >> >> > [main:ZooKeeperServer@751] > >> >> > - > >> >> > >> >> > maxSessionTimeout set to -1 > >> >> > >> >> > 2017-09-12 17:05:19,213 [myid:] - INFO > >> >> > >> >> > [main:NIOServerCnxnFactory@110] - binding to port > >> >> > 0.0.0.0/0.0.0.0:2181 > >> >> > >> >> > 2017-09-12 17:05:19,240 [myid:] - INFO > >> >> > [main:FileTxnSnapLog@238] - > >> >> > >> >> > Snapshotting: 0x0 to /tmp/zookeeper/version-2/snapshot.0 > >> >> > >> >> > 2017-09-12 17:05:19,673 [myid:] - INFO > >> >> > [NIOServerCxn.Factory:0.0.0.0/ > >> >> > >> >> > 0.0.0.0:2181:NIOServerCnxnFactory@213] - Accepted socket > >> >> > connection > >> >> > >> >> > from > >> >> > >> >> > /0:0:0:0:0:0:0:1:21754 > >> >> > >> >> > 2017-09-12 17:17:40,265 [myid:] - WARN > >> >> > [NIOServerCxn.Factory:0.0.0.0/ > >> >> > >> >> > 0.0.0.0:2181:NIOServerCnxn@349] - caught end of stream > >> >> > exception > >> >> > >> >> > EndOfStreamException: Unable to read additional data from > >> >> > client > >> >> > >> >> > sessionid 0x0, likely client has closed socket > >> >> > >> >> > at org.apache.zookeeper.server.NIOServerCnxn.doIO( > >> >> > >> >> > NIOServerCnxn.java:220) > >> >> > >> >> > at > >> >> > org.apache.zookeeper.server.NIOServerCnxnFactory.run( > >> >> > >> >> > NIOServerCnxnFactory.java:224) > >> >> > >> >> > at java.lang.Thread.run(Thread.java:745) > >> >> > >> >> > 2017-09-12 17:17:40,267 [myid:] - INFO > >> >> > [NIOServerCxn.Factory:0.0.0.0/ > >> >> > >> >> > 0.0.0.0:2181:NIOServerCnxn@1000] - Closed socket connection > >> >> > for client > >> >> > >> >> > /0:0:0:0:0:0:0:1:21754 (no session established for client) > >> >> > >> >> > 2017-09-12 17:18:06,236 [myid:] - INFO > >> >> > [NIOServerCxn.Factory:0.0.0.0/ > >> >> > >> >> > 0.0.0.0:2181:NIOServerCnxnFactory@213] - Accepted socket > >> >> > connection > >> >> > >> >> > from / > >> >> > >> >> > 127.0.0.1:48071 > >> >> > >> >> > 2017-09-12 17:18:06,247 [myid:] - INFO > >> >> > [NIOServerCxn.Factory:0.0.0.0/ > >> >> > >> >> > 0.0.0.0:2181:ZooKeeperServer@838] - Client attempting to > >> >> > establish new > >> >> > >> >> > session at /127.0.0.1:48071 > >> >> > >> >> > 2017-09-12 17:18:06,250 [myid:] - INFO > >> >> > [SyncThread:0:FileTxnLog@199] > >> >> > >> >> > - Creating new log file: log.1 > >> >> > >> >> > 2017-09-12 17:18:06,263 [myid:] - INFO > >> >> > >> >> > [SyncThread:0:ZooKeeperServer@604] > >> >> > >> >> > - Established session 0x15e770d61920000 with negotiated > >> >> > timeout > >> >> > 6000 > >> >> > >> >> > for client /127.0.0.1:48071 > >> >> > >> >> > 2017-09-12 17:18:06,321 [myid:] - INFO [ProcessThread(sid:0 > >> cport:-1):: > >> >> > >> >> > PrepRequestProcessor@617] - Got user-level KeeperException > >> >> > when > >> >> > >> >> > processing sessionid:0x15e770d61920000 type:create cxid:0x5 > >> >> > zxid:0x3 > >> >> > >> >> > txntype:-1 reqpath:n/a Error Path:/brokers > >> >> > Error:KeeperErrorCode = > >> >> > >> >> > NoNode for /brokers > >> >> > >> >> > 2017-09-12 17:18:06,342 [myid:] - INFO [ProcessThread(sid:0 > >> cport:-1):: > >> >> > >> >> > PrepRequestProcessor@617] - Got user-level KeeperException > >> >> > when > >> >> > >> >> > processing sessionid:0x15e770d61920000 type:create cxid:0xb > >> >> > zxid:0x7 > >> >> > >> >> > txntype:-1 reqpath:n/a Error Path:/config > >> >> > Error:KeeperErrorCode = > >> >> > >> >> > NoNode for /config > >> >> > >> >> > 2017-09-12 17:18:06,356 [myid:] - INFO [ProcessThread(sid:0 > >> cport:-1):: > >> >> > >> >> > PrepRequestProcessor@617] - Got user-level KeeperException > >> >> > when > >> >> > >> >> > processing sessionid:0x15e770d61920000 type:create cxid:0x13 > >> >> > zxid:0xc > >> >> > >> >> > txntype:-1 reqpath:n/a Error Path:/admin Error:KeeperErrorCode > >> >> > = > >> >> > >> >> > NoNode for /admin > >> >> > >> >> > 2017-09-12 17:18:06,457 [myid:] - INFO [ProcessThread(sid:0 > >> cport:-1):: > >> >> > >> >> > PrepRequestProcessor@617] - Got user-level KeeperException > >> >> > when > >> >> > >> >> > processing sessionid:0x15e770d61920000 type:create cxid:0x1b > >> >> > zxid:0x11 > >> >> > >> >> > txntype:-1 reqpath:n/a Error Path:/cluster > >> >> > Error:KeeperErrorCode = > >> >> > >> >> > NoNode for /cluster > >> >> > >> >> > 2017-09-12 17:18:06,973 [myid:] - INFO [ProcessThread(sid:0 > >> cport:-1):: > >> >> > >> >> > PrepRequestProcessor@617] - Got user-level KeeperException > >> >> > when > >> >> > >> >> > processing sessionid:0x15e770d61920000 type:setData cxid:0x25 > >> >> > >> >> > zxid:0x15 > >> >> > >> >> > txntype:-1 reqpath:n/a Error Path:/controller_epoch > >> >> > >> >> > Error:KeeperErrorCode = NoNode for /controller_epoch > >> >> > >> >> > 2017-09-12 17:18:07,050 [myid:] - INFO [ProcessThread(sid:0 > >> cport:-1):: > >> >> > >> >> > PrepRequestProcessor@617] - Got user-level KeeperException > >> >> > when > >> >> > >> >> > processing sessionid:0x15e770d61920000 type:delete cxid:0x34 > >> >> > zxid:0x17 > >> >> > >> >> > txntype:-1 reqpath:n/a Error > >> >> > Path:/admin/preferred_replica_election > >> >> > >> >> > Error:KeeperErrorCode = NoNode for > >> >> > /admin/preferred_replica_election > >> >> > >> >> > 2017-09-12 17:18:07,288 [myid:] - INFO [ProcessThread(sid:0 > >> cport:-1):: > >> >> > >> >> > PrepRequestProcessor@617] - Got user-level KeeperException > >> >> > when > >> >> > >> >> > processing sessionid:0x15e770d61920000 type:create cxid:0x3f > >> >> > zxid:0x18 > >> >> > >> >> > txntype:-1 reqpath:n/a Error Path:/brokers > >> >> > Error:KeeperErrorCode = > >> >> > >> >> > NodeExists for /brokers > >> >> > >> >> > 2017-09-12 17:18:07,290 [myid:] - INFO [ProcessThread(sid:0 > >> cport:-1):: > >> >> > >> >> > PrepRequestProcessor@617] - Got user-level KeeperException > >> >> > when > >> >> > >> >> > processing sessionid:0x15e770d61920000 type:create cxid:0x40 > >> >> > zxid:0x19 > >> >> > >> >> > txntype:-1 reqpath:n/a Error Path:/brokers/ids > >> >> > Error:KeeperErrorCode = > >> >> > >> >> > NodeExists for /brokers/ids > >> >> > >> >> > > >> >> > >> >> > > >> >> > >> >> > > >> >> > >> >> > -----Original Message----- > >> >> > >> >> > From: Yi Pan [mailto:nickpa...@gmail.com] > >> >> > >> >> > Sent: Tuesday, September 12, 2017 2:02 PM > >> >> > >> >> > To: dev@samza.apache.org<mailto:dev@samza.apache.org> > >> >> > >> >> > Subject: Re: Connection timed out error while installing > >> >> > "Hello > >> Samza" > >> >> > >> >> > > >> >> > >> >> > Hi, Anantharaman, > >> >> > >> >> > > >> >> > >> >> > Could you post your zookeeper startup logs here? > >> >> > >> >> > > >> >> > >> >> > On Tue, Sep 12, 2017 at 10:21 AM, Anantharaman, Srinatha > >> >> > (Contractor) > >> >> > >> >> > < srinatha_ananthara...@comcast.com<mailto:Srinatha_Anantharam > >> >> a...@comcast.com>> wrote: > >> >> > >> >> > > >> >> > >> >> > > It hangs while bringing up the service > >> >> > >> >> > > > >> >> > >> >> > > [root@codehdplak-po-r19p hello-samza]# pwd > >> >> > >> >> > > /app/home/eventsvc/samza-git/hello-samza > >> >> > >> >> > > [root@codehdplak-po-r19p hello-samza]# ./bin/grid install > >> >> > > zookeeper > >> >> > >> >> > > EXECUTING: install zookeeper > >> >> > >> >> > > Using previously downloaded file /root/.samza/download/ > >> >> > >> >> > > zookeeper-3.4.3.tar.gz [root@codehdplak-po-r19p > >> >> > > hello-samza]# > >> >> > >> >> > > ./bin/grid start zookeeper > >> >> > >> >> > > EXECUTING: start zookeeper > >> >> > >> >> > > JMX enabled by default > >> >> > >> >> > > Using config: > >> >> > >> >> > > /app/home/eventsvc/samza-git/hello-samza/deploy/zookeeper/ > >> >> > >> >> > > bin/../conf/zoo.cfg > >> >> > >> >> > > Starting zookeeper ... STARTED > >> >> > >> >> > > Waiting for zookeeper to start... > >> >> > >> >> > > > >> >> > >> >> > > ^C > >> >> > >> >> > > [root@codehdplak-po-r19p hello-samza]# ./bin/grid start > >> >> > > zookeeper^C > >> >> > >> >> > > [root@codehdplak-po-r19p hello-samza]# ./bin/grid install > >> >> > > kafka > >> >> > >> >> > > EXECUTING: install kafka > >> >> > >> >> > > Using previously downloaded file /root/.samza/download/kafka_2. > >> >> > >> >> > > 11-0.10.1.1.tgz > >> >> > >> >> > > [root@codehdplak-po-r19p hello-samza]# ./bin/grid start > >> >> > > kafka > >> >> > >> >> > > EXECUTING: start kafka > >> >> > >> >> > > Waiting for kafka to start... > >> >> > >> >> > > Ncat: Connection refused. > >> >> > >> >> > > .Ncat: Connection refused. > >> >> > >> >> > > .Ncat: Connection refused. > >> >> > >> >> > > .^C > >> >> > >> >> > > [root@codehdplak-po-r19p hello-samza]# > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > -----Original Message----- > >> >> > >> >> > > From: Navina Ramesh [mailto:nram...@linkedin.com] > >> >> > >> >> > > Sent: Tuesday, September 12, 2017 12:43 PM > >> >> > >> >> > > To: dev@samza.apache.org<mailto:dev@samza.apache.org> > >> >> > >> >> > > Subject: Re: Connection timed out error while installing > >> >> > > "Hello > >> Samza" > >> >> > >> >> > > > >> >> > >> >> > > Yes. You can clone apache/samza locally and build it with: > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > cd <samza-workspace> > >> >> > >> >> > > > >> >> > >> >> > > gradle -b bootstrap.gradle > >> >> > >> >> > > > >> >> > >> >> > > ./gradlew clean build -x test > >> >> > >> >> > > > >> >> > >> >> > > ./gradlew publishToMavenLocal ## This publishes a snapshot > >> version > >> >> of > >> >> > >> >> > > the latest apache/samza into your local maven repo > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > Then, head to hello-samza workspace and build again: > >> >> > >> >> > > > >> >> > >> >> > > cd <hello-samza-workspace> > >> >> > >> >> > > > >> >> > >> >> > > mvn clean package ## This should create a build target > >> >> > >> >> > > > >> >> > >> >> > > ./bin/grid install zookeeper > >> >> > >> >> > > > >> >> > >> >> > > ./bin/grid start zookeeper > >> >> > >> >> > > > >> >> > >> >> > > ./bin/grid install kafka > >> >> > >> >> > > > >> >> > >> >> > > ./bin/grid start kafka > >> >> > >> >> > > > >> >> > >> >> > > ./bin/grid install yarn > >> >> > >> >> > > > >> >> > >> >> > > ./bin/grid start yarn > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > mkdir -p deploy/samza > >> >> > >> >> > > > >> >> > >> >> > > tar -xvf ./target/hello-samza-*-SNAPSHOT-dist.tar.gz -C > >> >> > > deploy/samza > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > After this, you can follow steps in the tutorial to "Run" > >> >> > > the > >> >> > >> >> > > example Samza job. > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > HTH! Let me know if you need further help. > >> >> > >> >> > > > >> >> > >> >> > > Navina > >> >> > >> >> > > > >> >> > >> >> > > ________________________________ > >> >> > >> >> > > From: Anantharaman, Srinatha (Contractor) > >> >> > > <Srinatha_Anantharaman@ > >> >> > >> >> > > comcast.com> > >> >> > >> >> > > Sent: Tuesday, September 12, 2017 9:21:53 AM > >> >> > >> >> > > To: dev@samza.apache.org<mailto:dev@samza.apache.org> > >> >> > >> >> > > Subject: RE: Connection timed out error while installing > >> >> > > "Hello > >> Samza" > >> >> > >> >> > > > >> >> > >> >> > > Navina, > >> >> > >> >> > > > >> >> > >> >> > > Is there any other way we can install Hello Samza? > >> >> > >> >> > > > >> >> > >> >> > > Regards, > >> >> > >> >> > > ~Sri > >> >> > >> >> > > > >> >> > >> >> > > -----Original Message----- > >> >> > >> >> > > From: Navina Ramesh [mailto:nram...@linkedin.com] > >> >> > >> >> > > Sent: Tuesday, September 12, 2017 11:42 AM > >> >> > >> >> > > To: dev@samza.apache.org<mailto:dev@samza.apache.org> > >> >> > >> >> > > Subject: Re: Connection timed out error while installing > >> >> > > "Hello > >> Samza" > >> >> > >> >> > > > >> >> > >> >> > > Ok. I tried again for the "latest" branch in hello-samza and > >> >> > > it > >> >> > >> >> > > still works. > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > > While installing it says "Building samza from master..." > >> >> > >> >> > > > >> >> > >> >> > > It is expected to build from "master" in apache/samza repo. > >> >> > > So, the > >> >> > >> >> > > output line is expected. > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > It is weird that you are unable to connect. Is it possible > >> >> > > you are > >> >> > >> >> > > behind a firewall or something? Can you try to ping " > >> git.apache.org" > >> >> ? > >> >> > >> >> > > Or try the setup on a different box? > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > Navina > >> >> > >> >> > > > >> >> > >> >> > > ________________________________ > >> >> > >> >> > > From: Anantharaman, Srinatha (Contractor) > >> >> > > <Srinatha_Anantharaman@ > >> >> > >> >> > > comcast.com> > >> >> > >> >> > > Sent: Tuesday, September 12, 2017 8:33:30 AM > >> >> > >> >> > > To: dev@samza.apache.org<mailto:dev@samza.apache.org> > >> >> > >> >> > > Subject: RE: Connection timed out error while installing > >> >> > > "Hello > >> Samza" > >> >> > >> >> > > > >> >> > >> >> > > Navina, > >> >> > >> >> > > > >> >> > >> >> > > I tried again but still same error > >> >> > >> >> > > > >> >> > >> >> > > While installing it says "Building samza from master..." > >> >> > >> >> > > > >> >> > >> >> > > But when after I cloned I executed " git checkout latest" > >> >> > >> >> > > > >> >> > >> >> > > Regards, > >> >> > >> >> > > ~Sri > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > -----Original Message----- > >> >> > >> >> > > From: Navina Ramesh [mailto:nram...@linkedin.com] > >> >> > >> >> > > Sent: Tuesday, September 12, 2017 11:10 AM > >> >> > >> >> > > To: dev@samza.apache.org<mailto:dev@samza.apache.org> > >> >> > >> >> > > Subject: Re: Connection timed out error while installing > >> >> > > "Hello > >> Samza" > >> >> > >> >> > > > >> >> > >> >> > > Hi Anantharaman, > >> >> > >> >> > > > >> >> > >> >> > > It looks like a transient connection failure to connect to > >> >> > > Apache's > >> >> > >> >> > > git. I tried on my host and it seems to be working. > >> >> > >> >> > > > >> >> > >> >> > > Can you give it another shot? > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > If it still doesn't work, please let me know if you are > >> >> > > running the > >> >> > >> >> > > command under the "master" or "latest" branch of > samza-hello-samza. > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > Thanks! > >> >> > >> >> > > > >> >> > >> >> > > Navina > >> >> > >> >> > > > >> >> > >> >> > > ________________________________ > >> >> > >> >> > > From: Anantharaman, Srinatha (Contractor) > >> >> > > <Srinatha_Anantharaman@ > >> >> > >> >> > > comcast.com> > >> >> > >> >> > > Sent: Tuesday, September 12, 2017 7:45:37 AM > >> >> > >> >> > > To: dev@samza.apache.org<mailto:dev@samza.apache.org> > >> >> > >> >> > > Subject: Connection timed out error while installing "Hello > Samza" > >> >> > >> >> > > > >> >> > >> >> > > Hi, > >> >> > >> >> > > > >> >> > >> >> > > I am trying to install "Hello Samza" on a single node > >> >> > > Initially I > >> >> > >> >> > > have installed Kafka, Yarn and Zookeeper using bin/grid > >> >> > > install > >> >> > >> >> > > kafka/yarn/zookeeper When I am trying bin/grid bootstrap > >> >> > > getting > >> >> > >> >> > > connection timed out error It also mentions no kafka, yarn > >> >> > > and > >> >> > >> >> > > zookeeper installed > >> >> > >> >> > > > >> >> > >> >> > > Please find below the error message > >> >> > >> >> > > > >> >> > >> >> > > [root@codehdplak-po-r19p bin]# cd .. > >> >> > >> >> > > [root@codehdplak-po-r19p hello-samza]# bin/grid install > >> >> > > kafka > >> >> > >> >> > > EXECUTING: install kafka > >> >> > >> >> > > Using previously downloaded file /root/.samza/download/kafka_2. > >> >> > >> >> > > 11-0.10.1.1.tgz > >> >> > >> >> > > [root@codehdplak-po-r19p hello-samza]# bin/grid install yarn > >> >> > >> >> > > EXECUTING: install yarn > >> >> > >> >> > > Using previously downloaded file > >> >> > >> >> > > /root/.samza/download/hadoop-2.6.1.tar.gz > >> >> > >> >> > > [root@codehdplak-po-r19p hello-samza]# bin/grid install > >> >> > > zookeeper > >> >> > >> >> > > EXECUTING: install zookeeper > >> >> > >> >> > > Using previously downloaded file /root/.samza/download/ > >> >> > >> >> > > zookeeper-3.4.3.tar.gz [root@codehdplak-po-r19p > >> >> > > hello-samza]# > >> >> > >> >> > > bin/grid bootstrap Bootstrapping the system... > >> >> > >> >> > > EXECUTING: stop kafka > >> >> > >> >> > > No kafka server to stop > >> >> > >> >> > > EXECUTING: stop yarn > >> >> > >> >> > > no resourcemanager to stop > >> >> > >> >> > > no nodemanager to stop > >> >> > >> >> > > EXECUTING: stop zookeeper > >> >> > >> >> > > JMX enabled by default > >> >> > >> >> > > Using config: > >> >> > >> >> > > /app/home/eventsvc/samza-git/hello-samza/deploy/zookeeper/ > >> >> > >> >> > > bin/../conf/zoo.cfg > >> >> > >> >> > > Stopping zookeeper ... no zookeeper to stop (could not find > >> >> > > file > >> >> > >> >> > > /tmp/zookeeper/zookeeper_server.pid) > >> >> > >> >> > > EXECUTING: install samza > >> >> > >> >> > > Building samza from master... > >> >> > >> >> > > ~/.samza/download /app/home/eventsvc/samza-git/hello-samza > >> >> > >> >> > > Cloning into 'samza'... > >> >> > >> >> > > fatal: unable to connect to git.apache.org: > >> >> > >> >> > > git.apache.org[0: 54.84.58.65]: errno=Connection timed out > >> >> > >> >> > > > >> >> > >> >> > > > >> >> > >> >> > > Could you please help me to resolve this issue? > >> >> > >> >> > > > >> >> > >> >> > > Regards, > >> >> > >> >> > > ~Sri > >> >> > >> >> > > > >> >> > >> >> > > >> >> > >> > > >> > > >> > > > > > -- Jagadish V, Graduate Student, Department of Computer Science, Stanford University
