Hi Moon, Sorry for the late reply.
I would like to use bin/zeppelin-daemon.sh, because I want to keep it as the background process while I can do some other work in the container. I also tried to use `nohup bin/zeppelin.sh &` command, but still, once the interpreter process is running in the background, it hangs. You can easily produce the problem by using the official docker image of Zeppelin: https://hub.docker.com/r/apache/zeppelin Step 1: Change the last line of Dockfile to "CMD ["/bin/bash"]" Step 2: Run "bin/zeppelin-daemon.sh start" in the docker container. There might be some docker mechanism I am not aware of. Thank you. Hao On Wed, Apr 10, 2019 at 11:37 AM moon soo Lee <leemoon...@gmail.com> wrote: > Hao Ren, > > It looks like other interpreters are working even if Zeppelin is started > inside the container using "bin/zeppelin-daemon.sh start" command, except > for Spark interpreter. So I guess it is somehow related to Spark. > > One question, do you have any particular reason try to use > bin/zeppelin-daemon.sh script instead of bin/zeppelin.sh inside the > container? > > Thanks, > moon > > On Thu, Apr 4, 2019 at 11:38 PM Hao Ren <inv...@gmail.com> wrote: > >> Here is the ps tree for the two cases. >> And I have no idea why Zeppelin fork some sub processes for STTY when >> Zeppelin is hanging >> >> * Working >> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND >> root 1 0.0 0.0 1144 4 ? Ss 08:24 0:00 >> /dev/init -- /usr/local/bin/bootstrap.sh >> root 7 0.0 0.0 48696 3132 ? S 08:24 0:00 su -l >> h.ren >> h.ren 25 0.0 0.0 20144 3816 ? S 08:25 0:00 \_ -su >> h.ren 279 0.0 0.0 36144 3272 ? R+ 08:27 0:00 \_ >> ps auxf -G1 >> h.ren 79 6.0 1.5 6305268 520520 ? Sl 08:25 0:10 >> /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Dfile.encoding=UTF-8 -Xms1024m >> -Xmx1024m -XX:MaxPermSize=512m -Dlog4j >> h.ren 133 0.0 0.0 11492 3124 ? S 08:27 0:00 \_ >> /bin/bash /app/zeppelin/bin/interpreter.sh -d >> /app/zeppelin/interpreter/spark -c 172.17.0.2 -p 45475 -r : -l >> h.ren 145 0.0 0.0 11492 2112 ? S 08:27 0:00 \_ >> /bin/bash /app/zeppelin/bin/interpreter.sh -d >> /app/zeppelin/interpreter/spark -c 172.17.0.2 -p 45475 -r : >> h.ren 146 110 2.0 14354788 670188 ? Sl 08:27 0:18 >> \_ /usr/lib/jvm/java-8-openjdk-amd64/bin/java -cp >> /app/zeppelin/interpreter/spark/*:/app/zeppelin/lib/in >> >> * Hanging >> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND >> root 1 0.1 0.0 1144 4 ? Ss 12:56 0:00 >> /dev/init -- /usr/local/bin/bootstrap.sh >> root 7 0.0 0.0 48696 3128 ? S 12:56 0:00 su -l >> h.ren >> h.ren 25 0.0 0.0 20152 3832 ? S 12:56 0:00 \_ -su >> h.ren 208 0.0 0.0 36144 3172 ? R+ 12:57 0:00 \_ >> ps auxf -g1 >> h.ren 79 40.2 1.5 6238704 516152 ? Tl 12:57 0:09 >> /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Dfile.encoding=UTF-8 -Xms1024m >> -Xmx1024m -XX:MaxPermSize=512m -Dlog4j >> h.ren 131 0.0 0.0 11492 3088 ? T 12:57 0:00 \_ >> /bin/bash /app/zeppelin/bin/interpreter.sh -d >> /app/zeppelin/interpreter/spark -c 172.17.0.2 -p 37333 -r : -l >> h.ren 143 0.0 0.0 11492 1980 ? T 12:57 0:00 \_ >> /bin/bash /app/zeppelin/bin/interpreter.sh -d >> /app/zeppelin/interpreter/spark -c 172.17.0.2 -p 37333 -r : >> h.ren 144 24.3 0.8 12053996 287476 ? Tl 12:57 0:03 >> \_ /usr/lib/jvm/java-8-openjdk-amd64/bin/java -cp >> /app/zeppelin/interpreter/spark/*:/app/zeppelin/lib/in >> h.ren 205 0.0 0.0 4524 720 ? T 12:57 0:00 >> \_ sh -c stty -icanon min 1 -icrnl -inlcr -ixon < /dev/tty >> h.ren 206 0.0 0.0 6136 708 ? T 12:57 0:00 >> \_ stty -icanon min 1 -icrnl -inlcr -ixon >> >> On Thu, Apr 4, 2019 at 12:45 AM Hao Ren <inv...@gmail.com> wrote: >> >>> Hi, >>> I am testing zeppelin in docker container by using the official image: >>> https://hub.docker.com/r/apache/zeppelin/dockerfile >>> >>> I notice that the Dockfile above is using "bin/zeppelin.sh" to start >>> zeppelin, instead of "bin/zeppelin-daemon start". >>> >>> I am not clear on the difference between the two scripts. So I did some >>> tests. >>> >>> What I did: >>> * remove ENTRYPOINT and CMD from Dockfile >>> * build and run the container >>> * Inside the container terminal, use "bin/zeppelin-daemon.sh start" to >>> start Zeppelin >>> >>> The problem I met: >>> * Zeppelin can be launched >>> * However, when executing "spark.version" in notebook, Zeppelin hangs >>> * http://localhost:8080/#/interpreter does not show anything >>> >>> If I stop the daemon and use "bin/zeppelin.sh" to start Zeppelin, >>> everything works as expected. >>> >>> Further investigation shows that the two scripts seem to run the same >>> command, except "bin/zeppelin-daemon start" using "nohup nice" to run the >>> command in the background. So I tried to run "nohup nice bin/zeppelin.sh", >>> unsurprisingly, the problem mentioned above appears again. >>> >>> Once I bring the process to the foreground, the frozen page works again! >>> >>> Could anyone explain what has happened? I am so confused about this. It >>> might be related to the way background process works in the docker >>> container. I am new to docker and Zeppelin. >>> Any help is highly appreciated. >>> >>> Thank you. >>> >>> -- >>> Hao Ren >>> >>> Software Engineer in Machine Learning @ Criteo >>> >>> Paris, France >>> >> >> >> -- >> Hao Ren >> >> Software Engineer in Machine Learning @ Criteo >> >> Paris, France >> > -- Hao Ren Software Engineer in Machine Learning @ Criteo Paris, France
