Hi,
this looks actually quite good.
Is the JobManager web interface accessible on port 8081 ?
With the exec mode set " --executionMode LOCAL" there is no
explicit start of a TaskManager required. Its running in the same
JVM as the JobManager.
The Flink Query Interface needs to be started using another main
class: org.apache.flink.client.WebFrontend.
Let me know if you need more help!
Robert
On Sat, Mar 7, 2015 at 6:08 PM, Mihail Vieru
<[email protected]
<mailto:[email protected]>> wrote:
Hey Robert,
the logging level was INFO and I didn't get any errors.
After I switched to DEBUG I get the following output:
2015-03-07 17:46:52 DEBUG MutableMetricsFactory:42 - field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
with annotation
@org.apache.hadoop.metrics2.annotation.Metric(valueName=Time,
about=, value=[Rate of successful kerberos logins and latency
(milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
2015-03-07 17:46:52 DEBUG MutableMetricsFactory:42 - field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
with annotation
@org.apache.hadoop.metrics2.annotation.Metric(valueName=Time,
about=, value=[Rate of failed kerberos logins and latency
(milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
2015-03-07 17:46:52 DEBUG MetricsSystemImpl:220 - UgiMetrics,
User and group related metrics
2015-03-07 17:46:53 DEBUG KerberosName:87 - Kerberos krb5
configuration not found, setting default realm to empty
2015-03-07 17:46:53 DEBUG Groups:180 - Creating new Groups object
2015-03-07 17:46:53 DEBUG NativeCodeLoader:46 - Trying to
load the custom-built native-hadoop library...
2015-03-07 17:46:53 DEBUG NativeCodeLoader:55 - Failed to
load native-hadoop with error:
java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
2015-03-07 17:46:53 DEBUG NativeCodeLoader:56 -
java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2015-03-07 17:46:53 WARN NativeCodeLoader:62 - Unable to
load native-hadoop library for your platform... using
builtin-java classes where applicable
2015-03-07 17:46:53 DEBUG
JniBasedUnixGroupsMappingWithFallback:40 - Falling back to
shell based
2015-03-07 17:46:53 DEBUG
JniBasedUnixGroupsMappingWithFallback:44 - Group mapping
impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
2015-03-07 17:46:53 DEBUG Groups:66 - Group mapping
impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
cacheTimeout=300000
2015-03-07 17:46:53 DEBUG UserGroupInformation:177 - hadoop login
2015-03-07 17:46:53 DEBUG UserGroupInformation:126 - hadoop
login commit
2015-03-07 17:46:53 DEBUG UserGroupInformation:156 - using
local user:UnixPrincipal: vieru
2015-03-07 17:46:53 DEBUG UserGroupInformation:703 - UGI
loginUser:vieru (auth:SIMPLE)
2015-03-07 17:46:53 INFO JobManager:225 -
--------------------------------------------------------------------------------
2015-03-07 17:46:53 INFO JobManager:226 - Starting
JobManager (Version: <unknown>, Rev:84c11e6, Date:02.03.2015
<tel:02.03.2015> @ 14:53:15 CET)
2015-03-07 17:46:53 INFO JobManager:228 - Current user: vieru
2015-03-07 17:46:53 INFO JobManager:229 - JVM: Java
HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.7/24.60-b09
2015-03-07 17:46:53 INFO JobManager:235 - Startup Options:
2015-03-07 17:46:53 INFO JobManager:237 -
-Dlog4j.configuration=file:///home/vieru/dev/flink-from-src/flink/flink-dist/target/flink-0.9-SNAPSHOT-bin/flink-0.9-SNAPSHOT/conf/log4j.properties3
2015-03-07 17:46:53 INFO JobManager:237 - -Dfile.encoding=UTF-8
2015-03-07 17:46:53 INFO JobManager:237 -
-Xbootclasspath/p:/home/vieru/dev/scala-ide/configuration/org.eclipse.osgi/bundles/290/1/.cp/lib/scala-library.jar:/home/vieru/dev/scala-ide/configuration/org.eclipse.osgi/bundles/290/1/.cp/lib/scala-swing.jar:/home/vieru/dev/scala-ide/configuration/org.eclipse.osgi/bundles/290/1/.cp/lib/scala-actors.jar:/home/vieru/dev/scala-ide/configuration/org.eclipse.osgi/bundles/289/1/.cp/lib/scala-reflect.jar
2015-03-07 17:46:53 INFO JobManager:241 - Maximum heap size:
1682 <tel:1682> MiBytes
2015-03-07 17:46:53 INFO JobManager:242 - JAVA_HOME: not set
2015-03-07 17:46:53 INFO JobManager:243 -
--------------------------------------------------------------------------------
2015-03-07 17:46:53 INFO JobManager:774 - Loading
configuration from
/home/vieru/dev/flink-from-src/flink/flink-dist/target/flink-0.9-SNAPSHOT-bin/flink-0.9-SNAPSHOT/conf
2015-03-07 17:46:53 DEBUG GlobalConfiguration:254 - Loading
configuration property: jobmanager.rpc.address, localhost
2015-03-07 17:46:53 DEBUG GlobalConfiguration:254 - Loading
configuration property: jobmanager.rpc.port, 6123
2015-03-07 17:46:53 DEBUG GlobalConfiguration:254 - Loading
configuration property: jobmanager.heap.mb, 256
2015-03-07 17:46:53 DEBUG GlobalConfiguration:254 - Loading
configuration property: taskmanager.heap.mb, 512
2015-03-07 17:46:53 DEBUG GlobalConfiguration:254 - Loading
configuration property: taskmanager.numberOfTaskSlots, -1
2015-03-07 17:46:53 DEBUG GlobalConfiguration:254 - Loading
configuration property: parallelization.degree.default, 1
2015-03-07 17:46:53 DEBUG GlobalConfiguration:254 - Loading
configuration property: jobmanager.web.port, 8081
2015-03-07 17:46:53 DEBUG GlobalConfiguration:254 - Loading
configuration property: webclient.port, 8080
2015-03-07 17:46:53 INFO JobManager:650 - Security is not
enabled. Starting non-authenticated JobManager.
2015-03-07 17:46:53 INFO JobManager:681 - Starting JobManager
2015-03-07 17:46:53 INFO JobManager:684 - Starting
JobManager actor system at localhost:6123
2015-03-07 17:46:53 DEBUG JobManager:690 - Using akka
configuration
Config(SimpleConfigObject({"akka":{"actor":{"default-dispatcher":{"fork-join-executor":{"parallelism-factor":2},"throughput":15},"guardian-supervisor-strategy":"akka.actor.StoppingSupervisorStrategy","provider":"akka.remote.RemoteActorRefProvider"},"daemonic":"on","jvm-exit-on-fatal-error":"off","log-config-on-start":"off","log-dead-letters":"off","log-dead-letters-during-shutdown":"off","loggers":["akka.event.slf4j.Slf4jLogger"],"logging-filter":"akka.event.slf4j.Slf4jLoggingFilter","loglevel":"DEBUG","remote":{"gate-invalid-addresses-for":"5
s","log-remote-lifecycle-events":"off","netty":{"tcp":{"connection-timeout":"100
seconds","hostname":"127.0.0.1","maximum-frame-size":"10485760b","port":6123,"tcp-nodelay":"on","transport-class":"akka.remote.transport.netty.NettyTransport"}},"quarantine-systems-for":"off","startup-timeout":"100
seconds","transport-failure-detector":{"acceptable-heartbeat-pause":"6000
s","heartbeat-interval":"1000
s","threshold":300},"watch-failure-detector":{"acceptable-heartbeat-pause":"100
seconds","heartbeat-interval":"10
seconds","threshold":12}},"serialize-messages":"off","stdout-loglevel":"OFF"}}))
2015-03-07 17:46:53 INFO Slf4jLogger:80 - Slf4jLogger started
2015-03-07 17:46:53 DEBUG EventStream:77 - logger
log1-Slf4jLogger started
2015-03-07 17:46:53 DEBUG EventStream:77 - Default Loggers
started
2015-03-07 17:46:54 INFO Remoting:74 - Starting remoting
2015-03-07 17:46:54 INFO Remoting:74 - Remoting started;
listening on addresses :[akka.tcp://[email protected]:6123
<http://[email protected]:6123>]
2015-03-07 17:46:54 INFO JobManager:710 - Starting
JobManager actor
2015-03-07 17:46:54 INFO BlobServer:83 - Created BLOB server
storage directory
/tmp/blobStore-3ff4969b-e9a1-4ba7-bae9-1c3896b8bc90
2015-03-07 17:46:54 INFO BlobServer:122 - Started BLOB
server at 0.0.0.0:59575 <http://0.0.0.0:59575> - max
concurrent requests: 50 - max backlog: 1000
2015-03-07 17:46:54 DEBUG JobManager:715 - Starting
JobManager process reaper
2015-03-07 17:46:54 INFO JobManager:729 - Starting JobManger
web frontend
2015-03-07 17:46:54 INFO JobManager:110 - Starting
JobManager at akka://flink/user/jobmanager.
2015-03-07 17:46:54 INFO WebInfoServer:113 - Setting up web
info server, using web-root directory
file:/home/vieru/dev/flink-from-src/flink/flink-runtime/target/classes/web-docs-infoserver.
2015-03-07 17:46:54 DEBUG log:70 - Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.eclipse.jetty.util.log)
via org.eclipse.jetty.util.log.Slf4jLog
2015-03-07 17:46:54 DEBUG log:70 - Container
org.eclipse.jetty.server.Server@16f376a3 +
[email protected]:8081
<http://[email protected]:8081> as connector
2015-03-07 17:46:54 DEBUG log:70 - filterNameMap={}
2015-03-07 17:46:54 DEBUG log:70 - pathFilters=null
2015-03-07 17:46:54 DEBUG log:70 - servletFilterMap=null
2015-03-07 17:46:54 DEBUG log:70 -
servletPathMap={/jobsInfo=org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007}
2015-03-07 17:46:54 DEBUG log:70 -
servletNameMap={org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007=org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007}
2015-03-07 17:46:54 DEBUG log:70 - filterNameMap={}
2015-03-07 17:46:54 DEBUG log:70 - pathFilters=null
2015-03-07 17:46:54 DEBUG log:70 - servletFilterMap=null
2015-03-07 17:46:54 DEBUG log:70 -
servletPathMap={/jobsInfo=org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007,
/logInfo=org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298}
2015-03-07 17:46:54 DEBUG log:70 -
servletNameMap={org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007=org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007,
org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298=org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298}
2015-03-07 17:46:54 DEBUG log:70 - filterNameMap={}
2015-03-07 17:46:54 DEBUG log:70 - pathFilters=null
2015-03-07 17:46:54 DEBUG log:70 - servletFilterMap=null
2015-03-07 17:46:54 DEBUG log:70 -
servletPathMap={/jobsInfo=org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007,
/setupInfo=org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137,
/logInfo=org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298}
2015-03-07 17:46:54 DEBUG log:70 -
servletNameMap={org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007=org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007,
org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298=org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298,
org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137=org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137}
2015-03-07 17:46:54 DEBUG log:70 - filterNameMap={}
2015-03-07 17:46:54 DEBUG log:70 - pathFilters=null
2015-03-07 17:46:54 DEBUG log:70 - servletFilterMap=null
2015-03-07 17:46:54 DEBUG log:70 -
servletPathMap={/jobsInfo=org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007,
/setupInfo=org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137,
/logInfo=org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298,
/menu=org.apache.flink.runtime.jobmanager.web.MenuServlet-994616192}
2015-03-07 17:46:54 DEBUG log:70 -
servletNameMap={org.apache.flink.runtime.jobmanager.web.MenuServlet-994616192=org.apache.flink.runtime.jobmanager.web.MenuServlet-994616192,
org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007=org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007,
org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298=org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298,
org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137=org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137}
2015-03-07 17:46:54 DEBUG log:70 - Container
org.eclipse.jetty.server.Server@16f376a3 +
HandlerList@50dafd61 as handler
2015-03-07 17:46:54 DEBUG log:70 - Container
HandlerList@50dafd61 +
ServletContextHandler@593114bf@593114bf/,null as handler
2015-03-07 17:46:54 DEBUG log:70 - Container
HandlerList@50dafd61 + ResourceHandler@25493558 as handler
2015-03-07 17:46:54 DEBUG log:70 - Starting
org.eclipse.jetty.server.Server@16f376a3
2015-03-07 17:46:54 INFO log:55 - jetty-8.0.0.M1
2015-03-07 17:46:54 DEBUG log:70 - Container
org.eclipse.jetty.server.Server@16f376a3 +
qtp176836489{8<=0<=0/254,-1} as threadpool
2015-03-07 17:46:54 DEBUG log:70 - Starting
qtp176836489{8<=0<=0/254,-1}
2015-03-07 17:46:54 DEBUG log:70 - STARTED
qtp176836489{8<=7<=8/254,0}
2015-03-07 17:46:54 DEBUG log:70 - Starting HandlerList@50dafd61
2015-03-07 17:46:54 DEBUG log:70 - Starting
ServletContextHandler@593114bf@593114bf/,null
2015-03-07 17:46:54 DEBUG log:70 - Container
ServletContextHandler@593114bf@593114bf/,null +
SessionHandler@679a7ae9@ as handler
2015-03-07 17:46:54 DEBUG log:70 - Container
ServletHandler@4efb8c22@ +
org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007
as servlet
2015-03-07 17:46:54 DEBUG log:70 - Container
ServletHandler@4efb8c22@ +
org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298
as servlet
2015-03-07 17:46:54 DEBUG log:70 - Container
ServletHandler@4efb8c22@ +
org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137
as servlet
2015-03-07 17:46:54 DEBUG log:70 - Container
ServletHandler@4efb8c22@ +
org.apache.flink.runtime.jobmanager.web.MenuServlet-994616192
as servlet
2015-03-07 17:46:54 DEBUG log:70 - Container
ServletHandler@4efb8c22@ +
[/jobsInfo]=>org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007
as servletMapping
2015-03-07 17:46:54 DEBUG log:70 - Container
ServletHandler@4efb8c22@ +
[/logInfo]=>org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298
as servletMapping
2015-03-07 17:46:54 DEBUG log:70 - Container
ServletHandler@4efb8c22@ +
[/setupInfo]=>org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137
as servletMapping
2015-03-07 17:46:54 DEBUG log:70 - Container
ServletHandler@4efb8c22@ +
[/menu]=>org.apache.flink.runtime.jobmanager.web.MenuServlet-994616192
as servletMapping
2015-03-07 17:46:54 DEBUG log:70 - Container
SessionHandler@679a7ae9@ + ServletHandler@4efb8c22@ as handler
2015-03-07 17:46:54 DEBUG log:70 - Container
SessionHandler@679a7ae9@ +
org.eclipse.jetty.server.session.HashSessionManager@64ad34b0
as sessionManager
2015-03-07 17:46:54 DEBUG log:70 - Starting
SessionHandler@679a7ae9@
2015-03-07 17:46:54 DEBUG log:70 - Starting
org.eclipse.jetty.server.session.HashSessionManager@64ad34b0
2015-03-07 17:46:54 DEBUG log:70 - Container
org.eclipse.jetty.server.Server@16f376a3 +
org.eclipse.jetty.server.session.HashSessionIdManager@a3f075
as sessionIdManager
2015-03-07 17:46:54 DEBUG log:70 - Starting
org.eclipse.jetty.server.session.HashSessionIdManager@a3f075
2015-03-07 17:46:54 DEBUG log:70 - STARTED
org.eclipse.jetty.server.session.HashSessionIdManager@a3f075
2015-03-07 17:46:54 DEBUG log:70 - STARTED
org.eclipse.jetty.server.session.HashSessionManager@64ad34b0
2015-03-07 17:46:54 DEBUG log:70 - Starting
ServletHandler@4efb8c22@
2015-03-07 17:46:54 DEBUG log:70 - filterNameMap={}
2015-03-07 17:46:54 DEBUG log:70 - pathFilters=null
2015-03-07 17:46:54 DEBUG log:70 - servletFilterMap=null
2015-03-07 17:46:54 DEBUG log:70 -
servletPathMap={/jobsInfo=org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007,
/setupInfo=org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137,
/logInfo=org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298,
/menu=org.apache.flink.runtime.jobmanager.web.MenuServlet-994616192}
2015-03-07 17:46:54 DEBUG log:70 -
servletNameMap={org.apache.flink.runtime.jobmanager.web.MenuServlet-994616192=org.apache.flink.runtime.jobmanager.web.MenuServlet-994616192,
org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007=org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007,
org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298=org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298,
org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137=org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137}
2015-03-07 17:46:54 DEBUG log:70 - starting
ServletHandler@4efb8c22@
2015-03-07 17:46:54 DEBUG log:70 - STARTED
ServletHandler@4efb8c22@
2015-03-07 17:46:54 DEBUG log:70 - starting
SessionHandler@679a7ae9@
2015-03-07 17:46:54 DEBUG log:70 - STARTED
SessionHandler@679a7ae9@
2015-03-07 17:46:54 DEBUG log:70 - starting
ServletContextHandler@593114bf@593114bf/,null
2015-03-07 17:46:54 DEBUG log:70 - Starting
org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007
2015-03-07 17:46:54 DEBUG log:70 - STARTED
org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007
2015-03-07 17:46:54 DEBUG log:70 - Starting
org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298
2015-03-07 17:46:54 DEBUG log:70 - STARTED
org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298
2015-03-07 17:46:54 DEBUG log:70 - Starting
org.apache.flink.runtime.jobmanager.web.MenuServlet-994616192
2015-03-07 17:46:54 DEBUG log:70 - STARTED
org.apache.flink.runtime.jobmanager.web.MenuServlet-994616192
2015-03-07 17:46:54 DEBUG log:70 - Starting
org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137
2015-03-07 17:46:54 DEBUG log:70 - STARTED
org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137
2015-03-07 17:46:54 DEBUG log:70 - STARTED
ServletContextHandler@593114bf@593114bf/,null
2015-03-07 17:46:54 DEBUG log:70 - Starting
ResourceHandler@25493558
2015-03-07 17:46:54 DEBUG log:70 - starting
ResourceHandler@25493558
2015-03-07 17:46:54 DEBUG log:70 - STARTED
ResourceHandler@25493558
2015-03-07 17:46:54 DEBUG log:70 - starting HandlerList@50dafd61
2015-03-07 17:46:54 DEBUG log:70 - STARTED HandlerList@50dafd61
2015-03-07 17:46:54 DEBUG log:70 - starting
org.eclipse.jetty.server.Server@16f376a3
2015-03-07 17:46:54 DEBUG log:70 - Starting
[email protected]:8081
<http://[email protected]:8081>
2015-03-07 17:46:54 DEBUG log:70 - Starting
org.eclipse.jetty.server.nio.SelectChannelConnector$1@4b70ca96
2015-03-07 17:46:54 DEBUG log:70 - STARTED
org.eclipse.jetty.server.nio.SelectChannelConnector$1@4b70ca96
2015-03-07 17:46:54 INFO log:55 - Started
[email protected]:8081
<http://[email protected]:8081>
2015-03-07 17:46:54 DEBUG log:70 - STARTED
[email protected]:8081
<http://[email protected]:8081>
2015-03-07 17:46:54 DEBUG log:70 - STARTED
org.eclipse.jetty.server.Server@16f376a3
2015-03-07 17:46:54 INFO WebInfoServer:190 - Started web
info server for JobManager on null:8081
org.eclipse.jetty.server.Server@16f376a3 STOPPED
+-HandlerList@50dafd61 started
+-ServletContextHandler@593114bf@593114bf/,null started
| +-SessionHandler@679a7ae9 started
| +-ServletHandler@4efb8c22 started
|
+-[/jobsInfo]=>org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet-116914007{}
|
+-[/logInfo]=>org.apache.flink.runtime.jobmanager.web.LogfileInfoServlet-243118298{}
|
+-[/setupInfo]=>org.apache.flink.runtime.jobmanager.web.SetupInfoServlet-1420072137{}
|
+-[/menu]=>org.apache.flink.runtime.jobmanager.web.MenuServlet-994616192{}
+-ResourceHandler@25493558 started