Having a hard time trying to get my head around how to deploy my Flink programs
to a pre-configured, remote Flink cluster setup.
My Mavenized setup uses Spring Boot (to simplify class path handling and
generate pretty logs) to execute provision a StreamExecutionEnvironment with
Kafka sources and sinks. I can also run this quite effective the standard way
(`java -jar …`). What I’m unclear on is how I might go about distributing this
code to run on an existing Flink cluster setup. Where do I drop the jars? Do I
need to restart Flink to do so?
class AppRunner extends CommandLineRunner {
val log = LoggerFactory.getLogger(classOf[AppRunner])
override def run(args: String*): Unit = {
val env : StreamExecutionEnvironment =
StreamExecutionEnvironment.getExecutionEnvironment
val consumer = …
val producer = ...
val stream = env.addSource(consumer)
stream
…
// Do some stuff
…
.addSink(producer)
env.execute
}
}
…
@SpringBootApplication
object App {
@throws(classOf[Exception])
def main( args: Array[String] ) : Unit = {
SpringApplication.run(classOf[AppRunner], args: _*)
}
}
Try as I might, I couldn’t find any clear instructions on how to do this in the
documentation. The cluster documentation ends with starting it.
https://ci.apache.org/projects/flink/flink-docs-release-0.8/cluster_setup.html#starting-flink
<https://ci.apache.org/projects/flink/flink-docs-release-0.8/cluster_setup.html#starting-flink>
The Wikiedits example doesn’t involve any third party dependencies, so I’m not
clear on how to manage class path for it.
https://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/run_example_quickstart.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/run_example_quickstart.html>
Any help in getting me on the right, preferably best practices path would be
appreciated.
Prez Cannady
p: 617 500 3378
e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org>
GH: https://github.com/opencorrelate <https://github.com/opencorrelate>
LI: https://www.linkedin.com/in/revprez <https://www.linkedin.com/in/revprez>
