Re: Production Deployments

2017-05-12 Thread Ted Yu
This seems related FLINK-5966 On Fri, May 12, 2017 at 4:29 PM, Samuel Doyle wrote: > Is there some documentation describing best practices regarding production > deployments specifically recommended configuration and setup? From what I > have seen, it appears that YARN is the suggested way to ha

Production Deployments

2017-05-12 Thread Samuel Doyle
Is there some documentation describing best practices regarding production deployments specifically recommended configuration and setup? From what I have seen, it appears that YARN is the suggested way to handle a clustered deployment. I am not familiar with YARN so I am hoping that after coming u

Simple Example for Flink Java

2017-05-12 Thread Neelesh Salian
Hi, I have a java program in a jar that parses some Spark Api and does a distcp action. I intend to add a window / interval of 5 mins (arbitrary time) for this jar to execute and do the function. What's the easiest way to make a window and run the above jar in it and make sure it runs every 5 min

Re: JobManager Web UI

2017-05-12 Thread Shravan R
Thanks Shannon. On Thu, May 11, 2017 at 3:37 PM, Shannon Carey wrote: > Since YARN must support people running multiple Flink clusters, the > JobManager web UI binds to an ephemeral port by default (to prevent port > usage conflicts). Also, the AM (and web UI) may be placed on any of the > Yarn

Re: Using Kafka 0.10.x timestamps as a record value in Flink Streaming

2017-05-12 Thread Robert Metzger
Hi Jia, The Kafka 0.10 connector internally relies on the Kafka09 fetcher, but it is extensible / pluggable so that also the Kafka 0.9 Fetcher can read the event timestamps from Kafka 10. We don't expose the timestamp through the deserilaization API, because we set it internally in Flink. (there i

Flink Mesos task management

2017-05-12 Thread Jared Stehler
Is there a way to direct the mesos application master to scale task manager tasks up or down, or to redeploy them? For example, when I want to roll out a new docker image for task managers? -- Jared Stehler Chief Architect - Intellify Learning o: 617.701.6330 x703 signature.asc Description: