Flink 1.6.1 Kerberos configuration

2019-02-22 Thread Marke Builder
Hello, I'm using flink 1.6.1 for streaming. In addition I need access to an storage layer with kerberos auth. . I added the following parameter in the flink-conf.yml security.kerberos.login.use-ticket-cache: true security.kerberos.login.keytab: /.../*.keytab security.kerberos.login.principal: *@*

Re: Stream enrichment with static data, side inputs for DataStream

2019-02-22 Thread Artur Mrozowski
Hi Averell, thank you for your answer. Indeed it is what I did. And it worked just fine in this case. I am sure I'll be facing mor complex scenarios, so it'd be very interesting to hear more about what you try to achieve by using broadcast state. Regards On Thu, Feb 21, 2019 at 11:36 AM Averell

Re: Why don't Tuple types implement Comparable?

2019-02-22 Thread Stephen Connolly
On Thu, 21 Feb 2019 at 18:29, Frank Grimes wrote: > Hi, > > I've recently started to evaluate Flink and have found it odd that its > Tuple types, while Serializable, don't implement java.lang.Comparable. > This means that I either need to provide an KeySelector for many > operations or subtype th

Re: Why don't Tuple types implement Comparable?

2019-02-22 Thread Stephen Connolly
On Fri, 22 Feb 2019 at 10:16, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: > On Thu, 21 Feb 2019 at 18:29, Frank Grimes > wrote: > >> Hi, >> >> I've recently started to evaluate Flink and have found it odd that its >> Tuple types, while Serializable, don't implement java.lang.Compar

Re: Why don't Tuple types implement Comparable?

2019-02-22 Thread Stephen Connolly
On Fri, 22 Feb 2019 at 10:38, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: > > > On Fri, 22 Feb 2019 at 10:16, Stephen Connolly < > stephen.alan.conno...@gmail.com> wrote: > >> On Thu, 21 Feb 2019 at 18:29, Frank Grimes >> wrote: >> >>> Hi, >>> >>> I've recently started to evaluate

Re: Starting Flink cluster and running a job

2019-02-22 Thread Boris Lublinsky
Gordon, I double checked it several times. Here is the list Columns are [info] - Jar-Size including dependencies [info] - Jar-Size [info] - Number of transitive dependencies [info] - Number of direct dependencies [info] - ModuleID [info] Done updating. [info]TotSizeJarSize #TDe #Dep M

Re: SinkFunction.Context

2019-02-22 Thread Durga Durga
Excellent ! Thanks Rong.. On Thu, Feb 21, 2019 at 7:26 PM Rong Rong wrote: > Hi Durga, > > 1. currentProcessingTime: refers to this operator(SinkFunction)'s system > time at the moment of invoke > 1a. the time you are referring to as "flink window got the message" is the > currentProcessingTime

Re: Calculating over multiple streams...

2019-02-22 Thread Michael Latta
You may want to union the 3 streams prior to the process function if they are independently processed. Michael > On Feb 22, 2019, at 9:15 AM, Oytun Tez wrote: > > Hi everyone! > > I've been struggling with an implementation problem in the last days, which I > am almost sure caused by my mi

Re: Jira issue Flink-11127

2019-02-22 Thread Andrey Zagrebin
cc alek...@ververica.com On Fri, Feb 22, 2019 at 1:28 AM Boris Lublinsky < boris.lublin...@lightbend.com> wrote: > Adding metric-query port makes it a bit better, but there is still an error > > > 019-02-22 00:03:56,173 INFO > org.apache.flink.runtime.taskexecutor.TaskExecutor- Could

Re: Calculating over multiple streams...

2019-02-22 Thread Oytun Tez
Restructuring with your tip now, Michael, thank you! --- Oytun Tez *M O T A W O R D* The World's Fastest Human Translation Platform. oy...@motaword.com — www.motaword.com On Fri, Feb 22, 2019 at 11:23 AM Michael Latta wrote: > You may want to union the 3 streams prior to the process function

Re: Jira issue Flink-11127

2019-02-22 Thread Boris Lublinsky
And it works now The problem was that I was setting jobmaneger.rest.address jobmanager.rpc.address that was creating actor system on the local host Although I am still getting the below messages in the job manager periodically, but they seem to be harmless ERROR org.apache.flink.runtime.rest

Re: Flink 1.6.1 Kerberos configuration

2019-02-22 Thread Andrey Zagrebin
Hi Marke, which storage layer causes the problem? Not sure, but some implementations might use different approaches internally and not update ticket automatically or use hadoop/jaas security. Best, Andrey On Fri, Feb 22, 2019 at 9:45 AM Marke Builder wrote: > Hello, > > I'm using flink 1.6.1 f