Re: HashMap/HashSet Serialization Issue

Thanks very much. Regards, Charith On Fri, Jan 6, 2017 at 7:44 PM, Chen Qin wrote: > My understanding is HashMap doesn't work with Flink Native serialization > framework, though I might be wrong. > > This might worth reading > ​https://cwiki.apache.org/confluence/display/FLINK/Type+ > System,+T

Re: HashMap/HashSet Serialization Issue

My understanding is HashMap doesn't work with Flink Native serialization framework, though I might be wrong. This might worth reading ​ https://cwiki.apache.org/confluence/display/FLINK/Type+System,+Type+Extraction,+Serialization -Chen​ On Fri, Jan 6, 2017 at 6:06 PM, Charith Wickramarachchi < c

HashMap/HashSet Serialization Issue

Hi All, I am using flink-gelly and using a custom POJO type as the VertexValue/MessageType (I am using the vertex-centric model). The POJO contains HashMap/HashSet as members. While executing the job I am getting following Log message. 17:50:53,582 INFO org.apache.flink.api.java.typeutils.TypeEx

Re: Increasing parallelism skews/increases overall job processing time linearly

Hi! You are right, parallelism 2 should be faster than parallelism 1 ;-) As ChenQin pointed out, having only 2 Kafka Partitions may prevent further scaleout. Few things to check: - How are you connecting the FlatMap and CoFlatMap? Default, keyBy, broadcast? - Broadcast for example would multi

Re: Increasing parallelism skews/increases overall job processing time linearly

Just noticed there are only two partitions per topic. Regardless of how large parallelism set. Only two of those will get partition assigned at most. Sent from my iPhone > On Jan 6, 2017, at 02:40, Chakravarthy varaga > wrote: > > Hi All, > > Any updates on this? > > Best Regards > CVP

Re: failure-rate restart strategy not working?

I think you are right, enabling checkpointing should not override the cluster settings per se. This is probably an unwanted artifact of the was that configuration currently works: Setting explicitly set in the program trump the cluster-defaults (in the config). Since activating checkpointing sets

Re: failure-rate restart strategy not working?

I think I figured it out: the problem is due to Flink's behavior when a job has checkpointing enabled. When the job graph is created, if checkpointing is enabled but a restart strategy hasn't been programmatically configured, Flink changes the job graph's execution config to use the fixed delay

Re: Increasing parallelism skews/increases overall job processing time linearly

Hi All, Any updates on this? Best Regards CVP On Thu, Jan 5, 2017 at 1:21 PM, Chakravarthy varaga < chakravarth...@gmail.com> wrote: > > Hi All, > > I have a job as attached. > > I have a 16 Core blade running RHEL 7. The taskmanager default number of > slots is set to 1. The source is a ka

Re: Does Flink cluster security works in Flink 1.1.4 release?

I think you can also use Kerberos in the standalone mode in 1.1.x, but is is more tricky - you need do a "kinit" on every host where you launch a Flink process. Flink 1.2 has better Kerberos support. On Fri, Jan 6, 2017 at 4:19 AM, Zhangrucong wrote: > Hi Stephan: > > Thanks for your

Re: Som question about Flink stream sql

Hi Yuhong, The design doc is not ready. You can refer FLINK-4679. There are some discussions under the issue. - Jark Wu > 在 2017年1月6日，下午3:08，Hongyuhong 写道： > > Thanks Jark > > Do you have any design documents or instructions about row window? We are > interested in adding supports of win