Re: Error with extracted type from custom partitioner key

2021-06-17 Thread Arvid Heise
This could be a bug but I'd need to see more of the DataStream code to be sure. Could you share that code? On Sat, Jun 12, 2021 at 9:56 PM Ken Krugler wrote: > Hi Timo, > > Thanks, I’ll give the ResultTypeQueryable interface a try - my previous > experience registering custom Kryo serializers wa

Re: Error with extracted type from custom partitioner key

2021-06-12 Thread Ken Krugler
Hi Timo, Thanks, I’ll give the ResultTypeQueryable interface a try - my previous experience registering custom Kryo serializers wasn’t so positive. Though I’m still curious as to whether java.lang.ClassCastException I got was representative of a bug in Flink, or my doing something wrong. But w

Re: Error with extracted type from custom partitioner key

2021-06-04 Thread Timo Walther
Hi Ken, non-POJOs are serialized with Kryo. This might not give you optimal performance. You can register a custom Kryo serializer in ExecutionConfig to speed up the serialization. Alternatively, you can implement `ResultTypeQueryable` provide a custom type information with a custom serializ

Error with extracted type from custom partitioner key

2021-06-04 Thread Ken Krugler
Hi all, I'm using Flink 1.12 and a custom partitioner/partitioning key (batch mode, with a DataSet) to do a better job of distributing data to tasks. The classes look like: public class MyPartitioner implements Partitioner { ... } public class MyGroupingKey implements Comparable { ..