Re: Can't call getProducedType on Avro messages with array types

2018-01-05 Thread Aljoscha Krettek
Yes, there is some magic in the KryoSerializer and other serialisers that detect whether the flink-avro dependency is there and then use special TypeSerializers from there. (Specifically, this is AvroUtils which has a default implementation that doesn't do much and a special implementation call

Re: Can't call getProducedType on Avro messages with array types

2018-01-05 Thread Kyle Hamlin
So I just added the dependency but didn't change the getProducedType method and it worked fine. Would you expect that to be the case? On Fri, Jan 5, 2018 at 5:43 PM Aljoscha Krettek wrote: > Yes, that should do the trick. > > > On 5. Jan 2018, at 18:37, Kyle Hamlin wrote: > > I can add that dep

Re: Can't call getProducedType on Avro messages with array types

2018-01-05 Thread Aljoscha Krettek
Yes, that should do the trick. > On 5. Jan 2018, at 18:37, Kyle Hamlin wrote: > > I can add that dependency. So I would replace > > override def getProducedType: TypeInformation[T] = { > TypeExtractor.getForClass(implicitly[ClassTag[T]].runtimeClass.asInstanceOf[Class[T]]) > } > > with somethi

Re: Can't call getProducedType on Avro messages with array types

2018-01-05 Thread Kyle Hamlin
I can add that dependency. So I would replace override def getProducedType: TypeInformation[T] = { TypeExtractor.getForClass(implicitly[ClassTag[T]].runtimeClass.asInstanceOf[Class[T]]) } with something like: override def getProducedType: TypeInformation[T] = { new AvroTypeInfo(classOf[T]) }

Re: Can't call getProducedType on Avro messages with array types

2018-01-04 Thread Aljoscha Krettek
Hi, I think you might be able to use AvroTypeInfo which you can use by including the flink-avro dependencies. Is that an option for you? Best, Aljoscha > On 3. Jan 2018, at 21:34, Kyle Hamlin wrote: > > Hi, > > It appears that Kryo can't properly extract/deserialize Avro array types. I > ha