so i did  register the type with Kryo and the ProtobufSerializer. However I am 
still continuing to see the warnings. is this a bug in Flink?

env.config.registerTypeWithKryoSerializer(Trace.APITrace:: class.java ( 
http://class.java/ ) , ProtobufSerializer:: class.java ( http://class.java/ ) )

val stream: DataStreamSource<Trace.APITrace> = 
env.addSource(FlinkKafkaConsumer(Config.TOPIC_SPANS, ApiTraceSchema(), props))

Sent via Superhuman ( https://sprh.mn/?vip=prash...@astradot.com )

On Sat, Apr 24, 2021 at 8:48 AM, Yun Gao < yungao...@aliyun.com > wrote:

> 
> Hi Prashant,
> 
> 
> I think the warn is given when calling
> 
> 
> return TypeInformation.of(Trace.APITrace:: class. java ( http://class.java/
> ) )
> 
> 
> Currently flink does not have the native support
> for the protobuf types yet[1], thus it would use a
> generic serializer created by kryo.
> 
> 
> This should not affect the rightness of the program
> and should only affect its performance. One possible
> solution might be register custom serializer into the kryo
> serializer framework for protobuf classes, like the example in [2].
> 
> 
> Best,
> Yun
> 
> 
> [1] https:/ / issues. apache. org/ jira/ browse/ FLINK-11333 (
> https://issues.apache.org/jira/browse/FLINK-11333 )
> [2] https:/ / ci. apache. org/ projects/ flink/ flink-docs-stable/ dev/ 
> custom_serializers.
> html (
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/custom_serializers.html
> )
> 
> 
> 
>> ------------------Original Mail ------------------
>> *Sender:* Prashant Deva < prashant@ astradot. com ( prash...@astradot.com )
>> >
>> *Send Date:* Sat Apr 24 11:00:17 2021
>> *Recipients:* User < user@ flink. apache. org ( user@flink.apache.org ) >
>> *Subject:* pojo warning when using auto generated protobuf class
>> 
>>> I am seeing this warning msg when trying to use a custom protobuf
>>> de/serializer with kafka source with auto generated java protobuf class:
>>> 
>>> 
>>> 18:41:31.164 [main] INFO org. apache. flink. api. java. typeutils. 
>>> TypeExtractor
>>> ( http://org.apache.flink.api.java.typeutils.typeextractor/ ) - Class class
>>> com.xx.APITrace cannot be used as a POJO type because not all fields are
>>> valid POJO fields, and must be processed as GenericType. Please read the
>>> Flink documentation on "Data Types & Serialization" for details of the
>>> effect on performance.
>>> 
>>> 
>>> 
>>> here is my serializer. What am i doing wrong?
>>> 
>>> 
>>> class ApiTraceSchema: DeserializationSchema<Trace.APITrace>,
>>> SerializationSchema<Trace.APITrace> {
>>> 
>>> override fun getProducedType(): TypeInformation<Trace.APITrace> {
>>> return TypeInformation.of(Trace.APITrace:: class. java ( http://class.java/
>>> ) )
>>> }
>>> 
>>> override fun deserialize(message: ByteArray): Trace.APITrace {
>>> return Trace.APITrace.parseFrom(message)
>>> }
>>> 
>>> override fun isEndOfStream(nextElement: Trace.APITrace): Boolean {
>>> return false
>>> }
>>> 
>>> override fun serialize(element: Trace.APITrace): ByteArray {
>>> return element.toByteArray()
>>> }
>>> }
>>> 
>> 
>> 
> 
>

Reply via email to