Hi,
I am using Avro GenericRecord for most of IN/OUT types from my custom
functions. What I have noticed is that default Avro GenericRecord
serializer, also serializes Schema which makes messages very heavy and
hence impacts overall performance.  In my case I already know the schema
before hand and hence can use same to serialize / deserialize records
without schema which should reduce size by great factor. Are there any
guidelines on how this custom serializer can be registered? One way I see
could be to first "enableEnforceKryo()" on StreamingEnvironment and then
register custom serializer for GenericRecord Type as here [1]. However I am
not sure if it's right approach or there is better approach to achieve
same. Also I would need some customization while storing these messages in
state store (RocksDb in my case) which I think can be achieved as here [2].

Please note I can not use POJOs in my case as it's generalized framework on
top of Flink.

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/custom_serializers.html
[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/state/custom_serialization.html

Gagan

Reply via email to