Flink doesn’t have built-in support for serializing Sets. See this (stale) issue about the same: https://issues.apache.org/jira/browse/FLINK-16729
You could create a custom serializer for sets, see https://stackoverflow.com/questions/59800851/flink-serialization-of-java-util-list-and-java-util-map and https://nightlies.apache.org/flink/flink-docs-stable/api/java/org/apache/flink/api/common/typeutils/base/ListSerializer.html for details on how this was done for a list, but it’s not trivial. Or as a hack, use a Map<Integer, Void> and the existing support for map serialization via https://nightlies.apache.org/flink/flink-docs-stable/api/java/org/apache/flink/api/common/typeutils/base/MapSerializer.html — Ken > On Mar 20, 2024, at 10:04 AM, Sachin Mittal <sjmit...@gmail.com> wrote: > > Hi, > I have a Pojo class like this > > public class A { > public String str; > public Set<Integer> aSet; > public Map<String, Double> dMap; > } > > However when I start the flink program I get this message: > > org.apache.flink.api.java.typeutils.TypeExtractor [] - Field > A#dMap will be processed as GenericType. Please read the Flink documentation > on "Data Types & Serialization" for details of the effect on performance and > schema evolution. > > org.apache.flink.api.java.typeutils.TypeExtractor [] - Field > A#aSet will be processed as GenericType. Please read the Flink documentation > on "Data Types & Serialization" for details of the effect on performance and > schema evolution. > > Also in my code I have added > env.getConfig().disableGenericTypes(); > So I don't understand when I use Maps and Sets of primitive types why is > Flink not > able to use PojoSerializer for these fields and even when I have disabled > generics types. > why I am getting message that it will be processed as GenericType? > > Any help in understanding what I need to do to ensure all the fields of my > object are handled using PojoSerializer. > > Thanks > Sachin > > -------------------------- Ken Krugler http://www.scaleunlimited.com Custom big data solutions Flink & Pinot
