Hi Saleh, If you could show us the minimal code example of the issue (event classes), I think someone could help you to solve it.
Best regards, Alexey On Mon, Aug 14, 2023 at 9:23 AM <s...@sammar.sa> wrote: > Hi, > > According to this blog post > https://flink.apache.org/2020/04/15/flink-serialization-tuning-vol.-1-choosing-your-serializer-if-you-can/#pojoserializer > The "Must be processed as GenericType" message means that the POJO > serializer will not be used and instead, Kyro will be used. > > I created a simple POJO to test it again with a java Collection but I got > the same message. Disabling generic types throws an exception. > > I'm not sure how to use these types along with the POJO serializer or any > other fast serializer. > > Best regards, > Saleh. > > > > On 14 Aug 2023, at 4:59 AM, liu ron <ron9....@gmail.com> wrote: > > Hi, > > According to the test in [1], I think Flink can recognize Pojo class which > contains java List, so I think you can refer to the related Pojo class > implementation. > > [1] > https://github.com/apache/flink/blob/fc2b5d8f53a41695117f6eaf4c798cc183cf1e36/flink-core/src/test/java/org/apache/flink/api/java/typeutils/PojoTypeExtractionTest.java#L192 > > Best, > Ron > > <s...@sammar.sa> 于2023年8月13日周日 22:50写道： > >> Greetings, >> >> I am working on a project that needs to process around 100k events per >> second and I'm trying to improve performance. >> >> Most of the classes being used are POJOs but have a couple of fields >> using a `java.util` class, either `ArrayList`, `HashSet` or `SortedSet` >> etc. This forces Flink to use Kyro and throw these warnings: >> >> ``` >> class java.util.ArrayList does not contain a setter for field size >> Class class java.util.ArrayList cannot be used as a POJO type because not >> all fields are valid POJO fields, and must be processed as GenericType. >> Please read the Flink documentation on "Data Types & Serialization" for >> details of the effect on performance and schema evolution. >> ``` >> >> ``` >> No fields were detected for class java.util.HashSet so it cannot be used >> as a POJO type and must be processed as GenericType. Please read the Flink >> documentation on "Data Types & Serialization" for details of the effect on >> performance and schema evolution. >> I read through the documentation and stackoverflow and the conclusion is >> that I need to make a TypeInfoFactory and use it inside a TypeInfo >> annotation over my POJO. >> ``` >> >> My question is what do I need to do to get Flink to recognize my classes >> as POJOs and use the POJO serializer for better performance? >> I read through the documentation and stackoverflow and the conclusion is >> that I need to make a TypeInfoFactory and use it inside a TypeInfo >> annotation over my POJO. >> While this seems incredibly tedious and I keep thinking "there must be a >> better way", I would be fine with this solution if I could figure out how >> to do this for the Set types I'm using. >> >> Any help would be appreciated. > > >
