How do you determine which fields you want to use if you don't know the
names and types beforehand?
I would wrap the GenericRecord in my own type, implements the field
selection logic in hashCode/equals,
and unwrap them again in your functions.
On 14.11.2018 10:57, Gaurav Luthra wrote:
There is a data stream of some records, Lets call them "input records".
Now, I want to partition this data stream by using keyBy(). I want
partitioning based on one or more fields of "input record", But the number
and type of fields are not fixed.
So, Kindly tell me how should I achieve this partitioning based on "input
records" mentioned above?
Note: Technically, I am using Avro's GenericRecord as "input records". Means
I am using DataStream<GenericRecord>, which needs to be partitioned. And its
schema can be different for different jobs. So, I do not know the field
names and types to be provided in keyBy().
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
