Hi to all, I have a Flink job that produce json objects that I'd like to index in different Elasticsearch indices depending on the "type" attribute of my json object (e.g. "people", "places", etc..). Is there any previous attempt to do something like that in Flink? I was thinking to use the EsHadoopOutputFormat but it requires to specify the index name in the job conf..however, in my use case I'll know the target indices only once the computation finish so Flink can't know how many sinks there will be in the pre-flight phase..
My solution at the moment was to implement my own mapPartition function that instantiate a client to ES and index the json documents in the right index at the end of the job pipeline..is there any better approach to it? Best, Flavio
