Re:Re: Re: flatmap returns a custom class object

Hi, Tauseef. This error happens after the job is running, so I think at least it proves that there is no problem with the modified code. Now let's focus on the new error. The exception seems that flink can't read the metadata from kafka to get all of the partitions about this topic. Can you ch

Re:RE:Flink OpenSearch Connector - Avro JSON to JSON

Hi, Dale. Thanks for your professional explanation ;) -- Best！ Xuyang 在 2023-11-22 00:39:47，"Dale Lane" 写道： FYI in case it’s relevant for this discussion > I'm not sure what is the ` Avro JSON` means Avro supports two encoding mechanisms – binary encoding, and JSON enco

Re:Re: flatmap returns a custom class object

Hi, Tauseef. I modify you code and the following can work. Can you try that? ``` static class C1 { Metrics metrics; public C1(Metrics metrics) { this.metrics = metrics; } public Metrics getMetrics() { return metrics; } } static class Metrics { List

Re: Increasing maximum number of FlinkDeployments that the Operator can handle

The recipe we used to get this working was increasing kubernetes.operator.reconcile.interval and kubernetes.operator.observer.progress-check.interval which essentially made reconciliation slower but more smooth for applies across a large number of bundled FlinkDeployments. We also bumped kubernetes

RE:Flink OpenSearch Connector - Avro JSON to JSON

FYI in case it’s relevant for this discussion > I'm not sure what is the ` Avro JSON` means Avro supports two encoding mechanisms – binary encoding, and JSON encoding. [1] In other words, an Avro record, although normally represented as binary data, can instead be represented as a JSON document

Re: [DISCUSS][FLINK-32993] Datagen connector handles length-constrained fields according to the schema definition by default

Thanks Yubin and Jane for the discussion! +1 to fix this bug, although it's usually used as a test source, it's important to provide the correct behavior for users. for the invalid field length configured by users, I think it's better to raise an error instead of using default value silently. ta

Re:flatmap returns a custom class object

Hi, Tauseef. This is an example to use custom POJO with flatmap[1]. If possible, can you post your code and tag the flink version? [1] https://nightlies.apache.org/flink/flink-docs-master/docs/learn-flink/etl/#flatmap -- Best！ Xuyang At 2023-11-21 22:48:41, "Tauseef Janveka

Re:Flink OpenSearch Connector - Avro JSON to JSON

Hi, Praveen. `OpenSearch supports writing data in the JSON format, but in Flink its default data format is Avro JSON.` I'm not sure what is the ` Avro JSON` means. In Opensearch connector, there are multi formats it supports sucn as "avro", "format" and etc by adding corresponding jars, and th

flatmap returns a custom class object

Dear Team, I am getting the following error while using flatMap. Caused by: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: The return type of function 'defineWorkflow(OtelTransformerJob.java:75)' could not be determined automatically, due to type erasu

Re: [DISCUSS][FLINK-32993] Datagen connector handles length-constrained fields according to the schema definition by default

Hi Yubin, Thanks for driving this discussion. Perhaps a specific example can better illustrate the current issue. Considering the following DDL, f0 will always be generated with a default char length of 100, regardless of char(5), bcause the connector option 'fields.f0.length' is not specified [1