2020-03-02 10:30:08 UTC - Konstantinos Papalias: Thanks @Sijie Guo can you see
any pros and cons based on your experience on using the 1st over the 2nd and
vice versa ?Is it better to let Pulsar Server to submit functions into
Kubernetes instead of submitting them independently of the Server ?
----
2020-03-02 12:21:45 UTC - Sérgio Silveira: @Sérgio Silveira has joined the
channel
----
2020-03-03 03:15:51 UTC - John Duffie: @Sijie Guo @yijie I’ve walked through
the source for the connector. I have been unable to locate the solution to my
simple use case and suspect I simply misunderstand the basics.
What I need is a FlinkPulsarSource that produces an Avro GenericRecord
based on the message bytes and the schema from that same message.
What I see in the code are :
• FlinkPulsarSource that must know at startup the message schema
• FlinkPulsarRowSource - that generates the deserializer on the fly based on
the schema from the message (what I want) but it returns a Flink Row instead of
the intermediate GenericRecord
I see the pattern and suspect I can create a hybrid of the above 2 source
classes. Before I do, is there an alternative solution?
----
2020-03-03 03:20:39 UTC - Sijie Guo: @John Duffie: I see. so you need the
FlinkPulsarSource connector to support Pulsar AUTO_CONSUME schema which
generates GenericRecord. @yijie is that doable in current code base? Or do we
need to enhance the implementation?
----
2020-03-03 03:37:25 UTC - yijie: Ah, decoding to GenericRecord is not provided
currently. But I think both GenericRecord and Row provides similar behavior
that read field based on position? what are the features provided in
GenericRecord or what are the desired way to use the record ?
----
