Hi everyone, I want to connect to a proprietary data stream, which sends different types of messages (maybe interpreted as a table), intertwined in the stream. Every type of message (or table) can have a different schema, but for each type this schema is known when connecting (i.e., at runtime) and does not change.
I'm new to flink, so I have a few (stupid?) questions about this use case. I have created a custom SourceFunction which produces Rows read from this data stream. Then I use side outputs to split up this stream into multiple DataStream[Row]. Is this the right way to do it? What's the best way to add custom TypeInformation[Row] to each of those streams, so that I can easily map this to a table which can be accessed via the Table API? Or would I rather directly implement a ScanTableSource (I played with this, the SourceFunction approach was easier though)? I believe that Row is the best way to create this kind of schema at runtime, or is there a better alternative? Kind regards Roman
