Fyi.

This flip using index as child source option prefix because we may use the
same connector as hybrid child sources.
e.g.

create table hybrid_source(
 f0 varchar,
 f1 varchar,
 f2 bigint
) with(
 'connector'='hybrid',
 'sources'='filesystem,filesystem',
 '0.path' = '/tmp/a.csv',
 '0.format' = 'csv',
 '1.path' = '/tmp/b.csv',
 '1.format' = 'csv'"
);

In this case, we must distinguish the format and path option belonging to
which filesystem connector. But as Timo says, it's not clear. He suggest
another way like this:

CREATE TABLE hybrid_source WITH (
   'sources'='historical;realtime',   -- Config option of type string list
   'historical.connector' = 'filesystem',
   'historical.path' = '/tmp/a.csv',
   'historcal.format' = 'csv',
   'realtime.path' = '/tmp/b.csv',
   'realtime.format' = 'csv'"
)

`sources` option is user-custom name instead of the concrete connector
type. And we use this user-custom name as prefix, and using
prefix.connector to call concrete connector impl.

Reply via email to