[
https://issues.apache.org/jira/browse/FLINK-25804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ada Wong updated FLINK-25804:
-----------------------------
Description:
When we use multiple connectors could have class conflicts. This class conflict
can not be solved by shade.
The following is example code.
{code:java}
CREATE TABLE es6 (
user_id STRING,
user_name STRING,
PRIMARYKEY (user_id) NOT ENFORCED
) WITH (
'connector' = 'elasticsearch-6',
'hosts' = 'http://localhost:9200',
'index' = 'users',
'document-type' = 'foo'
);
CREATE TABLE es7 (
user_id STRING,
user_name STRING,
PRIMARYKEY (user_id) NOT ENFORCED
) WITH (
'connector' = 'elasticsearch-7',
'hosts' = 'http://localhost:9200',
'index' = 'users'
);
CREATE TABLE ods (
user_id STRING,
user_name STRING
) WITH (
'connector' = 'datagen'
);
INSERT INTO es6 SELECT user_id, user_name FROM ods;
INSERT INTO es7 SELECT user_id, user_name FROM ods;{code}
Inspired by PluginManager, PluginFileSystemFactory and
ClassLoaderFixingFileSystem class.
Could we create many ClassLoaderFixing* class to avoid class conflict. Such as
ClassLoaderFixingDynamicTableFactory, ClassLoaderFixingSink or
ClassLoaderFixingSinkFunction.
If we use ClassLoader fixing, each call SinkFunction#invoke will switch
ClassLoader by Thread#currentThread()#setContextClassLoader(). Does
setContextClassLoader() has heavy overhead of setContextClassLoader()?
was:
When we use multiple connectors could have class conflicts. This class conflict
can not be solved by shade.
The following is example code.
{code:java}
CREATE TABLE es6 (
user_id STRING,
user_name STRING,
PRIMARYKEY (user_id) NOT ENFORCED
) WITH (
'connector' = 'elasticsearch-6',
'hosts' = 'http://localhost:9200',
'index' = 'users',
'document-type' = 'foo'
);
CREATE TABLE es7 (
user_id STRING,
user_name STRING,
PRIMARYKEY (user_id) NOT ENFORCED
) WITH (
'connector' = 'elasticsearch-7',
'hosts' = 'http://localhost:9200',
'index' = 'users'
);
CREATE TABLE ods (
user_id STRING,
user_name STRING
) WITH (
'connector' = 'datagen'
);
INSERT INTO es6 SELECT user_id, user_name FROM ods;
INSERT INTO es7 SELECT user_id, user_name FROM ods;{code}
Inspired by PluginManager, PluginFileSystemFactory and
ClassLoaderFixingFileSystem class.
Could we create many ClassLoaderFixing* class to avoid class conflict. Such as
ClassLoaderFixingDynamicTableFactory, ClassLoaderFixingSink or
ClassLoaderFixingSinkFunction.
If we use ClassLoader fixing, each call SinkFunction#invoke will switch
ClassLoader by Thread#currentThread()#setContextClassLoader(). Does
setContextClassLoader() has heavy overhead of setContextClassLoader()?
> Loading and running connector code use separated ClassLoader.
> -------------------------------------------------------------
>
> Key: FLINK-25804
> URL: https://issues.apache.org/jira/browse/FLINK-25804
> Project: Flink
> Issue Type: Bug
> Components: API / Core, Connectors / Common, Table SQL / Runtime
> Affects Versions: 1.14.3
> Reporter: Ada Wong
> Priority: Blocker
>
> When we use multiple connectors could have class conflicts. This class
> conflict can not be solved by shade.
> The following is example code.
> {code:java}
> CREATE TABLE es6 (
> user_id STRING,
> user_name STRING,
> PRIMARYKEY (user_id) NOT ENFORCED
> ) WITH (
> 'connector' = 'elasticsearch-6',
> 'hosts' = 'http://localhost:9200',
> 'index' = 'users',
> 'document-type' = 'foo'
> );
> CREATE TABLE es7 (
> user_id STRING,
> user_name STRING,
> PRIMARYKEY (user_id) NOT ENFORCED
> ) WITH (
> 'connector' = 'elasticsearch-7',
> 'hosts' = 'http://localhost:9200',
> 'index' = 'users'
> );
> CREATE TABLE ods (
> user_id STRING,
> user_name STRING
> ) WITH (
> 'connector' = 'datagen'
> );
> INSERT INTO es6 SELECT user_id, user_name FROM ods;
> INSERT INTO es7 SELECT user_id, user_name FROM ods;{code}
>
> Inspired by PluginManager, PluginFileSystemFactory and
> ClassLoaderFixingFileSystem class.
> Could we create many ClassLoaderFixing* class to avoid class conflict. Such
> as ClassLoaderFixingDynamicTableFactory, ClassLoaderFixingSink or
> ClassLoaderFixingSinkFunction.
> If we use ClassLoader fixing, each call SinkFunction#invoke will switch
> ClassLoader by Thread#currentThread()#setContextClassLoader(). Does
> setContextClassLoader() has heavy overhead of setContextClassLoader()?
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
