Hi everyone,
If there are no objections, I would like to start a voting thread by
tomorrow. So this is the last call to give feedback for FLIP-122.
Cheers,
Jark
On Wed, 1 Apr 2020 at 16:30, zoudan wrote:
> Hi Jark，
> Thanks for the proposal.
> I like the idea that we put the version in ‘connec
Hi Jark，
Thanks for the proposal.
I like the idea that we put the version in ‘connector’ field. That will be
friendly for existing jobs as some of existing connectors may do not contains
‘connector.version’.
Best,
Dan Zou
Hi everyone,
In order to not postpone FLIP-95 further, I include the "removing
Factory#factoryVersion" in this FLIP.
I updated the "Proposed Changes" section to reflect the changes.
https://cwiki.apache.org/confluence/display/FLINK/FLIP-122%3A+New+Connector+Property+Keys+for+New+Factory
Please l
Hi, Dawid
Regarding to `connector.property-version`,
I totally agree with you we should implicitly add a "property-version=1"
(without 'connector.' prefix) property for future evolving. So I updated
FLIP for this.
However, I still doubt to use property version to distinguish old/new
factory. Becau
Hi Jark,
Thanks for the proposal. I'm +1 since it's more simple and clear for sql
users.
I have a question about this: does this affect descriptors and related
validators?
*Best Regards,*
*Zhenghua Gao*
On Mon, Mar 30, 2020 at 2:02 PM Jark Wu wrote:
> Hi everyone,
>
> I want to start a discus
Thanks for the update, Jark. +1 for your proposal.
Some minor feedback from my side:
sink.bulk-flush -> sink.buffer-flush?
zookeeper.znode.parent -> zookeeper.znode-parent?
for consistency.
username -> secrect.username?
password -> secrect.password?
How about we prefix options that should not
Hi,
As for the `connector.property-version`. I have doubts that having
deprecated keys mechanism is enough to fully support backwards
compatibility. I do think we are changing not just the key names, but
structure of the keys as well as the discovery mechanism and I think it
is really hard to fore
Hi everyone,
Thanks for the great feedbacks so far.
I updated the FLIP documentation according to the discussion. Changes
include:
- remove "version" key, and merge it into "connector"
- add "scan", "lookup", "sink" prefix to some property keys if they are
only used in that case.
- add a "New Pro
Hi Kurt,
I also prefer "-" as version delimiter now. I didn't remove the "_"
proposal by mistake, that's why I sent another email last night :)
Regarding to "property-version", I also think we shouldn't let users to
learn about this. And ConfigOption provides a good ability
to support deprecated k
Thanks Jark for bring up this discussion, +1 for this idea, I believe the user
has suffered from the verbose property key for long time.
Just one question, how do we handle the keys with wildcard, such as the
“connector.properties.*” in Kafka connector which would then hand-over to Kafka
client
Regarding to "property-version", even this is brought up by myself, but I'm
actually not a fan of this.
As a user, I think this property is too much for me to learn, and I
probably won't use this anyway.
If framework can follow a good procedure to upgrade properties, show some
error messages (but s
Regarding to version delimiter, I'm in favor of using "-", not "_". Not
only because we are using
"-" for config keys in other module, like CoreOptions, DeploymentOptions,
but also "-" is easier
to type than "_" which doesn't need to press "shift" ;-)
Best,
Kurt
On Tue, Mar 31, 2020 at 12:37 AM
Hi all,
Thanks for the feedbacks.
It seems that we have a conclusion to put the version into the factory
identifier. I'm also fine with this.
If we have this outcome, the interface of Factory#factoryVersion is not
needed anymore, this can simplify the learning cost of new factory.
We may need to
Hi all,
Thanks for the feedbacks.
It seems that we have a conclusion to put the version into the factory
identifier. I'm also fine with this.
If we have this outcome, the interface of Factory#factoryVersion is not
needed anymore, this can simplify the learning cost of new factory.
We may need to
Hi all,
I like the overall design of the FLIP.
As for the withstanding concerns. I kind of like the approach to put the
version into the factory identifier. I think it's the cleanest way to
say that this version actually applies to the connector itself and not
to the system it connects to. BTW, I
Hi Jark,
thanks for the FLIP. I don't have a strong opinion on
DynamicTableFactory#factoryVersion() for distiguishing connector
versions. We can also include it in the factory identifier itself. For
Kafka, I would then just use `kafka` because the naming "universal" was
just a helper solution
Hi Jark,
Thanks for starting the discussion. The FLIP LGTM generally.
Regarding connector version VS put version into connector's name,
I favor the latter personally, using one property to locate a connector can
make the error log more precise.
>From the users' side, using one property to match a
> It's not possible for the framework to throw such exception. Because the
framework doesn't know what versions do the connector support.
Not really, we can list all valid connectors framework could found. E.g.
user mistyped 'kafka-0.x', the error message will looks like:
we don't have any connec
Hi Kurt,
> 2) Lists all available connectors seems also quite straightforward, e.g
user provided a wrong "kafka-0.8", we tell user we have candidates of
"kakfa-0.11", "kafka-universal"
It's not possible for the framework to throw such exception. Because the
framework doesn't know what versions do
Thanks Jark for the proposal.
+1 to the general idea.
For "version", what about "kafka.version"? It is obvious to know its
meaning.
And I'd like to start a new topic:
Should we need to explicitly separate source from sink?
With the development of batch and streaming, more and more connectors hav
> 1) it's more readable to separete them into different keys, because they
are orthogonal concepts.
I'm not sure whether this is more clear, but using one key will be
definitely one line less.
> 2) the planner can give all the availble versions in the exception
message, if user uses a wrong versio
Hi Kurt,
That's good questions.
> the meaning of "version"
There are two versions in the old design. One is property version
"connector.property-version" which can be used for backward compatibility.
The other one is "connector.version" which defines the version of external
system, e.g. 0.11" for
Hi Jark,
Thanks for the proposal, I'm +1 to the general idea. However I have a
question about "version",
in the old design, the version seems to be aimed for tracking property
version, with different
version, we could evolve these step by step without breaking backward
compatibility. But in this
d
Hi Jark,
I am really looking forward to this feature. I think this feature
could simplify flink sql code,and at the same time ,
it could make the developer more easlier to config the flink sql WITH
options.
Now when I am using flink sql to write flink task , sometimes I think the
WITH options is
Hi everyone,
I want to start a discussion about further improve and simplify our current
connector porperty keys, aka WITH options. Currently, we have a
'connector.' prefix for many properties, but they are verbose, and we see a
big inconsistency between the properties when designing FLIP-107.
So
25 matches
Mail list logo
