If there is no more questions, I will start the voting thread tomorrow
On 2023/01/13 14:15:04 Joao Boto wrote:
> Hi flink devs,
>
> I'd like to start a discussion thread for FLIP-287[1].
> This comes from an offline discussion with @Lijie Wang, from FLIP-239[2]
> specially for the sink[3].
>
> B
Hi,
> I think we can get the serializer directly in InitContextImpl through
`getOperatorConfig().getTypeSerializerIn(0,
getUserCodeClassloader()).duplicate()`.
This should work, yes.
+1 to the updated FLIP so far. Thank you, Joao, for being on top of this!
Thanks,
Gordon
On Tue, May 30, 2023 a
Hi Lijie,
I left the two options to use whatever you want, but I can clean the FLIP to
have only one..
Updated the FLIP
Regards
On 2023/05/23 07:23:45 Lijie Wang wrote:
> Hi Joao,
>
> I noticed the FLIP currently contains the following 2 methods about type
> serializer:
>
> (1) TypeSerializ
Hi Joao,
I noticed the FLIP currently contains the following 2 methods about type
serializer:
(1) TypeSerializer createInputSerializer();
(2) TypeSerializer createSerializer(TypeInformation inType);
Is the method (2) still needed now?
Best,
Lijie
João Boto 于2023年5月19日周五 16:53写道：
> Updated
Updated the FLIP to use this option.
Hi,
+1 for `InitContext#createInputSerializer()` .
I think we can get the serializer directly in InitContextImpl through
`getOperatorConfig().getTypeSerializerIn(0,
getUserCodeClassloader()).duplicate()`.
Best,
Lijie.
João Boto 于2023年4月24日周一 19:04写道：
> Hi @Gordon,
>
> `InitContext#createInput
Hello Joao Boto,
Can you please add Gordon’s suggested approach as a third option. Also, it will
be great to list the pros and cons of each approach in the FLIP
Thanks,
Raman Verma
On 2023/04/24 11:04:26 João Boto wrote:
> Hi @Gordon,
>
> `InitContext#createInputSerializer()` is a great option
Hi @Gordon,
`InitContext#createInputSerializer()` is a great option and will solve more
than one problem, but I cant find a way to get the TypeInformation on
InitContextImpl (I can be missing something)
On current (legacy) implementations we rely on interface ´
InputTypeConfigurable´ to get th
Do we have to introduce `InitContext#createSerializer(TypeInformation)`
which returns TypeSerializer, or is it sufficient to only provide
`InitContext#createInputSerializer()` which returns TypeSerializer?
I had the impression that buffering sinks like JDBC only need the
latter. @Joao, could you c
Good point! @Gordon
Introducing an `InitContext#createSerializer(TypeInformation)` looks a
better option to me, so we do not need to introduce an unmodifiable
`ExecutionConfig` at this moment.
Hope that we can make `ExecutionConfig` a read-only interface in
Flink 2.0. It is exposed in `RuntimeCont
Hi,
Sorry for chiming in late.
I'm not so sure that exposing ExecutionConfig / ReadExecutionConfig
directly through Sink#InitContext is the right thing to do.
1. A lot of the read-only getter methods on ExecutionConfig are irrelevant
for sinks. Expanding the scope of the InitContext interface wi
We can let the `InitContext` return `ExecutionConfig` in the interface.
However, a `ReadableExecutionConfig` implementation should be returned
so that exceptions will be thrown if users tries to modify the
`ExecutionConfig`.
We can rework all the setters of `ExecutionConfig` to internally invoke a
Hi Zhu,
Thanks for you time for reviewing this.
Extending ´ExecutionConfig´ will allow to modify the values in the config (this
is what we want to prevent with Option2)
To extend the ExecutionConfig is not simpler to do Option1 (expose
ExecutionConfig directly).
Regards
On 2023/04/03 09:42
Hi João,
Thanks for creating this FLIP!
I'm overall +1 for it to unblock the migration of sinks to SinkV2.
Yet I think it's better to let the `ReadableExecutionConfig` extend
`ExecutionConfig`, because otherwise we have to introduce a new method
`TypeInformation#createSerializer(ReadableExecution
I have update the FLIP with the 2 options that we have discussed..
Option 1: Expose ExecutionConfig directly on InitContext
this have a minimal impact as we only have to expose the new methods
Option 2: Expose ReadableExecutionConfig on InitContext
with this option we have more impact as we need
Hi Konstantin,
I checked the usage of ExecutionConfig in Kinesis and KafKa sinks:
- Kinesis sink: ExecutionConfig is not used by Kinesis sink. The one that
uses getAutoWatermarkInterval is the Kinesis Source, Joao may have made a
mistake.
- Kafka sink: isObjectReuseEnabled and ExecutionConfig are
Hi everyone,
if I am not mistaken of the sinks mentioned by Joao Kafka, Kinesis &
Kinesis already use the Sink2 API. How were those implemented without
exposing the ExecutionConfig?
Best,
Konstantin
Am Mi., 1. Feb. 2023 um 12:28 Uhr schrieb Lijie Wang <
wangdachui9...@gmail.com>:
> +1 for Opt
+1 for Option 2, if we can abstract an "ReadableExecutionConfig"
interface(contains all is/get mehtod), and let ExecutionConfig implements
ReadableExecutionConfig
Best,
Lijie
João Boto 于2023年1月17日周二 20:39写道：
> Hi all,
>
> As establish a read-only contract seems to be consensual approach, talkin
Hi all,
As establish a read-only contract seems to be consensual approach, talking to
Lijie we saw two ways for doing this..
Option 1: UnmodifiableExecutionConfig that extends ExecutionConfig (just like
the UnmodifiableConfiguration)
Pros:
- we have all the get methods
- don't need to change T
Hi Joao,
Thanks for driving this FLIP, +1 for exposing a read-only ExecutionConfig,
users/developers are not expected to modify the ExecutionConfig in Sink.
Can we directly introduce the read-only ExecutionConfig in this FLIP? No
need for a separate FLIP, because currently it will only be used in
Hi Jing Ge,
Thanks for your response..
Making the review left above about all connectors, I realise that we need the
full ExecutionConfig as it is needed to generate the serializer correctly if
objectReuse is enabled as we call
TypeInformation.createSerializer(ExecutionConfig config)
On the Po
Hi all,
After an offline talk with Gunnar I review all connectors on branch
release_1.16 (still has virtually all connectors) we can see some of them using
ExecutionConfig..
The basic use is for creating serializers:
TypeInformation.createSerializer(ExecutionConfig config), and since it is a
F
Hi Joao,
Thanks for bringing this up. Exposing internal domain instances depends on
your requirements. Technically, it is even possible to expose the
RuntimeContext [1] (must be considered very carefully). Since you mentioned
that you only need to know if objectReuse is enabled, how about just exp
Hi Gunnar,
Thanks for your time and response...
I think the problem you want to solve is the exposure of the ExecutionConfig
(that can be mutated) no?
The configuration is not mutated, we only need to know if objectReuse is
enable.
This is already expose on RuntimeContext we think to keep it s
Hey Joao,
Thanks for this FLIP! One question on the proposed interface changes:
is it expected that the configuration is *mutated* via the InitContext
passed to Sink::createWriter()? If that's not the case, how about
establishing a read-only contract representing the current
configuration and pass
Hi flink devs,
I'd like to start a discussion thread for FLIP-287[1].
This comes from an offline discussion with @Lijie Wang, from FLIP-239[2]
specially for the sink[3].
Basically to expose the ExecutionConfig and JobId on SinkV2#InitContext.
This changes are necessary to correct migrate the cur
26 matches
Mail list logo
