Thanks Jiangang for bringing this up, and Steven & Peter for the feedback.
I was part of the preliminary offline discussions before this proposal went
public. So maybe I can help clarify things a bit.
In short, despite the phrase "control mode" might be a bit misleading, what
we truly want to do
Hey Community
Assuming there are 3 groups,
A, B, C
Each group represents a set of data about employees and salaries.
Group A ( 0-20K $)
Group B (20K$ - 50K$)
Group C ( > 50K$)
Is it possible to process stream data from single source containing information
about employees and salaries and split
In flink-es connector 6.*, you can set the socket timeout by implementing a
customized RestClientFactory。 Here is the code snippet.
@Override
public void configureRestClientBuilder(RestClientBuilder restClientBuilder)
{
restClientBuilder
.setRequestConfigCallback(
Hi, Kai,
I think the exception should be thrown from
RetryRejectedExecutionFailureHandler as you configure the
'failure-handler' to 'retry-rejected'. It will retry the action that
failed with EsRejectedExecutionException and throw all other failures.
AFAIK, there is no way to configure the connec
Hi Chris,
Interval Join should clean state which is not joined during interval and you
don't need to set state TTL. (Actually, the states used in interval join are
not exposed out and you cannot set TTL for those state as TTL is only public
for user self-described states.)
The checkpoint size
Thanks for the reply Yun. I strangely don't see any nulls. And infact this
exception comes on the first few records and then job starts processing
normally.
Also, I don't see any reason for Concurrent access to the state in my code.
Could more CPU cores than task slots to the Task Manager be th
Hi, Luck,
I may not fully understand your requirements. If you just want to test
the performance of typical streaming jobs with the Flink, you can
refer to the nexmark[1]. If you just care about the performance
regression of your specific production jobs, I don't know there is
such a framework.
[
Thank you for the reply. I have checked the post you mentioned. The dynamic
config may be useful sometimes. But it is hard to keep data consistent in
flink, for example, what if the dynamic config will take effect when
failover. Since dynamic config is a desire for users, maybe flink can
support it
Thanks Yingjie for the great effort!
This is really helpful to Flink Batch users!
Best,
Jingsong
On Mon, Jun 7, 2021 at 10:11 AM Yingjie Cao wrote:
> Hi devs & users,
>
> The FLIP-148[1] has been released with Flink 1.13 and the final
> implementation has some differences compared with the ini
Hi devs & users,
The FLIP-148[1] has been released with Flink 1.13 and the final
implementation has some differences compared with the initial proposal in
the FLIP document. To avoid potential misunderstandings, I have updated the
FLIP document[1] accordingly and I also drafted another document[2]
--
best,
Zhipeng
Hi Marco,
It seems to me that the imbalance problem and the state is independent for this
issue: the data distribution
is only decided by the KeySelector used. The only limitation for state is that
the keyed state is bind to the
KeySelector used across the tasks. If the imbalance is the root p
Hi Thoms,
Very thanks for reporting the exceptions, and it seems to be not work as
expected to me...
Could you also show us the dag of the job ? And does some operators in the
source task
use multiple-threads to emit records?
Best,
Yun
--Original Mail --
Sen
Hi team,
I want to use avro-confluent to encode the data using SQL but the schema
registered by the encoder hard code the schema name to 'record'. is it
possible to dictate the name?
--
Regards,
Tao
14 matches
Mail list logo
