ighack created FLINK-23480:
--
Summary: about proxy-user problem
Key: FLINK-23480
URL: https://issues.apache.org/jira/browse/FLINK-23480
Project: Flink
Issue Type: Improvement
Components: AP
How do you feel about proceeding without an additional vote?
I agree, we added quite a bit in there, but none of the added content
changes what has been voted so far. A big part of the added content can
be considered as clarifying the implementation. I'd be inclined to
proceed without a vote. WDYT
Dawid Wysakowicz created FLINK-23479:
Summary:
IncrementalAggregateJsonPlanTest.testIncrementalAggregateWithSumCountDistinctAndRetraction
fail
Key: FLINK-23479
URL: https://issues.apache.org/jira/browse/FLINK
Dawid Wysakowicz created FLINK-23478:
Summary: KubernetesSharedInformerITCase.testWatchWithBlockHandler
fails on Azure
Key: FLINK-23478
URL: https://issues.apache.org/jira/browse/FLINK-23478
Proje
Harshvardhan created FLINK-23477:
Summary: wasbs/s3 FileSystem support
Key: FLINK-23477
URL: https://issues.apache.org/jira/browse/FLINK-23477
Project: Flink
Issue Type: Bug
Compone
Thanks for sharing the thoughts Chesnay, and I overall agree with you. We
can't give a default value suitable for all jobs, but we can figure out
whether the current default value is too large for most of the jobs, and
that is the guideline for this topic. Configurability is reserved for the
others
退订
Chesnay Schepler created FLINK-23476:
Summary: Snapshot deployments are broken
Key: FLINK-23476
URL: https://issues.apache.org/jira/browse/FLINK-23476
Project: Flink
Issue Type: Bug
I'm wondering if this discussion isn't going in the wrong direction.
It is clear that we cannot support all use-case with the defaults, so
let's not try that. We won't find it.
And I would argue that is also not their purpose; they are configurable
for a reason.
I would say the defaults should p
Hi Guowei,
I think the whole discussion here started as a means to avoid exposing
MailboxExecutor to the user. Your preferred way would be to improve AsyncIO
to support batching or implement AsyncSink as batching+AsyncIO. Here are
some thoughts.
1) We should take a step back and note that we act
Dawid Wysakowicz created FLINK-23475:
Summary: Handle BroadcastState with finished operators
Key: FLINK-23475
URL: https://issues.apache.org/jira/browse/FLINK-23475
Project: Flink
Issue T
Hi Till,
Very thanks for the comments and tips! We will update the FLIP with the new
details and we
should need a new vote since the FLIP is updated a large part, I think we would
start the
vote right after we also solve the remaining UnionListState problem and have a
double
check on that we h
Dawid Wysakowicz created FLINK-23474:
Summary: InputStatus should not contain END_OF_RECOVERY
Key: FLINK-23474
URL: https://issues.apache.org/jira/browse/FLINK-23474
Project: Flink
Issue
Dawid Wysakowicz created FLINK-23473:
Summary: Do not create transaction in TwoPhaseCommitSinkFunction
after finish()
Key: FLINK-23473
URL: https://issues.apache.org/jira/browse/FLINK-23473
Projec
Thanks for your input, team. Good catch, Chesney.
Till, we will address said comments.
All in all now we can close the vote successfully with the binding +1 votes
of Gyula, Konstantin and Till.
On Mon, Jul 12, 2021 at 10:42 AM Till Rohrmann wrote:
> Thanks for starting the vote Marton.
>
> I h
Thanks, Till. There are many reasons to reduce the heartbeat interval and
timeout. But I am not sure what values are suitable. In our cases, the GC
time and big job can be related factors. Since most flink jobs are pipeline
and a total failover can cost some time, we should tolerate some stop-world
Maciej Bryński created FLINK-23472:
--
Summary: Math functions with DECIMAL argument should return
DECIMAL result
Key: FLINK-23472
URL: https://issues.apache.org/jira/browse/FLINK-23472
Project: Flink
Yun Tang created FLINK-23471:
Summary: Try best to ensure all operators and state manager handle
the checkpoint notification
Key: FLINK-23471
URL: https://issues.apache.org/jira/browse/FLINK-23471
Project
Hi, Steffen
Thank you for your detailed explanation.
>>>But whether a sink is overloaded not only depends on the queue size. It
also depends on the number of in-flight async requests
1. How about chaining two AsyncIOs? One is for controlling the size of the
buffer elements; The other is for cont
Timo Walther created FLINK-23470:
Summary: Use blocking shuffles but pipeline within a slot
Key: FLINK-23470
URL: https://issues.apache.org/jira/browse/FLINK-23470
Project: Flink
Issue Type:
+1 (binding)
@Anton it is usually a good practice to start a new mailing list thread for
the vote. It should refer to the discussion thread and have a subject line
of the form "[VOTE] FLIP-183: Dynamic buffer size adjustment". Next time,
let's do it like this.
Cheers,
Till
On Wed, Jul 21, 2021 a
Thanks everyone for this discussion. I think this is very helpful.
I do agree with Piotr's proposal to separate state and the lifecycle of a
StreamOperator. That way the finished state can be used to recover a
StreamOperator or to rescale the topology. I also believe that this will
make the implem
Thanks for your inputs Gen and Arnaud.
I do agree with you, Gen, that we need better guidance for our users on
when to change the heartbeat configuration. I think this should happen in
any case. I am, however, not so sure whether we can give hard threshold
like 5000 tasks, for example, because as
Dawid Wysakowicz created FLINK-23469:
Summary: StateBackendContext does not close underlying MiniCluster
Key: FLINK-23469
URL: https://issues.apache.org/jira/browse/FLINK-23469
Project: Flink
Dawid Wysakowicz created FLINK-23468:
Summary: Benchmarks networkThroughput for OpenSSL do not run
Key: FLINK-23468
URL: https://issues.apache.org/jira/browse/FLINK-23468
Project: Flink
I
Shengkai Fang created FLINK-23467:
-
Summary: Remove the usage of the yaml file in pyflink e2e test
Key: FLINK-23467
URL: https://issues.apache.org/jira/browse/FLINK-23467
Project: Flink
Issue
Hi Piotr,
Very thanks for the explanation! and very sorry that initially I should wrongly
understand the problem Dawid proposed.
> And what should we do now? We can of course commit all transactions until
> checkpoint 43.
> But should we keep waiting for `notyifyCheckpointComplete(44)`?
> ha
Hey,
We are using the `MailboxExecutor` to block calls to `write` in case the sink
is somehow overloaded. Overloaded basically means that the sink cannot persist
messages quickly enough into the respective destination.
But whether a sink is overloaded not only depends on the queue size. It also
The only histogram implementation available to use are those by
dropwizard, and they do some lock-free synchronization stuff that so far
we wanted to keep out of hot paths (this applis to both reading and
writing); we have however never made benchmarks.
But it is reasonable to assume that they a
Hi all,
@Steven Wu
> Regarding "lastFetchTime" latency metric, I found Gauge to be less
> informative as it only captures the last sampling value for each metric
> publish interval (e.g. 60s).
> * Can we make it a histogram? Histograms are more expensive though.
> * Timer [1, 2] is cheaper as it
Hi Guowei,
> Thank Dawid and Piotr for sharing the problem. +1 to EndInput/Finish can
be
called repeatedly.
Just to clarify. It's not about calling `finish()` and `endInput()`
repeatedly, but about (from the perspective of operator's state)
1. seeing `finish()`
2. checkpoint X triggered and compl
Dawid Wysakowicz created FLINK-23466:
Summary: UnalignedCheckpointITCase hangs on Azure
Key: FLINK-23466
URL: https://issues.apache.org/jira/browse/FLINK-23466
Project: Flink
Issue Type:
Hello,
From a user perspective: we have some (rare) use cases where we use “coarse
grain” datasets, with big beans and tasks that do lengthy operation (such as ML
training). In these cases we had to increase the time out to huge values
(heartbeat.timeout: 50) so that our app is not killed.
Hi Dawid, Piotr, Steven,
Very thanks for pointing out these issues and very thanks for the discussion !
Failure before notifyCheckpointComplete()
For this issue I would agree with what Piotr has proposed. I tried to use some
operators like sink / window as example and currently I also do not fou
34 matches
Mail list logo
