Hi Lijie,
Regarding the shuffle mode, I think it would be reasonable to also support
"pipeline shuffle" if possible.
"pipeline shuffle" is a essential for OLAP/MPP computing, although this has
not been much exposed to users for now, I know a few companies that uses
Flink as a MPP computing engine
Hi Yuxia,
I made a mistake in the above response.
The runtime filter can work well with all shuffle mode. However, hybrid
shuffle and blocking shuffle are currently recommended for batch jobs
(piepline shuffle is not recommended).
One more thing to mention here is that we will force the edge bet
Hi Gordon,
Thanks for your feedback as always.
Why not just Map?
I think it makes sense to relocate the bootstrapServer field in the Kafka
properties (ClusterMetadata abstraction), since that is intuitive based
on how it is defined in the Kafka clients library. It also makes the
uniqueness of a
Thanks Feng for the FLIP.
+1(binding)
Cheers,
Dong
On Wed, Jun 14, 2023 at 10:35 AM Feng Jin wrote:
> Hi everyone
>
> Thanks for all the feedback about the FLIP-295: Support lazy initialization
> of catalogs and persistence of catalog configurations[1].
> [2] is the discussion thread.
>
>
> I'
I agree that Public APIs should require a longer migration period. I think
that is why the FLIP requires at least 2 minor releases (compared to 1
minor release for PublicEvolving and 1 patch release for Experimental).
I think the key of stability guarantees is about not breaking the
commitments we
+1 (non-binding)
Best,
Jane
On Wed, Jun 14, 2023 at 10:41 AM Feng Jin wrote:
> +1 (no-binding)
>
>
> Best,
> Feng
>
>
> On Wed, Jun 14, 2023 at 7:02 AM Jing Ge
> wrote:
>
> > +1(binding)
> >
> > Best Regards,
> > Jing
> >
> > On Tue, Jun 13, 2023 at 9:03 AM Mang Zhang wrote:
> >
> > > +1 (no-
Hi Yuxia,
Thanks for your feedback. The answers of your questions are as follows:
1. Yes, the row count comes from statistic of underlying table(Or estimated
based on the statistic of underlying table, if the build side or probe side
is not TableScan). If the statistic unavailable, we will not i
Hi Aitozi,
Thanks for your reply! Gives sql users more flexibility to get
asynchronous processing capabilities via lateral join table function +1 for
this
For `JavaAsyncTableFunc0` in flip, can you use a scenario like RPC call as
an example?
For the name of this query hint, 'LATERAL' (include i
Thanks, Hong!
I understand that if the user case is to simply write sth to an external
service, Async Sink is a good option that provides features like batching,
state management and rate limiting. I have some follow up questions:
1. Is there any problem if we use Async Function for such a user c
Hi Lu,
Thanks for your question. See below for my understanding.
I would recommend using the Async Sink if you are writing to the external
service as the final output of your job graph, and if you don’t have the
ordered requirement that updates to the external system must be done before
update
Hi Mason,
Thanks for addressing my comments. I agree that option 3 seems more
reasonable.
> Reorganize the metadata in a Map in
`KafkaStream` where the String is the proposed
`KafkaClusterIdentifier.name` field.
Why not just Map?
Regarding naming, I like DynamicKafkaSource as that's what I imme
Hi, Flink dev and users
If I want to async write to an external service, which API shall I use,
AsyncFunction or Async Sink?
My understanding after checking the code are:
1. Both APIs guarantee at least once write to external service. As both
API internally stores in-flight requests in the
chenyuexin created FLINK-32341:
--
Summary: Fix resource requirements rest API response empty result
Key: FLINK-32341
URL: https://issues.apache.org/jira/browse/FLINK-32341
Project: Flink
Issue T
+1 (binding)
Best Regards,
Jing
On Wed, Jun 14, 2023 at 3:28 PM Rui Fan <1996fan...@gmail.com> wrote:
> +1(binding)
>
> Best,
> Rui Fan
>
> On Wed, Jun 14, 2023 at 16:24 Hang Ruan wrote:
>
> > +1 (non-binding)
> >
> > Thanks for Feng driving it.
> >
> > Best,
> > Hang
> >
> > Feng Jin 于2023年6月
+1 (binding)
Best Regards,
Jing
On Wed, Jun 14, 2023 at 4:07 PM Benchao Li wrote:
> +1 (binding)
>
> Shammon FY 于2023年6月14日周三 19:52写道：
>
> > Hi all:
> >
> > Thanks for all the feedback for FLIP-294: Support Customized Catalog
> > Modification Listener [1]. I would like to start a vote for it a
Sergii Nazarov created FLINK-32340:
--
Summary: NPE in K8s operator which brakes current and subsequent
deployments
Key: FLINK-32340
URL: https://issues.apache.org/jira/browse/FLINK-32340
Project: Flin
+1 (binding)
Shammon FY 于2023年6月14日周三 19:52写道：
> Hi all:
>
> Thanks for all the feedback for FLIP-294: Support Customized Catalog
> Modification Listener [1]. I would like to start a vote for it according to
> the discussion in thread [2].
>
> The vote will be open for at least 72 hours(excludin
Thanks for the explanation, Matthias.
In the example you raised, would it be better to just keep both YARN and
K8S support in the new major version, but with YARN support deprecated if
we want to? We can say for YARN we will only provide bug fixes but no
feature development anymore. Given these tw
Hi all,
@Yukia,I updated the FLIP to include the aggregation of the staked
operations that we discussed below PTAL.
Best
Etienne
Le 13/06/2023 à 16:31, Etienne Chauchot a écrit :
Hi Yuxia,
Thanks for your feedback. The number of potentially stacked operations
depends on the configured le
+1(binding)
Best,
Rui Fan
On Wed, Jun 14, 2023 at 16:24 Hang Ruan wrote:
> +1 (non-binding)
>
> Thanks for Feng driving it.
>
> Best,
> Hang
>
> Feng Jin 于2023年6月14日周三 10:36写道：
>
> > Hi everyone
> >
> > Thanks for all the feedback about the FLIP-295: Support lazy
> initialization
> > of catalo
One (made-up) example from the top of my head would have been that the
community decides to focus fully on Kubernetes without considering Yarn
anymore because of some must-have feature on the Kubernetes side. At the
same time there are still some users for whom it would be tricky to migrate
from Ya
Thanks Lijie for starting this discussion. Excited to see runtime filter is to
be implemented in Flink.
I have few questions about it:
1: As the FLIP said, `if the ndv cannot be estimated, use row count instead`.
So, does row count comes from the statistic from underlying table? What if the
the
Hi all:
Thanks for all the feedback for FLIP-294: Support Customized Catalog
Modification Listener [1]. I would like to start a vote for it according to
the discussion in thread [2].
The vote will be open for at least 72 hours(excluding weekends, until June
19, 19:00 PM GMT) unless there is an ob
Matthias Pohl created FLINK-32339:
-
Summary: Align lock usage in DefaultLeaderElectionService
Key: FLINK-32339
URL: https://issues.apache.org/jira/browse/FLINK-32339
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32338:
Summary: Add FailsOnJava17 annotation
Key: FLINK-32338
URL: https://issues.apache.org/jira/browse/FLINK-32338
Project: Flink
Issue Type: Sub-task
Sergey Nuyanzin created FLINK-32337:
---
Summary: SQL array_union could return wrong result
Key: FLINK-32337
URL: https://issues.apache.org/jira/browse/FLINK-32337
Project: Flink
Issue Type: B
Chesnay Schepler created FLINK-32336:
Summary: PartitionITCase#ComparablePojo should be public
Key: FLINK-32336
URL: https://issues.apache.org/jira/browse/FLINK-32336
Project: Flink
Issue
Hi Matthias,
Thanks for the feedback.
Do you have an example of behavioral change in mind? Not sure I fully
understand the concern for behavioral change here. From what I understand,
any user sensible change in an existing API, regardless of its kind (API
signature or behavioral change), can alwa
Thanks Lijie start this discussion. Runtime Filter is a common optimization
to improve the join performance that has been adopted by many computing
engines such as Spark, Doris, etc... Flink is a streaming batch computing
engine, and we are continuously optimizing the performance of batches.
Runtim
Hi devs
Ron Liu, Gen Luo and I would like to start a discussion about FLIP-324:
Introduce Runtime Filter for Flink Batch Jobs[1]
Runtime Filter is a common optimization to improve join performance. It is
designed to dynamically generate filter conditions for certain Join queries
at runtime to red
Hi Xintong,
Thanks for the comment. Please see the replies below:
1. Do we allow deprecation & removal of APIs without adding a new one as a
> replacement? The examples in the table give me an impression that marking
> an API as `@Deprecated` should only happen at the same time of introducing
> a
Thanks for starting this discussion, Becket. A few good points were raised.
Here's what I want to add:
Stefan raised the point of behavioral stability (in contrast to API
stability). That might be a reason for users to not be able to go ahead
with a major version bump. Working around behavioral ch
Jiang Xin created FLINK-32335:
-
Summary: Fix the Flink ML unittest failure
Key: FLINK-32335
URL: https://issues.apache.org/jira/browse/FLINK-32335
Project: Flink
Issue Type: New Feature
+1 (non-binding)
Thanks for Feng driving it.
Best,
Hang
Feng Jin 于2023年6月14日周三 10:36写道：
> Hi everyone
>
> Thanks for all the feedback about the FLIP-295: Support lazy initialization
> of catalogs and persistence of catalog configurations[1].
> [2] is the discussion thread.
>
>
> I'd like to st
Hi Lincoln
Very thanks for your valuable question. I will try to answer your
questions inline.
>Does the async udtf bring any additional benefits besides a
lighter implementation？
IMO, async udtf is more than a lighter implementation. It can act as a
general way for sql users to use the asyn
Thanks for bringing up this discussion, Becket.
My two cents:
1. Do we allow deprecation & removal of APIs without adding a new one as a
replacement? The examples in the table give me an impression that marking
an API as `@Deprecated` should only happen at the same time of introducing
a new repla
Nicolas Fraison created FLINK-32334:
---
Summary: Operator failed to create taskmanager deployment because
it already exist
Key: FLINK-32334
URL: https://issues.apache.org/jira/browse/FLINK-32334
Proje
Hi Jing,
Thanks for the feedback. Please see the answers to your questions below:
*"Always add a "Since X.X.X" comment to indicate when was a class /
> interface / method marked as deprecated."*
> Could you describe it with a code example? Do you mean Java comments?
It is just a comment such as
38 matches
Mail list logo
