Hi,
From my point of view, I am a bit against using SQL hint to set state TTL as
FlinkSQL could be translated to several stateful operators. If we want to let
different state could have different TTL configs within one operator, the SQL
hint solution could not work. A better way is to allow a g
Yao Zhang created FLINK-31603:
-
Summary: Line break should be removed in create table
with-clauses, load module with-clauses and table hints for both keys and values
Key: FLINK-31603
URL: https://issues.apache.org/jir
Checking with the community again, if anyone explored this before.
Thanks.
On Fri, Mar 17, 2023 at 1:56 PM Elkhan Dadashov
wrote:
> Dear Flink developers,
>
> Wanted to check, if there is a way to control the parallelism of
> auto-generated Flink operators of the FlinkSQL job graph?
>
> In Jav
jackylau created FLINK-31602:
Summary: Add ARRAY_POSITION supported in SQL & Table API
Key: FLINK-31602
URL: https://issues.apache.org/jira/browse/FLINK-31602
Project: Flink
Issue Type: Improveme
Thanks Leonard for the suggestion.
- Check that all POM files point to the same version
- Build the source with Maven
- Ran end-to-end tests locally and succeeded
- Tested read from and write to MongoDB by local cluster.
- Tested lookup table feature by local cluster.
Tested read from and writ
Hi Shammon and Shuo,
Thanks for your valuable comments!
Some thoughts:
@Shuo
> I think it's more properly to say that hint does not affect the
equivalenceof execution plans (hash agg vs sort agg), not the equivalence
of execution
results, e.g., users can set 'scan.startup.mode' for kafka connect
Thanks all for your feedback.
@Shammon FY
My gut feeling is that the end user shouldn't care about whether it'll delete
direcotry or move to Trash directory with the TRUNCATE TABLE statement. They
only need to know it will delete all rows from a table.
To me, I think delete directory or move to
"If all new messages in old partitions should be consumed, all new messages
in new partitions should also be consumed."
Sorry, I wrote the last sentence incorrectly.
On Fri, Mar 24, 2023 at 11:15 AM Hongshun Wang
wrote:
> Hi Shammon,
>
> Thanks for your advise! I learn a lot about TIMESTAMP/SP
Hi Shammon,
Thanks for your advise! I learn a lot about TIMESTAMP/SPECIFIC_OFFSET.
That's interesting.
However, I have a different opinion.
If a user employs the SPECIFIC_OFFSET strategy and enables auto-discovery,
they will be able to find new partitions beyond the specified offset.
Otherwise,
Hi jane
Thanks for initializing this discussion. Configure TTL per operator can
help users manage state more effectively.
I think the `compiled json plan` proposal may need to consider the impact
on the user's submission workflow. Generally, Flink jobs support two types
of submission: SQL and jar
Roman Khachatryan created FLINK-31601:
-
Summary: While waiting for resources, resources check might be
scheduled unlimited number of times (Adaptive Scheduler)
Key: FLINK-31601
URL: https://issues.apache.org/j
Alex Sorokoumov created FLINK-31600:
---
Summary: Fix producer leaks in flink-connector-kafka tests
Key: FLINK-31600
URL: https://issues.apache.org/jira/browse/FLINK-31600
Project: Flink
Issue
Alex Sorokoumov created FLINK-31599:
---
Summary: Update Kafka dependency in flink-connector-kafka to 3.4.0
Key: FLINK-31599
URL: https://issues.apache.org/jira/browse/FLINK-31599
Project: Flink
Jark Wu created FLINK-31598:
---
Summary: Cleanup usage of deprecated TableEnvironment#registerTable
Key: FLINK-31598
URL: https://issues.apache.org/jira/browse/FLINK-31598
Project: Flink
Issue Type:
Jark Wu created FLINK-31597:
---
Summary: Cleanup usage of deprecated
TableEnvironment#registerFunction
Key: FLINK-31597
URL: https://issues.apache.org/jira/browse/FLINK-31597
Project: Flink
Issue Ty
Jark Wu created FLINK-31596:
---
Summary: Cleanup usage of deprecated methods in TableEnvironment
Key: FLINK-31596
URL: https://issues.apache.org/jira/browse/FLINK-31596
Project: Flink
Issue Type: Tec
Bo Cui created FLINK-31595:
--
Summary: MiniBatchLocalGroupAggFunction produces wrong aggregate
results with state clean up
Key: FLINK-31595
URL: https://issues.apache.org/jira/browse/FLINK-31595
Project: Flin
Matthias Pohl created FLINK-31594:
-
Summary: Updates the docs stable version
Key: FLINK-31594
URL: https://issues.apache.org/jira/browse/FLINK-31594
Project: Flink
Issue Type: Sub-task
Matthias Pohl created FLINK-31593:
-
Summary: Update reference data for Migration Tests
Key: FLINK-31593
URL: https://issues.apache.org/jira/browse/FLINK-31593
Project: Flink
Issue Type: Sub-t
Correction: “users can set 'scan.startup.mode' for kafka connector” -> “users
can set 'scan.startup.mode' for kafka connector by dynamic table option”
Shuo Cheng 于2023年3月23日 周四21:50写道：
> Hi Jane,
> Thanks for driving this, operator level state ttl is absolutely a desired
> feature. I would share
Hi Jane,
Thanks for driving this, operator level state ttl is absolutely a desired
feature. I would share my opinion as following:
If the scope of this proposal is limited as an enhancement for compiled
json plan, it makes sense. I think it does not conflict with configuring
state ttl
in other way
Gyula Fora created FLINK-31592:
--
Summary: Configurable deletion propagation for Flink resources
Key: FLINK-31592
URL: https://issues.apache.org/jira/browse/FLINK-31592
Project: Flink
Issue Type:
Chesnay Schepler created FLINK-31591:
Summary: Extend JobGraphWriter to persist requirements
Key: FLINK-31591
URL: https://issues.apache.org/jira/browse/FLINK-31591
Project: Flink
Issue T
Chesnay Schepler created FLINK-31590:
Summary: Allow setting JobResourceRequirements through
JobMasterGateway
Key: FLINK-31590
URL: https://issues.apache.org/jira/browse/FLINK-31590
Project: Flink
Hi Hongshun
Thanks for driving this discussion. Automatically discovering partitions
without losing data sounds great!
Currently flink supports kafka source with different startup modes, such as
EARLIEST, LATEST, TIMESTAMP, SPECIFIC_OFFSETS and GROUP_OFFSET.
If I understand correctly, you will s
Hi Yisha,
Thank you for the valuable feedback! I appreciate that you find this
proposal beneficial. I'd like to answer your questions and explain the
reason why we don't prefer to use SQL hints.
> 1. Hints may not only aim to inspire the planner. For example, we have
dynamic options for tables an
liuzhuo created FLINK-31589:
---
Summary: Reduce the pressure of the list pod method on the k8s
cluster
Key: FLINK-31589
URL: https://issues.apache.org/jira/browse/FLINK-31589
Project: Flink
Issue Ty
Rui Fan created FLINK-31588:
---
Summary: The unaligned checkpoint type is wrong at subtask level
Key: FLINK-31588
URL: https://issues.apache.org/jira/browse/FLINK-31588
Project: Flink
Issue Type: Bug
Congratulations. Well done!
From: Matthias Pohl via user
Sent: Thursday, March 23, 2023 12:28 PM
To: Jing Ge
Cc: Leonard Xu ; dev ;
annou...@apache.org ; user ;
user-zh
Subject: Re: [ANNOUNCE] Apache Flink 1.17.0 released
EXTERNAL EMAIL
Thanks for making th
Thanks for making this release getting over the finish line.
One additional thing:
Feel free to reach out to the release managers (or respond to this thread)
with feedback on the release process. Our goal is to constantly improve the
release process. Feedback on what could be improved or things th
Hi, yuxia.
Thanks for starting the discussion.
I think it's a nice improvement to support TRUNCATE TABLE statement because
many other mature engines supports it.
I have some questions.
1. because table has different types, whether we will support view or
temporary tables?
2. some other engines s
Hi all,
I am happy to announce that FLIP-301: Hybrid Shuffle supports Remote
Storage[1] has been accepted. The FLIP was voted in this thread[2].
There are 4 binding votes and 3 non-binding votes:
- Wencong Liu (non-binding)
- Weihua Hu (non-binding)
- Junrui Lee (non-binding)
- weijie guo (bindi
Excellent work! Congratulations! Appreciate the hard work and contributions
of everyone in the Apache Flink community who helped make this release
possible. Looking forward to those new features. Cheers!
Best regards,
Jing
On Thu, Mar 23, 2023 at 10:24 AM Leonard Xu wrote:
> The Apache Flink co
Hi Jane,
Thanks for driving this, and your solution is absolutely creative. In my
company, there also exist some scenarios in which users want to config state
TTL at operator level, especially for window operators which regard TTL as
allow lateness.
To support this scenarios, we implemented a
Hi, yuxia,
Thanks for starting the discussion.
I wonder what the behavior is when we truncate a table which is used as a
source. Source table and sink table may have different table options.
IMO, the truncate sql should be supported no matter which kind the table is.
Best,
Hang
Shammon FY 于202
Qingsheng Ren created FLINK-31586:
-
Summary: CLONE - CLONE - Propose a pull request for website updates
Key: FLINK-31586
URL: https://issues.apache.org/jira/browse/FLINK-31586
Project: Flink
Qingsheng Ren created FLINK-31583:
-
Summary: Build Release Candidate: 1.17.0-rc3
Key: FLINK-31583
URL: https://issues.apache.org/jira/browse/FLINK-31583
Project: Flink
Issue Type: New Feature
Qingsheng Ren created FLINK-31587:
-
Summary: CLONE - CLONE - Vote on the release candidate
Key: FLINK-31587
URL: https://issues.apache.org/jira/browse/FLINK-31587
Project: Flink
Issue Type: S
Qingsheng Ren created FLINK-31585:
-
Summary: CLONE - CLONE - Stage source and binary releases on
dist.apache.org
Key: FLINK-31585
URL: https://issues.apache.org/jira/browse/FLINK-31585
Project: Flink
Qingsheng Ren created FLINK-31584:
-
Summary: CLONE - CLONE - Build and stage Java and Python artifacts
Key: FLINK-31584
URL: https://issues.apache.org/jira/browse/FLINK-31584
Project: Flink
I
Qingsheng Ren created FLINK-31582:
-
Summary: CLONE - Vote on the release candidate
Key: FLINK-31582
URL: https://issues.apache.org/jira/browse/FLINK-31582
Project: Flink
Issue Type: Sub-task
Qingsheng Ren created FLINK-31580:
-
Summary: CLONE - Stage source and binary releases on
dist.apache.org
Key: FLINK-31580
URL: https://issues.apache.org/jira/browse/FLINK-31580
Project: Flink
Qingsheng Ren created FLINK-31581:
-
Summary: CLONE - Propose a pull request for website updates
Key: FLINK-31581
URL: https://issues.apache.org/jira/browse/FLINK-31581
Project: Flink
Issue Ty
Qingsheng Ren created FLINK-31579:
-
Summary: CLONE - Build and stage Java and Python artifacts
Key: FLINK-31579
URL: https://issues.apache.org/jira/browse/FLINK-31579
Project: Flink
Issue Typ
Qingsheng Ren created FLINK-31578:
-
Summary: Build Release Candidate: 1.17.0-rc2
Key: FLINK-31578
URL: https://issues.apache.org/jira/browse/FLINK-31578
Project: Flink
Issue Type: New Feature
Jeff Yang created FLINK-31577:
-
Summary: Upgrade documentation to improve Operator
Key: FLINK-31577
URL: https://issues.apache.org/jira/browse/FLINK-31577
Project: Flink
Issue Type: Improvement
The Apache Flink community is very happy to announce the release of Apache
Flink 1.17.0, which is the first release for the Apache Flink 1.17 series.
Apache Flink® is an open-source unified stream and batch data processing
framework for distributed, high-performing, always-available, and accurat
+1 to support custom restart strategies in a different FLIP.
It's fine to have a different plugin for custom restart strategy.
If so, since we do not treat the FLIP-304 plugin as a common failure
handler, but instead mainly targets to add labels to errors, I would
+1 for the name `FailureEnricher`
Hi, everyone
I'm closing this vote now. I will follow up with the result in another
email. Thanks
Best,
Yuxin
Yun Tang 于2023年3月21日周二 10:58写道：
> +1 (binding)
>
> Best
> Yun Tang
>
> From: Zhu Zhu
> Sent: Tuesday, March 21, 2023 10:07
> To: dev@flink.apache.org
>
> One additional remark on introducing it as an async operation: We would
> need a new configuration parameter to define the timeout for such a
> listener call, wouldn't we?
>
This could be left up to the implementor to handle.
What about adding an extra method getNamespace() to the Listener in
Sounds good. Two points I want to add:
- Listener execution should be independent — however we need a way to
> enforce a Label key/key-prefix is only assigned to a single Listener,
> thinking of a validation step both at Listener init and runtime stages
>
What about adding an extra method getNa
51 matches
Mail list logo
