Thank you for bringing this up!
+1
On Mon, Jun 13, 2022 at 1:48 PM Rui Fan <1996fan...@gmail.com> wrote:
> Strongly +1
>
> Best,
> Rui Fan
>
> On Mon, Jun 13, 2022 at 7:35 PM Martijn Visser
> wrote:
>
> > > BTW, from my knowledge, nothing would happen for HashMapStateBackend,
> > which does no
+1
It's basically enabled in most scenarios in production environments.
For HashMapStateBackend, it will adopt a full checkpoint even if we enable
incremental checkpoint. It will also support incremental checkpoint after
[1]. It's compatible.
BTW, I think we may also need to improve the documentati
yutao created FLINK-28041:
-
Summary: table store cannot distinguish filesystem Scheme when
system have 'hadoop classpath'
Key: FLINK-28041
URL: https://issues.apache.org/jira/browse/FLINK-28041
Project: Fli
+1 (not binding)
I am a little bit (too) late to this discussion. This is an important
feature. Thanks for driving this! BTW, the [2] thread does not point to
your original kickoff email.
Best regards,
Jing
On Tue, Jun 14, 2022 at 8:33 AM yue ma wrote:
> Thanks for Nicholas driving this.
> +1
Chesnay Schepler created FLINK-28042:
Summary: Create an extension for resetting HiveConf
Key: FLINK-28042
URL: https://issues.apache.org/jira/browse/FLINK-28042
Project: Flink
Issue Type
Juntao Hu created FLINK-28043:
-
Summary: "Invalid lambda deserialization" in AvroParquetReaders
Key: FLINK-28043
URL: https://issues.apache.org/jira/browse/FLINK-28043
Project: Flink
Issue Type:
Hi Godfrey, thanks for starting this discussion, this is a great feature
for batch users.
The FLIP looks good to me in general.
I only have 2 comments:
1) How do users know whether the given table or partition contains required
statistics?
Do we need to provide DESC EXTENDED statement like Spar
Gabor Somogyi created FLINK-28044:
-
Summary: Add hadoop filesystems configuration possibility to all
deployment targets
Key: FLINK-28044
URL: https://issues.apache.org/jira/browse/FLINK-28044
Project:
Alexander Fedulov created FLINK-28045:
-
Summary: [umbrella] Deprecate SourceFunction API
Key: FLINK-28045
URL: https://issues.apache.org/jira/browse/FLINK-28045
Project: Flink
Issue Type:
Alexander Fedulov created FLINK-28046:
-
Summary: Annotate SourceFunction as deprecated
Key: FLINK-28046
URL: https://issues.apache.org/jira/browse/FLINK-28046
Project: Flink
Issue Type: S
Alexander Fedulov created FLINK-28047:
-
Summary: Deprecate StreamExecutionEnvironment#readFiles(*) methods
in favor of FileSource#forRecordStreamFormat/forBulkFileFormat.
Key: FLINK-28047
URL: https://issues.a
Alexander Fedulov created FLINK-28048:
-
Summary: Introduce Source API alternative to FiniteTestSource
Key: FLINK-28048
URL: https://issues.apache.org/jira/browse/FLINK-28048
Project: Flink
Alexander Fedulov created FLINK-28049:
-
Summary: Introduce FLIP-208 functionality to stop Source based on
consumed records
Key: FLINK-28049
URL: https://issues.apache.org/jira/browse/FLINK-28049
P
Alexander Fedulov created FLINK-28050:
-
Summary: Introduce Source API alternative to
SourceExecutionContext#fromCollection(*) methods
Key: FLINK-28050
URL: https://issues.apache.org/jira/browse/FLINK-28050
Alexander Fedulov created FLINK-28051:
-
Summary: Introduce Source API alternative to
ExternallyInducedSource
Key: FLINK-28051
URL: https://issues.apache.org/jira/browse/FLINK-28051
Project: Flink
+1 for the suggestion. We have use the incremental checkpoint in our
production for a long time.
Hangxiang Yu 于2022年6月14日周二 15:41写道：
> +1
> It's basically enabled in most scenarios in production environments.
> For HashMapStateBackend, it will adopt a full checkpoint even if we enable
> incremen
Chesnay Schepler created FLINK-28052:
Summary: Remove AdaptiveSchedulerTest#RunFailedJobListener
Key: FLINK-28052
URL: https://issues.apache.org/jira/browse/FLINK-28052
Project: Flink
Iss
Hi Devs!
I would like to discuss/propose a new release flow for the Kubernetes
operator patch releases.
For the operator the main logic (and the bugs) are part of the operator
docker image and the helm charts associated with it. It would be nice to be
able to have lightweight patch releases that
Shengkai Fang created FLINK-28053:
-
Summary: Introduce queue to execute request in sequence
Key: FLINK-28053
URL: https://issues.apache.org/jira/browse/FLINK-28053
Project: Flink
Issue Type:
On 14/06/2022 14:10, Gyula Fóra wrote:
Another alternative would be to simply not have these patch releases and
encourage users to pick up the latest automatic image builds from
https://ghcr.io/apache/flink-kubernetes-operator for the current release
branch, but I feel that production users woul
On 14/06/2022 14:10, Gyula Fóra wrote:
For the operator the main logic (and the bugs) are part of the operator
docker image and the helm charts associated with it. It would be nice to be
able to have lightweight patch releases that only contain the docker
image + updated Helm chart.
This would a
Thanks Chesnay, these are exactly the questions I would like to clarify
because I don't really understand the limitations/boundaries of the apache
release process.
Is there a strict requirement to have a source release accompany the docker
image? I will have to look this up.
Gyula
On Tue, Jun 14
I think what you are referring to is here:
https://www.apache.org/legal/release-policy.html#source-packages
Based on this we probably cannot simply release the docker image. We could
decide to not release the maven artifacts though, but that seems to be a
minor difference and probably not worth it
Yes, pretty much.
Mind you that the 72h voting duration is a recommendation by the ASF;
it's not a strict rule.
AFAICT we also haven't locked this down in our bylaws, apart from
requiring 3 votes.
On 14/06/2022 14:25, Gyula Fóra wrote:
I think what you are referring to is here:
https://www.a
My bad, our bylaws actually state that release votes must have a minimum
3 days duration.
On 14/06/2022 14:46, Chesnay Schepler wrote:
Yes, pretty much.
Mind you that the 72h voting duration is a recommendation by the ASF;
it's not a strict rule.
AFAICT we also haven't locked this down in our
Alexander Fedulov created FLINK-28054:
-
Summary: Introduce Source API utilities to control records emitted
per checkpoint
Key: FLINK-28054
URL: https://issues.apache.org/jira/browse/FLINK-28054
Pr
Alexander Fedulov created FLINK-28055:
-
Summary: Introduce Source API utilities for throttling number of
records emitted per second.
Key: FLINK-28055
URL: https://issues.apache.org/jira/browse/FLINK-28055
Alexander Fedulov created FLINK-28056:
-
Summary: Introduce high level API for cases that do not require
specific SourceEnumerator logic.
Key: FLINK-28056
URL: https://issues.apache.org/jira/browse/FLINK-28056
Hi Becket,
thanks for your feedback. As proposed, I started an umbrella ticket [1] to
collect all the
steps needed. Please add any missing items.
Judging by the discussion on this thread I propose to open a vote on the
first two
subtasks since they are immediately actionable [2] [3].
[1] https:/
Nicolas Ferrario created FLINK-28057:
Summary: LD_PRELOAD is hardcoded to x64 on flink-docker
Key: FLINK-28057
URL: https://issues.apache.org/jira/browse/FLINK-28057
Project: Flink
Issue
lucas.wu created FLINK-28058:
Summary: add option SKIP_FIRST_DATA_ROW
Key: FLINK-28058
URL: https://issues.apache.org/jira/browse/FLINK-28058
Project: Flink
Issue Type: New Feature
Comp
Hi Alex,
I guess you are meaning to start a new voting thread following the FLIP
discussion&voting concept. Looking forward to it!
Best regards,
Jing
On Tue, Jun 14, 2022 at 3:35 PM Alexander Fedulov
wrote:
> Hi Becket,
>
> thanks for your feedback. As proposed, I started an umbrella ticket [1
Matyas Orhidi created FLINK-28059:
-
Summary: Parallelize e2e tests
Key: FLINK-28059
URL: https://issues.apache.org/jira/browse/FLINK-28059
Project: Flink
Issue Type: Improvement
Com
Christian Lorenz created FLINK-28060:
Summary: Kafka Commit on checkpointing fails repeatedly after a
broker restart
Key: FLINK-28060
URL: https://issues.apache.org/jira/browse/FLINK-28060
Project
Hi Jing,
I don't think we do pure deprecation FLIPs. I am also OK if we consider
consensus in this thread enough to proceed with opening the actual
deprecation PRs for the first two items of the umbrella ticket. In that
case, we can skip the vote.
What do people prefer? Do you think we need to do
Hi Steven,
FYI, I've added your requirement to the list of subtasks for
deprecating the SourceFunction API [1] [2].
[1] https://issues.apache.org/jira/browse/FLINK-28045
[2] https://issues.apache.org/jira/browse/FLINK-28054
Best,
Alexander Fedulov
On Tue, Jun 7, 2022 at 6:03 PM Steven Wu wrot
Hi,
After reading all discussions posted in this thread and the source code of
DataGeneratorSource which unfortunately used "Source" instead of
"SourceFunction" in its name, issues could summarized as following:
1. The current DataGeneratorSource based on SourceFunction is a stateful
source conne
Hi everyone,
I would like to give you a brief update of the Flink 1.16 release sync
meeting of 2022-06-14.
At this moment, 3 features have been completed for this release and 41
features are still expected to make it. We have 6 more weeks remaining
until the release branch is cut (at the end of J
Hi Alex and Jing,
According to the Flink Bylaws and FLIP guide, this change would be a
"change that impacts the public interfaces of the project". I do see little
value in creating a special deprecation FLIP, but I do think it requires a
[VOTE] thread which follows the FLIP approval process. So ba
Jing Ge created FLINK-28061:
---
Summary: create new tech blog for connector development based on
Source API
Key: FLINK-28061
URL: https://issues.apache.org/jira/browse/FLINK-28061
Project: Flink
Iss
Hi Martijn,
Yes, that is exactly what I tried to say implicitly in a polite manner. :))
Best regards,
Jing
On Tue, Jun 14, 2022 at 8:12 PM Martijn Visser
wrote:
> Hi Alex and Jing,
>
> According to the Flink Bylaws and FLIP guide, this change would be a
> "change that impacts the public inter
Hi everyone,
Thank you for all the effort that went into this discussion. The split
level watermark alignment will be an important feature for Flink that
will address operational problems for various use cases. From reading
through this thread it appears that not too much remains to bring this
FLI
Alex, thanks a lot for capturing the checkpoint lockstep emitting source.
It doesn't have to be the same wrapper class. It could be another wrapper
source (like ManualSource) where users supply the exact records emitted by
the source per checkpoint.
On Tue, Jun 14, 2022 at 10:18 AM Jing Ge wrote:
Jiangfei Liu created FLINK-28062:
Summary: Flink SQL Upsert-Kafka can not support Flink1.14.x With
Sink Buffer
Key: FLINK-28062
URL: https://issues.apache.org/jira/browse/FLINK-28062
Project: Flink
thanks for helping with some inputs, yes I am using rich function and
handling objects created in open, and also and network calls are getting
called in a run.
but currently, I got stuck running this same task on *all task managers*
(nodes), when I submit the job, this task1(static data task) runs
Jingsong Lee created FLINK-28063:
Summary: Optimize PredicateBuilder.in for lots of parameters
Key: FLINK-28063
URL: https://issues.apache.org/jira/browse/FLINK-28063
Project: Flink
Issue Typ
Jingsong Lee created FLINK-28064:
Summary: PredicateBuilder.in should accept null parameters
Key: FLINK-28064
URL: https://issues.apache.org/jira/browse/FLINK-28064
Project: Flink
Issue Type:
zhuyufeng created FLINK-28065:
-
Summary: A never reached code in ProcessMemoryUtils Class
Key: FLINK-28065
URL: https://issues.apache.org/jira/browse/FLINK-28065
Project: Flink
Issue Type: Bug
Jingsong Lee created FLINK-28066:
Summary: Use FileSystem.createRecoverableWriter in FileStoreCommit
Key: FLINK-28066
URL: https://issues.apache.org/jira/browse/FLINK-28066
Project: Flink
Iss
Jingsong Lee created FLINK-28067:
Summary: Introduce a tablestore HiveCatalog
Key: FLINK-28067
URL: https://issues.apache.org/jira/browse/FLINK-28067
Project: Flink
Issue Type: Sub-task
Jingsong Lee created FLINK-28068:
Summary: Control the total memory of a sink task
Key: FLINK-28068
URL: https://issues.apache.org/jira/browse/FLINK-28068
Project: Flink
Issue Type: Improveme
Thanks for bringing this up.
I am +1 on making incremental checkpoints by default for RocksDB, but not
universally for all state backends.
Besides being widely used in prod, enabling incremental checkpoint for
RocksDB by default is also a pre-requisite when enabling task-local by
default FLINK-15
SunShun created FLINK-28069:
---
Summary: Cannot attach SSL JKS file for Kafka connector
Key: FLINK-28069
URL: https://issues.apache.org/jira/browse/FLINK-28069
Project: Flink
Issue Type: Bug
Affe
I don't really understand how task2 reads static data from task1,
but I think you can integrate the logic of getting static data from http in
task1 into task2 and keep only one kind of task.
Best,
Weihua
On Wed, Jun 15, 2022 at 10:07 AM Great Info wrote:
> thanks for helping with some inputs,
Sergey Nuyanzin created FLINK-28070:
---
Summary: [JUnit5 Migration] Migrate ScalaAPICompletenessTestBase
to Junit5
Key: FLINK-28070
URL: https://issues.apache.org/jira/browse/FLINK-28070
Project: Flin
Dian Fu created FLINK-28071:
---
Summary: Support missing built-in functions in Table API
Key: FLINK-28071
URL: https://issues.apache.org/jira/browse/FLINK-28071
Project: Flink
Issue Type: Sub-task
Congratulations, Jingsong!
Best,
Yun Gao
--
From:Jing Zhang
Send Time:2022 Jun. 14 (Tue.) 11:05
To:dev
Subject:Re: [ANNOUNCE] New Apache Flink PMC Member - Jingsong Lee
Congratulations, Jingsong!
Best,
Jing Zhang
Leonard Xu 于2
Jingsong Lee created FLINK-28072:
Summary: Set Hadoop FileSystem for Orc reader
Key: FLINK-28072
URL: https://issues.apache.org/jira/browse/FLINK-28072
Project: Flink
Issue Type: Improvement
Jingsong Lee created FLINK-28073:
Summary: Table Store supports Flink 1.14
Key: FLINK-28073
URL: https://issues.apache.org/jira/browse/FLINK-28073
Project: Flink
Issue Type: Improvement
Thanks everyone.
It's great to be with you in the Flink community!
Best,
Jingsong
On Wed, Jun 15, 2022 at 2:11 PM Yun Gao wrote:
>
> Congratulations, Jingsong!
>
> Best,
> Yun Gao
>
>
> --
> From:Jing Zhang
> Send Time:2022 Jun. 1
60 matches
Mail list logo
