Thanks Wei for this work! +1 to bring up the VOTE thread.
> 在 2020年3月30日，下午2:43，jincheng sun 写道：
>
> Hi Wei,
>
> +1, Thanks for this discussion which is crucial for SQL users to use
> PyFlink. Would be great to bring up the VOTE thread.
>
> Best,
> Jincheng
>
>
> Wei Zhong 于2020年3月30日周一 下午2
Hi all,
I would like to start the vote for FLIP-95 [1], which is discussed and
reached a consensus in the discussion thread [2].
The vote will be open until April 2nd (72h), unless there is an
objection or not enough votes.
Thanks,
Timo
[1]
https://cwiki.apache.org/confluence/display/FLINK/
>
> The links work for me now. Someone might have fixed them. Never mind.
>
Actually, I fixed the links after seeing your email. Thanks for reporting.
Best,
Gary
On Mon, Mar 30, 2020 at 3:48 AM Xintong Song wrote:
> @ZhuZhu
>
> The links work for me now. Someone might have fixed them. Never mi
Nikola created FLINK-16860:
--
Summary: TableException: Failed to push filter into table source!
when upgrading flink to 1.9.2
Key: FLINK-16860
URL: https://issues.apache.org/jira/browse/FLINK-16860
Project: F
Hi Kurt,
That's good questions.
> the meaning of "version"
There are two versions in the old design. One is property version
"connector.property-version" which can be used for backward compatibility.
The other one is "connector.version" which defines the version of external
system, e.g. 0.11" for
+1 from my side.
Thanks Timo for driving this.
Best,
Jark
On Mon, 30 Mar 2020 at 15:36, Timo Walther wrote:
> Hi all,
>
> I would like to start the vote for FLIP-95 [1], which is discussed and
> reached a consensus in the discussion thread [2].
>
> The vote will be open until April 2nd (72h),
Thanks for the feedbacks everyone!
Overall, everyone who replied is positive about creating a separate repo
for the Stateful Functions Dockerfiles.
Since such an action does not necessarily need a vote (as defined by the
project bylaws), and we'd like the images to be available as soon as
possible
+1 (non-binding)
Jark Wu 于2020年3月30日周一 下午3:57写道：
> +1 from my side.
>
> Thanks Timo for driving this.
>
> Best,
> Jark
>
> On Mon, 30 Mar 2020 at 15:36, Timo Walther wrote:
>
> > Hi all,
> >
> > I would like to start the vote for FLIP-95 [1], which is discussed and
> > reached a consensus in th
Xianxun Ye created FLINK-16861:
--
Summary: Adding delegation token to the AM container Failed
Key: FLINK-16861
URL: https://issues.apache.org/jira/browse/FLINK-16861
Project: Flink
Issue Type: Bu
Chesnay Schepler created FLINK-16862:
Summary: Use proper example url in quickstarts
Key: FLINK-16862
URL: https://issues.apache.org/jira/browse/FLINK-16862
Project: Flink
Issue Type: Imp
Hi Wei,
Thanks a lot for the proposal! +1 for the VOTE.
Best,
Hequn
On Mon, Mar 30, 2020 at 3:31 PM Dian Fu wrote:
> Thanks Wei for this work! +1 to bring up the VOTE thread.
>
> > 在 2020年3月30日，下午2:43，jincheng sun 写道：
> >
> > Hi Wei,
> >
> > +1, Thanks for this discussion which is crucial f
> 1) it's more readable to separete them into different keys, because they
are orthogonal concepts.
I'm not sure whether this is more clear, but using one key will be
definitely one line less.
> 2) the planner can give all the availble versions in the exception
message, if user uses a wrong versio
Thanks Jark for the proposal.
+1 to the general idea.
For "version", what about "kafka.version"? It is obvious to know its
meaning.
And I'd like to start a new topic:
Should we need to explicitly separate source from sink?
With the development of batch and streaming, more and more connectors hav
+1
Best,
Kurt
On Mon, Mar 30, 2020 at 4:08 PM Benchao Li wrote:
> +1 (non-binding)
>
> Jark Wu 于2020年3月30日周一 下午3:57写道：
>
> > +1 from my side.
> >
> > Thanks Timo for driving this.
> >
> > Best,
> > Jark
> >
> > On Mon, 30 Mar 2020 at 15:36, Timo Walther wrote:
> >
> > > Hi all,
> > >
> > > I
+1
Best,
Jingsong Lee
On Mon, Mar 30, 2020 at 4:41 PM Kurt Young wrote:
> +1
>
> Best,
> Kurt
>
>
> On Mon, Mar 30, 2020 at 4:08 PM Benchao Li wrote:
>
> > +1 (non-binding)
> >
> > Jark Wu 于2020年3月30日周一 下午3:57写道：
> >
> > > +1 from my side.
> > >
> > > Thanks Timo for driving this.
> > >
> > >
Repo has been created:
https://github.com/apache/flink-statefun-docker
https://gitbox.apache.org/repos/asf?p=flink-statefun-docker.git
On Mon, Mar 30, 2020 at 3:57 PM Tzu-Li (Gordon) Tai
wrote:
> Thanks for the feedbacks everyone!
> Overall, everyone who replied is positive about creating a sepa
Hi Kurt,
> 2) Lists all available connectors seems also quite straightforward, e.g
user provided a wrong "kafka-0.8", we tell user we have candidates of
"kakfa-0.11", "kafka-universal"
It's not possible for the framework to throw such exception. Because the
framework doesn't know what versions do
Hi Timo,
Agree with you that streaming queries is our top priority,
but I think there are too many things need to discuss for multiline
statements:
e.g.
1. what's the behaivor of DDL and DML mixing for async execution:
create table t1 xxx;
create table t2 xxx;
insert into t2 select * from t1 where
At the moment the RM does not have a user code class loader and I agree
with Stephan that it should stay like this. This, however, does not mean
that we cannot support pluggable components in the RM. As long as the
plugins are on the system's class path, it should be fine for the RM to
load them. F
Hi Sivaprasanna,
thanks for starting this discussion. In general I like the idea to remove
duplications and move common code to a shared module. As a recommendation,
I would exclude the whole part about Flink's Hadoop compatibility modules
because they are legacy code and hardly used anymore. This
On 18.03.20 14:45, Flavio Pompermaier wrote:
what do you think if we exploit this job-submission sprint to address also
the problem discussed in https://issues.apache.org/jira/browse/FLINK-10862?
That's a good idea! What should we do? It seems that most committers on
the issue were in favour o
Hello Till,
I agree with having the scope limited and more concentrated. I can file a
Jira and get started with the code changes, as and when someone has some
bandwidth, the review can also be done. What do you think?
Cheers,
Sivaprasanna
On Mon, Mar 30, 2020 at 3:00 PM Till Rohrmann wrote:
>
I also agree that the pluggable ExternalResourceDriver should be loaded by
the cluster class loader. Despite the plugin might be implemented by users,
external resources (as part of task executor resources) should be cluster
configurations, unlike job-level user codes such as UDFs, because the task
I would recommend to wait until a committer has signed up for reviewing
your changes before preparing any PR.
Otherwise the chances are high that you invest a lot of time but the
changes never get in.
On 30/03/2020 11:42, Sivaprasanna wrote:
Hello Till,
I agree with having the scope limited a
Hi @Till, @Xintong
I think even without the credential concerns, replacing the interfaces
with configuration options is a good idea from my side.
- Currently, I don't see any external resource does not compatible
with this mechanism
- It reduces the burden of users to implement a plugin themselves
lining created FLINK-16863:
--
Summary: add lastModified as a field of LogInfo
Key: FLINK-16863
URL: https://issues.apache.org/jira/browse/FLINK-16863
Project: Flink
Issue Type: Improvement
I would personally like to see a way of describing a Flink job/pipeline
(including its parameters and types) in order to enable better UIs, then
the important thing is to make things consistent and aligned with the new
client developments and exploit this new dev sprint to fix such issues.
On Mon,
Wenlong Lyu created FLINK-16864:
---
Summary: Add idle metrics for Task
Key: FLINK-16864
URL: https://issues.apache.org/jira/browse/FLINK-16864
Project: Flink
Issue Type: Improvement
Com
zhisheng created FLINK-16865:
Summary: 【Flink Kafka Connector】Restore from Savepoint，if add new
Kafka topic，Flink will consume the new topic from earlist，it may cause
duplicate data sink
Key: FLINK-16865
URL: https:/
> It's not possible for the framework to throw such exception. Because the
framework doesn't know what versions do the connector support.
Not really, we can list all valid connectors framework could found. E.g.
user mistyped 'kafka-0.x', the error message will looks like:
we don't have any connec
+1(non-binding)
Best,
Leonard Xu
> 在 2020年3月30日，16:43，Jingsong Li 写道：
>
> +1
>
> Best,
> Jingsong Lee
>
> On Mon, Mar 30, 2020 at 4:41 PM Kurt Young wrote:
>
>> +1
>>
>> Best,
>> Kurt
>>
>>
>> On Mon, Mar 30, 2020 at 4:08 PM Benchao Li wrote:
>>
>>> +1 (non-binding)
>>>
>>> Jark Wu 于
Hi Folks,
I am very excited to announce the integration work of flink on apache
zeppelin notebook is completed. You can now run flink jobs via datastream
api, table api, sql, pyflink in apache apache zeppelin notebook. Download
it here http://zeppelin.apache.org/download.html),
Here's some highli
If there is no need for the ExternalResourceDriver on the RM side, then it
is always a good idea to keep it simple and don't introduce it. One can
always change things once one realizes that there is a need for it.
Cheers,
Till
On Mon, Mar 30, 2020 at 12:00 PM Yangze Guo wrote:
> Hi @Till, @Xin
This is great news Jeff! Thanks a lot for sharing it with the community.
Looking forward trying Flink on Zeppelin out :-)
Cheers,
Till
On Mon, Mar 30, 2020 at 2:47 PM Jeff Zhang wrote:
> Hi Folks,
>
> I am very excited to announce the integration work of flink on apache
> zeppelin notebook is c
Jimmy Wong created FLINK-16868:
--
Summary: Table/SQL doesn't support custom trigger
Key: FLINK-16868
URL: https://issues.apache.org/jira/browse/FLINK-16868
Project: Flink
Issue Type: New Feature
Till Rohrmann created FLINK-16867:
-
Summary: Simplify default timeout configuration
Key: FLINK-16867
URL: https://issues.apache.org/jira/browse/FLINK-16867
Project: Flink
Issue Type: Improvem
Till Rohrmann created FLINK-16866:
-
Summary: Make job submission non-blocking
Key: FLINK-16866
URL: https://issues.apache.org/jira/browse/FLINK-16866
Project: Flink
Issue Type: Improvement
Hi Jark,
Thanks for starting the discussion. The FLIP LGTM generally.
Regarding connector version VS put version into connector's name,
I favor the latter personally, using one property to locate a connector can
make the error log more precise.
>From the users' side, using one property to match a
Hi Dawid,
thanks for updating the FLIP. One minor comment from my side, should we
move the LIKE clause to the very end?
CREATE TABLE X () WITH () LIKE ...
Otherwise, the LIKE clause looks a bit lost if there are options
afterwards. Otherwise, +1 for start a vote from my side.
Regards,
Timo
Hi All,
Thanks for this FLIP, Yadong. This is a very good improvement to the
Flink's UI.
It looks like there are still couple of things to resolve before the final
vote.
- I also find the non-heap title in configuration confusing because there
are also other non-heap types of memory. The "off-hea
Nikola created FLINK-16869:
--
Summary: Flink UI does not show version
Key: FLINK-16869
URL: https://issues.apache.org/jira/browse/FLINK-16869
Project: Flink
Issue Type: Bug
Components: Runt
Hi Jark,
thanks for the FLIP. I don't have a strong opinion on
DynamicTableFactory#factoryVersion() for distiguishing connector
versions. We can also include it in the factory identifier itself. For
Kafka, I would then just use `kafka` because the naming "universal" was
just a helper solution
Hi all,
I like the overall design of the FLIP.
As for the withstanding concerns. I kind of like the approach to put the
version into the factory identifier. I think it's the cleanest way to
say that this version actually applies to the connector itself and not
to the system it connects to. BTW, I
Hi Godfrey,
maybe I wasn't expressing my biggest concern enough in my last mail.
Even in a singleline and sync execution, I think that streaming queries
should not block the execution. Otherwise it is not possible to call
collect() or print() on them afterwards.
"there are too many things ne
Nikola created FLINK-16870:
--
Summary: OrcTableSource throws NullPointerException
Key: FLINK-16870
URL: https://issues.apache.org/jira/browse/FLINK-16870
Project: Flink
Issue Type: Bug
Comp
Niels Basjes created FLINK-16871:
Summary: Make more build information available during runtime.
Key: FLINK-16871
URL: https://issues.apache.org/jira/browse/FLINK-16871
Project: Flink
Issue T
Nikola created FLINK-16872:
--
Summary: Flink UI shows server error on selecting back pressure on
a batch job
Key: FLINK-16872
URL: https://issues.apache.org/jira/browse/FLINK-16872
Project: Flink
I
+1 on allowing user defined resourceId for taskmanager
On Sun, Mar 29, 2020 at 7:24 PM Yang Wang wrote:
> Hi Konstantin,
>
> I think it is a good idea. Currently, our users also report a similar issue
> with
> resourceId of standalone cluster. When we start a standalone cluster now,
> the `TaskM
Hi all,
Thanks for the feedbacks.
It seems that we have a conclusion to put the version into the factory
identifier. I'm also fine with this.
If we have this outcome, the interface of Factory#factoryVersion is not
needed anymore, this can simplify the learning cost of new factory.
We may need to
Hi all,
Thanks for the feedbacks.
It seems that we have a conclusion to put the version into the factory
identifier. I'm also fine with this.
If we have this outcome, the interface of Factory#factoryVersion is not
needed anymore, this can simplify the learning cost of new factory.
We may need to
Hi,
I didn't follow the full discussion.
But I share the same concern with Timo that streaming queries should always
be async.
Otherwise, I can image it will cause a lot of confusion and problems if
users don't deeply keep the "sync" in mind (e.g. client hangs).
Besides, the streaming mode is stil
Why are there unnecessary ';' lines in the code style of any files? For
example, line 41 of the file Flink / Flink Runtime / SRC / main / Java / org /
Apache / Flink / Runtime / client / clientutils.java.
Thanks.
Hi Gordon,
+1 (non-binding)
* Maven build from source...check
* Python build from source...check
* Went through Walkthrough based on local builds...check
Cheers,
Konstantin
On Mon, Mar 30, 2020 at 5:52 AM Tzu-Li (Gordon) Tai
wrote:
> Hi everyone,
>
> Please review and vote on the *release ca
Regarding to version delimiter, I'm in favor of using "-", not "_". Not
only because we are using
"-" for config keys in other module, like CoreOptions, DeploymentOptions,
but also "-" is easier
to type than "_" which doesn't need to press "shift" ;-)
Best,
Kurt
On Tue, Mar 31, 2020 at 12:37 AM
Regarding to "property-version", even this is brought up by myself, but I'm
actually not a fan of this.
As a user, I think this property is too much for me to learn, and I
probably won't use this anyway.
If framework can follow a good procedure to upgrade properties, show some
error messages (but s
Because it is a Enum. For java Enums, you need to first define its values,
followed by a semicolon, then the methods. In this case, there's no values
because we are only using Enum to keep ClientUtils singleton, but you still
need the semicolon before defining methods. You could see that removing
t
Concise and comprehensive.Thank you.
At 2020-03-31 09:45:35, "Xintong Song" wrote:
>Because it is a Enum. For java Enums, you need to first define its values,
>followed by a semicolon, then the methods. In this case, there's no values
>because we are only using Enum to keep ClientUtils singleton,
Hi Jeff,
Thanks for the great work and sharing it with the community! Very impressive
and will try it out.
Regards,
Dian
> 在 2020年3月30日，下午9:16，Till Rohrmann 写道：
>
> This is great news Jeff! Thanks a lot for sharing it with the community.
> Looking forward trying Flink on Zeppelin out :-)
>
Hi everyone,
I've updated the FLIP accordingly. The key change is replacing two
resource allocation interfaces to config options.
If there are no further comments, I would like to start a voting
thread by tomorrow.
Best,
Yangze Guo
On Mon, Mar 30, 2020 at 9:15 PM Till Rohrmann wrote:
>
> If the
Hi all,
I would like to start the vote for FLIP-114[1] which is discussed and reached
consensus in the discussion thread[2].
The vote will be open for at least 72 hours. I'll try to close it by 2020-04-03
03:00 UTC, unless there is an objection or not enough votes.
Best,
Wei
[1]
https://cwik
Yu Li created FLINK-16873:
-
Summary: Document the suggested memory size for statefun docker
machine
Key: FLINK-16873
URL: https://issues.apache.org/jira/browse/FLINK-16873
Project: Flink
Issue Type:
+1 (non-binding)
Checked sums and signatures: OK
Checked no binaries in source distribution: OK
Checked RAT and end-to-end tests (8u101, 11.0.4): OK
Checked version in pom/README/setup.py files: OK
Checked quick start: OK
Checked Greeter local docker-compose examples: OK
Checked Ridesharing local
Yangze Guo created FLINK-16874:
--
Summary: Respect the dynamic options when calculating memory
options in taskmanager.sh
Key: FLINK-16874
URL: https://issues.apache.org/jira/browse/FLINK-16874
Project: Fl
Hi all,
I would like to start the vote for FLIP-113 [1], which is discussed and
reached a consensus in the discussion thread [2].
The vote will be open until April 2nd (72h), unless there is an
objection or not enough votes.
Thanks,
Danny Chan
[1]
https://cwiki.apache.org/confluence/display/FLI
Thanks for the continuous efforts for engaging in Flink ecosystem Jeff!
Glad to see the progressive achievement. Wish more users try it out in practice.
Best,
Zhijiang
--
From:Dian Fu
Send Time:2020 Mar. 31 (Tue.) 10:15
To:Jeff Z
Thanks Jeff very much, that is very impressive.
Zeppelin is very convenient development platform.
Best,
Jingsong Lee
On Tue, Mar 31, 2020 at 11:58 AM Zhijiang
wrote:
>
> Thanks for the continuous efforts for engaging in Flink ecosystem Jeff!
> Glad to see the progressive achievement. Wish more
Thanks Jark for bring up this discussion, +1 for this idea, I believe the user
has suffered from the verbose property key for long time.
Just one question, how do we handle the keys with wildcard, such as the
“connector.properties.*” in Kafka connector which would then hand-over to Kafka
client
FYI - I've also updated the website Downloads page to include this release.
Please also consider that for your reviews:
https://github.com/apache/flink-web/pull/318
On Tue, Mar 31, 2020 at 3:42 AM Konstantin Knauf
wrote:
> Hi Gordon,
>
> +1 (non-binding)
>
> * Maven build from source...check
> *
+1 (binding)
** Legal **
- checksums and GPG files match corresponding release files
- Source distribution does not contain binaries, contents are sane (no
.git* / .travis* / generated html content files)
- Bundled source LICENSEs and NOTICE looks good. Mentions bundled
font-awesome dependency in
jinhai created FLINK-16875:
--
Summary: Tracking replace QueryConfig and TableConfig
implementation with ConfigOption
Key: FLINK-16875
URL: https://issues.apache.org/jira/browse/FLINK-16875
Project: Flink
Yun Tang created FLINK-16876:
Summary: Make TtlTimeProvider configurable when creating keyed
state backend
Key: FLINK-16876
URL: https://issues.apache.org/jira/browse/FLINK-16876
Project: Flink
Hi, Timo & Jark
Thanks for your explanation.
Agree with you that async execution should always be async,
and sync execution scenario can be covered by async execution.
It helps provide an unified entry point for batch and streaming.
I think we can also use sync execution for some testing.
So, I a
Thanks for the nice suggestion Till.
The section 'Bulk Slot Allocation' is updated.
Thanks,
Zhu Zhu
Gary Yao 于2020年3月30日周一 下午3:38写道：
> >
> > The links work for me now. Someone might have fixed them. Never mind.
> >
>
> Actually, I fixed the links after seeing your email. Thanks for reporting.
>
Thank you all for the feedback! Sorry for the belated reply.
@Till
I'm +1 for your two ideas and I'd like to move these two out of the
scope of this FLIP since the pipelined region scheduling is an ongoing
work now.
I also agree that we should not make the InstanceID in
TaskExecutorConnection bein
While checking the release, I found a 77
MB statefun-parent-2.0.0-source-release.zip file in the maven staging repo:
https://repository.apache.org/content/repositories/orgapacheflink-1343/org/apache/flink/statefun-parent/2.0.0/
It seems that the file contains all ruby dependencies in docs/ from je
Hi Robert,
I think you're right. There should be no tarballs / jars packaged for
statefun-parent actually, only the pom file since that's the parent module
which only has pom packaging.
I'm looking into it.
On Tue, Mar 31, 2020 at 2:23 PM Robert Metzger wrote:
> While checking the release, I fo
Hi everyone,
I'd like to start the vote of FLIP-84[1] again, because we have some
feedbacks. The feedbacks are all about new introduced methods, here is the
discussion thread [2].
The vote will be open for at least 72 hours. Unless there is an objection,
I will try to close it by Apr 3, 2020 06:3
Found the culprit:
The Stateful Functions project uses the Apache POM as the parent POM, and
uses the `apache-release` build profile to build the staging jars.
The problem arises because the `apache-release` build profile itself
bundles a source release distribution to be released to Maven.
This
Jingsong Lee created FLINK-16877:
Summary: SingleDirectoryWriter should not produce file when no
input record
Key: FLINK-16877
URL: https://issues.apache.org/jira/browse/FLINK-16877
Project: Flink
Hi Kurt,
I also prefer "-" as version delimiter now. I didn't remove the "_"
proposal by mistake, that's why I sent another email last night :)
Regarding to "property-version", I also think we shouldn't let users to
learn about this. And ConfigOption provides a good ability
to support deprecated k
Thank you for looking into this.
I'm fine with keeping this RC open, but re-vote on a new maven staging
repository.
On Tue, Mar 31, 2020 at 8:42 AM Tzu-Li (Gordon) Tai
wrote:
> Found the culprit:
>
> The Stateful Functions project uses the Apache POM as the parent POM, and
> uses the `apache-re
Sounds good, I'll post a new link to this vote thread, which will have the
problem fixed in a new maven staging repository.
On Tue, Mar 31, 2020 at 2:51 PM Robert Metzger wrote:
> Thank you for looking into this.
>
> I'm fine with keeping this RC open, but re-vote on a new maven staging
> reposi
Robert Metzger created FLINK-16878:
--
Summary: flink-table-planner contains unwanted dependency
org.apiguardian.api
Key: FLINK-16878
URL: https://issues.apache.org/jira/browse/FLINK-16878
Project: Fli
Hi everyone,
Thanks for the great feedbacks so far.
I updated the FLIP documentation according to the discussion. Changes
include:
- remove "version" key, and merge it into "connector"
- add "scan", "lookup", "sink" prefix to some property keys if they are
only used in that case.
- add a "New Pro
84 matches
Mail list logo
