Terry Wang created FLINK-17152:
--
Summary: FunctionDefinitionUtil generate wrong resultType and acc
type of AggregateFunctionDefinition
Key: FLINK-17152
URL: https://issues.apache.org/jira/browse/FLINK-17152
Hi Everyone,
I'd like to discuss about releasing a more full-featured Flink
distribution. The motivation is that there is friction for SQL/Table API
users that want to use Table connectors which are not there in the
current Flink Distribution. For these users the workflow is currently
roughly
Big +1 from my side.
>From my experience, missing connector & format jar is the TOP 1 problem
which
SQL users will probably run into. Similar questions raised in Flink's
Dingtalk group
almost every 1 or 2 days. And I have personally answered dozens of such
question.
Sometimes it's still not enough
Hi Aljoscha,
Big +1 for the fat flink distribution, where do you plan to put these
connectors ? opt or lib ?
Aljoscha Krettek 于2020年4月15日周三 下午3:30写道：
> Hi Everyone,
>
> I'd like to discuss about releasing a more full-featured Flink
> distribution. The motivation is that there is friction for SQ
Hi Dian,
creating a new 1.9 bug fix release is a very good idea. +1 for creating it
soon. Also thanks for volunteering as our release manager.
Cheers,
Till
On Fri, Apr 10, 2020 at 7:27 AM Dian Fu wrote:
> Hi Jincheng,
>
> Thanks a lot for offering help. It would be very helpful. Thanks again!
Hi community,
I have a question about flink on yarn ha , if active resourcemanager
changed, what is the flink task staus. Is flink task running normally?
Should I must restart my flink task to run？
Thanks to your reply.
Best,
LakeShen
+1 to the proposal. I also found the "download additional jar" step is
really verbose when I prepare webinars.
At least, I think the flink-csv and flink-json should in the distribution,
they are quite small and don't have other dependencies.
Best,
Jark
On Wed, 15 Apr 2020 at 15:44, Jeff Zhang w
Normally, Yarn RM switch should not cause any problem to the running Flink
instance. Unless the RM switch takes too long and Flink happens to request
new containers during that time, it might lead to resource allocation
timeout.
Thank you~
Xintong Song
On Wed, Apr 15, 2020 at 3:49 PM LakeShen
Ok, if there can be multiple resources of the same type then we definitely
need the name as a differentiator.
I agree that such an interface won't give compile time checks but I think
that it could be easier to use from a user's perspective because there is
no explicit casting required.
public in
@Till
If we add "Class externalResourceType" param, what if there are
multiple subtypes in the ExternalResourceInfos set of one external
resource? It seems user has to set the T to ExternalResourceInfo and
the mechanism is useless at this case.
Best,
Yangze Guo
On Wed, Apr 15, 2020 at 3:57 PM Til
Hi David,
making the training materials available on flink.apache.org would increase
the reach and improve its visibility. Since this is very helpful material
for our users +1 for contributing the training material.
If we decide not maintain different versions, then we might be able to
highlight
Great to see that will have the first RC for Flink 1.10.1 soon. Thanks a
lot for driving this effort Yu!
Cheers,
Till
On Sun, Apr 12, 2020 at 5:03 PM Yu Li wrote:
> Thanks Weike and all others for the efforts!
>
> Here comes the latest status, we are in good shape and plan to produce RC1
> next
>
> I agree that such an interface won't give compile time checks but I think
> that it could be easier to use from a user's perspective because there is
> no explicit casting required.
> public interface RuntimeContext {
> Set getExternalResourceInfos(String
> resourceName, Class externalReso
Is this something we expect to happen? If there is an external
resource implementation which shows this behaviour I assume that it is
quite hard to use for a user.
In this case, externalResourceType could also be set
to ExternalResourceInfo.class or any common super type. Then the API won't
give y
> One minor note: I think the value of the returned map does not need to use a
> bounded wildcard type because for the user it won't make a difference.
Since the Driver now use a bounded wildcard type, it seems we could
not hold the return value(Set) with
Set. Am I right?
Best,
Yangze Guo
On We
xiaodao created FLINK-17153:
---
Summary: flink sql convert constant to char type cause loop
Key: FLINK-17153
URL: https://issues.apache.org/jira/browse/FLINK-17153
Project: Flink
Issue Type: Bug
I think Set
getExternalResourceInfos(String resourceName, Class
externalResourceType) is not less flexible than the other API since you can
always pass in ExternalResourceInfo.class as the second argument.
The benefit I see for the user is that he does not have to do the
instanceof checks and typ
+1 for the release and for Dian being the RM.
Thanks Jincheng for your continuous efforts on helping the releasing.
Best,
Hequn
On Wed, Apr 15, 2020 at 3:45 PM Till Rohrmann wrote:
> Hi Dian,
>
> creating a new 1.9 bug fix release is a very good idea. +1 for creating it
> soon. Also thanks for
Yang Wang created FLINK-17154:
-
Summary: Make ResourceManager#internalDeregisterApplication and
implementation asynchronous
Key: FLINK-17154
URL: https://issues.apache.org/jira/browse/FLINK-17154
Project:
+1 to this proposal.
Missing connector jars is also a big problem for PyFlink users. Currently,
after a Python user has installed PyFlink using `pip`, he has to manually copy
the connector fat jars to the PyFlink installation directory for the connectors
to be used if he wants to run jobs local
True, the user can always pass in ExternalResourceInfo.class to retain the
flexibility.
As long as the flexibility is not harmed, I'm ok with both. It's probably
better to do the type checking and exception handling for users.
Thank you~
Xintong Song
On Wed, Apr 15, 2020 at 4:23 PM Till Rohrma
Thanks for the explanation. I do not have a strong opinion regarding
this interface. So, if it is better from your perspective, I'm +1 for
this. I just saying it may not help a lot regarding the type-safe.
Regarding the bounded wildcard type, yes, it's the implementation
detail. If it won't make a
jackylau created FLINK-17155:
Summary: flink in memory catalog can not support hive udf
Key: FLINK-17155
URL: https://issues.apache.org/jira/browse/FLINK-17155
Project: Flink
Issue Type: Bug
Big +1.
This will improve user experience (special for Flink new users).
We answered so many questions about "class not found".
Best,
Godfrey
Dian Fu 于2020年4月15日周三 下午4:30写道：
> +1 to this proposal.
>
> Missing connector jars is also a big problem for PyFlink users. Currently,
> after a Python us
This is good news Yangze. Decreasing the size of our id's is a really nice
side effect :-)
Hence, +1 from my side as well.
Cheers,
Till
On Tue, Apr 14, 2020 at 9:54 AM Zhu Zhu wrote:
> Thanks for doing this benchmark @Yangze Guo .
> The result looks promising. So I would +1 to refactor Execut
Roman Khachatryan created FLINK-17156:
-
Summary: Add checkpoint cancellation to Unaligner
Key: FLINK-17156
URL: https://issues.apache.org/jira/browse/FLINK-17156
Project: Flink
Issue Type
Thanks for the feedback, @ZhuZhu and @Till
Thanks for the deeper analysis of the size of TDD, @ZhuZhu.
If there is no further concern in the next 24 hours, I'll start voting
for this FLIP.
Best,
Yangze Guo
On Wed, Apr 15, 2020 at 4:56 PM Till Rohrmann wrote:
>
> This is good news Yangze. Decrea
Jark Wu created FLINK-17157:
---
Summary: TaskMailboxProcessorTest.testIdleTime failed on travis
Key: FLINK-17157
URL: https://issues.apache.org/jira/browse/FLINK-17157
Project: Flink
Issue Type: Bug
Hi, Everyone.
Google is running its Season of Docs [1] program again this year. The goal
of the program is to pair open source organizations/projects with
professional technical writers to improve their documentation.
The Flink community submitted an application in 2019 (led by Konstantin)
[2,3],
+1 to have a 1.10.1 RC soon. It has been a long time since 1.10.0 is
released.
Best,
Jark
On Wed, 15 Apr 2020 at 16:10, Till Rohrmann wrote:
> Great to see that will have the first RC for Flink 1.10.1 soon. Thanks a
> lot for driving this effort Yu!
>
> Cheers,
> Till
>
> On Sun, Apr 12, 2020 a
Hi all,
My vote is +1 (binding).
Thanks for participating and giving your votes.
Hereby, the vote is closed and the FLIP is accepted with no -1/vetos.
+1's:
3 binding (Ufuk, Till, Andrey)
4 non-binding (Yang, Canbin, Ismaël, Yangze)
Thanks,
Andrey
On Fri, Apr 10, 2020 at 8:40 AM Yangze Guo w
Big +1.
I like "fat" and "slim".
For csv and json, like Jark said, they are quite small and don't have other
dependencies. They are important to kafka connector, and important
to upcoming file system connector too.
So can we move them to both "fat" and "slim"? They're so important, and
they're so
Timo Walther created FLINK-17158:
Summary: Watermark strategy property cannot be expressed in YAML
Key: FLINK-17158
URL: https://issues.apache.org/jira/browse/FLINK-17158
Project: Flink
Issue
Chesnay Schepler created FLINK-17159:
Summary: ES6 ElasticsearchSinkITCase unstable
Key: FLINK-17159
URL: https://issues.apache.org/jira/browse/FLINK-17159
Project: Flink
Issue Type: Bug
Andrey Zagrebin created FLINK-17160:
---
Summary: FLIP-111: Docker image unification
Key: FLINK-17160
URL: https://issues.apache.org/jira/browse/FLINK-17160
Project: Flink
Issue Type: Improvem
Andrey Zagrebin created FLINK-17161:
---
Summary: Document the official docker hub image and examples of
how to run
Key: FLINK-17161
URL: https://issues.apache.org/jira/browse/FLINK-17161
Project: Flin
Andrey Zagrebin created FLINK-17162:
---
Summary: Document examples of how to extend the official docker
hub image
Key: FLINK-17162
URL: https://issues.apache.org/jira/browse/FLINK-17162
Project: Flink
Andrey Zagrebin created FLINK-17163:
---
Summary: Remove flink-contrib/docker-flink
Key: FLINK-17163
URL: https://issues.apache.org/jira/browse/FLINK-17163
Project: Flink
Issue Type: Sub-task
Andrey Zagrebin created FLINK-17164:
---
Summary: Extend entry point script and docs with job cluster mode
and user job artefacts
Key: FLINK-17164
URL: https://issues.apache.org/jira/browse/FLINK-17164
Andrey Zagrebin created FLINK-17165:
---
Summary: Remove flink-container/docker
Key: FLINK-17165
URL: https://issues.apache.org/jira/browse/FLINK-17165
Project: Flink
Issue Type: Sub-task
Andrey Zagrebin created FLINK-17166:
---
Summary: Modify the log4j-console.properties to also output logs
into the files for WebUI
Key: FLINK-17166
URL: https://issues.apache.org/jira/browse/FLINK-17166
Andrey Zagrebin created FLINK-17167:
---
Summary: Extend entry point script and docs with history server
mode
Key: FLINK-17167
URL: https://issues.apache.org/jira/browse/FLINK-17167
Project: Flink
Yu Li created FLINK-17168:
-
Summary: TPC-DS end-to-end test failed on travis due to failed to
download data generator
Key: FLINK-17168
URL: https://issues.apache.org/jira/browse/FLINK-17168
Project: Flink
+1 to create a new 1.9 bug fix release. Thanks Dian for volunteering as our
RM.
Best Regards,
Yu
On Wed, 15 Apr 2020 at 16:25, Hequn Cheng wrote:
> +1 for the release and for Dian being the RM.
> Thanks Jincheng for your continuous efforts on helping the releasing.
>
> Best,
> Hequn
>
> On Wed
Regarding to the specific solution, I'm not sure about the "fat" and "slim"
solution though. I get the idea
that we can make the slim one even more lightweight than current
distribution, but what about the "fat"
one? Do you mean that we would package all connectors and formats into
this? I'm not su
I don't see a lot of value in having multiple distributions.
The simple reality is that no fat distribution we could provide would
satisfy all use-cases, so why even try.
If users commonly run into issues for certain jars, then maybe those
should be added to the current distribution.
Personal
Hi,
I think we should first reach an consensus on "what problem do we want to
solve?"
(1) improve first experience? or (2) improve production experience?
As far as I can see, with the above discussion, I think what we want to
solve is the "first experience".
And I think the slim jar is still the
+1 to create a new 1.9 bugfix release. and FLINK-16576[1] has merged into
master, filed a pr for release-1.9 already
[1] https://issues.apache.org/jira/browse/FLINK-16576
Best,
Congxian
Yu Li 于2020年4月15日周三 下午9:16写道：
> +1 to create a new 1.9 bug fix release. Thanks Dian for volunteering as our
Jark Wu created FLINK-17169:
---
Summary: Refactor BaseRow to use RowKind instead of byte header
Key: FLINK-17169
URL: https://issues.apache.org/jira/browse/FLINK-17169
Project: Flink
Issue Type: Sub-
+1 for releasing 1.9.3 soon.
Thanks Dian for driving this!
Best,
Jark
On Wed, 15 Apr 2020 at 22:11, Congxian Qiu wrote:
> +1 to create a new 1.9 bugfix release. and FLINK-16576[1] has merged into
> master, filed a pr for release-1.9 already
>
> [1] https://issues.apache.org/jira/browse/FLINK-16
+1 for this idea.
I think there would existed many details to discuss once community ready to
host the materials:
1. How to judge whether a lab exercise should be added? There would be many
user cases for streaming computation, I think we should need a outline for the
knowledge map of Flink
Thanks you all for the positive feedback and thanks a lot Congxian and Yu for
working on FLINK-16576.
I'll create the first RC after FLINK-16576 is merged to release-1.9.
Feel free to let to know if there are any other issues you'd like to be
included in this release.
Thanks,
Dian
> 在 2020年4月
Hi everyone,
thanks for starting this discussion Niels.
I like the idea of getting rid of parsing a Properties instance. On the
other hand, I also understand that people are concerned about disrupting
the workflows of our devs.
Maybe we can find a compromise between both approaches. For example,
Hi,
I am thinking both "improve first experience" and "improve production
experience".
I'm thinking about what's the common mode of Flink?
Streaming job use Kafka? Batch job use Hive?
Hive 1.2.1 dependencies can be compatible with most of Hive server
versions. So Spark and Presto have built-in H
+1. It's very useful for Flink newcomers.
Best,
Jingsong Lee
On Wed, Apr 15, 2020 at 10:23 PM Yun Tang wrote:
> +1 for this idea.
>
> I think there would existed many details to discuss once community ready
> to host the materials:
>
>1. How to judge whether a lab exercise should be added?
Hi Till,
+1 to define an interface and load it at runtime if we can do.
No disrupting the workflows of devs and throw an exception with good
description look good to me.
This also force us to do a good dependent class abstract.
Best,
Jingsong Lee
On Wed, Apr 15, 2020 at 10:31 PM Till Rohrmann w
It doesn't have to be a properties file, nor do we necessarily have to
do any manual parsing.
It could just be a JSON file that we point Jackson at. Naturally we
could also generate it with Jackson.
You'd have a POJO for all the fields with sane defaults (an analogue to
the proposed generated PO
Vasii Cosmin Radu created FLINK-17170:
-
Summary: Cannot stop streaming job with savepoint which uses
kinesis consumer
Key: FLINK-17170
URL: https://issues.apache.org/jira/browse/FLINK-17170
Projec
Is the only really new method on the public APIs
getExternalResourceInfos(..) on the RuntimeContext? I'm generally quite
skeptical about adding anything to that interface but the method seems ok.
Side note for the configuration keys: the pattern is similar to metrics
configuration. There we ha
I'm not advocating for a specific approach. The point I wanted to make is
that there are solutions which allow us to get rid of the problematic
parsing and not disrupting the workflow. If the Jackson JSON file approach
works for Niels, then I'm fine with that as well.
Cheers,
Till
On Wed, Apr 15,
Hi all,
Regarding slim and fat distributions, I think different kinds of jobs may
prefer different type of distribution:
For DataStream job, I think we may not like fat distribution containing
connectors because user would always need to depend on the connector in
user code, it is easy to include
Nico Kruber created FLINK-17171:
---
Summary: Blink planner fails to compile Table program with POJO
source
Key: FLINK-17171
URL: https://issues.apache.org/jira/browse/FLINK-17171
Project: Flink
Till, Yun, etal,
Now that we've established the community's interest in engaging with this
content, I've started a new thread on the dev list for discussion of the
details.
I've said a bit there already regarding ongoing maintenance, and CI for the
exercises.
Best,
David
On Wed, Apr 15, 2020 at
Thank you all for the very positive response to our proposal to contribute
the training materials that have been at training.ververica.com to the
Apache Flink project. Now I’d like to begin the more detailed discussion of
how to go about this.
In that earlier thread I mentioned that we were thinki
Gary Yao created FLINK-17172:
Summary: Re-enable debug level logging in Jepsen Tests
Key: FLINK-17172
URL: https://issues.apache.org/jira/browse/FLINK-17172
Project: Flink
Issue Type: Bug
Hi David,
I am happy to get the repo created for you.
If we go with the documentation (vs flink.apache.org) do you think we
should remove any of the existing content? There is already a getting
started section with quickstarts and walkthroughs and a concepts section.
In particular, the concepts s
Aljoscha, I know you've been working on the docs recently. Are you working
on the concepts section? I don't want to just drop it if someone has work
in the pipeline.
Seth
On Wed, Apr 15, 2020 at 2:19 PM Seth Wiesman wrote:
> Hi David,
>
> I am happy to get the repo created for you.
>
> If we g
I agree with Aljoscha. It is important to keep API the same style. And we
probably should move the long discussion to the [DISCUSS] thread.
Thanks,
Jiangjie (Becket) Qin
On Wed, Apr 15, 2020 at 11:27 PM Aljoscha Krettek
wrote:
> Is the only really new method on the public APIs
> getExternalRes
Hi Aljoscha,
Thanks for your advice. +1 to align the config pattern.
I also agree that we need to move the long discussion to the [DISCUSS]
thread. Sorry if it bothers you.
Best,
Yangze Guo
On Thu, Apr 16, 2020 at 7:52 AM Becket Qin wrote:
>
> I agree with Aljoscha. It is important to keep API
Looking forward the first RC of Flink 1.10.1 .
Good job Yu!
Best,
Jincheng
Jark Wu 于2020年4月15日周三 下午6:28写道：
> +1 to have a 1.10.1 RC soon. It has been a long time since 1.10.0 is
> released.
>
> Best,
> Jark
>
> On Wed, 15 Apr 2020 at 16:10, Till Rohrmann wrote:
>
> > Great to see that will
Danny Chen created FLINK-17173:
--
Summary: Supports query hint to config "IdleStateRetentionTime"
per query in SQL
Key: FLINK-17173
URL: https://issues.apache.org/jira/browse/FLINK-17173
Project: Flink
Thanks a lot for driving this, Yu! Looking forward for the first RC of 1.10.1.
> 在 2020年4月16日，上午10:24，jincheng sun 写道：
>
> Looking forward the first RC of Flink 1.10.1 .
> Good job Yu!
>
> Best,
> Jincheng
>
>
>
> Jark Wu 于2020年4月15日周三 下午6:28写道：
>
>> +1 to have a 1.10.1 RC soon. It has be
Thanks a lot for your great work Yu! Looking forward to the RC.
Best, Hequn
On Thu, Apr 16, 2020 at 10:35 AM Dian Fu wrote:
> Thanks a lot for driving this, Yu! Looking forward for the first RC of
> 1.10.1.
>
> > 在 2020年4月16日，上午10:24，jincheng sun 写道：
> >
> > Looking forward the first RC of Fli
Yiwen Tu created FLINK-17174:
Summary: Conversion to relational algebra failed to preserve
datatypes when using Table API but succeed when using SQL query
Key: FLINK-17174
URL: https://issues.apache.org/jira/browse/FL
Bowen Li created FLINK-17175:
Summary: StringUtils.arrayToString() should consider Object[]
lastly
Key: FLINK-17175
URL: https://issues.apache.org/jira/browse/FLINK-17175
Project: Flink
Issue Ty
Canbin Zheng created FLINK-17176:
Summary: Slow down Pod recreation in
KubernetesResourceManager#PodCallbackHandler
Key: FLINK-17176
URL: https://issues.apache.org/jira/browse/FLINK-17176
Project: Fli
Canbin Zheng created FLINK-17177:
Summary: Handle Pod ERROR event correctly in
KubernetesResourceManager#onError
Key: FLINK-17177
URL: https://issues.apache.org/jira/browse/FLINK-17177
Project: Flink
Lijie Wang created FLINK-17178:
--
Summary: Provide "ALL" cache strategy in LookupFunction
Key: FLINK-17178
URL: https://issues.apache.org/jira/browse/FLINK-17178
Project: Flink
Issue Type: New F
Yiwen Tu created FLINK-17179:
Summary: Set row time attribute not working in connector descriptor
Key: FLINK-17179
URL: https://issues.apache.org/jira/browse/FLINK-17179
Project: Flink
Issue Type
Gary Yao created FLINK-17180:
Summary: Implement PipelinedRegion interface for SchedulingTopology
Key: FLINK-17180
URL: https://issues.apache.org/jira/browse/FLINK-17180
Project: Flink
Issue Type
Gary Yao created FLINK-17181:
Summary: Simplify generic Types in Topology Interface
Key: FLINK-17181
URL: https://issues.apache.org/jira/browse/FLINK-17181
Project: Flink
Issue Type: Sub-task
81 matches
Mail list logo
