victor.jiang created FLINK-16794:
Summary: ClassNotFoundException caused by
ClassLoader.getSystemClassLoader using impertinently
Key: FLINK-16794
URL: https://issues.apache.org/jira/browse/FLINK-16794
Robert Metzger created FLINK-16795:
--
Summary: End to end tests timeout on Azure
Key: FLINK-16795
URL: https://issues.apache.org/jira/browse/FLINK-16795
Project: Flink
Issue Type: Task
Huang Xingbo created FLINK-16796:
Summary: Fix The Bug of Python UDTF in SQL Query
Key: FLINK-16796
URL: https://issues.apache.org/jira/browse/FLINK-16796
Project: Flink
Issue Type: Bug
Xintong Song created FLINK-16798:
Summary: Logs from BashJavaUtils are not properly preserved and
passed into TM logs.
Key: FLINK-16798
URL: https://issues.apache.org/jira/browse/FLINK-16798
Project:
Jeff Zhang created FLINK-16797:
--
Summary: Flink doesn't merge multiple sinks into one DAG
Key: FLINK-16797
URL: https://issues.apache.org/jira/browse/FLINK-16797
Project: Flink
Issue Type: Bug
Jun Zhang created FLINK-16799:
-
Summary: add hive partition limit when read from hive
Key: FLINK-16799
URL: https://issues.apache.org/jira/browse/FLINK-16799
Project: Flink
Issue Type: Improvemen
Zhenghua Gao created FLINK-16800:
Summary: TypeMappingUtils#checkPhysicalLogicalTypeCompatible
didn't deal with nested types
Key: FLINK-16800
URL: https://issues.apache.org/jira/browse/FLINK-16800
Pro
Robert Metzger created FLINK-16801:
--
Summary: PostgresCatalogITCase fails with IOException
Key: FLINK-16801
URL: https://issues.apache.org/jira/browse/FLINK-16801
Project: Flink
Issue Type:
Rui Li created FLINK-16802:
--
Summary: Set schema info in JobConf for Hive readers
Key: FLINK-16802
URL: https://issues.apache.org/jira/browse/FLINK-16802
Project: Flink
Issue Type: Task
Co
Hi Godfrey,
having control over the job after submission is a requirement that was
requested frequently (some examples are [1], [2]). Users would like to
get insights about the running or completed job. Including the jobId,
jobGraph etc., the JobClient summarizes these properties.
It is good
Rui Li created FLINK-16803:
--
Summary: Need to make sure partition inherit table spec when
writing to Hive tables
Key: FLINK-16803
URL: https://issues.apache.org/jira/browse/FLINK-16803
Project: Flink
Thanks for the update Danny. +1 for this proposal.
Regards,
Timo
On 26.03.20 04:51, Danny Chan wrote:
Thanks everyone who engaged in this discussion ~
Our goal is "Supports Dynamic Table Options for Flink SQL". After an
offline discussion with Kurt, Timo and Dawid, we have made the final
concl
Hi Danny,
Thanks for the updates. I have 2 comments regarding to latest document:
1) I think we also need `*supportedHintOptions*` for `*TableFormatFactory*`
2) IMO "dynamic-table-options.enabled" should belong to `
*OptimizerConfigOptions*`
Best,
Kurt
On Thu, Mar 26, 2020 at 4:40 PM Timo Wal
Dawid Wysakowicz created FLINK-16804:
Summary: Deprecate String based Expression DSL in Table
Key: FLINK-16804
URL: https://issues.apache.org/jira/browse/FLINK-16804
Project: Flink
Issue
Thanks for the suggestion, @Stephan, @Becket and @Xintong.
I've updated the FLIP accordingly. I do not add a
ResourceInfoProvider. Instead, I introduce the ExternalResourceDriver,
which takes the responsibility of all relevant operations on both RM
and TM sides.
After a rethink about decoupling th
Hi Becket,
Regarding "PushDown/NestedPushDown which is internal to optimizer":
Those concepts cannot be entirely internal to the optimizer, at some
point the optimizer needs to pass them into the connector specific code.
This code will then convert it to e.g. Parque expressions. So there must
Hi Danny,
Regarding to `supportedHintOptions()` interface, I suggest to use the
inverted version, `unsupportedHintOptions()`.
Because I think the disallowed list is much smaller.
In addition, it's hard to list all the properties under
`connector.properties.*`.
But we know `connector.properties.boo
Thanks Kurt for the suggestion ~
In my opinion:
- There is no need for TableFormatFactory#supportedHintOptions because all
the format options can be configured dynamically, they have no security
issues
- Dynamic table options is not an optimization, it is more like an
execution behavior from my si
Thanks Jark for the feedback ~
I actually have a discussion offline with Timo and we think the black-list
options has implicit rick with the growing new table options, a black-list
there means all the new introduced options are default to be configurable
dynamically, if the user forget to add it i
Hi everyone,
We would like to start a discussion thread on "FLIP-118: Improve
Flink’s ID system"[1].
This FLIP mainly discusses the following issues, target to enhance the
readability of IDs in log and help user to debug in case of failures:
- Enhance the readability of the string literals of ID
Thanks Danny for update.
+1 to "dynamic-table-options.enabled" should belong to
`*OptimizerConfigOptions*`.
Hint looks like optimizer in my opinion. Actually optimizer affect
execution, but it is optimizer, not directly related to execution.
+1 to `unsupportedHintOptions`, we already list all opt
Hi everyone,
it is not only about security concerns. Hint options should be
well-defined. We had a couple of people that were concerned about
changing the semantics with a concept that is called "hint". These
options are more like "debugging options" while someone is developing a
connector or
Thanks for updating the FLIP Yadong.
What is the difference between managedMemory and managedMemoryTotal
and networkMemory and networkMemoryTotal in the REST response? If they are
duplicates, then we might be able to remove one.
Apart from that, the proposal looks good to me.
Pulling also Andrey
Robert Metzger created FLINK-16805:
--
Summary: StreamingKafkaITCase fails with "Could not instantiate
instance using default factory."
Key: FLINK-16805
URL: https://issues.apache.org/jira/browse/FLINK-16805
Thanks Robert for this dev blog post. It's a good read.
Cheers,
Till
On Mon, Mar 23, 2020 at 10:24 PM Arvid Heise wrote:
> Thank you Robert! (also thanks for incorporating my feedback so swiftly)
>
> On Mon, Mar 23, 2020 at 8:54 PM Seth Wiesman wrote:
>
> > Very interesting! No questions but t
Hi Timo,
Regarding "connector developers just need to know how to write an
> ExpressionToParquetFilter":
>
This is the entire purpose of the DynamicTableSource/DynamicTableSink.
> The bridging between SQL concepts and connector specific concepts.
> Because this is the tricky part. How to get fro
Piotr Nowojski created FLINK-16806:
--
Summary: Implement Input selection for MultipleInputStreamOperator
Key: FLINK-16806
URL: https://issues.apache.org/jira/browse/FLINK-16806
Project: Flink
Thanks for driving this effort Robert. I'd be in favour of disabling Travis
for PRs once AZP is decently stable.
Cheers,
Till
On Wed, Mar 25, 2020 at 8:28 PM Robert Metzger wrote:
> Thank you for your responses.
>
> @Yu Li: In the current master, the log upload always fails, if the e2e job
> fa
Thanks for the clarification Robert.
Since the first step plan is to replace the travis PR runs, I checked all
PR builds from 2020-01-01 (PR#10735-11526) [1], and below is the result:
* Travis FAILURE: 298
* Travis SUCCESS: 649 (68.5%)
* Azure FAILURE: 420
* Azure SUCCESS: 571 (57.6%)
Since the
Nice, thanks a lot!
On Thu, Mar 26, 2020 at 10:21 AM Yangze Guo wrote:
> Thanks for the suggestion, @Stephan, @Becket and @Xintong.
>
> I've updated the FLIP accordingly. I do not add a
> ResourceInfoProvider. Instead, I introduce the ExternalResourceDriver,
> which takes the responsibility of a
Hi Timo,
I agree with you that streaming queries mostly need async execution.
In fact, our original plan is only introducing sync methods in this FLIP,
and async methods (like "executeSqlAsync") will be introduced in the future
which is mentioned in the appendix.
Maybe the async methods also need
Chesnay Schepler created FLINK-16807:
Summary: Improve reporting of errors during resource initialization
Key: FLINK-16807
URL: https://issues.apache.org/jira/browse/FLINK-16807
Project: Flink
Chesnay Schepler created FLINK-16808:
Summary: Consolidated logging in FactoryUtils
Key: FLINK-16808
URL: https://issues.apache.org/jira/browse/FLINK-16808
Project: Flink
Issue Type: Impr
zhuqi created FLINK-16809:
-
Summary: Add Caller Context in Flink
Key: FLINK-16809
URL: https://issues.apache.org/jira/browse/FLINK-16809
Project: Flink
Issue Type: Improvement
Reporter: z
Hi Becket,
Generally I don't think connector developers should bother with
understanding any of the SQL concepts.
I am not sure if we understand "connector developer" the same way. Let
me describe how I see the process of writing a new source (that can be
used in both Table & DataStream API)
1.
Hi Godfrey,
executing streaming queries must be our top priority because this is
what distinguishes Flink from competitors. If we change the execution
behavior, we should think about the other cases as well to not break the
API a third time.
I fear that just having an async execute method wi
By the way: I have disabled the test in the Hadoop 2.4.1 build, so my
original problem is resolved.
I'm not convinced that posting on the user@ ml is the right approach.
The last few questions to the user@ list asking for feedback have not
really been answered. I believe that we have deprecated th
Hi Timo and Dawid,
Thanks for the patient explanation. I just had a phone call with Kurt and
Jark. I do see there are a few abstractions that we only see the use case
in SQL so far. Therefore while thinking of a Source abstraction that may be
shared with different use cases semantics is theoretica
Hi Becket,
thanks for your feedback and the healthy discussion.
I think the connector story will still keep many of us busy in the next
time. It would be great if concepts from SQL can positively influence
the design of Source/Sink abstractions. Esp. we should think about some
guidelines of h
Hi Manas,
First of all I think your understanding of how the session windows work
is correct.
I tend to slightly disagree that the end for a session window is wrong.
It is my personal opinion though. I see it this way that a TimeWindow in
case of a session window is the session itself. The sessio
Hi Flink devs,
As part of a Stateful Functions release, we would like to publish Stateful
Functions Docker images to Dockerhub as an official image.
Some background context on Stateful Function images, for those who are not
familiar with the project yet:
- Stateful Function images are built o
+1 to a separate repository.
It seems to be best practice in the docker community.
And since it does not add overhead, why not go with the best practice?
Best,
Stephan
On Thu, Mar 26, 2020 at 4:15 PM Tzu-Li (Gordon) Tai
wrote:
> Hi Flink devs,
>
> As part of a Stateful Functions release, we w
+1 for a separate repository.
Thanks,
Igal
On Thursday, March 26, 2020, Stephan Ewen wrote:
> +1 to a separate repository.
>
> It seems to be best practice in the docker community.
> And since it does not add overhead, why not go with the best practice?
>
> Best,
> Stephan
>
>
> On Thu, Mar 26,
+1 to use a dedicated repository. All reasons listed in the proposal make
sense to me.
Best Regards,
Yu
On Thu, 26 Mar 2020 at 23:56, Igal Shilman wrote:
> +1 for a separate repository.
>
> Thanks,
> Igal
>
> On Thursday, March 26, 2020, Stephan Ewen wrote:
>
> > +1 to a separate repository.
+1.
The repo creation process is a light-weight, automated process on the ASF
side. When Patrick Lucas contributed docker-flink back to the Flink
community (as flink-docker), there was virtually no overhead in creating
the repository. Reusing build scripts should still be possible at the cost
of s
Hi everyone,
Please review and vote on the release candidate #0 for the version 2.0.0 of
Apache Flink Stateful Functions,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
**Testing Guideline**
You can find here [1] a doc that we can us
Hi everyone,
I'm a bit late to the party. I think the current proposal looks good.
Concerning the ExternalResourceDriver interface defined in the FLIP [1], I
would suggest to not include the decorator calls for Kubernetes and Yarn in
the base interface. Instead I would suggest to segregate the de
+1 for a separate repository.
Cheers,
Till
On Thu, Mar 26, 2020 at 5:13 PM Ufuk Celebi wrote:
> +1.
>
> The repo creation process is a light-weight, automated process on the ASF
> side. When Patrick Lucas contributed docker-flink back to the Flink
> community (as flink-docker), there was virtua
Hi community,
In the past releases, we have been working on refactoring Flink's scheduler
with the goal of making the scheduler extensible [1]. We have rolled out
most of the intended refactoring in Flink 1.10, and we think it is now time
to leverage our newly introduced abstractions to implement
Hi Yangze,
thanks for creating this FLIP. I think it is a very good improvement
helping our users and ourselves understanding better what's going on in
Flink.
Creating the ResourceIDs with host information/pod name is a good idea.
Also deriving ExecutionGraph IDs from their superset ID is a good
Bowen Li created FLINK-16810:
Summary: add back PostgresCatalogITCase
Key: FLINK-16810
URL: https://issues.apache.org/jira/browse/FLINK-16810
Project: Flink
Issue Type: Sub-task
Compone
Bowen Li created FLINK-16811:
Summary: introduce JDBCRowConverter
Key: FLINK-16811
URL: https://issues.apache.org/jira/browse/FLINK-16811
Project: Flink
Issue Type: Improvement
Componen
Bowen Li created FLINK-16812:
Summary: introduce PostgresRowConverter
Key: FLINK-16812
URL: https://issues.apache.org/jira/browse/FLINK-16812
Project: Flink
Issue Type: Sub-task
Compone
Bowen Li created FLINK-16813:
Summary: JDBCInputFormat doesn't correctly map Short
Key: FLINK-16813
URL: https://issues.apache.org/jira/browse/FLINK-16813
Project: Flink
Issue Type: Sub-task
Bowen Li created FLINK-16814:
Summary: StringUtils.arrayToString() doesn't convert byte[]
correctly
Key: FLINK-16814
URL: https://issues.apache.org/jira/browse/FLINK-16814
Project: Flink
Issue T
Bowen Li created FLINK-16815:
Summary: add e2e tests for reading from postgres with
JDBCTableSource and PostgresCatalog
Key: FLINK-16815
URL: https://issues.apache.org/jira/browse/FLINK-16815
Project: Fli
Bowen Li created FLINK-16816:
Summary: planner doesn't parse timestamp array correctly
Key: FLINK-16816
URL: https://issues.apache.org/jira/browse/FLINK-16816
Project: Flink
Issue Type: Bug
Bowen Li created FLINK-16817:
Summary: StringUtils.arrayToString() doesn't convert byte[][]
correctly
Key: FLINK-16817
URL: https://issues.apache.org/jira/browse/FLINK-16817
Project: Flink
Issue
+1 for a separate repository.
The dedicated `flink-docker` repo works fine now. We can do it similarly.
Best,
Hequn
On Fri, Mar 27, 2020 at 1:16 AM Till Rohrmann wrote:
> +1 for a separate repository.
>
> Cheers,
> Till
>
> On Thu, Mar 26, 2020 at 5:13 PM Ufuk Celebi wrote:
>
> > +1.
> >
> > T
Jun Zhang created FLINK-16818:
-
Summary: Optimize data skew when flink write data to hive dynamic
partition table
Key: FLINK-16818
URL: https://issues.apache.org/jira/browse/FLINK-16818
Project: Flink
Also, here is the documentation for Stateful Functions for those who were
wondering:
master - https://ci.apache.org/projects/flink/flink-statefun-docs-master/
release-2.0 -
https://ci.apache.org/projects/flink/flink-statefun-docs-release-2.0/
This is not yet visible directly from the Flink website
Thanks for updating the FLIP, Yangze.
I agree with Till that we probably want to separate the K8s/Yarn decorator
calls. Users can still configure one driver class, and we can use
`instanceof` to check whether the driver implemented K8s/Yarn specific
interfaces.
Moreover, I'm not sure about exposi
Hi everyone,
I'd like to start the vote of FLIP-115 [1], which introduce Filesystem
table factory in table. This FLIP is discussed in the thread[2].
The vote will be open for at least 72 hours. Unless there is an objection,
I will try to close it by March 30, 2020 03:00 UTC if we have received
su
Thanks for the feedback, @Till and @Xintong.
Regarding separating the interface, I'm also +1 with it.
Regarding the resource allocation interface, true, it's dangerous to
give much access to user codes. Changing the return type to Map makes sense to me. AFAIK, it is compatible
with all the first-
Sorry for the late response.
I have shared my suggestions with Yadong & Lining offline. I think it would
be better to also post them here, for the public record.
- I'm not sure about displaying Total Process Memory Used. Currently, we
do not have a good way to monitor all memory footprints
Thanks for driving this discussion, Zhu Zhu & Gary.
I found that the image link in this FLIP is not working well. When I
open that link, Google doc told me that I have no access privilege.
Could you take a look at that issue?
Best,
Yangze Guo
On Fri, Mar 27, 2020 at 1:38 AM Gary Yao wrote:
>
>
Xingxing Di created FLINK-16819:
---
Summary: Got KryoException while using UDAF in flink1.9
Key: FLINK-16819
URL: https://issues.apache.org/jira/browse/FLINK-16819
Project: Flink
Issue Type: Bug
Bowen Li created FLINK-16820:
Summary: support reading array of timestamp, data, and time in
JDBCTableSource
Key: FLINK-16820
URL: https://issues.apache.org/jira/browse/FLINK-16820
Project: Flink
Zhijiang created FLINK-16821:
Summary: Run Kubernetes test failed with invalid named "minikube"
Key: FLINK-16821
URL: https://issues.apache.org/jira/browse/FLINK-16821
Project: Flink
Issue Type:
Hi Dawid,
Thank you for the response, I see your point. I was perhaps thinking only
from the perspective of my use case where I think such a definition makes
sense and did not account for the general case.
Regards,
Manas
On Thu, Mar 26, 2020 at 8:40 PM Dawid Wysakowicz
wrote:
> Hi Manas,
>
> F
Thanks everyone for the feedback ~
- For the global config option belongs to `ExecutionConfigOptions` or
`OptimizerConfigOptions`, i have to strong objections, switch
to `OptimizerConfigOptions` is okey to me and i have updated the WIKI
- For use while-list or black-list, i have opinion with Timo,
Sorry, i meant white-list ~
Danny Chan 于2020年3月27日周五 下午12:40写道：
> Thanks everyone for the feedback ~
>
> - For the global config option belongs to `ExecutionConfigOptions` or
> `OptimizerConfigOptions`, i have to strong objections, switch
> to `OptimizerConfigOptions` is okey to me and i have up
Thanks for reporting this Yangze.
I have update the permission to those images. Everyone are able to view
them now.
Thanks,
Zhu Zhu
Yangze Guo 于2020年3月27日周五 上午11:25写道：
> Thanks for driving this discussion, Zhu Zhu & Gary.
>
> I found that the image link in this FLIP is not working well. When I
Till, Stephen, & Others,
I have created a discuss thread a few days back. Attaching the link here.
Appreciate if you could take a look.
https://lists.apache.org/thread.html/rf885987160bede5911a7f61923307a6d5ae07f850da0a90555728e5f%40%3Cdev.flink.apache.org%3E
Please let me know if you want me to
godfrey he created FLINK-16822:
--
Summary: The config set by SET command does not work
Key: FLINK-16822
URL: https://issues.apache.org/jira/browse/FLINK-16822
Project: Flink
Issue Type: Bug
Adam N D DENG created FLINK-16823:
-
Summary: The functioin TIMESTAMPDIFF doesn't perform expected
result
Key: FLINK-16823
URL: https://issues.apache.org/jira/browse/FLINK-16823
Project: Flink
-1
Already discovered that the source distribution NOTICE file is missing
mentions for font-awesome.
The source of that is bundled under "docs/page/font-awesome/fonts", and is
licensed with SIL OFL 1.1 license, which makes it a requirement to be
listed in the NOTICE file.
I'll open a new RC2 with
77 matches
Mail list logo
