Hi Matyas,
According to the release schedule defined in [1], it seems that the feature
freeze of v1.2 may occur at the beginning of September, is this correct? If
this is the case, I think it should be reasonable to make it in v1.2 for
Python support.
Regards,
Dian
[1]
https://cwiki.apache.org/c
+1 (binding)
- Verified the checksum and signature
- Installed the Python package and runs an example documented here[1]
- Reviewed the website PR and LGTM overall
Regards,
Dian
[1]
https://nightlies.apache.org/flink/flink-ml-docs-master/docs/operators/clustering/kmeans/#examples
On Mon, Jul 4,
Yun Gao created FLINK-28386:
---
Summary: Trigger an immediate checkpoint after all sources finished
Key: FLINK-28386
URL: https://issues.apache.org/jira/browse/FLINK-28386
Project: Flink
Issue Type:
Jingsong Lee created FLINK-28387:
Summary: Organizing Options in table store
Key: FLINK-28387
URL: https://issues.apache.org/jira/browse/FLINK-28387
Project: Flink
Issue Type: Improvement
Hi everyone,
Thanks for all the feedback so far. Based on the discussion [1], we seem to
have consensus. So, I would like to re-start a vote on FLIP-218 [2].
The vote will last for at least 72 hours unless there is an objection or
insufficient votes.
[1] https://lists.apache.org/threa
Both sql and py support is requested frequently. I guess we should aim to
support both in v1.2.
Matyas
On Tue, Jul 5, 2022 at 6:26 AM Gyula Fóra wrote:
> Thank you for the info and help Dian :)
>
> Gyula
>
> On Tue, 5 Jul 2022 at 05:14, Yang Wang wrote:
>
> > Thanks Dian for the confirmation a
hi Team, We recently started migrating flink from 1.14 to 1.15. We came
across a challenge that till 1.14 ParquetColumnarRowInputFormat constructor
was publicly accessible. But in version 1.15 it is accessible only within
the package. As per documentation, constructors are allowed to be accessed
pu
Xin Hao created FLINK-28385:
---
Summary: Change the validator to return an error if the Jar URI is
an empty string
Key: FLINK-28385
URL: https://issues.apache.org/jira/browse/FLINK-28385
Project: Flink
Hi everyone, Thank you to all those who participated in the discussion, the
program has been gradually revised and improved, everyone has reached a
consensus.
I will relaunch vote soon.
--
Best regards,
Mang Zhang
At 2022-07-05 11:54:07, "yuxia" wrote:
Thanks for updating.
Now, the
+1 for the feature.
Jing Zhang 于2022年7月5日周二 11:43写道：
> Hi all,
>
> I'd like to start a vote for FLIP-245: Source Supports Speculative
> Execution For Batch Job[1] on the discussion thread [2].
>
> The vote will last for at least 72 hours unless there is an objection or
> insufficient votes.
>
>
Thank you for the info and help Dian :)
Gyula
On Tue, 5 Jul 2022 at 05:14, Yang Wang wrote:
> Thanks Dian for the confirmation and nice help.
>
> Best,
> Yang
>
> Dian Fu 于2022年7月5日周二 09:27写道：
>
> > @Yang, Yes, you are right. Python jobs could be seen as special JAR jobs
> > whose main class i
Thanks for updating.
Now, the FLIP looks good to me.
Best regards,
Yuxia
发件人: "zhangmang1"
收件人: luoyu...@alumni.sjtu.edu.cn
抄送: "dev" , "Martijn Visser" ,
imj...@gmail.com
发送时间: 星期二, 2022年 7 月 05日 上午 11:35:35
主题: Re:Re: Re: Re: Re:Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE
Hi everyone,
I'm sorry to bother you all, but since FLIP-218[1] has been updated, I'm going
to relaunch VOTE.
The main contents of the modification are:
1. remove rtas from option name
2. no longer introduce AtomicCatalog, add javadocs description for Catalog
interface:
If Catalog needs to suppor
Hi all,
I'd like to start a vote for FLIP-245: Source Supports Speculative
Execution For Batch Job[1] on the discussion thread [2].
The vote will last for at least 72 hours unless there is an objection or
insufficient votes.
Best,
Jing Zhang
[1]
https://cwiki.apache.org/confluence/display/FLINK
Hi yuxia,
I updated the FLIP and adjusted your concern for RTAS and AtomicCatalog, not
sure if it solved your concern, looking forward to your reply!
--
Best regards,
Mang Zhang
At 2022-07-05 11:26:22, "Jark Wu" wrote:
>Thanks for the update, the FLIP looks good to me now.
>
>Best,
>
Hello, sorry to bother but I want to know what’s the progress of this FLIP, we
have facing the similar problem in Flink and FlinkSQL batch mode because of the
resource shortage momentary and I think this may help us. Thanks!
On 2022/05/30 16:11:24 "Afonso, Nuno" wrote:
> Hello everyone,
>
>
Hello, sorry to bother but I want to know what’s the progress of this FLIP, we
have facing the similar problem in Flink and FlinkSQL batch mode because of the
resource shortage momentary and I think this may help us. Thanks!
On 2022/05/30 16:11:24 "Afonso, Nuno" wrote:
> Hello everyone,
>
>
Hi everyone,
Thanks a lot for all the feedback!
I will open a vote for it since there is no more concern.
Best,
Jing Zhang
Jing Zhang 于2022年7月5日周二 11:31写道：
> Hi ZhuZhu, Jiangjie,
>
> Thanks a lot for your feedback.
>
> I agree that it's better to support most existing events.
> I have updated
Hi ZhuZhu, Jiangjie,
Thanks a lot for your feedback.
I agree that it's better to support most existing events.
I have updated the FLIP to cover how to deal with the
RequestSplitEvent/SourceEventWrapper/ReaderRegistrationEvent.
The ReportedWatermarkEvent is only used in watermark alignment.
Water
Thanks for the update, the FLIP looks good to me now.
Best,
Jark
On Tue, 5 Jul 2022 at 10:57, Mang Zhang wrote:
> Hi Jark,
> Regarding the two issues of concern to yuxia, we did some offline
> discussions and adjusted the implementation plan.
>
> >1) RTAS
> RTAS is not supported in FLIP, so we
Thanks Dian for the confirmation and nice help.
Best,
Yang
Dian Fu 于2022年7月5日周二 09:27写道：
> @Yang, Yes, you are right. Python jobs could be seen as special JAR jobs
> whose main class is always `org.apache.flink.client.python.PythonDriver`.
> What we could do in Flink K8s operator is to make it
Hi Jark,
Regarding the two issues of concern to yuxia, we did some offline discussions
and adjusted the implementation plan.
>1) RTAS
RTAS is not supported in FLIP, so we will remove rtas from option name and do
option forward compatibility when it is supported in the future.
>2) AtomicCatal
Jingsong Lee created FLINK-28384:
Summary: Add CatalogLock support for HiveCatalog in table store
Key: FLINK-28384
URL: https://issues.apache.org/jira/browse/FLINK-28384
Project: Flink
Issue
@Yang, Yes, you are right. Python jobs could be seen as special JAR jobs
whose main class is always `org.apache.flink.client.python.PythonDriver`.
What we could do in Flink K8s operator is to make it more convenient and
handle properly for the different kinds of dependencies[1].
@Gyula, I can help
+1 for the feature.
Jark Wu 于2022年7月4日周一 17:33写道：
> Hi Mang,
>
> I left a comment in the DISCUSS thread.
>
> Best,
> Jark
>
> On Mon, 4 Jul 2022 at 15:24, Rui Fan <1996fan...@gmail.com> wrote:
>
> > Hi.
> >
> > Thanks Mang for this FLIP. I think it will be useful for users.
> >
> > +1(non-bindin
Yes, that sounds reasonable to me. That said, supporting custom events
might still be preferable if that does not complicate the design too much.
It would be good to avoid having a tricky feature availability matrix when
we add new features to the project.
Thanks,
Jiangjie (Becket) Qin
On Mon,
Hi Alex,
While it is true that the RuntimeContext gives access to all the stuff the
framework can provide, it seems a little overkilling for the SourceReader.
It is probably OK to expose all the read-only information in the
RuntimeContext to the SourceReader, but we may want to hide the "write"
me
Hi Becket,
I updated and extended FLIP-238 accordingly.
Here is also my POC branch [1].
DataGeneratorSourceV3 is the class that I currently converged on [2]. It is
based on the expanded SourceReaderContext.
A couple more relevant classes [3] [4]
Would appreciate it if you could take a quick look
Hi Konstantin,
I agree this issue has been around for a while now, but Flink 1.15's
savepoint ownership changes make it much more likely to cause issues.
Since it is not a regression of 1.15.0 we can fix it in the next patch
release.
+0 from me too.
Thanks
On Mon, 4 Jul 2022, 07:45 Jingsong Li
Hi Martijn,
Thanks a lot supporting the FLIP! Regarding your question, yes, Elasticsearch
has a new client in v8 (deprecating HighLevelRestClient from now on) and, no,
there won't be any impact on Opensearch (the connectors use Opensearch's
own HighLevelRestClient, no deprecations yet).
Thank y
Hi Roc,
Thanks for the FLIP and opening the discussion. I have a couple of initial
questions/remarks:
* The FLIP contains information for both Source and Sink, but nothing
explicitly on the Lookup functionality. I'm assuming we also want to have
that implementation covered while porting this to t
Arseniy Tashoyan created FLINK-28383:
Summary: HDFS source: Trying to access closed classloader
Key: FLINK-28383
URL: https://issues.apache.org/jira/browse/FLINK-28383
Project: Flink
Issu
Hi Andriy,
Thanks for creating the FLIP and opening the discussion. In general +1 for
the FLIP, looking forward!
One question: I recall that in order for Flink to support Elasticsearch 8,
the connector needs to be overhauled to use Elastic's new client because
only the new client has a compatible
Yingjie Cao created FLINK-28382:
---
Summary: Introduce new compression algorithms of higher
compression ratio
Key: FLINK-28382
URL: https://issues.apache.org/jira/browse/FLINK-28382
Project: Flink
Jingsong Lee created FLINK-28381:
Summary: Add Spark Reader documentation
Key: FLINK-28381
URL: https://issues.apache.org/jira/browse/FLINK-28381
Project: Flink
Issue Type: Sub-task
Yingjie Cao created FLINK-28380:
---
Summary: Produce one intermediate dataset for multiple consumers
consuming the same data
Key: FLINK-28380
URL: https://issues.apache.org/jira/browse/FLINK-28380
Project
Hi zoucao,
Regarding the pruning function, maybe a simple filter function is enough,
e.g. `FilterFunction partitionFilter`.
Besides, it would be better to state clearly how the new
`applyPartitionPuringFunction`
method affects batch/bounded table sources. From my understanding,
this method won't
Hi zoucao,
The FileSource does support streaming reading [1].
Best regards,
Martijn
[1]
https://nightlies.apache.org/flink/flink-docs-release-1.15/api/java/org/apache/flink/connector/file/src/FileSource.html
Op ma 4 jul. 2022 om 05:58 schreef godfrey he :
> Hi zoucao,
>
> Look forward your FL
Martijn Visser created FLINK-28379:
--
Summary: Process produced no output for 900 seconds after/while
running org.apache.flink.runtime.blob.BlobClientSslTest
Key: FLINK-28379
URL: https://issues.apache.org/jira/br
Hi Becket,
Exposing the RuntimeContext is potentially even more useful.
Do you think it is worth having both currentParallelism() and
getRuntimeContext() methods?
One can always call getNumberOfParallelSubtasks() on the RuntimeContext
directly if we expose it.
Best,
Alexander Fedulov
On Mon, J
FYI, this has been fixed 4 days ago [1].
Cheers,
Jark
[1]:
https://issues.apache.org/jira/browse/INFRA-23425?focusedCommentId=17560597&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17560597
On Wed, 29 Jun 2022 at 10:37, Xintong Song wrote:
> Hi devs,
>
> I jus
Hi Mang,
I left a comment in the DISCUSS thread.
Best,
Jark
On Mon, 4 Jul 2022 at 15:24, Rui Fan <1996fan...@gmail.com> wrote:
> Hi.
>
> Thanks Mang for this FLIP. I think it will be useful for users.
>
> +1(non-binding)
>
> Best wishes
> Rui Fan
>
> On Mon, Jul 4, 2022 at 3:01 PM Mang Zhang w
Hi Mang,
I'm not sure whether your response has addressed Yuxia's concern or not.
Would be better to receive a confirmation from participants before starting
the vote.
Actually, I have the same feeling with Yuxia's reply.
1) RTAS
If it's hard to have a consistent behavior for RTAS between stream
Yingjie Cao created FLINK-28378:
---
Summary: Use larger data reading buffer size for sort-shuffle
Key: FLINK-28378
URL: https://issues.apache.org/jira/browse/FLINK-28378
Project: Flink
Issue Type
Hi Jiangjie,
Yes you are that the goals of watermark alignment and speculative
execution do not conflict. For the example you gave, we can make it
work by only aligning watermarks for executions that are pipelined
connected (i.e. in the same execution attempt level pipelined region).
Even not cons
Yingjie Cao created FLINK-28377:
---
Summary: Enable to request less number of data reading buffers for
sort-shuffle if there is no enough data
Key: FLINK-28377
URL: https://issues.apache.org/jira/browse/FLINK-28377
Yingjie Cao created FLINK-28376:
---
Summary: Reduce the number of IO threads for sort-shuffle
Key: FLINK-28376
URL: https://issues.apache.org/jira/browse/FLINK-28376
Project: Flink
Issue Type: Su
hehuiyuan created FLINK-28375:
-
Summary: Whether to consider adding other data type to support for
last_value function
Key: FLINK-28375
URL: https://issues.apache.org/jira/browse/FLINK-28375
Project: Flin
Yingjie Cao created FLINK-28374:
---
Summary: Some further improvements of sort-shuffle
Key: FLINK-28374
URL: https://issues.apache.org/jira/browse/FLINK-28374
Project: Flink
Issue Type: Improveme
Hi Zhu,
I agree that if we are talking about a single execution region with
blocking shuffle, watermark alignment may not be that helpful as the
subtasks are running independently of each other.
That said, I don't think watermark alignment and speculative execution
necessarily conflict with each
Yingjie Cao created FLINK-28373:
---
Summary: Read larger size of data sequentially for sort-shuffle
Key: FLINK-28373
URL: https://issues.apache.org/jira/browse/FLINK-28373
Project: Flink
Issue Ty
Chesnay Schepler created FLINK-28372:
Summary: Investigate Akka Artery
Key: FLINK-28372
URL: https://issues.apache.org/jira/browse/FLINK-28372
Project: Flink
Issue Type: Technical Debt
I had another thought and now I think watermark alignment is actually
conceptually conflicted with speculative execution.
This is because the idea of watermark alignment is to limit the progress
of all sources to be around the progress of the slowest source in the
watermark group. However, speculat
Hi.
Thanks Mang for this FLIP. I think it will be useful for users.
+1(non-binding)
Best wishes
Rui Fan
On Mon, Jul 4, 2022 at 3:01 PM Mang Zhang wrote:
> Hi everyone,
>
>
>
>
> Thanks for all the feedback so far. Based on the discussion [1], we seem
> to have consensus. So, I would like to s
Michael Groys created FLINK-28371:
-
Summary: Upgrade netty version in flink-akka-rpc
Key: FLINK-28371
URL: https://issues.apache.org/jira/browse/FLINK-28371
Project: Flink
Issue Type: Technic
Leonard Xu created FLINK-28370:
--
Summary: Add close method for KafkaRecordSerializationSchema
Key: FLINK-28370
URL: https://issues.apache.org/jira/browse/FLINK-28370
Project: Flink
Issue Type: I
Hi everyone,
Thanks for all the feedback so far. Based on the discussion [1], we seem to
have consensus. So, I would like to start a vote on FLIP-218 [2].
The vote will last for at least 72 hours unless there is an objection or
insufficient votes.
[1] https://lists.apache.org/thread/m
57 matches
Mail list logo
