Hi Jingsong,
Although it would be nice if the accumulators in GlobalAggregateManager is
fault-tolerant, we could still take advantage of managed state to guarantee
the semantic and use the accumulators to implement distributed barrier or
lock to solve the distributed access problem.
Best,
Shimin
Hi devs,
I'd like to start a discussion thread on the topic how we provide
retrieval services in non-high-availability scenario. To clarify
terminology, non-high-availability scenario refers to
StandaloneHaServices and EmbeddedHaServices.
***The problem***
We notice that retrieval services of cu
Till Rohrmann created FLINK-14008:
-
Summary: Flink Scala 2.12 binary distribution contains incorrect
NOTICE-binary file
Key: FLINK-14008
URL: https://issues.apache.org/jira/browse/FLINK-14008
Project:
Till Rohrmann created FLINK-14009:
-
Summary: Cron jobs broken due to verifying incorrect NOTICE-binary
file
Key: FLINK-14009
URL: https://issues.apache.org/jira/browse/FLINK-14009
Project: Flink
Thanks a lot everyone for the warm welcome!
Cheers,
Kostas
On Mon, Sep 9, 2019 at 4:54 AM Yun Gao wrote:
>
> Congratulations, Kostas!
>
> Best,
> Yun
>
>
>
>
> --
> From:Becket Qin
> Send Time:2019 Sep. 9 (Mon.)
Congrats, Kostas! Well deserved.
Best,
Jingsong Lee
--
From:Kostas Kloudas
Send Time:2019年9月9日(星期一) 15:50
To:dev ; Yun Gao
Subject:Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC
Thanks a lot everyone for the warm welcome!
Che
Thank you Bowen and Becket.
What's the take from Flink community? Shall we wait for FLIP-27 or shall we
proceed to next steps? And what the next steps are? :-)
Thanks,
Sijie
On Thu, Sep 5, 2019 at 2:43 PM Bowen Li wrote:
> Hi,
>
> I think having a Pulsar connector in Flink can be a good mutual
Yes you are correct Dominik. The committed Kafka offsets tell you what the
program has read as input from the Kafka topic. But depending on the actual
program logic this does not mean that you have output the results of
processing these input events up to this point. As you have said, there are
Fli
TisonKun created FLINK-14010:
Summary: Dispatcher & JobManagers don't give up leadership when AM
is shut down
Key: FLINK-14010
URL: https://issues.apache.org/jira/browse/FLINK-14010
Project: Flink
Hi guys,
There is an issue about supporting Elasticsearch 7.x.[1]
Based on our validation and discussion. We found that Elasticsearch 7.x
does not guarantee API compatibility. Therefore, it does not have the
ability to provide a universal connector like Kafka. It seems that we have
to provide a ne
Alex created FLINK-14011:
Summary: Make some fields final and initialize them during
construction in AsyncWaitOperator
Key: FLINK-14011
URL: https://issues.apache.org/jira/browse/FLINK-14011
Project: Flink
Hi Enrico,
Sorry for the late reply. I think your understanding is correct.
The best way to do it is to write your own ParquetBulkWriter and the
corresponding factory.
Out of curiosity, I guess that in the BucketingSink you were using the
AvroKeyValueSinkWriter, right?
Cheers,
Kostas
On Fri, Au
Hi Tison,
thanks for starting this discussion. I think your mail includes multiple
points which are worth being treated separately (might even make sense to
have separate discussion threads). Please correct me if I understood things
wrongly:
1. Adding new non-ha HAServices:
Based on your descrip
Daebeom Lee created FLINK-14012:
---
Summary: Failed to start job for consuming Secure Kafka after the
job cancel
Key: FLINK-14012
URL: https://issues.apache.org/jira/browse/FLINK-14012
Project: Flink
Thanks for confirming.
We have a
```
public class ParquetSinkWriter implements Writer
```
that handles the serialization of the data.
We implemented it starting from:
https://medium.com/hadoop-noob/flink-parquet-writer-d127f745b519
https://stackoverflow.com/questions/48098011/how-to-use-apache-fli
Hi all,
Thanks all for so much feedbacks received in the doc so far.
I saw a general agreement on using computed column to support proctime
attribute and extract timestamps.
So we will prepare a computed column FLIP and share in the dev ML soon.
Feel free to leave more comments!
Best,
Jark
On
+1 (non-binding)
- built from source successfully (mvn clean install -DskipTests)
- checked gpg signature and hashes of the source release and binary release
packages
- All artifacts have been deployed to the maven central repository
- no new dependencies were added since 1.8.1
- run a couple of
Thanks Jincheng a lot for the remind and thanks all for the voting. I'm closing
the vote now.
So far, the vote has received:
- 5 binding +1 votes (Jincheng, Hequn, Jark, Shaoxuan, Becket)
- 5 non-binding +1 votes (Wei, Xingbo, Terry, Yu, Jeff)
- No 0/-1 votes
There are more than 3 binding +
Dian Fu created FLINK-14013:
---
Summary: Support Flink Python User-Defined Stateless Function for
Table
Key: FLINK-14013
URL: https://issues.apache.org/jira/browse/FLINK-14013
Project: Flink
Issue T
Dian Fu created FLINK-14014:
---
Summary: Introduce PythonScalarFunctionRunner to handle the
communication with Python worker for Python ScalarFunction execution
Key: FLINK-14014
URL: https://issues.apache.org/jira/browse/
Dian Fu created FLINK-14015:
---
Summary: Introduce PythonScalarFunctionOperator as a standalone
StreamOperator for Python ScalarFunction execution
Key: FLINK-14015
URL: https://issues.apache.org/jira/browse/FLINK-14015
Dian Fu created FLINK-14016:
---
Summary: Introduce RelNodes FlinkLogicalPythonScalarFunctionExec
and DataStreamPythonScalarFunctionExec which are containers for Python
PythonScalarFunctions
Key: FLINK-14016
URL: https://
Dian Fu created FLINK-14017:
---
Summary: Support to start up Python worker in process mode
Key: FLINK-14017
URL: https://issues.apache.org/jira/browse/FLINK-14017
Project: Flink
Issue Type: Sub-task
Dian Fu created FLINK-14018:
---
Summary: Add Python building blocks to make sure the basic
functionality of Python ScalarFunction could work
Key: FLINK-14018
URL: https://issues.apache.org/jira/browse/FLINK-14018
Dian Fu created FLINK-14019:
---
Summary: Python environment and dependency management
Key: FLINK-14019
URL: https://issues.apache.org/jira/browse/FLINK-14019
Project: Flink
Issue Type: Sub-task
Dian Fu created FLINK-14020:
---
Summary: User Apache Arrow as the serializer for data transmission
between Java operator and Python harness
Key: FLINK-14020
URL: https://issues.apache.org/jira/browse/FLINK-14020
Dian Fu created FLINK-14021:
---
Summary: Add rules to push down the Python ScalarFunctions
contained in the join condition of Correlate node
Key: FLINK-14021
URL: https://issues.apache.org/jira/browse/FLINK-14021
Dian Fu created FLINK-14022:
---
Summary: Add validation check for places where Python
ScalarFunction cannot be used
Key: FLINK-14022
URL: https://issues.apache.org/jira/browse/FLINK-14022
Project: Flink
Dian Fu created FLINK-14023:
---
Summary: Support accessing job parameters in Python user-defined
functions
Key: FLINK-14023
URL: https://issues.apache.org/jira/browse/FLINK-14023
Project: Flink
Issu
Dian Fu created FLINK-14024:
---
Summary: Support use-defined metrics in Python user-defined
functions
Key: FLINK-14024
URL: https://issues.apache.org/jira/browse/FLINK-14024
Project: Flink
Issue Typ
Dian Fu created FLINK-14025:
---
Summary: Support to run the Python worker in docker mode
Key: FLINK-14025
URL: https://issues.apache.org/jira/browse/FLINK-14025
Project: Flink
Issue Type: Sub-task
Dian Fu created FLINK-14026:
---
Summary: Manage the resource of Python worker properly
Key: FLINK-14026
URL: https://issues.apache.org/jira/browse/FLINK-14026
Project: Flink
Issue Type: Sub-task
Dian Fu created FLINK-14027:
---
Summary: Add documentation for Python user-defined functions
Key: FLINK-14027
URL: https://issues.apache.org/jira/browse/FLINK-14027
Project: Flink
Issue Type: Sub-tas
Dian Fu created FLINK-14028:
---
Summary: Support logging aggregation in Python user-defined
functions
Key: FLINK-14028
URL: https://issues.apache.org/jira/browse/FLINK-14028
Project: Flink
Issue Typ
Piyush Narang created FLINK-14029:
-
Summary: Update Flink's Mesos scheduling behavior to reject all
expired offers
Key: FLINK-14029
URL: https://issues.apache.org/jira/browse/FLINK-14029
Project: Flin
Congrats, Kostas!
Thanks,
Biao /'bɪ.aʊ/
On Mon, 9 Sep 2019 at 16:07, JingsongLee
wrote:
> Congrats, Kostas! Well deserved.
>
> Best,
> Jingsong Lee
>
>
> --
> From:Kostas Kloudas
> Send Time:2019年9月9日(星期一) 15:50
> To:dev ; Yun G
Leonard Xu created FLINK-14030:
--
Summary: Nonequivalent conversion happens in Table planner
Key: FLINK-14030
URL: https://issues.apache.org/jira/browse/FLINK-14030
Project: Flink
Issue Type: Bu
Jimmy Wong created FLINK-14031:
--
Summary: flink-examples should add blink dependency on
flink-examples-table
Key: FLINK-14031
URL: https://issues.apache.org/jira/browse/FLINK-14031
Project: Flink
One thing that I just came across: Some of these options should also have a
corresponding value for the JobManager, like JVM overhead, metaspace,
direct memory.
On Fri, Sep 6, 2019 at 4:34 AM Xintong Song wrote:
> Thanks all for the votes.
> So far, we have
>
>- 4 binding +1 votes (Stephan,
Yun Tang created FLINK-14032:
Summary: Make the cache size of RocksDBPriorityQueueSetFactory
configurable
Key: FLINK-14032
URL: https://issues.apache.org/jira/browse/FLINK-14032
Project: Flink
I
Zhenqiu Huang created FLINK-14033:
-
Summary: Distributed caches are not registered in Yarn Per Job
Cluster Mode
Key: FLINK-14033
URL: https://issues.apache.org/jira/browse/FLINK-14033
Project: Flink
Niels van Kaam created FLINK-14034:
--
Summary: In FlinkKafkaProducer, KafkaTransactionState should be
made public or invoke should be made final
Key: FLINK-14034
URL: https://issues.apache.org/jira/browse/FLINK-14
Hi all,
During the discussion of how to support Hive built-in functions in Flink in
FLIP-57 [1], an idea of "modular built-in functions" was brought up with
examples of "Extension" in Postgres [2] and "Plugin" in Presto [3]. Thus
I'd like to kick off a discussion to see if we should adopt such an
Congxian Qiu(klion26) created FLINK-14035:
-
Summary: Introduce/Change some log for snapshot to better analysis
checkpoint problem
Key: FLINK-14035
URL: https://issues.apache.org/jira/browse/FLINK-14035
Hi Sijie,
If we agree that the goal is to have Pulsar connector in 1.10, how about we
do the following:
0. Start a FLIP to add Pulsar connector to Flink main repo as it is a new
public interface to Flink main repo.
1. Start to review the Pulsar sink right away as there is no change to the
sink in
Leonard Xu created FLINK-14036:
--
Summary: function log(f0,f1) in Table API do not support decimal
type
Key: FLINK-14036
URL: https://issues.apache.org/jira/browse/FLINK-14036
Project: Flink
liupengcheng created FLINK-14037:
Summary: Deserializing the input/output formats failed: unread
block data
Key: FLINK-14037
URL: https://issues.apache.org/jira/browse/FLINK-14037
Project: Flink
liupengcheng created FLINK-14038:
Summary: ExecutionGraph deploy failed due to akka timeout
Key: FLINK-14038
URL: https://issues.apache.org/jira/browse/FLINK-14038
Project: Flink
Issue Type:
Hi Jingsong,
Good point!
1. If it doesn't matter which task performs the finalize work, then I think
task-0 suggested by Jark is a very good solution.
2. If it requires the last finished task to perform the finalize work, then we
have to consider other solutions.
WRT fault-tolerant of Stream
+1 (binding)
- build from source and passed all tests locally
- checked the difference between 1.8.1 and 1.8.2, no legal risk found
- went through all commits checked in between 1.8.1 and 1.8.2, make
sure all the issues set the proper "fixVersion" property
Best,
Kurt
On Mon, Sep 9, 2019 at 8:45
Hi Jingsong,
Thank you for bringing this discussion. Since I don't have much experience
of Flink table/SQL, I'll ask some questions from runtime or engine
perspective.
> ... where we describe how to partition support in flink and how to
integrate to hive partition.
FLIP-27 [1] introduces "partit
51 matches
Mail list logo
