Re: [ANNOUNCE] Welcome Stefan Richter as a new committer

2017-02-15 Thread Shaoxuan Wang
Woohoo, Just saw this (was travelling). Congratulations, Stefan! Looking forward to the promising future of the backend state. On Mon, Feb 13, 2017 at 5:20 PM, Stefan Richter wrote: > Thanks a lot! I feel very happy and will try help the Flink community as > good as I can :-) > > Best, > Stefan

RE: [ANNOUNCE] Welcome Jark Wu and Kostas Kloudas as committers

2017-02-15 Thread Vasudevan, Ramkrishna S
Congrats Jark and Kostas. Regards Ram -Original Message- From: Kostas Kloudas [mailto:k.klou...@data-artisans.com] Sent: Monday, February 13, 2017 3:06 PM To: dev@flink.apache.org Subject: Re: [ANNOUNCE] Welcome Jark Wu and Kostas Kloudas as committers Thanks a lot guys! Looking forwa

RE: [ANNOUNCE] Welcome Stefan Richter as a new committer

2017-02-15 Thread Vasudevan, Ramkrishna S
Congrats Stefan Richter. Regards Ram -Original Message- From: Stefan Richter [mailto:s.rich...@data-artisans.com] Sent: Monday, February 13, 2017 2:50 PM To: dev@flink.apache.org Cc: u...@flink.apache.org; srich...@apache.org Subject: Re: [ANNOUNCE] Welcome Stefan Richter as a new commit

[jira] [Created] (FLINK-5813) code generation for user-defined aggregate functions

2017-02-15 Thread Shaoxuan Wang (JIRA)
Shaoxuan Wang created FLINK-5813: Summary: code generation for user-defined aggregate functions Key: FLINK-5813 URL: https://issues.apache.org/jira/browse/FLINK-5813 Project: Flink Issue Type

Re: [DISCUSS] Table API / SQL indicators for event and processing time

2017-02-15 Thread Fabian Hueske
Thanks everybody for the comments. Actually, I think we do not have much choice when deciding whether to use attributes or functions. Consider the following join query: SELECT​ ​t1.amount​,​ ​t2.rate FROM​ ​ table1 ​AS​ t1, ​ ​ table2 ​AS​ ​t2 WHERE ​ t1.currency = t2.currency AND t2.rowtim

Flink Job Exception

2017-02-15 Thread Govindarajan Srinivasaraghavan
Hi All, I'm trying to run a streaming job with flink 1.2 version and there are 3 task managers with 12 task slots. Irrespective of the parallelism that I give it always fails with the below error and I found a JIRA link corresponding to this issue. Can I know by when this will be resolved since I'

Reliable Distributed FS support (HCFS)

2017-02-15 Thread Vijay Srinivasaraghavan
Hello, Regarding the Filesystem abstraction support, we are planning to use a distributed file system which complies with Hadoop Compatible File System (HCFS) standard in place of standard HDFS. According to the documentation (https://ci.apache.org/projects/flink/flink-docs-release-1.3/internals

[jira] [Created] (FLINK-5812) Clean up FileSystem

2017-02-15 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-5812: --- Summary: Clean up FileSystem Key: FLINK-5812 URL: https://issues.apache.org/jira/browse/FLINK-5812 Project: Flink Issue Type: Improvement Components:

[jira] [Created] (FLINK-5811) Harden YarnClusterDescriptorTest

2017-02-15 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-5811: Summary: Harden YarnClusterDescriptorTest Key: FLINK-5811 URL: https://issues.apache.org/jira/browse/FLINK-5811 Project: Flink Issue Type: Bug Comp

[jira] [Created] (FLINK-5810) Harden SlotManager

2017-02-15 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-5810: Summary: Harden SlotManager Key: FLINK-5810 URL: https://issues.apache.org/jira/browse/FLINK-5810 Project: Flink Issue Type: Sub-task Components: D

RE: [DISCUSS] Table API / SQL indicators for event and processing time

2017-02-15 Thread Radu Tudoran
Hi, My initial thought would be that it makes more sense to thave procTime() and rowTime() only as functions which in fact are to be used as markers. Having the value (even from special system attributes does not make sense in some scenario such as the ones for creating windows, e.g., If you ha

[jira] [Created] (FLINK-5809) Add registerNextWatermarkCallback() to TimerService

2017-02-15 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-5809: --- Summary: Add registerNextWatermarkCallback() to TimerService Key: FLINK-5809 URL: https://issues.apache.org/jira/browse/FLINK-5809 Project: Flink Issue

[jira] [Created] (FLINK-5808) Missing verification for setParallelism and setMaxParallelism

2017-02-15 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-5808: --- Summary: Missing verification for setParallelism and setMaxParallelism Key: FLINK-5808 URL: https://issues.apache.org/jira/browse/FLINK-5808 Project: Flink

[jira] [Created] (FLINK-5807) improved wording for doc home page

2017-02-15 Thread David Anderson (JIRA)
David Anderson created FLINK-5807: - Summary: improved wording for doc home page Key: FLINK-5807 URL: https://issues.apache.org/jira/browse/FLINK-5807 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-5806) TaskExecutionState toString format have wrong key

2017-02-15 Thread shijinkui (JIRA)
shijinkui created FLINK-5806: Summary: TaskExecutionState toString format have wrong key Key: FLINK-5806 URL: https://issues.apache.org/jira/browse/FLINK-5806 Project: Flink Issue Type: Bug

Re: Handling skewness and Heterogeniety

2017-02-15 Thread Anis Nasir
Dear Fabian, Can you have a look into this issue. What actions will be required to resolve this one? https://issues.apache.org/jira/browse/FLINK-1725 Regards, Anis On Wed, Feb 15, 2017 at 6:36 PM, Fabian Hueske wrote: > Hi Anis, > > Flink uses regular hash-partitioning to shuffle records an

[jira] [Created] (FLINK-5805) improve docs for ProcessFunction

2017-02-15 Thread David Anderson (JIRA)
David Anderson created FLINK-5805: - Summary: improve docs for ProcessFunction Key: FLINK-5805 URL: https://issues.apache.org/jira/browse/FLINK-5805 Project: Flink Issue Type: Improvement

Re: Handling skewness and Heterogeniety

2017-02-15 Thread Fabian Hueske
Hi Anis, Flink uses regular hash-partitioning to shuffle records and does not have a mechanism to counter data skew (other than scaling out). Heterogeneous hardware can (to some extend) be addressed by adapting the number of processing slots (or task managers) per machine, i.e., configure fewer sl

Re: [DISCUSS] Table API / SQL indicators for event and processing time

2017-02-15 Thread Timo Walther
Hi all, at first I also thought that built-in functions (rowtime() and proctime()) are the easiest solution. However, I think to be future-proof we should make them system attributes; esp. to relate them to a corresponding table in case of multiple tables. Logically they are attributes of eac