[jira] [Created] (FLINK-8366) Use Row instead of String as key when process upsert results

2018-01-05 Thread Hequn Cheng (JIRA)
Hequn Cheng created FLINK-8366: -- Summary: Use Row instead of String as key when process upsert results Key: FLINK-8366 URL: https://issues.apache.org/jira/browse/FLINK-8366 Project: Flink Issue

[jira] [Created] (FLINK-8367) Port SubtaskCurrentAttemptDetailsHandler to new REST endpoint

2018-01-05 Thread Biao Liu (JIRA)
Biao Liu created FLINK-8367: --- Summary: Port SubtaskCurrentAttemptDetailsHandler to new REST endpoint Key: FLINK-8367 URL: https://issues.apache.org/jira/browse/FLINK-8367 Project: Flink Issue Type

[jira] [Created] (FLINK-8368) Port SubtaskExecutionAttemptDetailsHandler to new REST endpoint

2018-01-05 Thread Biao Liu (JIRA)
Biao Liu created FLINK-8368: --- Summary: Port SubtaskExecutionAttemptDetailsHandler to new REST endpoint Key: FLINK-8368 URL: https://issues.apache.org/jira/browse/FLINK-8368 Project: Flink Issue Ty

[jira] [Created] (FLINK-8369) Port SubtaskExecutionAttemptAccumulatorsHandler to new REST endpoint

2018-01-05 Thread Biao Liu (JIRA)
Biao Liu created FLINK-8369: --- Summary: Port SubtaskExecutionAttemptAccumulatorsHandler to new REST endpoint Key: FLINK-8369 URL: https://issues.apache.org/jira/browse/FLINK-8369 Project: Flink Iss

Re: Using SQL with dynamic tables where rows are updated

2018-01-05 Thread Fabian Hueske
Hi Ghassan, Flink's Table API / SQL does not support the upsert ingestion mode (updating table rows by key) yet but only append mode, i.e, each event of the data stream is appended to the table. Hence, it is not possible to implement your use case using SQL. An upsert ingestion mode will be added

[jira] [Created] (FLINK-8370) Port AbstractAggregatingMetricsHandler to RestServerEndpoint

2018-01-05 Thread Gary Yao (JIRA)
Gary Yao created FLINK-8370: --- Summary: Port AbstractAggregatingMetricsHandler to RestServerEndpoint Key: FLINK-8370 URL: https://issues.apache.org/jira/browse/FLINK-8370 Project: Flink Issue Type:

[jira] [Created] (FLINK-8371) Buffers are not recycled in a non-spilled SpillableSubpartition upon release

2018-01-05 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-8371: -- Summary: Buffers are not recycled in a non-spilled SpillableSubpartition upon release Key: FLINK-8371 URL: https://issues.apache.org/jira/browse/FLINK-8371 Project: Flink

[jira] [Created] (FLINK-8372) NPEs stopping containers via YarnResourceManager and adding Tasks via TaskSlotTable

2018-01-05 Thread Sebastian Bauersfeld (JIRA)
Sebastian Bauersfeld created FLINK-8372: --- Summary: NPEs stopping containers via YarnResourceManager and adding Tasks via TaskSlotTable Key: FLINK-8372 URL: https://issues.apache.org/jira/browse/FLINK-8372

[jira] [Created] (FLINK-8373) Inconsistencies in some FileSystem directory functions

2018-01-05 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-8373: --- Summary: Inconsistencies in some FileSystem directory functions Key: FLINK-8373 URL: https://issues.apache.org/jira/browse/FLINK-8373 Project: Flink Issue Type

[jira] [Created] (FLINK-8374) Unstable Yarn tests due to Akka Shutdown Exception Logging

2018-01-05 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-8374: --- Summary: Unstable Yarn tests due to Akka Shutdown Exception Logging Key: FLINK-8374 URL: https://issues.apache.org/jira/browse/FLINK-8374 Project: Flink Issue

Re: Using SQL with dynamic tables where rows are updated

2018-01-05 Thread Hequn Cheng
hi Ghassan, TableSource in Flink doesn't support primary key now, but you can achieve it by doing a group by manually. Such as: val resultTable = tableEnv.sql( " SELECT ProductID, AVG(Rating) FROM ( SELECT ReviewID, LAST_VALUE(ProductID), LAST_VALUE(Approved) , L

Re: Using SQL with dynamic tables where rows are updated

2018-01-05 Thread Fabian Hueske
Ha, that's a neat workaround! Thanks for sharing Hequn! When doing this, you should however, ensure that all records with the same key arrive from the same input task to avoid inconsistent behavior due to records arriving out of order. This would be the case if you ingest the table directly from a

[jira] [Created] (FLINK-8375) Remove unnecessary synchronization in ResultPartition

2018-01-05 Thread Piotr Nowojski (JIRA)
Piotr Nowojski created FLINK-8375: - Summary: Remove unnecessary synchronization in ResultPartition Key: FLINK-8375 URL: https://issues.apache.org/jira/browse/FLINK-8375 Project: Flink Issue T

[jira] [Created] (FLINK-8376) Remove potentially unnecessary synchronization in SpillableSubpartition

2018-01-05 Thread Piotr Nowojski (JIRA)
Piotr Nowojski created FLINK-8376: - Summary: Remove potentially unnecessary synchronization in SpillableSubpartition Key: FLINK-8376 URL: https://issues.apache.org/jira/browse/FLINK-8376 Project: Flin

Re: Using SQL with dynamic tables where rows are updated

2018-01-05 Thread Fabian Hueske
Also, if you expect your the key update records to be out of order, you might want to add a ProcessFunction on a keyed stream that filters records with smaller timestamps than the highest observed timestamp. This would prevent a record to be overridden by an earlier version with a smaller timestamp

Re: [DISCUSS] Service Authorization (SSL client authentication)

2018-01-05 Thread Till Rohrmann
Thanks for drafting the service authorization FLIP Eron. It is a very important feature which Flink is still lacking and hinders some people to deploy Flink in their cluster environments. The overall design looks really good to me. I would suggest to make a FLIP out of it by adding it to the Flink

[jira] [Created] (FLINK-8377) Document DataViews for user-defined aggregate functions

2018-01-05 Thread Timo Walther (JIRA)
Timo Walther created FLINK-8377: --- Summary: Document DataViews for user-defined aggregate functions Key: FLINK-8377 URL: https://issues.apache.org/jira/browse/FLINK-8377 Project: Flink Issue Typ

[jira] [Created] (FLINK-8378) Serialization and Deserialization of Kafka Avro messages from and to Row with Confluent Schema Registry

2018-01-05 Thread Christophe Philemotte (JIRA)
Christophe Philemotte created FLINK-8378: Summary: Serialization and Deserialization of Kafka Avro messages from and to Row with Confluent Schema Registry Key: FLINK-8378 URL: https://issues.apache.org/jir

[jira] [Created] (FLINK-8379) Improve type checking for DataView

2018-01-05 Thread Timo Walther (JIRA)
Timo Walther created FLINK-8379: --- Summary: Improve type checking for DataView Key: FLINK-8379 URL: https://issues.apache.org/jira/browse/FLINK-8379 Project: Flink Issue Type: Improvement

Re: Using SQL with dynamic tables where rows are updated

2018-01-05 Thread Ghassan Yammine
Thanks very much! Regards, Ghassan > On Jan 5, 2018, at 7:55 AM, Hequn Cheng wrote: > > hi Ghassan, > > TableSource in Flink doesn't support primary key now, but you can achieve > it by doing a group by manually. > > Such as: > val resultTable = tableEnv.sql( > " > SELECT Prod

[jira] [Created] (FLINK-8380) Dynamic BucketingSink paths based on ingested Kafka topics

2018-01-05 Thread Kyle Hamlin (JIRA)
Kyle Hamlin created FLINK-8380: -- Summary: Dynamic BucketingSink paths based on ingested Kafka topics Key: FLINK-8380 URL: https://issues.apache.org/jira/browse/FLINK-8380 Project: Flink Issue Ty

[jira] [Created] (FLINK-8381) Document more flexible schema definition

2018-01-05 Thread Timo Walther (JIRA)
Timo Walther created FLINK-8381: --- Summary: Document more flexible schema definition Key: FLINK-8381 URL: https://issues.apache.org/jira/browse/FLINK-8381 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-8382) sheduleRunAsync with a positive schedule delay does not work in JobMaster

2018-01-05 Thread Zhu Zhu (JIRA)
Zhu Zhu created FLINK-8382: -- Summary: sheduleRunAsync with a positive schedule delay does not work in JobMaster Key: FLINK-8382 URL: https://issues.apache.org/jira/browse/FLINK-8382 Project: Flink

[jira] [Created] (FLINK-8383) flink-mesos build failing: duplicate Jackson relocation in shaded jar

2018-01-05 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-8383: -- Summary: flink-mesos build failing: duplicate Jackson relocation in shaded jar Key: FLINK-8383 URL: https://issues.apache.org/jira/browse/FLINK-8383 Proj