[jira] [Created] (FLINK-23686) KafkaSource metric "commitsSucceeded" should count per-commit instead of per-partition

2021-08-09 Thread Qingsheng Ren (Jira)
Qingsheng Ren created FLINK-23686: - Summary: KafkaSource metric "commitsSucceeded" should count per-commit instead of per-partition Key: FLINK-23686 URL: https://issues.apache.org/jira/browse/FLINK-23686

Re: [ANNOUNCE] Apache Flink 1.13.2 released

2021-08-09 Thread Yu Li
Thanks Yun Tang for being our release manager and everyone else who made the release possible! Best Regards, Yu On Fri, 6 Aug 2021 at 13:52, Yun Tang wrote: > > The Apache Flink community is very happy to announce the release of Apache > Flink 1.13.2, which is the second bugfix release for the

[jira] [Created] (FLINK-23687) Add Sql query hint to enable LookupJoin shuffle by join key of left input

2021-08-09 Thread JING ZHANG (Jira)
JING ZHANG created FLINK-23687: -- Summary: Add Sql query hint to enable LookupJoin shuffle by join key of left input Key: FLINK-23687 URL: https://issues.apache.org/jira/browse/FLINK-23687 Project: Flink

Re: [VOTE] FLIP-180: Adjust StreamStatus and Idleness definition

2021-08-09 Thread Till Rohrmann
+1 (binding) Cheers, Till On Thu, Aug 5, 2021 at 9:09 PM Arvid Heise wrote: > Dear devs, > > I'd like to open a vote on FLIP-180: Adjust StreamStatus and Idleness > definition [1] which was discussed in this thread [2]. > The vote will be open for at least 72 hours unless there is an objection

[jira] [Created] (FLINK-23688) Make JaasModule.getAppConfigurationEntries public

2021-08-09 Thread Gabor Somogyi (Jira)
Gabor Somogyi created FLINK-23688: - Summary: Make JaasModule.getAppConfigurationEntries public Key: FLINK-23688 URL: https://issues.apache.org/jira/browse/FLINK-23688 Project: Flink Issue Typ

Re: [DISCUSS] FLIP-173: Support DAG of algorithms (Flink ML)

2021-08-09 Thread 青雉(祁明良)
Hi all, This is mingliang, a machine learning engineer in recommendation area. I see there’s discussion about “heterogeneous topologies in training and inference.” Actually this is a very common case in recommendation system especially in CTR prediction tasks. For training task, usually data is

[jira] [Created] (FLINK-23689) TestProcessingTimeService.setCurrentTime(long) not delay the firing timer by 1ms delay

2021-08-09 Thread Jiayi Liao (Jira)
Jiayi Liao created FLINK-23689: -- Summary: TestProcessingTimeService.setCurrentTime(long) not delay the firing timer by 1ms delay Key: FLINK-23689 URL: https://issues.apache.org/jira/browse/FLINK-23689 Pr

[jira] [Created] (FLINK-23690) Processing timers can be triggered more efficiently

2021-08-09 Thread Jiayi Liao (Jira)
Jiayi Liao created FLINK-23690: -- Summary: Processing timers can be triggered more efficiently Key: FLINK-23690 URL: https://issues.apache.org/jira/browse/FLINK-23690 Project: Flink Issue Type: I

[jira] [Created] (FLINK-23691) flink sql submit job blob parallelism

2021-08-09 Thread Peihui He (Jira)
Peihui He created FLINK-23691: - Summary: flink sql submit job blob parallelism Key: FLINK-23691 URL: https://issues.apache.org/jira/browse/FLINK-23691 Project: Flink Issue Type: Improvement

Incompatible RAW types in Table API

2021-08-09 Thread Dominik Wosiński
Hey all, I think I've hit some weird issue in Flink TypeInformation generation. I have the following code: val stream: DataStream[Event] = ... tableEnv.createTemporaryView("TableName",stream) val table = tableEnv .sqlQuery("SELECT id, timestamp, eventType from TableName") tableEnvironment.toAppen

Re: [ANNOUNCE] Apache Flink 1.13.2 released

2021-08-09 Thread Till Rohrmann
Thanks Yun Tang for being our release manager and the great work! Also thanks a lot to everyone who contributed to this release. Cheers, Till On Mon, Aug 9, 2021 at 9:48 AM Yu Li wrote: > Thanks Yun Tang for being our release manager and everyone else who made > the release possible! > > Best R

Re: Incompatible RAW types in Table API

2021-08-09 Thread Till Rohrmann
Hi Dominik, I am pulling in Timo who might know more about this. Cheers, Till On Mon, Aug 9, 2021 at 3:21 PM Dominik Wosiński wrote: > Hey all, > > I think I've hit some weird issue in Flink TypeInformation generation. I > have the following code: > > val stream: DataStream[Event] = ... > tabl

Re: Incompatible RAW types in Table API

2021-08-09 Thread Dominik Wosiński
It should be `id` instead of `licence` in the error, I've copy-pasted it incorrectly :< I've also tried additional thing, i.e. creating the ScalarFunction that does mapping of one avro generated enum to additional avro generated enum: @FunctionHint( input = Array( new DataTypeHint(value = "

[jira] [Created] (FLINK-23692) Clarify text on "Cancel Job" confirmation buttons

2021-08-09 Thread Jira
Márton Balassi created FLINK-23692: -- Summary: Clarify text on "Cancel Job" confirmation buttons Key: FLINK-23692 URL: https://issues.apache.org/jira/browse/FLINK-23692 Project: Flink Issue T

[jira] [Created] (FLINK-23693) Add principal creation possibilities to SecureTestEnvironment

2021-08-09 Thread Gabor Somogyi (Jira)
Gabor Somogyi created FLINK-23693: - Summary: Add principal creation possibilities to SecureTestEnvironment Key: FLINK-23693 URL: https://issues.apache.org/jira/browse/FLINK-23693 Project: Flink

Re: [VOTE] FLIP-180: Adjust StreamStatus and Idleness definition

2021-08-09 Thread Stephan Ewen
+1 (binding) On Mon, Aug 9, 2021 at 12:08 PM Till Rohrmann wrote: > +1 (binding) > > Cheers, > Till > > On Thu, Aug 5, 2021 at 9:09 PM Arvid Heise wrote: > > > Dear devs, > > > > I'd like to open a vote on FLIP-180: Adjust StreamStatus and Idleness > > definition [1] which was discussed in this

Re: [VOTE] FLIP-180: Adjust StreamStatus and Idleness definition

2021-08-09 Thread Piotr Nowojski
+1 (binding) Piotrek pon., 9 sie 2021 o 18:00 Stephan Ewen napisał(a): > +1 (binding) > > On Mon, Aug 9, 2021 at 12:08 PM Till Rohrmann > wrote: > > > +1 (binding) > > > > Cheers, > > Till > > > > On Thu, Aug 5, 2021 at 9:09 PM Arvid Heise wrote: > > > > > Dear devs, > > > > > > I'd like to o

Re: [VOTE] FLIP-180: Adjust StreamStatus and Idleness definition

2021-08-09 Thread Tzu-Li (Gordon) Tai
+1 (binding) On Tue, Aug 10, 2021 at 12:15 AM Piotr Nowojski wrote: > +1 (binding) > > Piotrek > > pon., 9 sie 2021 o 18:00 Stephan Ewen napisał(a): > > > +1 (binding) > > > > On Mon, Aug 9, 2021 at 12:08 PM Till Rohrmann > > wrote: > > > > > +1 (binding) > > > > > > Cheers, > > > Till > > > >

Re: Incompatible RAW types in Table API

2021-08-09 Thread Timo Walther
Hi Dominik, `toAppendStream` is soft deprecated in Flink 1.13 and will be deprecated in Flink 1.13. It uses the old type system and might not match perfectly with the other reworked type system in new functions and sources. For SQL, a lot of Avro classes need to be treated as RAW types. But w

Re: Incompatible RAW types in Table API

2021-08-09 Thread Timo Walther
Sorry, I meant "will be deprecated in Flink 1.14" On 09.08.21 19:32, Timo Walther wrote: Hi Dominik, `toAppendStream` is soft deprecated in Flink 1.13 and will be deprecated in Flink 1.13. It uses the old type system and might not match perfectly with the other reworked type system in new fun

Re: [ANNOUNCE] Apache Flink 1.13.2 released

2021-08-09 Thread Xintong Song
Thanks Yun and everyone~! Thank you~ Xintong Song On Mon, Aug 9, 2021 at 10:14 PM Till Rohrmann wrote: > Thanks Yun Tang for being our release manager and the great work! Also > thanks a lot to everyone who contributed to this release. > > Cheers, > Till > > On Mon, Aug 9, 2021 at 9:48 AM Yu

Re: [DISCUSS] FLIP-173: Support DAG of algorithms (Flink ML)

2021-08-09 Thread Becket Qin
Thanks Mingliang. It is super helpful to get your input. At this point, there are two ways mentioned in the FLIP to support heterogeneous topology in training and inference phase. 1. Create two separate DAGs or code for training and inference respectively. 2. An encapsulation API called Graph, whi

[jira] [Created] (FLINK-23694) KafkaWriterITCase hangs on azure

2021-08-09 Thread Xintong Song (Jira)
Xintong Song created FLINK-23694: Summary: KafkaWriterITCase hangs on azure Key: FLINK-23694 URL: https://issues.apache.org/jira/browse/FLINK-23694 Project: Flink Issue Type: Bug Co

[jira] [Created] (FLINK-23695) DispatcherFailoverITCase.testRecoverFromCheckpointAfterJobGraphRemovalOfTerminatedJobFailed fails due to timeout

2021-08-09 Thread Xintong Song (Jira)
Xintong Song created FLINK-23695: Summary: DispatcherFailoverITCase.testRecoverFromCheckpointAfterJobGraphRemovalOfTerminatedJobFailed fails due to timeout Key: FLINK-23695 URL: https://issues.apache.org/jira/bro

[jira] [Created] (FLINK-23696) RMQSourceTest.testRedeliveredSessionIDsAck fails on azure

2021-08-09 Thread Xintong Song (Jira)
Xintong Song created FLINK-23696: Summary: RMQSourceTest.testRedeliveredSessionIDsAck fails on azure Key: FLINK-23696 URL: https://issues.apache.org/jira/browse/FLINK-23696 Project: Flink Iss

Re: [DISCUSS] FLIP-173: Support DAG of algorithms (Flink ML)

2021-08-09 Thread 青雉(祁明良)
Vote for option 2. It is similar to what we are doing with Tensorflow. 1. Define the graph in training phase 2. Export model with different input/output spec for online inference Thanks, Mingliang On Aug 10, 2021, at 9:39 AM, Becket Qin mailto:becket@gmail.com>> wrote: estimatorInputs 本?件

[jira] [Created] (FLINK-23697) flink standalone Isolation is poor, why not do as spark does

2021-08-09 Thread jackylau (Jira)
jackylau created FLINK-23697: Summary: flink standalone Isolation is poor, why not do as spark does Key: FLINK-23697 URL: https://issues.apache.org/jira/browse/FLINK-23697 Project: Flink Issue T

Re: [DISCUSS] FLIP-173: Support DAG of algorithms (Flink ML)

2021-08-09 Thread Dong Lin
Thank you Mingliang for providing the comments. Currently option-1 proposes Graph/GraphModel/GraphBuilder to build an Estimator from a graph of Estimator/Transformer, where Estimator could generate the model (as a Transformer) directly. On the other hand, option-2 proposes AlgoOperator that can be