Re: How heap state backend serialization happens during snapshotting

2021-08-26 Thread JING ZHANG
Hello, Let me share my views on this issue, correct me if there is anything wrong. To get a state handle, you have to create a StateDescriptor. This holds the name of the state, the type of the values that the state holds and other information. You could pass the type serializer or the type inform

[jira] [Created] (FLINK-24013) Add IssueNavigationLink for IDEA

2021-08-26 Thread gaoyajun02 (Jira)
gaoyajun02 created FLINK-24013: -- Summary: Add IssueNavigationLink for IDEA Key: FLINK-24013 URL: https://issues.apache.org/jira/browse/FLINK-24013 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-173: Support DAG of algorithms (Flink ML)

2021-08-26 Thread Zhipeng Zhang
Thanks for the post, Dong :) We welcome everyone to drop us an email on Flink ML. Let's work together to build machine learning on Flink :) Dong Lin 于2021年8月25日周三 下午8:58写道: > Hi everyone, > > Based on the feedback received in the online/offline discussion in the > past few weeks, we (Zhepeng,

[jira] [Created] (FLINK-24012) PulsarSourceITCase.testTaskManagerFailure fails due to NoResourceAvailableException

2021-08-26 Thread Xintong Song (Jira)
Xintong Song created FLINK-24012: Summary: PulsarSourceITCase.testTaskManagerFailure fails due to NoResourceAvailableException Key: FLINK-24012 URL: https://issues.apache.org/jira/browse/FLINK-24012 P

[jira] [Created] (FLINK-24011) SET 'sql-client.verbose' = 'true'; on Flink SQL client does not give full stack trace

2021-08-26 Thread James Kim (Jira)
James Kim created FLINK-24011: - Summary: SET 'sql-client.verbose' = 'true'; on Flink SQL client does not give full stack trace Key: FLINK-24011 URL: https://issues.apache.org/jira/browse/FLINK-24011 Proje

[jira] [Created] (FLINK-24010) HybridSourceReader needs to forward checkpoint notifications

2021-08-26 Thread Thomas Weise (Jira)
Thomas Weise created FLINK-24010: Summary: HybridSourceReader needs to forward checkpoint notifications Key: FLINK-24010 URL: https://issues.apache.org/jira/browse/FLINK-24010 Project: Flink

[jira] [Created] (FLINK-24009) Upgrade to Apache pom parent 24

2021-08-26 Thread Kevin Ratnasekera (Jira)
Kevin Ratnasekera created FLINK-24009: - Summary: Upgrade to Apache pom parent 24 Key: FLINK-24009 URL: https://issues.apache.org/jira/browse/FLINK-24009 Project: Flink Issue Type: Improve

How heap state backend serialization happens during snapshotting

2021-08-26 Thread Muhammad Haseeb Asif
Hello, I have been trying to understand how serialization works for the Flink Heap Keyed state backend when a checkpoint happens. I want to know how Flink knows which type of serializer to use to serialize the values for different states during the snapshot. I have debugged both RocksDB and he

[jira] [Created] (FLINK-24008) Support state cleanup based on unique keys

2021-08-26 Thread Nico Kruber (Jira)
Nico Kruber created FLINK-24008: --- Summary: Support state cleanup based on unique keys Key: FLINK-24008 URL: https://issues.apache.org/jira/browse/FLINK-24008 Project: Flink Issue Type: New Feat

[jira] [Created] (FLINK-24007) Support Avro timestamp conversion with precision greater than three

2021-08-26 Thread Xingcan Cui (Jira)
Xingcan Cui created FLINK-24007: --- Summary: Support Avro timestamp conversion with precision greater than three Key: FLINK-24007 URL: https://issues.apache.org/jira/browse/FLINK-24007 Project: Flink

[jira] [Created] (FLINK-24006) MailboxExecutorImplTest#testIsIdle does not test the correct behaviour

2021-08-26 Thread Fabian Paul (Jira)
Fabian Paul created FLINK-24006: --- Summary: MailboxExecutorImplTest#testIsIdle does not test the correct behaviour Key: FLINK-24006 URL: https://issues.apache.org/jira/browse/FLINK-24006 Project: Flink

Re: 【Announce】Zeppelin 0.10.0 is released, Flink on Zeppelin Improved

2021-08-26 Thread Till Rohrmann
Cool, thanks for letting us know Jeff. Hopefully, many users use Zeppelin together with Flink. Cheers, Till On Thu, Aug 26, 2021 at 4:47 AM Leonard Xu wrote: > Thanks Jeff for the great work ! > > Best, > Leonard > > 在 2021年8月25日,22:48,Jeff Zhang 写道: > > Hi Flink users, > > We (Zeppelin commun

[jira] [Created] (FLINK-24005) Resource requirements declaration may be incorrect if JobMaster disconnects with a TaskManager with available slots in the SlotPool

2021-08-26 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-24005: --- Summary: Resource requirements declaration may be incorrect if JobMaster disconnects with a TaskManager with available slots in the SlotPool Key: FLINK-24005 URL: https://issues.apache.org/

[jira] [Created] (FLINK-24003) Lookback mode doesn't work when mixing use of Python Table API and Python DataStream API

2021-08-26 Thread Dian Fu (Jira)
Dian Fu created FLINK-24003: --- Summary: Lookback mode doesn't work when mixing use of Python Table API and Python DataStream API Key: FLINK-24003 URL: https://issues.apache.org/jira/browse/FLINK-24003 Projec

[jira] [Created] (FLINK-24004) Introduce a source/sink testing harness

2021-08-26 Thread Jira
Ingo Bürk created FLINK-24004: - Summary: Introduce a source/sink testing harness Key: FLINK-24004 URL: https://issues.apache.org/jira/browse/FLINK-24004 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-24002) Support count window with the window TVF in planner

2021-08-26 Thread JING ZHANG (Jira)
JING ZHANG created FLINK-24002: -- Summary: Support count window with the window TVF in planner Key: FLINK-24002 URL: https://issues.apache.org/jira/browse/FLINK-24002 Project: Flink Issue Type: S

[jira] [Created] (FLINK-24001) Support evaluating individual window table-valued function in runtime

2021-08-26 Thread JING ZHANG (Jira)
JING ZHANG created FLINK-24001: -- Summary: Support evaluating individual window table-valued function in runtime Key: FLINK-24001 URL: https://issues.apache.org/jira/browse/FLINK-24001 Project: Flink

Re: [VOTE] Apache Flink Stateful Functions 3.1.0, release candidate #1

2021-08-26 Thread Stephan Ewen
+1 (binding) - Build project on Java 8 (maven command line) with full end to end tests - Compiled and ran tests in IDE (IntelliJ) with Java 11 (possible after some manual config, see comment below to automate this) - No binaries in the distribution - Verified license and notice files - C

[jira] [Created] (FLINK-24000) window aggregate support allow lateness

2021-08-26 Thread JING ZHANG (Jira)
JING ZHANG created FLINK-24000: -- Summary: window aggregate support allow lateness Key: FLINK-24000 URL: https://issues.apache.org/jira/browse/FLINK-24000 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-23999) Support evaluating individual window table-valued function

2021-08-26 Thread JING ZHANG (Jira)
JING ZHANG created FLINK-23999: -- Summary: Support evaluating individual window table-valued function Key: FLINK-23999 URL: https://issues.apache.org/jira/browse/FLINK-23999 Project: Flink Issue

Re: [VOTE] Apache Flink Stateful Functions 3.1.0, release candidate #1

2021-08-26 Thread Arvid Heise
+1 (binding) - Built from downloaded sources with Java 8 (mvn install -Prun-e2e-tests) - Verified signatures and hashes Best, Arvid On Thu, Aug 26, 2021 at 8:32 AM Tzu-Li (Gordon) Tai wrote: > +1 (binding) > > - Built from source with Java 11 and Java 8 (mvn clean install > -Prun-e2e-tests) >

[jira] [Created] (FLINK-23998) Scopt not mentioned in flink-dist NOTICE file

2021-08-26 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23998: Summary: Scopt not mentioned in flink-dist NOTICE file Key: FLINK-23998 URL: https://issues.apache.org/jira/browse/FLINK-23998 Project: Flink Issue T

[jira] [Created] (FLINK-23997) SQL windowing table-valued function improvement

2021-08-26 Thread JING ZHANG (Jira)
JING ZHANG created FLINK-23997: -- Summary: SQL windowing table-valued function improvement Key: FLINK-23997 URL: https://issues.apache.org/jira/browse/FLINK-23997 Project: Flink Issue Type: Bug

Re: Scala 2.12 End to End Tests

2021-08-26 Thread Danny Cranmer
I have raised the Jira [1]. If no one picks this up I will try to find some time next week to make a start. Thanks. [1] https://issues.apache.org/jira/browse/FLINK-23996 On Tue, Aug 24, 2021 at 11:37 AM Till Rohrmann wrote: > Great, thanks Danny. > > On Mon, Aug 23, 2021 at 9:41 PM Danny Cranm

[jira] [Created] (FLINK-23996) Make Scala 2.12 default build target

2021-08-26 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-23996: - Summary: Make Scala 2.12 default build target Key: FLINK-23996 URL: https://issues.apache.org/jira/browse/FLINK-23996 Project: Flink Issue Type: Improvemen

[jira] [Created] (FLINK-23995) Hive dialect: `insert overwrite table partition if not exists` will throw exception when tablename is like 'database.table'

2021-08-26 Thread luoyuxia (Jira)
luoyuxia created FLINK-23995: Summary: Hive dialect: `insert overwrite table partition if not exists` will throw exception when tablename is like 'database.table' Key: FLINK-23995 URL: https://issues.apache.org/ji

[jira] [Created] (FLINK-23994) ArrayDataSerializer and MapDataSerializer doesn't handle correctly for Null values

2021-08-26 Thread Dian Fu (Jira)
Dian Fu created FLINK-23994: --- Summary: ArrayDataSerializer and MapDataSerializer doesn't handle correctly for Null values Key: FLINK-23994 URL: https://issues.apache.org/jira/browse/FLINK-23994 Project: Fli

[jira] [Created] (FLINK-23993) Describe eventually-consistency of materializing upserts

2021-08-26 Thread Nico Kruber (Jira)
Nico Kruber created FLINK-23993: --- Summary: Describe eventually-consistency of materializing upserts Key: FLINK-23993 URL: https://issues.apache.org/jira/browse/FLINK-23993 Project: Flink Issue

[jira] [Created] (FLINK-23992) Update doc version in release-1.13 to 1.13.2

2021-08-26 Thread Yun Tang (Jira)
Yun Tang created FLINK-23992: Summary: Update doc version in release-1.13 to 1.13.2 Key: FLINK-23992 URL: https://issues.apache.org/jira/browse/FLINK-23992 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-23991) Specifying yarn.staging-dir fail when staging scheme is different from default fs scheme

2021-08-26 Thread Junfan Zhang (Jira)
Junfan Zhang created FLINK-23991: Summary: Specifying yarn.staging-dir fail when staging scheme is different from default fs scheme Key: FLINK-23991 URL: https://issues.apache.org/jira/browse/FLINK-23991

[jira] [Created] (FLINK-23990) Replace custom monaco editor component

2021-08-26 Thread Jira
Ingo Bürk created FLINK-23990: - Summary: Replace custom monaco editor component Key: FLINK-23990 URL: https://issues.apache.org/jira/browse/FLINK-23990 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-23989) Flink SQL visibility

2021-08-26 Thread wangzhihao (Jira)
wangzhihao created FLINK-23989: -- Summary: Flink SQL visibility Key: FLINK-23989 URL: https://issues.apache.org/jira/browse/FLINK-23989 Project: Flink Issue Type: New Feature Component

[jira] [Created] (FLINK-23988) Test corner cases

2021-08-26 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-23988: -- Summary: Test corner cases Key: FLINK-23988 URL: https://issues.apache.org/jira/browse/FLINK-23988 Project: Flink Issue Type: Sub-task Componen

[jira] [Created] (FLINK-23987) Move Powered By section into separate page

2021-08-26 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23987: Summary: Move Powered By section into separate page Key: FLINK-23987 URL: https://issues.apache.org/jira/browse/FLINK-23987 Project: Flink Issue Type