[jira] [Created] (FLINK-20071) Add a dedicated stage to build the PyFlink wheel packges in azure-pipelines.yml
Dian Fu created FLINK-20071: --- Summary: Add a dedicated stage to build the PyFlink wheel packges in azure-pipelines.yml Key: FLINK-20071 URL: https://issues.apache.org/jira/browse/FLINK-20071 Project: Flink Issue Type: Improvement Components: Build System Reporter: Dian Fu Assignee: Dian Fu Fix For: 1.12.0 Currently, we reuse the nightly pipeline to build the "PyFlink wheel packages" during release. There are some drawbacks as following: - As there are many stages in the nightly pipeline and so it takes quite a long time to build the "PyFlink wheel packages" as there is no much resource available in the personal azure account. It takes me about 5 to 6 hours to build the "PyFlink wheel packages" during preparing the RC1 of 1.12.0. - It's wired that there are so many errors [1] for the stages and it confuses release managers who are not familiar with process. [1] https://dev.azure.com/dianfu/Flink/_build/results?buildId=203&view=results -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-20072) Add documentation for FLIP-107
Timo Walther created FLINK-20072: Summary: Add documentation for FLIP-107 Key: FLINK-20072 URL: https://issues.apache.org/jira/browse/FLINK-20072 Project: Flink Issue Type: Sub-task Components: Documentation Reporter: Timo Walther Assignee: Timo Walther Add documentation for FLIP-107: - Connector/format metadata in general - Kafka key/value and metadata - Debezium metadata -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-20073) Add kerberos setup documentation for native k8s integration
Yangze Guo created FLINK-20073: -- Summary: Add kerberos setup documentation for native k8s integration Key: FLINK-20073 URL: https://issues.apache.org/jira/browse/FLINK-20073 Project: Flink Issue Type: Task Reporter: Yangze Guo In FLINK-18971, we support to mount kerberos conf as ConfigMap and Keytab as Secrete. We need to add a user doc for it. Maybe on the Kerberos page. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Apache Flink Stateful Functions 2.2.1, release candidate #1
+1 (binding)
Compiled source and ran all tests (JDK 8)
Ran example (greeter) with a docker compose setup
Release contains no binaries.
Checked LICENSE and NOTICE for source release
Checked LICENSE and NOTICE for flink-statefun-distribution binary release
On Fri, Nov 6, 2020 at 11:46 PM Igal Shilman wrote:
> +1 (non binding)
>
> Legal:
> - Checked signatures
> - Checked that no binaries were included in source artifacts
>
> Build
> - Built source `mvn clean install -Prun-e2e-tests` with jdk8
> - Built source mvn clean install -Prun-e2e-tests' with jdk11
> - Built the python SDK
> - Built the Docker Image
>
> Functional:
> - Verified that FLINK-19692 can no longer be reproduced.
> - Run a randomized smoke test ("random clicker"), with a high volume
> producer, and state verification in the presence of task manager failures.
> This program run successfully on this RC, and failed (as expected) on the
> previous version.
>
> Thanks,
> Igal.
>
> On Fri, Nov 6, 2020 at 9:48 AM Tzu-Li (Gordon) Tai
> wrote:
>
> > +1 (binding)
> >
> > Legal:
> > - Checked signatures
> > - Checked that no binaries were included in source artifacts
> > - Build source `mvn clean install -Prun-e2e-tests`
> > - No dependency changes since 2.2.0, so assuming that NOTICE files are
> > correct (require no changes)
> >
> > Functional:
> > - Verified that FLINK-19692 can no longer be reproduced with a StateFun
> app
> > that deterministically floods the feedback loop with events. Run with a
> > docker-compose setup with parallelism=4, checkpointing to S3, verified
> for
> > all state backends.
> > - Checked that state bootstrap API by running the state bootstrap
> example,
> > and restoring the greeter example from the generated savepoint.
> >
> > On Thu, Nov 5, 2020 at 3:24 AM Robert Metzger
> wrote:
> >
> > > +1 binding
> > >
> > > Checks:
> > > - source sha and signature
> > > - mvn clean install passed on source archive
> > > - staging artifacts have correct version, also in quickstart, NOTICE
> > files
> > > seem correct
> > >
> > >
> > >
> > > On Wed, Nov 4, 2020 at 11:20 AM Tzu-Li (Gordon) Tai <
> tzuli...@apache.org
> > >
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > Please review and vote on the release candidate #1 for the version
> > 2.2.1
> > > of
> > > > Apache Flink Stateful Functions,
> > > > as follows:
> > > > [ ] +1, Approve the release
> > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > >
> > > > ***Testing Guideline***
> > > >
> > > > You can find here [1] a page in the project wiki on instructions for
> > > > testing.
> > > > To cast a vote, it is not necessary to perform all listed checks,
> > > > but please mention which checks you have performed when voting.
> > > >
> > > > ***Release Overview***
> > > >
> > > > As an overview, the release consists of the following:
> > > > a) Stateful Functions canonical source distribution, to be deployed
> to
> > > the
> > > > release repository at dist.apache.org
> > > > b) Stateful Functions Python SDK distributions to be deployed to PyPI
> > > > c) Maven artifacts to be deployed to the Maven Central Repository
> > > > d) New Dockerfiles for the release
> > > > e) Release announcement to be published to the Flink website
> > > >
> > > > ***Staging Areas to Review***
> > > >
> > > > The staging areas containing the above mentioned artifacts are as
> > > follows,
> > > > for your review:
> > > > * All artifacts for a) and b) can be found in the corresponding dev
> > > > repository at dist.apache.org [2]
> > > > * All artifacts for c) can be found at the Apache Nexus Repository
> [3]
> > > >
> > > > All artifacts are signed with the key
> > > > 1C1E2394D3194E1944613488F320986D35C33D6A [4]
> > > >
> > > > Other links for your review:
> > > > * JIRA release notes [5]
> > > > * source code tag "release-2.2.1-rc1" [6]
> > > > * PR for the new Dockerfiles [7]
> > > > * PR for the release announcement blog post [8]
> > > >
> > > > ***Vote Duration***
> > > >
> > > > The voting time will run for 72 hours, lasting until Nov. 7th, 10am
> > CET.
> > > > It is adopted by majority approval, with at least 3 PMC affirmative
> > > votes.
> > > >
> > > > Thanks,
> > > > Gordon
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/Verifying+a+Flink+Stateful+Functions+Release
> > > > [2]
> > > https://dist.apache.org/repos/dist/dev/flink/flink-statefun-2.2.1-rc1/
> > > > [3] https://repository.apache.org/content/repositories/org apache
> > > > flink-1401/
> > > > <
> > https://repository.apache.org/content/repositories/orgapacheflink-1401/
> > > >
> > > > [4] https://dist.apache.org/repos/dist/release/flink/KEYS
> > > > [5]
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12349291
> > > > [6]
> > > >
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=flink-statefun.git;a=commit;h=3d53612fccfe143acbdd1686a942b29c96c6bbd8
> > > > [7] https://git
[jira] [Created] (FLINK-20074) ChangelogSourceITCase.testRegularJoin fail
Jingsong Lee created FLINK-20074: Summary: ChangelogSourceITCase.testRegularJoin fail Key: FLINK-20074 URL: https://issues.apache.org/jira/browse/FLINK-20074 Project: Flink Issue Type: Bug Affects Versions: 1.12.0 Reporter: Jingsong Lee Fix For: 1.12.0 INSTANCE: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=9379&view=logs&s=ae4f8708-9994-57d3-c2d7-b892156e7812&j=e25d5e7e-2a9c-5589-4940-0b638d75a414 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-20075) When the JobMaster is initialized, the JobID of the JobGraph is retrieved again in the createlotpool method
ZhangAiping created FLINK-20075: --- Summary: When the JobMaster is initialized, the JobID of the JobGraph is retrieved again in the createlotpool method Key: FLINK-20075 URL: https://issues.apache.org/jira/browse/FLINK-20075 Project: Flink Issue Type: Improvement Components: Runtime / Coordination Affects Versions: 1.11.2 Reporter: ZhangAiping In the initialization of JobMaster, the JobID is obtained twice. For the first time, the JobID is obtained and recorded when the log is recorded; The second time when you createSlotPool, you get it from the JobGraph again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[RESULT] [VOTE] Apache Flink Stateful Functions 2.2.1, release candidate #1
Thanks for testing and voting everyone!
The vote has passed with 4 +1 votes, 3 of which are binding:
+1:
- Robert Metzger (binding)
- Tzu-Li (Gordon) Tai (binding)
- Igal Shilman
- Stephan Ewen (binding)
There are no 0 or -1 votes.
I'll announce the release in a separate email once the artifacts are
released.
Cheers,
Gordon
On Tue, Nov 10, 2020 at 6:31 PM Stephan Ewen wrote:
> +1 (binding)
>
> Compiled source and ran all tests (JDK 8)
> Ran example (greeter) with a docker compose setup
> Release contains no binaries.
> Checked LICENSE and NOTICE for source release
> Checked LICENSE and NOTICE for flink-statefun-distribution binary release
>
>
> On Fri, Nov 6, 2020 at 11:46 PM Igal Shilman wrote:
>
> > +1 (non binding)
> >
> > Legal:
> > - Checked signatures
> > - Checked that no binaries were included in source artifacts
> >
> > Build
> > - Built source `mvn clean install -Prun-e2e-tests` with jdk8
> > - Built source mvn clean install -Prun-e2e-tests' with jdk11
> > - Built the python SDK
> > - Built the Docker Image
> >
> > Functional:
> > - Verified that FLINK-19692 can no longer be reproduced.
> > - Run a randomized smoke test ("random clicker"), with a high volume
> > producer, and state verification in the presence of task manager
> failures.
> > This program run successfully on this RC, and failed (as expected) on the
> > previous version.
> >
> > Thanks,
> > Igal.
> >
> > On Fri, Nov 6, 2020 at 9:48 AM Tzu-Li (Gordon) Tai
> > wrote:
> >
> > > +1 (binding)
> > >
> > > Legal:
> > > - Checked signatures
> > > - Checked that no binaries were included in source artifacts
> > > - Build source `mvn clean install -Prun-e2e-tests`
> > > - No dependency changes since 2.2.0, so assuming that NOTICE files are
> > > correct (require no changes)
> > >
> > > Functional:
> > > - Verified that FLINK-19692 can no longer be reproduced with a StateFun
> > app
> > > that deterministically floods the feedback loop with events. Run with a
> > > docker-compose setup with parallelism=4, checkpointing to S3, verified
> > for
> > > all state backends.
> > > - Checked that state bootstrap API by running the state bootstrap
> > example,
> > > and restoring the greeter example from the generated savepoint.
> > >
> > > On Thu, Nov 5, 2020 at 3:24 AM Robert Metzger
> > wrote:
> > >
> > > > +1 binding
> > > >
> > > > Checks:
> > > > - source sha and signature
> > > > - mvn clean install passed on source archive
> > > > - staging artifacts have correct version, also in quickstart, NOTICE
> > > files
> > > > seem correct
> > > >
> > > >
> > > >
> > > > On Wed, Nov 4, 2020 at 11:20 AM Tzu-Li (Gordon) Tai <
> > tzuli...@apache.org
> > > >
> > > > wrote:
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > Please review and vote on the release candidate #1 for the version
> > > 2.2.1
> > > > of
> > > > > Apache Flink Stateful Functions,
> > > > > as follows:
> > > > > [ ] +1, Approve the release
> > > > > [ ] -1, Do not approve the release (please provide specific
> comments)
> > > > >
> > > > > ***Testing Guideline***
> > > > >
> > > > > You can find here [1] a page in the project wiki on instructions
> for
> > > > > testing.
> > > > > To cast a vote, it is not necessary to perform all listed checks,
> > > > > but please mention which checks you have performed when voting.
> > > > >
> > > > > ***Release Overview***
> > > > >
> > > > > As an overview, the release consists of the following:
> > > > > a) Stateful Functions canonical source distribution, to be deployed
> > to
> > > > the
> > > > > release repository at dist.apache.org
> > > > > b) Stateful Functions Python SDK distributions to be deployed to
> PyPI
> > > > > c) Maven artifacts to be deployed to the Maven Central Repository
> > > > > d) New Dockerfiles for the release
> > > > > e) Release announcement to be published to the Flink website
> > > > >
> > > > > ***Staging Areas to Review***
> > > > >
> > > > > The staging areas containing the above mentioned artifacts are as
> > > > follows,
> > > > > for your review:
> > > > > * All artifacts for a) and b) can be found in the corresponding dev
> > > > > repository at dist.apache.org [2]
> > > > > * All artifacts for c) can be found at the Apache Nexus Repository
> > [3]
> > > > >
> > > > > All artifacts are signed with the key
> > > > > 1C1E2394D3194E1944613488F320986D35C33D6A [4]
> > > > >
> > > > > Other links for your review:
> > > > > * JIRA release notes [5]
> > > > > * source code tag "release-2.2.1-rc1" [6]
> > > > > * PR for the new Dockerfiles [7]
> > > > > * PR for the release announcement blog post [8]
> > > > >
> > > > > ***Vote Duration***
> > > > >
> > > > > The voting time will run for 72 hours, lasting until Nov. 7th, 10am
> > > CET.
> > > > > It is adopted by majority approval, with at least 3 PMC affirmative
> > > > votes.
> > > > >
> > > > > Thanks,
> > > > > Gordon
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/Verifying+a+Flink+St
[jira] [Created] (FLINK-20076) DispatcherTest.testOnRemovedJobGraphDoesNotCleanUpHAFiles does not test the desired functionality
Matthias created FLINK-20076:
Summary: DispatcherTest.testOnRemovedJobGraphDoesNotCleanUpHAFiles
does not test the desired functionality
Key: FLINK-20076
URL: https://issues.apache.org/jira/browse/FLINK-20076
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Affects Versions: 1.11.2, 1.10.2
Reporter: Matthias
Assignee: Matthias
Fix For: 1.12.0
{{DispatcherTest.testOnRemovedJobGraphDoesNotCleanUpHAFiles}} succeeds but due
to different reasons: The used {{TestingJobGraphStore}} is not started. An
{{IllegalStateException}} prevents the code from reaching the set
`removeJobGraphFuture` to get triggered. Hence, the test succeeds but not for
the reason the test was implemented for.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-20077) Cannot register a view with MATCH_RECOGNIZE clause
Dawid Wysakowicz created FLINK-20077:
Summary: Cannot register a view with MATCH_RECOGNIZE clause
Key: FLINK-20077
URL: https://issues.apache.org/jira/browse/FLINK-20077
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Affects Versions: 1.11.0, 1.12.0
Reporter: Dawid Wysakowicz
Assignee: Dawid Wysakowicz
Fix For: 1.12.0
{code}
TableEnvironment env =
TableEnvironment.create(EnvironmentSettings.newInstance()
.useBlinkPlanner()
.build());
env.executeSql("" +
"CREATE TEMPORARY TABLE data (\n" +
"id INT,\n" +
"ts AS PROCTIME()\n" +
") WITH (\n" +
"'connector' = 'datagen',\n" +
"'rows-per-second' = '3',\n" +
"'fields.id.kind' = 'sequence',\n" +
"'fields.id.start' = '100',\n" +
"'fields.id.end' = '200'\n" +
")");
env.executeSql("" +
"CREATE TEMPORARY VIEW events AS \n" +
"SELECT 1 AS key, id, MOD(id, 10) AS measurement, ts
\n" +
"FROM data");
env.executeSql("" +
"CREATE TEMPORARY VIEW foo AS \n" +
"SELECT * \n" +
"FROM events MATCH_RECOGNIZE (\n" +
"PARTITION BY key \n" +
"ORDER BY ts ASC \n" +
"MEASURES \n" +
" this_step.id as startId,\n" +
" next_step.id as nextId,\n" +
" this_step.ts AS ts1,\n" +
" next_step.ts AS ts2,\n" +
" next_step.measurement - this_step.measurement AS
diff \n" +
"AFTER MATCH SKIP TO NEXT ROW \n" +
"PATTERN (this_step next_step)\n" +
"DEFINE this_step AS TRUE,\n" +
"next_step AS TRUE\n" +
")");
env.executeSql("SELECT * FROM foo");
{code}
fails with:
{code}
java.lang.AssertionError
at
org.apache.calcite.sql.SqlMatchRecognize$SqlMatchRecognizeOperator.createCall(SqlMatchRecognize.java:274)
at
org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.result(SqlShuttle.java:117)
at
org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.result(SqlShuttle.java:101)
at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:67)
at
org.apache.flink.table.planner.utils.Expander$Expanded$1.visit(Expander.java:153)
at
org.apache.flink.table.planner.utils.Expander$Expanded$1.visit(Expander.java:130)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
at
org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:134)
at
org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:101)
at org.apache.calcite.sql.SqlOperator.acceptCall(SqlOperator.java:879)
at
org.apache.calcite.sql.SqlSelectOperator.acceptCall(SqlSelectOperator.java:133)
at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:66)
at
org.apache.flink.table.planner.utils.Expander$Expanded$1.visit(Expander.java:153)
at
org.apache.flink.table.planner.utils.Expander$Expanded$1.visit(Expander.java:130)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
at
org.apache.flink.table.planner.utils.Expander$Expanded.substitute(Expander.java:168)
at
org.apache.flink.table.planner.operations.SqlToOperationConverter.convertViewQuery(SqlToOperationConverter.java:728)
at
org.apache.flink.table.planner.operations.SqlToOperationConverter.convertCreateView(SqlToOperationConverter.java:699)
at
org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:226)
at
org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:78)
at
org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:658)
at
org.apache.flink.table.planner.runtime.stream.table.FunctionITCase.test(FunctionITCase.java:165)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
Re: [DISCUSS] Planning the 1.12 release testing
On 04.11.20 20:05, Robert Metzger wrote: For testing Flink in a coordinated way, and to allow the broader community to participate in the testing more easily, I've created a wiki page to collect testing tasks: https://cwiki.apache.org/confluence/display/FLINK/1.12+Release+-+Community+Testing If you have implemented a new feature, or you've changed something that needs testing, please put down a reference to your change, or testing instructions. Maybe that's just me, but for most of the entries in the Table I wouldn't know what/how to test it. I think most entries would benefit from some notes that outline what the implementors expect to be tested. Though we need to be careful because we don't want to limit too much, which might make them miss some important problems Aljoscha
[jira] [Created] (FLINK-20078) Factor out an ExecutionGraph factory method for DefaultExecutionTopology
Andrey Zagrebin created FLINK-20078: --- Summary: Factor out an ExecutionGraph factory method for DefaultExecutionTopology Key: FLINK-20078 URL: https://issues.apache.org/jira/browse/FLINK-20078 Project: Flink Issue Type: Improvement Components: Runtime / Coordination Reporter: Andrey Zagrebin Based on [this PR discussion|https://github.com/apache/flink/pull/13958#discussion_r519676104]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] Planning the 1.12 release testing
This is a fair point. I would recommend people to put instructions into the Wiki, or ask here if something is unclear! On Tue, Nov 10, 2020 at 2:23 PM Aljoscha Krettek wrote: > On 04.11.20 20:05, Robert Metzger wrote: > > For testing Flink in a coordinated way, and to allow the broader > community > > to participate in the testing more easily, I've created a wiki page to > > collect testing tasks: > > > > > https://cwiki.apache.org/confluence/display/FLINK/1.12+Release+-+Community+Testing > > > > If you have implemented a new feature, or you've changed something that > > needs testing, please put down a reference to your change, or testing > > instructions. > > Maybe that's just me, but for most of the entries in the Table I > wouldn't know what/how to test it. I think most entries would benefit > from some notes that outline what the implementors expect to be tested. > Though we need to be careful because we don't want to limit too much, > which might make them miss some important problems > > Aljoscha >
[jira] [Created] (FLINK-20079) Modified UnalignedCheckpointITCase...MassivelyParallel fails
Roman Khachatryan created FLINK-20079: - Summary: Modified UnalignedCheckpointITCase...MassivelyParallel fails Key: FLINK-20079 URL: https://issues.apache.org/jira/browse/FLINK-20079 Project: Flink Issue Type: Bug Components: Runtime / Task Affects Versions: 1.11.3 Reporter: Roman Khachatryan Assignee: Roman Khachatryan Fix For: 1.12.0, 1.11.3 In FLINK-19681, UnalignedCheckpointITCase was updated to put more backpressure. This revealed a bug introduced in FLINK-19907: resultPartitions can be requested before the operator chain is initialized. A proper fix would be to initialize the chain after the outputs but before the inputs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Register processing time timers when Operator.close() is called
Hi team, I'm writing my custom Operator as a high fan-out operation and I use processing time timers to defer processing some inputs When timers are firing, the operator will continue to process the deferred elements. One typical use case for my Operator is like: ImpulseOperator -> my Operator -> downstream where the watermark of ImpulseOperator advances to MAX_TIMESTAMP immediately. One problem I have is that after my operator.close() is called, it's still possible for me to set processing time timers and wait for these timers to be fired. But it seems like Flink pauses invoking processing timers once one operator.close() is called in the new version. I'm curious why Flink decides to do so and any workaround I can do for my operator? Thanks for your help!
[jira] [Created] (FLINK-20080) Exceeded checkpoint tolerable failure threshold
Alex Z created FLINK-20080: -- Summary: Exceeded checkpoint tolerable failure threshold Key: FLINK-20080 URL: https://issues.apache.org/jira/browse/FLINK-20080 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing Affects Versions: 1.11.2 Reporter: Alex Z -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] FLIP-145: Support SQL windowing table-valued function (2nd)
+1 (binding) On Tue, 10 Nov 2020 at 14:59, Jark Wu wrote: > Hi all, > > There is new feedback on the FLIP-145. So I would like to start a new vote > for FLIP-145 [1], > which has been discussed and reached consensus in the discussion thread > [2]. > > The vote will be open until 15:00 (UTC+8) 13th Nov. (72h), unless there is > an objection or not enough votes. > > Best, > Jark > > [1]: > https://cwiki.apache.org/confluence/display/FLINK/FLIP-145%3A+Support+SQL+windowing+table-valued+function > [2]: > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-145-Support-SQL-windowing-table-valued-function-td45269.html >
[jira] [Created] (FLINK-20081) ExecutorNotifier should run handler in the main thread when receive an exception from the callable.
Jiangjie Qin created FLINK-20081:
Summary: ExecutorNotifier should run handler in the main thread
when receive an exception from the callable.
Key: FLINK-20081
URL: https://issues.apache.org/jira/browse/FLINK-20081
Project: Flink
Issue Type: Bug
Components: Connectors / Common
Reporter: Jiangjie Qin
Currently the {{ExecutorNotifier}} runs the {{handler}} in the worker thread if
there is an exception thrown from the {{callable}}. This breaks the threading
model and prevents an exception from bubbling up to fail the job.
Another issue is that right now, when an exception bubbles up from the
{{SourceCoordinator}}, the UncaughtExceptionHandler will call System.exit(-17)
and kill the JM. This is too much. Instead, we should just fail the job to
trigger a failover.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [VOTE] FLIP-145: Support SQL windowing table-valued function (2nd)
+1 (binding) Jark Wu 于2020年11月11日周三 上午10:13写道： > +1 (binding) > > On Tue, 10 Nov 2020 at 14:59, Jark Wu wrote: > > > Hi all, > > > > There is new feedback on the FLIP-145. So I would like to start a new > vote > > for FLIP-145 [1], > > which has been discussed and reached consensus in the discussion thread > > [2]. > > > > The vote will be open until 15:00 (UTC+8) 13th Nov. (72h), unless there > is > > an objection or not enough votes. > > > > Best, > > Jark > > > > [1]: > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-145%3A+Support+SQL+windowing+table-valued+function > > [2]: > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-145-Support-SQL-windowing-table-valued-function-td45269.html > > >
[jira] [Created] (FLINK-20082) Cannot start Flink application due to "cannot assign instance of java.lang.invoke.SerializedLambda to type scala.Function1
ChangZhuo Chen (陳昌倬) created FLINK-20082: Summary: Cannot start Flink application due to "cannot assign instance of java.lang.invoke.SerializedLambda to type scala.Function1 Key: FLINK-20082 URL: https://issues.apache.org/jira/browse/FLINK-20082 Project: Flink Issue Type: Bug Components: API / DataStream, API / Scala Affects Versions: 1.11.2 Environment: * Flink 1.11.2 * Java 11 * Scala 2.12.11 Reporter: ChangZhuo Chen (陳昌倬) Attachments: image-20201110-060934.png Hi, * Our Flink application (Java 11 + Scala 2.12) has the following problem when executing it. It cannot be run in Flink cluster. * The problem is similar to https://issues.apache.org/jira/browse/SPARK-25047, so maybe the same fix shall be implemented in Flink? !image-20201110-060934.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-20083) OrcFsStreamingSinkITCase times out
Robert Metzger created FLINK-20083:
--
Summary: OrcFsStreamingSinkITCase times out
Key: FLINK-20083
URL: https://issues.apache.org/jira/browse/FLINK-20083
Project: Flink
Issue Type: Improvement
Components: Connectors / FileSystem, Formats (JSON, Avro, Parquet,
ORC, SequenceFile)
Affects Versions: 1.12.0
Reporter: Robert Metzger
https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8661&view=logs&j=66592496-52df-56bb-d03e-37509e1d9d0f&t=ae0269db-6796-5583-2e5f-d84757d711aa
{code}
[ERROR]
OrcFsStreamingSinkITCase>FsStreamingSinkITCaseBase.testPart:84->FsStreamingSinkITCaseBase.test:120->FsStreamingSinkITCaseBase.check:133
» TestTimedOut
[ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 26.871
s <<< FAILURE! - in org.apache.flink.orc.OrcFsStreamingSinkITCase
[ERROR] testPart(org.apache.flink.orc.OrcFsStreamingSinkITCase) Time elapsed:
20.052 s <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 20 seconds
at java.lang.Thread.sleep(Native Method)
at
org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.sleepBeforeRetry(CollectResultFetcher.java:231)
at
org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.next(CollectResultFetcher.java:119)
at
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:103)
at
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:77)
at
org.apache.flink.table.planner.sinks.SelectTableSinkBase$RowIteratorWrapper.hasNext(SelectTableSinkBase.java:115)
at
org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:355)
at java.util.Iterator.forEachRemaining(Iterator.java:115)
at
org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:114)
at
org.apache.flink.table.planner.runtime.stream.FsStreamingSinkITCaseBase.check(FsStreamingSinkITCaseBase.scala:133)
at
org.apache.flink.table.planner.runtime.stream.FsStreamingSinkITCaseBase.test(FsStreamingSinkITCaseBase.scala:120)
at
org.apache.flink.table.planner.runtime.stream.FsStreamingSinkITCaseBase.testPart(FsStreamingSinkITCaseBase.scala:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-20084) Fix NPE when generating watermark for record whose rowtime field is null after watermark push down
Shengkai Fang created FLINK-20084:
-
Summary: Fix NPE when generating watermark for record whose
rowtime field is null after watermark push down
Key: FLINK-20084
URL: https://issues.apache.org/jira/browse/FLINK-20084
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Affects Versions: 1.12.0
Reporter: Shengkai Fang
Fix For: 1.12.0
The problem is from the class
{{PushWatermarkIntoTableSourceScanRuleBase$DefaultWatermarkGeneratorSupplier$DefaultWatermarkGenerator#onEvent}}.
It doesn't test whether the calculated watermark is null before set.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-20085) Remove RemoteFunctionStateMigrator code paths from StateFun
Tzu-Li (Gordon) Tai created FLINK-20085:
---
Summary: Remove RemoteFunctionStateMigrator code paths from
StateFun
Key: FLINK-20085
URL: https://issues.apache.org/jira/browse/FLINK-20085
Project: Flink
Issue Type: Bug
Components: Stateful Functions
Reporter: Tzu-Li (Gordon) Tai
Assignee: Tzu-Li (Gordon) Tai
Fix For: statefun-2.3.0
The {{RemoteFunctionStateMigrator}} was added to allow savepoints with versions
<= 2.1.0 to have a migration path for upgrading to versions >= 2.2.0. The
binary format of remote function state was changed due to demultiplexed remote
state, introduced in 2.2.0.
With 2.2.0 already released with the new formats, it is now safe to fully
remove this migration path.
For users, what this means that it would not be possible to directly upgrade
from 2.0.x / 2.1.x to 2.3.x+. They'd have to perform incremental upgrades via
2.2.x, by restoring first with 2.2.x and then taking another savepoint, before
upgrading to 2.3.x.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
