[jira] [Created] (FLINK-16593) Misspelled words in GlobalAggregateManager.java
forideal created FLINK-16593: Summary: Misspelled words in GlobalAggregateManager.java Key: FLINK-16593 URL: https://issues.apache.org/jira/browse/FLINK-16593 Project: Flink Issue Type: Bug Components: Runtime / Task Affects Versions: 1.11.0 Reporter: forideal -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16594) Misspelled words in GlobalAggregateManager.java
forideal created FLINK-16594: Summary: Misspelled words in GlobalAggregateManager.java Key: FLINK-16594 URL: https://issues.apache.org/jira/browse/FLINK-16594 Project: Flink Issue Type: Bug Components: Runtime / Task Affects Versions: 1.11.0 Reporter: forideal -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16595) Support extra hadoop filesystem URLs for which to request delegation tokens
fa zheng created FLINK-16595: Summary: Support extra hadoop filesystem URLs for which to request delegation tokens Key: FLINK-16595 URL: https://issues.apache.org/jira/browse/FLINK-16595 Project: Flink Issue Type: Improvement Components: Deployment / YARN Affects Versions: 1.10.0 Reporter: fa zheng Fix For: 1.11.0 When the cluster has multiple nameservices, the client only can obtain the tokens of default nameservice. We should add an configuration in YarnConfigOptions to obtain all nameservices refer to spark. [yarn-specific-kerberos-configuration|https://spark.apache.org/docs/latest/running-on-yarn.html#yarn-specific-kerberos-configuration]. I encountered this problem when the directory of yarn logs is in another nameservice. It will lead a long waiting for application to run and the log can not be aggregated finally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16596) Support Enum-Values as part of a Key
Felix Wollschläger created FLINK-16596: -- Summary: Support Enum-Values as part of a Key Key: FLINK-16596 URL: https://issues.apache.org/jira/browse/FLINK-16596 Project: Flink Issue Type: Improvement Reporter: Felix Wollschläger See: FLINK-11774 h2. Description: The hashCode implementation of Enum-Values is guaranteed to be stable inside a JVM, but not accross multiple JVMs. This leads to failures when restoring from a checkpoint/savepoint containing Keyed-State on Keys with Enums as a part of the key. For users of Flink there is a workaround to solve this problem: Don't rely on the hashCode-Implementation of Enum-Values in the hashCode-Implementation of the actual key. Use the ordinal() or name().hashCode() instead of the Enum hashCode. h2. Goals of this improvement: Implement a way to handle Enum-Values, by either handling Enum-Values on a internal level or implementing a abstract Base KeySelector class that users of flink can choose if their key contains a Enum-Value. Code to reproduce a failure and first thoughts can be found in FLINK-11774 . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16597) tryCleanupOnError should happen before close
Jiayi Liao created FLINK-16597:
--
Summary: tryCleanupOnError should happen before close
Key: FLINK-16597
URL: https://issues.apache.org/jira/browse/FLINK-16597
Project: Flink
Issue Type: Improvement
Components: Runtime / Task
Affects Versions: 1.10.0
Reporter: Jiayi Liao
If {{DataSinkTask}} fails or throws an exception, the {{OutputFormat}} will
call {{tryCleanupOnError}} before {{close}}. But the calling order is reverse
when {{DataSinkTask}} is cancelled, which doesn't make much sense that we can
still clean the output format after it's closed.
I'm not very sure if this is a bug. But this does mislead our developers when
implementing the {{OutputFormat}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: How to change the flink web-ui jobServer?
Ok, thanks! Arvid Arvid Heise 于2020年3月10日周二 下午4:14写道： > Hi LakeShen, > > you can change the port with > > conf.setInteger(RestOptions.PORT, 8082); > > or if want to be on the safe side specify a range > > conf.setString(RestOptions.BIND_PORT, "8081-8099"); > > > On Mon, Mar 9, 2020 at 10:47 AM LakeShen > wrote: > >> Hi community, >>now I am moving the flink job to k8s,and I plan to use the ingress >> to show the flink web ui , the problem is that fink job server aren't >> correct, so I want to change the flink web-ui jobserver ,I don't find the >> any method to change it ,are there some method to do that? >>Thanks to your reply. >> >> Best wishes, >> LakeShen >> >
[jira] [Created] (FLINK-16598) Respect the rest port exposed by Service in Fabric8FlinkKubeClient#getRestEndpoint
Canbin Zheng created FLINK-16598: Summary: Respect the rest port exposed by Service in Fabric8FlinkKubeClient#getRestEndpoint Key: FLINK-16598 URL: https://issues.apache.org/jira/browse/FLINK-16598 Project: Flink Issue Type: Bug Components: Deployment / Kubernetes Affects Versions: 1.10.0 Reporter: Canbin Zheng Fix For: 1.11.0 Steps to reproduce the problem: # Start a Kubernetes session cluster, by default, the rest port exposed by the rest Service is 8081. # Submit a job to the session cluster, but specify a different rest port via -Drest.port=8082 As we can see, the job could never be submitted to the existing session cluster since we retrieve the Endpoint from the Flink configuration object instead of the created Service. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: How to change the flink web-ui jobServer?
IIRC Flink on Kubernetes doesn't support configure rest port as port range. Maybe Yang(in cc) can give more information and if so, our current logic only take care of RestOptions.PORT but not RestOptions.BIND_PORT, which will be a bug. Best, tison. LakeShen 于2020年3月15日周日 上午11:25写道： > Ok, thanks! Arvid > > Arvid Heise 于2020年3月10日周二 下午4:14写道： > >> Hi LakeShen, >> >> you can change the port with >> >> conf.setInteger(RestOptions.PORT, 8082); >> >> or if want to be on the safe side specify a range >> >> conf.setString(RestOptions.BIND_PORT, "8081-8099"); >> >> >> On Mon, Mar 9, 2020 at 10:47 AM LakeShen >> wrote: >> >>> Hi community, >>>now I am moving the flink job to k8s,and I plan to use the >>> ingress to show the flink web ui , the problem is that fink job server >>> aren't correct, so I want to change the flink web-ui jobserver ,I don't >>> find the any method to change it ,are there some method to do that? >>>Thanks to your reply. >>> >>> Best wishes, >>> LakeShen >>> >>
[jira] [Created] (FLINK-16599) Throw Unknown identifier when having nested computed column
YufeiLiu created FLINK-16599:
Summary: Throw Unknown identifier when having nested computed
column
Key: FLINK-16599
URL: https://issues.apache.org/jira/browse/FLINK-16599
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Affects Versions: 1.10.0
Reporter: YufeiLiu
{code:sql}
CREATE TABLE source {
origin STRING,
col1 AS FUN_1(origin),
col2 AS FUN_2(col1),
col3 AS FUN_3(col1)
}
{code}
Throws {{Unknown identifier 'col1' }}.
I konw maybe is not a bug, because of computed column implementation logic. But
I think it's a little bit inconvenient case I have to write like this.
{code:sql}
CREATE TABLE source {
origin STRING,
col2 AS FUN_2(FUN_1(origin)),
col3 AS FUN_3(FUN_1(origin))
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-16600) Respect the rest.bind-port for the Kubernetes setup
Canbin Zheng created FLINK-16600:
Summary: Respect the rest.bind-port for the Kubernetes setup
Key: FLINK-16600
URL: https://issues.apache.org/jira/browse/FLINK-16600
Project: Flink
Issue Type: Bug
Components: Deployment / Kubernetes
Affects Versions: 1.10.0
Reporter: Canbin Zheng
Fix For: 1.11.0
Our current logic only takes care of {{RestOptions.PORT}} but not
{{RestOptions.BIND_PORT, }}which is a bug.
For example, when one sets the {{RestOptions.BIND_PORT}} to a value other than
{{RestOptions.PORT}}, jobs could not be submitted to the existing session
cluster deployed via the kubernetes-session.sh.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-16601) Corret the way to get Endpoint address for NodePort rest Service
Canbin Zheng created FLINK-16601:
Summary: Corret the way to get Endpoint address for NodePort rest
Service
Key: FLINK-16601
URL: https://issues.apache.org/jira/browse/FLINK-16601
Project: Flink
Issue Type: Bug
Components: Deployment / Kubernetes
Affects Versions: 1.10.0
Reporter: Canbin Zheng
Fix For: 1.11.0
Currently, if one sets the type of the rest-service to {{NodePort}}, then the
way to get the Endpoint address is by calling the method of
'KubernetesClient.getMasterUrl().getHost()'. This solution works fine for the
case of the non-managed Kubernetes cluster but not for the managed ones.
For the managed Kubernetes cluster setups, the Kubernetes masters are deployed
in a pool different from the Kubernetes nodes and do not expose NodePort for a
NodePort Service.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
