Want contributor permission

[hongton122]

Hi,


I want to contribute to Apache Flink.
Would you please give me the contributor permission?
My JIRA ID is XD122






Thx;




 

Re: [DISCUSS] Graceful Shutdown Handling by UDFs.

[Konstantin Knauf]

Hi Klou,

+1 for this proposal. I am missing any mention of "cancel" in the design
document though. In my understanding we are not planning to deprecate
"cancel" completely (just cancel-with-savepoint, which is superseded by
"stop"). In any case we should consider it in the design document. It seems
to me that "cancel" should be consider an ungraceful shutdown, so that the
Job could be restarted from last (retained) checkpoint (as right now).

Cheers,

Konstantin

On Thu, Jul 4, 2019 at 3:21 PM Kostas Kloudas  wrote:

> Hi all,
>
> In many cases, UDFs (User Defined Functions) need to be able to perform
> application-specific actions when they stop in an orderly manner.
> Currently, Flink's UDFs, and more specifically the RichFunction which
> exposes lifecycle-related hooks, only have a close() method which is called
> in any case of job termination. This includes any form of orderly
> termination (STOP or End-Of-Stream) and termination due to an error.
>
>
> The FLIP in [1] and the design document in [2] propose the addition of an
> interface that will allow UDFs that implement it to perform application
> specific logic in the case of graceful termination. These cases include
> DRAIN and SUSPEND for streaming jobs (see FLIP-34), but also reaching the
> End-Of-Stream for jobs with finite sources.
>
> Let's have a lively discussion to solve this issue that has been around for
> quite some time.
>
> Cheers,
> Kostas
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs
>
> [2]
>
> https://docs.google.com/document/d/1SXfhmeiJfWqi2ITYgCgAoSDUv5PNq1T8Zu01nR5Ebog/edit?usp=sharing
>


-- 

Konstantin Knauf | Solutions Architect

+49 160 91394525


Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2010


--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--

Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen

[jira] [Created] (FLINK-13133) Correct error in PubSub documentation

[Richard Deurwaarder (JIRA)]

Richard Deurwaarder created FLINK-13133:
---

 Summary: Correct error in PubSub documentation
 Key: FLINK-13133
 URL: https://issues.apache.org/jira/browse/FLINK-13133
 Project: Flink
  Issue Type: Sub-task
Affects Versions: 1.9.0
Reporter: Richard Deurwaarder
Assignee: Richard Deurwaarder
 Fix For: 1.9.0


In the documentation for PubsubSink 
([https://ci.apache.org/projects/flink/flink-docs-master/dev/connectors/pubsub.html#pubsub-sink])
 incorrectly uses *De*serialisationSchema, it should use a SerializationSchema



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[ANNOUNCE] Weekly Community Update 2019/27

[Konstantin Knauf]

Dear community,

this week's community digest with news on Flink 1.9.0 and Flink 1.8.1, our
Travis setup, Flink on PyPi, and a couple of new initiatives around the
DataStream API.

As always, please feel free to add additional updates and news to this
thread!

Flink Development
===

* [releases] The feature freeze for *Flink 1.9.0 *has been announced on
Friday. The release branch will only be cut next Thursday though to allow
for some finishing strokes mainly on the planned features for the Table
API. [1]

* [releases] *Flink 1.8.1* has been released. The release blog post [2]
contains a list of all resolved issues.

* [development process] Over the last weeks there has been an ongoing
discussion on how to deal with the lack of *resources available for Flink's
CI (Travis)*. This week Chesnay has started a voting thread to move away
from ASF's Travis account to a Travis account sponsored by Ververica. This
got a lot of approvals, but the actual implementation is not trivial as we
can not link an external Travis account to the apache/flink repository. The
approach currently poc'ed by Chensay involves us fetching every PR to a
ververica-controlled repository and publishing the results back to
apache/flink. [3]

* [python] Flink 1.9.0 will contain a first version of a *Python Table API*.
Consequently, Jincheng started a discussion to publish it to* PyPi*. The
name will simply be "apache-flink". Currently, the main question is whether
the python package should bundle the binary distribution of Apache Flink
(Java/Scala) it depends on. It seems more feedback is needed to move this
forward. [4]

* [datastream-api] With FLIP-34 the Flink community introduced a the "stop"
command for graceful shutdown of a Flink Job. UDFs currently only have a
close() method, which is called for any type of shutdown. Now, Klou started
a discussion about adding an *optional interface*, which methods would only
be called during *gracelful shutdown* scenarios (but e.g. not in the case
of failure). [5,6]

* [datastream-api] Xingcan has started a VOTE on how to proceed with
the *split/select
API.* Voting Time ended one ago. More opinions wouldn't hurt to move this
forward. [7]

* [state management] After an offline sync between Vino and Kurt the
discussion on "*local keyed state" *and* "local aggregations"* covered last
week will now first focus on the proposal of "local keyed state" and
"localKeyBy()" as this was the core of Vino's original proposal.  [8]

* [logging] An old discussion thread [9] by Vino on separating the logs
between Flink Jobs within the same cluster was revived a bit this week in
the context of a new ticket [10]. A lot of the (interesting) discussion has
moved to this ticket for now.

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Feature-freeze-for-Apache-Flink-1-9-0-release-td29751.html
[2] https://flink.apache.org/news/2019/07/02/release-1.8.1.html
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-solve-unstable-build-capacity-problem-on-TravisCI-tp29881.html
[4]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Publish-the-PyFlink-into-PyPI-td30095.html
[5]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Graceful-Shutdown-Handling-by-UDFs-td30229.html
[6]
https://docs.google.com/document/d/1SXfhmeiJfWqi2ITYgCgAoSDUv5PNq1T8Zu01nR5Ebog/edit
[7]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-How-to-Deal-with-Split-Select-in-DataStream-API-td30233.html
[8]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-Local-Aggregation-in-Flink-td29307.html
[9] https://issues.apache.org/jira/browse/FLINK-12953
[10]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-framework-and-user-log-separation-td27441.html

Notable Bugs
===

* [FLINK-13063] [1.8.1] [1.7.2] [1.6.4] In an operator chain containing an
AsyncOperator records can be lost or duplicated due to a bug in the
AsyncOperator. Until the mailbox model has been fully implemented, the fix
ensures that the AsyncOperator can only be the head of an operator. Fixed
in 1.7.3 and 1.8.2. [11]

* [FLINK-12889] [1.8.1] [1.7.2] In the case where a TM fails with an OOME
during checkpointing, its Tasks might not reach a proper failed state,
because the failure handling mechanism is affected by the OOME, too. Fixed
in 1.73 and 1.8.2. [12]

* [FLINK-13059] The CassandraSink might become deadlocked during shutdown,
when an exception happens during asynchronously sending requests to
Cassandra. A PR is available but needs a review. [13]

* [FLINK-12122] [1.6.4] [ 1.7.2] [ 1.8.1] Not a new bug, but a continuous
source of questions: Since FLIP-6 tasks are not spread out evenly among all
TaskManagers as before, which can result in very uneven loads if the number
of available TaskSlots exceeds the parallelism of the submitted job. Right
now it looks like a fix is targeted for 1.9.1 and 1.8.2 [14]

[11] ht

Apply for being a contributor

[????]

Hi, Apache


I want to contribute to Apache Flink.
Would you please give me the contributor permission?
My JIRA ID is WangHW.

[jira] [Created] (FLINK-13134) override default hadoop version from 2.4.1 to 2.7.5 in flink-connector-hive

[Bowen Li (JIRA)]

Bowen Li created FLINK-13134:


 Summary: override default hadoop version from 2.4.1 to 2.7.5 in 
flink-connector-hive
 Key: FLINK-13134
 URL: https://issues.apache.org/jira/browse/FLINK-13134
 Project: Flink
  Issue Type: Sub-task
  Components: Connectors / Hive
Reporter: Bowen Li
Assignee: Bowen Li
 Fix For: 1.9.0


Hive 2.3.4 relies on Hadoop 2.7.2 or later version. The default hadoop version 
in Flink globally is 2.4.1 which misses some classes in 2.7.2 and thus doesn't 
meet the requirement. Not sure why Travis CI succeeded though, maybe because 
the build machine has all versioned hadoops cached locally.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-13135) unify configs for meta-objects in HiveCatalog

[Bowen Li (JIRA)]

Bowen Li created FLINK-13135:


 Summary: unify configs for meta-objects in HiveCatalog
 Key: FLINK-13135
 URL: https://issues.apache.org/jira/browse/FLINK-13135
 Project: Flink
  Issue Type: Sub-task
  Components: Connectors / Hive
Reporter: Bowen Li
Assignee: Bowen Li
 Fix For: 1.9.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[RESULT][VOTE] Migrate to sponsored Travis account

[Chesnay Schepler]

The vote has passed unanimously in favor of migrating to a separate 
Travis account.


I will now set things up such that no PullRequest is no longer run on 
the ASF servers.

This is a major setup in reducing our usage of ASF resources.
For the time being we'll use free Travis plan for flink-ci (i.e. 5 
workers, which is the same the ASF gives us). Over the course of the 
next week we'll setup the Ververica subscription to increase this limit.


From now now, a bot will mirror all new and updated PullRequests to a 
mirror repository (https://github.com/flink-ci/flink-ci) and write an 
update into the PR once the build is complete.
I have ran the bots for the past 3 days in parallel to our existing 
Travis and it was working without major issues.


The biggest change that contributors will see is that there's no longer 
a icon next to each commit. We may revisit this in the future.


I'll setup a repo with the source of the bot later.

On 04/07/2019 10:46, Chesnay Schepler wrote:
I've raised a JIRA 
with INFRA to 
inquire whether it would be possible to switch to a different Travis 
account, and if so what steps would need to be taken.
We need a proper confirmation from INFRA since we are not in full 
control of the flink repository (for example, we cannot access the 
settings page).


If this is indeed possible, Ververica is willing sponsor a Travis 
account for the Flink project.

This would provide us with more than enough resources than we need.

Since this makes the project more reliant on resources provided by 
external companies I would like to vote on this.


Please vote on this proposal, as follows:
[ ] +1, Approve the migration to a Ververica-sponsored Travis account, 
provided that INFRA approves
[ ] -1, Do not approach the migration to a Ververica-sponsored Travis 
account


The vote will be open for at least 24h, and until we have confirmation 
from INFRA. The voting period may be shorter than the usual 3 days 
since our current is effectively not working.


On 04/07/2019 06:51, Bowen Li wrote:
Re: > Are they using their own Travis CI pool, or did the switch to 
an entirely different CI service?


I reached out to Wes and Krisztián from Apache Arrow PMC. They are 
currently moving away from ASF's Travis to their own in-house metal 
machines at [1] with custom CI application at [2]. They've seen 
significant improvement w.r.t both much higher performance and 
basically no resource waiting time, "night-and-day" difference 
quoting Wes.


Re: > If we can just switch to our own Travis pool, just for our 
project, then this might be something we can do fairly quickly?


I believe so, according to [3] and [4]


[1] https://ci.ursalabs.org/ 
[2] https://github.com/ursa-labs/ursabot
[3] 
https://docs.travis-ci.com/user/migrate/open-source-repository-migration

[4] https://docs.travis-ci.com/user/migrate/open-source-on-travis-ci-com



On Wed, Jul 3, 2019 at 12:01 AM Chesnay Schepler > wrote:


    Are they using their own Travis CI pool, or did the switch to an
    entirely different CI service?

    If we can just switch to our own Travis pool, just for our
    project, then
    this might be something we can do fairly quickly?

    On 03/07/2019 05:55, Bowen Li wrote:
    > I responded in the INFRA ticket [1] that I believe they are
    using a wrong
    > metric against Flink and the total build time is a completely
    different
    > thing than guaranteed build capacity.
    >
    > My response:
    >
    > "As mentioned above, since I started to pay attention to Flink's
    build
    > queue a few tens of days ago, I'm in Seattle and I saw no build
    was kicking
    > off in PST daytime in weekdays for Flink. Our teammates in China
    and Europe
    > have also reported similar observations. So we need to evaluate
    how the
    > large total build time came from - if 1) your number and 2) our
    > observations from three locations that cover pretty much a full
    day, are
    > all true, I **guess** one reason can be that - highly likely the
    extra
    > build time came from weekends when other Apache projects may be
    idle and
    > Flink just drains hard its congested queue.
    >
    > Please be aware of that we're not complaining about the lack of
    resources
    > in general, I'm complaining about the lack of **stable, 
dedicated**

    > resources. An example for the latter one is, currently even if
    no build is
    > in Flink's queue and I submit a request to be the queue head in 
PST

    > morning, my build won't even start in 6-8+h. That is an absurd
    amount of
    > waiting time.
    >
    > That's saying, if ASF INFRA decides to adopt a quota system and
    grants
    > Flink five DEDICATED servers that runs all the time only for
    Flink, that'll
    > be PERFECT and can totally solve our problem now.
    >
    > Please be aware of that we're not complaining

Apply for being a contributor

[????]

Hi, 


I want to contribute to Apache Flink.
Would you please give me the contributor permission?
My JIRA ID is WangHW, my fullname is WangHengWei.


Thanks very much.

[DISCUSS] ARM support for Flink

[Xiyuan Wang]

 Hi Flink:
  Recently we meet a problem that we have to test and run Flink on ARM
arch. While after searching Flink community, I didn’t find an official ARM
release version.

Since Flink is made by Java and Scala language which can be ran
cross-platform usually, I think Flink can be built and ran on ARM directly
as well. Then with my local test, Flink was built and deployed success as
expected. But some tests were failed due to ARM arch. For example:

1. MemoryArchitectureTest.testArchitectureNotUnknown:34 Values should be
different. Actual: UNKNOWN
2. [ERROR]
testIterator(org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest)
 Time elapsed: 0.234 s  <<< ERROR!
java.io.IOException: Could not load the native RocksDB library
at
org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest.testIteratorHelper(RocksDBRocksStateKeysIteratorTest.java:90)
at
org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest.testIterator(RocksDBRocksStateKeysIteratorTest.java:63)
Caused by: java.lang.UnsatisfiedLinkError:
/tmp/rocksdb-lib-81ca7930b92af2cca143a050c0338d34/librocksdbjni-linux64.so:
/tmp/rocksdb-lib-81ca7930b92af2cca143a050c0338d34/librocksdbjni-linux64.so:
cannot open shared object file: No such file or directory (Possible cause:
can't load AMD 64-bit .so on a AARCH64-bit platform)
…

  Since the test isn’t passed totally, we are not sure if Flink 100%
support ARM or not. Is it possible for Flink to support ARM release
officially? I guess it may be not a very huge work basing on Java. I notice
that Flink now uses trivis-ci which is X86 only for build & test check. Is
it possible to add an ARM arch CI as well? It can be non-voting first. Then
we can keep monitoring and fixing ARM related error. One day it’s stable
enough, we can remove the non-voting tag and create Flink ARM release.

  There is an open source CI community called OpenLab[1] which can provide
CI function and ARM resource to Flink by free. I’m one of the OpenLab
members. If Flink commun think ARM support is fine, I can keep helping
Flink to build and maintain the ARM CI job. There is an  POC for Flink ARM
build job made by me on OpenLab system[2] and a live demo which built and
run on an ARM VM[3]. You can take a look first.

Eager to get everybody’s feedback. Any question is welcome.

Thanks.

[1]: https://openlabtesting.org/
[2]: https://github.com/theopenlab/flink/pull/1
[3]: http://114.115.168.52:8081/#/overview

Re: [RESULT][VOTE] Migrate to sponsored Travis account

[Congxian Qiu]

As we used flink bot to trigger the CI test, could we add a command for
flink bot to retrigger the CI(sometimes we may encounter some flaky tests)

Best,
Congxian


Chesnay Schepler  于2019年7月8日周一 上午5:01写道：

> The vote has passed unanimously in favor of migrating to a separate
> Travis account.
>
> I will now set things up such that no PullRequest is no longer run on
> the ASF servers.
> This is a major setup in reducing our usage of ASF resources.
> For the time being we'll use free Travis plan for flink-ci (i.e. 5
> workers, which is the same the ASF gives us). Over the course of the
> next week we'll setup the Ververica subscription to increase this limit.
>
>  From now now, a bot will mirror all new and updated PullRequests to a
> mirror repository (https://github.com/flink-ci/flink-ci) and write an
> update into the PR once the build is complete.
> I have ran the bots for the past 3 days in parallel to our existing
> Travis and it was working without major issues.
>
> The biggest change that contributors will see is that there's no longer
> a icon next to each commit. We may revisit this in the future.
>
> I'll setup a repo with the source of the bot later.
>
> On 04/07/2019 10:46, Chesnay Schepler wrote:
> > I've raised a JIRA
> > with INFRA to
> > inquire whether it would be possible to switch to a different Travis
> > account, and if so what steps would need to be taken.
> > We need a proper confirmation from INFRA since we are not in full
> > control of the flink repository (for example, we cannot access the
> > settings page).
> >
> > If this is indeed possible, Ververica is willing sponsor a Travis
> > account for the Flink project.
> > This would provide us with more than enough resources than we need.
> >
> > Since this makes the project more reliant on resources provided by
> > external companies I would like to vote on this.
> >
> > Please vote on this proposal, as follows:
> > [ ] +1, Approve the migration to a Ververica-sponsored Travis account,
> > provided that INFRA approves
> > [ ] -1, Do not approach the migration to a Ververica-sponsored Travis
> > account
> >
> > The vote will be open for at least 24h, and until we have confirmation
> > from INFRA. The voting period may be shorter than the usual 3 days
> > since our current is effectively not working.
> >
> > On 04/07/2019 06:51, Bowen Li wrote:
> >> Re: > Are they using their own Travis CI pool, or did the switch to
> >> an entirely different CI service?
> >>
> >> I reached out to Wes and Krisztián from Apache Arrow PMC. They are
> >> currently moving away from ASF's Travis to their own in-house metal
> >> machines at [1] with custom CI application at [2]. They've seen
> >> significant improvement w.r.t both much higher performance and
> >> basically no resource waiting time, "night-and-day" difference
> >> quoting Wes.
> >>
> >> Re: > If we can just switch to our own Travis pool, just for our
> >> project, then this might be something we can do fairly quickly?
> >>
> >> I believe so, according to [3] and [4]
> >>
> >>
> >> [1] https://ci.ursalabs.org/ 
> >> [2] https://github.com/ursa-labs/ursabot
> >> [3]
> >>
> https://docs.travis-ci.com/user/migrate/open-source-repository-migration
> >> [4]
> https://docs.travis-ci.com/user/migrate/open-source-on-travis-ci-com
> >>
> >>
> >>
> >> On Wed, Jul 3, 2019 at 12:01 AM Chesnay Schepler  >> > wrote:
> >>
> >> Are they using their own Travis CI pool, or did the switch to an
> >> entirely different CI service?
> >>
> >> If we can just switch to our own Travis pool, just for our
> >> project, then
> >> this might be something we can do fairly quickly?
> >>
> >> On 03/07/2019 05:55, Bowen Li wrote:
> >> > I responded in the INFRA ticket [1] that I believe they are
> >> using a wrong
> >> > metric against Flink and the total build time is a completely
> >> different
> >> > thing than guaranteed build capacity.
> >> >
> >> > My response:
> >> >
> >> > "As mentioned above, since I started to pay attention to Flink's
> >> build
> >> > queue a few tens of days ago, I'm in Seattle and I saw no build
> >> was kicking
> >> > off in PST daytime in weekdays for Flink. Our teammates in China
> >> and Europe
> >> > have also reported similar observations. So we need to evaluate
> >> how the
> >> > large total build time came from - if 1) your number and 2) our
> >> > observations from three locations that cover pretty much a full
> >> day, are
> >> > all true, I **guess** one reason can be that - highly likely the
> >> extra
> >> > build time came from weekends when other Apache projects may be
> >> idle and
> >> > Flink just drains hard its congested queue.
> >> >
> >> > Please be aware of that we're not complaining about the lack of
> >> resources
> >> > in genera

Re: [DISCUSS] ARM support for Flink

[Dian Fu]

Hi Xiyuan,

Thanks for bring the discussion. 

WRT the exception, it's because the native bundled in the rocksdb jar file 
isn't compiled with cross platform support. You can refer [1] for how to build 
rocksdb which has ARM platform.

WRT ARM support, the rocksdb currently used in Flink is hosted in the Ververica 
git [2], so it won't be difficult to make it support ARM. However, I guess this 
git exists just for temporary [3], not because we want to add much feature in 
rocksdb.

[1] https://github.com/facebook/rocksdb/issues/678 

[2] https://github.com/dataArtisans/frocksdb 

[3] https://issues.apache.org/jira/browse/FLINK-10471 


Regards,
Dian

> 在 2019年7月8日，上午9:17，Xiyuan Wang  写道：
> 
> Hi Flink:
>  Recently we meet a problem that we have to test and run Flink on ARM
> arch. While after searching Flink community, I didn’t find an official ARM
> release version.
> 
> Since Flink is made by Java and Scala language which can be ran
> cross-platform usually, I think Flink can be built and ran on ARM directly
> as well. Then with my local test, Flink was built and deployed success as
> expected. But some tests were failed due to ARM arch. For example:
> 
> 1. MemoryArchitectureTest.testArchitectureNotUnknown:34 Values should be
> different. Actual: UNKNOWN
> 2. [ERROR]
> testIterator(org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest)
> Time elapsed: 0.234 s  <<< ERROR!
> java.io.IOException: Could not load the native RocksDB library
> at
> org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest.testIteratorHelper(RocksDBRocksStateKeysIteratorTest.java:90)
> at
> org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest.testIterator(RocksDBRocksStateKeysIteratorTest.java:63)
> Caused by: java.lang.UnsatisfiedLinkError:
> /tmp/rocksdb-lib-81ca7930b92af2cca143a050c0338d34/librocksdbjni-linux64.so:
> /tmp/rocksdb-lib-81ca7930b92af2cca143a050c0338d34/librocksdbjni-linux64.so:
> cannot open shared object file: No such file or directory (Possible cause:
> can't load AMD 64-bit .so on a AARCH64-bit platform)
> …
> 
>  Since the test isn’t passed totally, we are not sure if Flink 100%
> support ARM or not. Is it possible for Flink to support ARM release
> officially? I guess it may be not a very huge work basing on Java. I notice
> that Flink now uses trivis-ci which is X86 only for build & test check. Is
> it possible to add an ARM arch CI as well? It can be non-voting first. Then
> we can keep monitoring and fixing ARM related error. One day it’s stable
> enough, we can remove the non-voting tag and create Flink ARM release.
> 
>  There is an open source CI community called OpenLab[1] which can provide
> CI function and ARM resource to Flink by free. I’m one of the OpenLab
> members. If Flink commun think ARM support is fine, I can keep helping
> Flink to build and maintain the ARM CI job. There is an  POC for Flink ARM
> build job made by me on OpenLab system[2] and a live demo which built and
> run on an ARM VM[3]. You can take a look first.
> 
> Eager to get everybody’s feedback. Any question is welcome.
> 
> Thanks.
> 
> [1]: https://openlabtesting.org/
> [2]: https://github.com/theopenlab/flink/pull/1
> [3]: http://114.115.168.52:8081/#/overview

Re: Apply for being a contributor

[jincheng sun]

Hi WangHW,

Welcome to the Flink community!
I gave you contributor permissions for Jira.

Best,
Jincheng

疯琴 <35023...@qq.com> 于2019年7月8日周一 上午9:17写道：

> Hi,
>
>
> I want to contribute to Apache Flink.
> Would you please give me the contributor permission?
> My JIRA ID is WangHW, my fullname is WangHengWei.
>
>
> Thanks very much.

[jira] [Created] (FLINK-13136) Fix documentation error about stop job with restful api

[vinoyang (JIRA)]

vinoyang created FLINK-13136:


 Summary: Fix documentation error about stop job with restful api
 Key: FLINK-13136
 URL: https://issues.apache.org/jira/browse/FLINK-13136
 Project: Flink
  Issue Type: Improvement
  Components: Runtime / REST
Affects Versions: 1.9.0
Reporter: vinoyang
Assignee: vinoyang


Currently, 
[https://ci.apache.org/projects/flink/flink-docs-master/monitoring/rest_api.html#jobs-jobid-1]
 does not support "stop" mode. If users use stop mode will throw an exception:
{code:java}
The "stop" command has been removed. Please use "stop-with-savepoint" instead.
{code}
IMO, We should remove it.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [VOTE] How to Deal with Split/Select in DataStream API

[Xingcan Cui]

Hi all,

Thanks for your participation.

In this thread, we got one +1 for option 1 and option 3, respectively. In the 
original thread[1], we got two +1 for option 1, one +1 for option 2, and five 
+1 and one -1 for option 3.

To summarize,

Option 1 (port side output to flatMap and deprecate split/select): three +1
Option 2 (introduce a new split/select and deprecate existing one): one +1
Option 3 ("correct" the existing split/select): six +1 and one -1

It seems that most people involved are in favor of "correcting" the existing 
split/select. However, this will definitely break the API compatibility, in a 
subtle way.

IMO, the real behavior of consecutive split/select's has never been thoroughly 
clarified. Even in the community, it hard to say that we come into a consensus 
on its real semantics[2-4]. Though the initial design is not ambiguous, there's 
no doubt that its concept has drifted. 

As the split/select is quite an ancient API, I cc'ed this to more members. It 
couldn't be better if you can share your opinions on this.

Thanks,
Xingcan

[1] 
https://lists.apache.org/thread.html/f94ea5c97f96c705527dcc809b0e2b69e87a4c5d400cb7c61859e1f4@%3Cdev.flink.apache.org%3E
[2] https://issues.apache.org/jira/browse/FLINK-1772
[3] https://issues.apache.org/jira/browse/FLINK-5031
[4] https://issues.apache.org/jira/browse/FLINK-11084


> On Jul 5, 2019, at 12:04 AM, 杨力  wrote:
> 
> I prefer the 1) approach. I used to carry fields, which is needed only for 
> splitting, in the outputs of flatMap functions. Replacing it with outputTags 
> would simplify data structures.
> 
> Xingcan Cui mailto:xingc...@gmail.com>> 于 2019年7月5日周五 
> 上午2:20写道：
> Hi folks,
> 
> Two weeks ago, I started a thread [1] discussing whether we should discard 
> the split/select methods (which have been marked as deprecation since v1.7) 
> in DataStream API. 
> 
> The fact is, these methods will cause "unexpected" results when using 
> consecutively (e.g., ds.split(a).select(b).split(c).select(d)) or multi-times 
> on the same target (e.g., ds.split(a).select(b), ds.split(c).select(d)). The 
> reason is that following the initial design, the new split/select logic will 
> always override the existing one on the same target operator, rather than 
> append to it. Some users may not be aware of that, but if you do, a current 
> solution would be to use the more powerful side output feature [2].
> 
> FLINK-11084  added some 
> restrictions to the existing split/select logic and suggest to replace it 
> with side output in the future. However, considering that the side output is 
> currently only available in the process function layer and the split/select 
> could have been widely used in many real-world applications, we'd like to 
> start a vote andlisten to the community on how to deal with them.
> 
> In the discussion thread [1], we proposed three solutions as follows. All of 
> them are feasible but have different impacts on the public API.
> 
> 1) Port the side output feature to DataStream API's flatMap and replace 
> split/select with it.
> 
> 2) Introduce a dedicated function in DataStream API (with the "correct" 
> behavior but a different name) that can be used to replace the existing 
> split/select.
> 
> 3) Keep split/select but change the behavior/semantic to be "correct".
> 
> Note that this is just a vote for gathering information, so feel free to 
> participate and share your opinions.
> 
> The voting time will end on July 7th 17:00 EDT.
> 
> Thanks,
> Xingcan
> 
> [1] 
> https://lists.apache.org/thread.html/f94ea5c97f96c705527dcc809b0e2b69e87a4c5d400cb7c61859e1f4@%3Cdev.flink.apache.org%3E
>  
> 
> [2] 
> https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/side_output.html
>  
> 

[jira] [Created] (FLINK-13137) Remove the legacy flink python docs

[Huang Xingbo (JIRA)]

Huang Xingbo created FLINK-13137:


 Summary: Remove the legacy flink python docs
 Key: FLINK-13137
 URL: https://issues.apache.org/jira/browse/FLINK-13137
 Project: Flink
  Issue Type: Bug
  Components: API / Python
Reporter: Huang Xingbo


 Batch (DataSet API)/Transformations and Batch (DataSet API)/Zipping Elements 
have legacy flink python related documentation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-13138) Make SqlCommandParser.SqlCommand as public

[Jeff Zhang (JIRA)]

Jeff Zhang created FLINK-13138:
--

 Summary: Make SqlCommandParser.SqlCommand as public
 Key: FLINK-13138
 URL: https://issues.apache.org/jira/browse/FLINK-13138
 Project: Flink
  Issue Type: Improvement
  Components: Table SQL / Client
Affects Versions: 1.9.0
Reporter: Jeff Zhang


So that downstream project can reuse the sql parse code in sql-client. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)