[jira] [Created] (FLINK-13334) Remove legacy implementation of slot sharing

[TisonKun (JIRA)]

TisonKun created FLINK-13334:


 Summary: Remove legacy implementation of slot sharing
 Key: FLINK-13334
 URL: https://issues.apache.org/jira/browse/FLINK-13334
 Project: Flink
  Issue Type: Sub-task
  Components: Runtime / Coordination
Affects Versions: 1.10.0
Reporter: TisonKun
 Fix For: 1.10.0


cc [~till.rohrmann] [~srichter]

>From my investigation currently Flink use {{SlotSharingManager}} and 
>{{SlotSharingGroupId}} for achieving slot sharing. And thus 
>{{SlotSharingGroupAssignment}} {{SlotSharingGroup}} and {{SharedSlot}} are all 
>legacy concept.

Notice that the ongoing scheduler re-design touches frequently tests based on 
legacy slot/instance logic or even uses it for testing. I'd like to nudge this 
process for totally remove legacy code from our code base.

Also I attach a patch on FLINK-12179 that remove {{Instance}}. With current 
contribution workflow your shepherds are significant :- ) 




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Created] (FLINK-13335) Align the SQL DDL with FLIP-37

[Timo Walther (JIRA)]

Timo Walther created FLINK-13335:


 Summary: Align the SQL DDL with FLIP-37
 Key: FLINK-13335
 URL: https://issues.apache.org/jira/browse/FLINK-13335
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / API
Reporter: Timo Walther
Assignee: Timo Walther


At a first glance it does not seem that the newly introduced DDL is compliant 
with FLIP-37. We should ensure consistent behavior esp. also for corner cases.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Re: [DISCUSS] Drop stale class Program

[Biao Liu]

Hi Zili,

Thank you for bring us this discussion.

My gut feeling is +1 for dropping it.
Usually it costs some time to deprecate a public (actually it's
`PublicEvolving`) API. Ideally it should be marked as `Deprecated` first.
Then it might be abandoned it in some later version.

I'm not sure how big the burden is to make it compatible with the enhanced
client API. If it's a critical blocker, I support dropping it radically in
1.10. Of course a survey is necessary. And the result of survey is
acceptable.



Zili Chen  于2019年7月19日周五 下午1:44写道：

> Hi devs,
>
> Participating the thread "Flink client api enhancement"[1], I just notice
> that inside submission codepath of Flink we always has a branch dealing
> with the case that main class of user program is a subclass of
> o.a.f.api.common.Program, which is defined as
>
> @PublicEvolving
> public interface Program {
>   Plan getPhan(String... args);
> }
>
> This class, as user-facing interface, asks user to implement #getPlan
> which return a almost Flink internal class. FLINK-10862[2] pointed out
> this confusion.
>
> Since our codebase contains quite a bit code handling this stale class,
> and also it obstructs the effort enhancing Flink cilent api,
> I'd like to propose dropping it. Or we can start a survey on user list
> to see if there is any user depending on this class.
>
> best,
> tison.
>
> [1]
>
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> [2] https://issues.apache.org/jira/browse/FLINK-10862
>

Re: flink-mapr-fs failed in travis

[Chesnay Schepler]

I did modify the .travis.yml do activate the unsafe-mapr-repo profile; 
did I modified the wrong profile?...



On 19/07/2019 07:57, Jark Wu wrote:

It seems that it is introduced by this commit:
https://github.com/apache/flink/commit/5c36c650e6520d92191ce2da33f7dcae774319f6
Hi @Chesnay Schepler  , do we need to add
"-Punsafe-mapr-repo" to the ".travis.yml"?

Best,
Jark

On Fri, 19 Jul 2019 at 10:58, JingsongLee 
wrote:


Hi everyone:

flink-mapr-fs failed in travis, and I retried many times, and also failed.
Anyone has idea about this?

01:32:54.755 [ERROR] Failed to execute goal on project flink-mapr-fs:
Could not resolve dependencies for project
org.apache.flink:flink-mapr-fs:jar:1.10-SNAPSHOT: Failed to collect
dependencies at com.mapr.hadoop:maprfs:jar:5.2.1-mapr: Failed to read
artifact descriptor for com.mapr.hadoop:maprfs:jar:5.2.1-mapr: Could not
transfer artifact com.mapr.hadoop:maprfs:pom:5.2.1-mapr from/to
mapr-releases (https://repository.mapr.com/maven/):
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target -> [Help 1]

https://api.travis-ci.org/v3/job/560790299/log.txt

Best, Jingsong Lee

Re: flink-mapr-fs failed in travis

[Chesnay Schepler]

Ah, I added it to the common options in the travis_manv_watchdog.sh .

On 19/07/2019 09:58, Chesnay Schepler wrote:
I did modify the .travis.yml do activate the unsafe-mapr-repo profile; 
did I modified the wrong profile?...



On 19/07/2019 07:57, Jark Wu wrote:

It seems that it is introduced by this commit:
https://github.com/apache/flink/commit/5c36c650e6520d92191ce2da33f7dcae774319f6 


Hi @Chesnay Schepler  , do we need to add
"-Punsafe-mapr-repo" to the ".travis.yml"?

Best,
Jark

On Fri, 19 Jul 2019 at 10:58, JingsongLee 


wrote:


Hi everyone:

flink-mapr-fs failed in travis, and I retried many times, and also 
failed.

Anyone has idea about this?

01:32:54.755 [ERROR] Failed to execute goal on project flink-mapr-fs:
Could not resolve dependencies for project
org.apache.flink:flink-mapr-fs:jar:1.10-SNAPSHOT: Failed to collect
dependencies at com.mapr.hadoop:maprfs:jar:5.2.1-mapr: Failed to read
artifact descriptor for com.mapr.hadoop:maprfs:jar:5.2.1-mapr: Could 
not

transfer artifact com.mapr.hadoop:maprfs:pom:5.2.1-mapr from/to
mapr-releases (https://repository.mapr.com/maven/):
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable 
to find

valid certification path to requested target -> [Help 1]

https://api.travis-ci.org/v3/job/560790299/log.txt

Best, Jingsong Lee

[jira] [Created] (FLINK-13336) Remove the legacy batch fault tolerance page and redirect it to the new task failure recovery page

[Zhu Zhu (JIRA)]

Zhu Zhu created FLINK-13336:
---

 Summary: Remove the legacy batch fault tolerance page and redirect 
it to the new task failure recovery page
 Key: FLINK-13336
 URL: https://issues.apache.org/jira/browse/FLINK-13336
 Project: Flink
  Issue Type: Task
  Components: Documentation
Affects Versions: 1.9.0
Reporter: Zhu Zhu


The [batch fault tolerance 
page|[https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/batch/fault_tolerance.html]]
 is describing a deprecated way to configure restart strategies. We should 
remove it and redirect it to the [task failure recovery 
page|[https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/restart_strategies.html]]
 for latest description of the fault tolerance configurations, including 
restart strategies and failover strategies.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Re: [DISCUSS] Drop stale class Program

[Flavio Pompermaier]

In my experience a basic "official" (but optional) program description
would be very useful indeed (in order to ease the integration with other
frameworks).

Of course it should be extended and integrated with the REST services and
the Web UI (when defined) in order to be useful..
It ease to show to the user what a job does and which parameters it
requires (optional or mandatory) and with a proper help description.
Indeed, when we write a Flink job we implement the following interface:

public interface FlinkJob {
  String getDescription();
  List getParameters();
 boolean isStreamingOrBatch();
}

public class ClusterJobParameter {

  private String paramName;
  private String paramType = "string";
  private String paramDesc;
  private String paramDefaultValue;
  private Set choices;
  private boolean mandatory;
}

This really helps to launch a Flink job by a frontend (if the rest services
returns back those infos).

Best,
Flavio

On Fri, Jul 19, 2019 at 9:57 AM Biao Liu  wrote:

> Hi Zili,
>
> Thank you for bring us this discussion.
>
> My gut feeling is +1 for dropping it.
> Usually it costs some time to deprecate a public (actually it's
> `PublicEvolving`) API. Ideally it should be marked as `Deprecated` first.
> Then it might be abandoned it in some later version.
>
> I'm not sure how big the burden is to make it compatible with the enhanced
> client API. If it's a critical blocker, I support dropping it radically in
> 1.10. Of course a survey is necessary. And the result of survey is
> acceptable.
>
>
>
> Zili Chen  于2019年7月19日周五 下午1:44写道：
>
> > Hi devs,
> >
> > Participating the thread "Flink client api enhancement"[1], I just notice
> > that inside submission codepath of Flink we always has a branch dealing
> > with the case that main class of user program is a subclass of
> > o.a.f.api.common.Program, which is defined as
> >
> > @PublicEvolving
> > public interface Program {
> >   Plan getPhan(String... args);
> > }
> >
> > This class, as user-facing interface, asks user to implement #getPlan
> > which return a almost Flink internal class. FLINK-10862[2] pointed out
> > this confusion.
> >
> > Since our codebase contains quite a bit code handling this stale class,
> > and also it obstructs the effort enhancing Flink cilent api,
> > I'd like to propose dropping it. Or we can start a survey on user list
> > to see if there is any user depending on this class.
> >
> > best,
> > tison.
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > [2] https://issues.apache.org/jira/browse/FLINK-10862
> >
>

[jira] [Created] (FLINK-13337) Do not need to backup and restore streamEnv config in BatchExecutor

[XuPingyong (JIRA)]

XuPingyong created FLINK-13337:
--

 Summary: Do not need to backup and restore streamEnv config in 
BatchExecutor
 Key: FLINK-13337
 URL: https://issues.apache.org/jira/browse/FLINK-13337
 Project: Flink
  Issue Type: Task
  Components: Table SQL / Planner
Affects Versions: 1.9.0, 1.10.0
Reporter: XuPingyong
 Fix For: 1.9.0, 1.10.0






--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Re: flink-mapr-fs failed in travis

[Chesnay Schepler]

I think I found the issue; I forgot to update travis_controller.sh .

On 19/07/2019 10:02, Chesnay Schepler wrote:

Ah, I added it to the common options in the travis_manv_watchdog.sh .

On 19/07/2019 09:58, Chesnay Schepler wrote:
I did modify the .travis.yml do activate the unsafe-mapr-repo 
profile; did I modified the wrong profile?...



On 19/07/2019 07:57, Jark Wu wrote:

It seems that it is introduced by this commit:
https://github.com/apache/flink/commit/5c36c650e6520d92191ce2da33f7dcae774319f6 


Hi @Chesnay Schepler  , do we need to add
"-Punsafe-mapr-repo" to the ".travis.yml"?

Best,
Jark

On Fri, 19 Jul 2019 at 10:58, JingsongLee 


wrote:


Hi everyone:

flink-mapr-fs failed in travis, and I retried many times, and also 
failed.

Anyone has idea about this?

01:32:54.755 [ERROR] Failed to execute goal on project flink-mapr-fs:
Could not resolve dependencies for project
org.apache.flink:flink-mapr-fs:jar:1.10-SNAPSHOT: Failed to collect
dependencies at com.mapr.hadoop:maprfs:jar:5.2.1-mapr: Failed to read
artifact descriptor for com.mapr.hadoop:maprfs:jar:5.2.1-mapr: 
Could not

transfer artifact com.mapr.hadoop:maprfs:pom:5.2.1-mapr from/to
mapr-releases (https://repository.mapr.com/maven/):
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable 
to find

valid certification path to requested target -> [Help 1]

https://api.travis-ci.org/v3/job/560790299/log.txt

Best, Jingsong Lee

[NOTICE] SSL issue when building flink-mapr-fs

[Chesnay Schepler]

Hello,

the Flink PMC was a while ago informed about a security risk in our 
build process, as we were accessing various maven repositories without 
HTTPS. This issue was resolved in FLINK-12578 for 1.7 on-wards.


However, there is an ongoing issue with the MapR repository where you 
may run into an SSLException when building the flink-mapr-fs module. 
MapR has been made aware of this issue, but we're still waiting for a 
solution.


If you run into this you can use the "unsafe-mapr-repo" profile to 
revert to the previous behavior. Please also comment either here or in 
FLINK-12578 so that we can gauge how many devs are affected; in case 
this is wide-spread we may have to look at other alternatives (like 
excluding it from the build process by default).


Regards,

Chesnay

Re: [DISCUSS] Drop stale class Program

[Zili Chen]

Hi Biao,

Thanks for your reply.

For the "burden" part, inside PackagedProgram and ClusterClient
we currently contains branches handling whether the mainClass of
user job jar is subclass of Program. Any effort under client api
enhancement should be compatible with such codepaths unless we
drop it.

The class Program is only active in batch codepath, while the
so-called interactive mode, i.e., executing the main method of
user's main class, is using widely and valid in both batch and
streaming codepath.

Previously we have drop flink-storm directly and I'm afraid this
Program class is much less known and used than flink-storm.
Keeping compatible with such class is an unnecessary overhead.

However, the enhancement of client api is still under discussion
so it would be ok to deprecate at first and propose removal when
necessary.

Either drop it or deprecate it prevents users from being confused
how to use this stale class. But given the community has dropped
stale flink-storm, flink-libarary/python|ml, I'd like to propose
drop this class if in the survey we receiving no usage report.

Best,
tison.


Flavio Pompermaier  于2019年7月19日周五 下午4:21写道：

> In my experience a basic "official" (but optional) program description
> would be very useful indeed (in order to ease the integration with other
> frameworks).
>
> Of course it should be extended and integrated with the REST services and
> the Web UI (when defined) in order to be useful..
> It ease to show to the user what a job does and which parameters it
> requires (optional or mandatory) and with a proper help description.
> Indeed, when we write a Flink job we implement the following interface:
>
> public interface FlinkJob {
>   String getDescription();
>   List getParameters();
>  boolean isStreamingOrBatch();
> }
>
> public class ClusterJobParameter {
>
>   private String paramName;
>   private String paramType = "string";
>   private String paramDesc;
>   private String paramDefaultValue;
>   private Set choices;
>   private boolean mandatory;
> }
>
> This really helps to launch a Flink job by a frontend (if the rest services
> returns back those infos).
>
> Best,
> Flavio
>
> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu  wrote:
>
> > Hi Zili,
> >
> > Thank you for bring us this discussion.
> >
> > My gut feeling is +1 for dropping it.
> > Usually it costs some time to deprecate a public (actually it's
> > `PublicEvolving`) API. Ideally it should be marked as `Deprecated` first.
> > Then it might be abandoned it in some later version.
> >
> > I'm not sure how big the burden is to make it compatible with the
> enhanced
> > client API. If it's a critical blocker, I support dropping it radically
> in
> > 1.10. Of course a survey is necessary. And the result of survey is
> > acceptable.
> >
> >
> >
> > Zili Chen  于2019年7月19日周五 下午1:44写道：
> >
> > > Hi devs,
> > >
> > > Participating the thread "Flink client api enhancement"[1], I just
> notice
> > > that inside submission codepath of Flink we always has a branch dealing
> > > with the case that main class of user program is a subclass of
> > > o.a.f.api.common.Program, which is defined as
> > >
> > > @PublicEvolving
> > > public interface Program {
> > >   Plan getPhan(String... args);
> > > }
> > >
> > > This class, as user-facing interface, asks user to implement #getPlan
> > > which return a almost Flink internal class. FLINK-10862[2] pointed out
> > > this confusion.
> > >
> > > Since our codebase contains quite a bit code handling this stale class,
> > > and also it obstructs the effort enhancing Flink cilent api,
> > > I'd like to propose dropping it. Or we can start a survey on user list
> > > to see if there is any user depending on this class.
> > >
> > > best,
> > > tison.
> > >
> > > [1]
> > >
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> > >
> >
>

Re: [DISCUSS] Drop stale class Program

[Biao Liu]

To Flavio, good point for the integration suggestion.

I think it should be considered in the "Flink client api enhancement"
discussion. But the outdated API should be deprecated somehow.

Flavio Pompermaier  于2019年7月19日周五 下午4:21写道：

> In my experience a basic "official" (but optional) program description
> would be very useful indeed (in order to ease the integration with other
> frameworks).
>
> Of course it should be extended and integrated with the REST services and
> the Web UI (when defined) in order to be useful..
> It ease to show to the user what a job does and which parameters it
> requires (optional or mandatory) and with a proper help description.
> Indeed, when we write a Flink job we implement the following interface:
>
> public interface FlinkJob {
>   String getDescription();
>   List getParameters();
>  boolean isStreamingOrBatch();
> }
>
> public class ClusterJobParameter {
>
>   private String paramName;
>   private String paramType = "string";
>   private String paramDesc;
>   private String paramDefaultValue;
>   private Set choices;
>   private boolean mandatory;
> }
>
> This really helps to launch a Flink job by a frontend (if the rest services
> returns back those infos).
>
> Best,
> Flavio
>
> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu  wrote:
>
> > Hi Zili,
> >
> > Thank you for bring us this discussion.
> >
> > My gut feeling is +1 for dropping it.
> > Usually it costs some time to deprecate a public (actually it's
> > `PublicEvolving`) API. Ideally it should be marked as `Deprecated` first.
> > Then it might be abandoned it in some later version.
> >
> > I'm not sure how big the burden is to make it compatible with the
> enhanced
> > client API. If it's a critical blocker, I support dropping it radically
> in
> > 1.10. Of course a survey is necessary. And the result of survey is
> > acceptable.
> >
> >
> >
> > Zili Chen  于2019年7月19日周五 下午1:44写道：
> >
> > > Hi devs,
> > >
> > > Participating the thread "Flink client api enhancement"[1], I just
> notice
> > > that inside submission codepath of Flink we always has a branch dealing
> > > with the case that main class of user program is a subclass of
> > > o.a.f.api.common.Program, which is defined as
> > >
> > > @PublicEvolving
> > > public interface Program {
> > >   Plan getPhan(String... args);
> > > }
> > >
> > > This class, as user-facing interface, asks user to implement #getPlan
> > > which return a almost Flink internal class. FLINK-10862[2] pointed out
> > > this confusion.
> > >
> > > Since our codebase contains quite a bit code handling this stale class,
> > > and also it obstructs the effort enhancing Flink cilent api,
> > > I'd like to propose dropping it. Or we can start a survey on user list
> > > to see if there is any user depending on this class.
> > >
> > > best,
> > > tison.
> > >
> > > [1]
> > >
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> > >
> >
>

Re: [DISCUSS] Drop stale class Program

[Zili Chen]

Hi Flavio,

Thanks for your reply and share.

I agree with that an "official" program description
would be helpful as you described.

However, this thread mainly focuses on drop the stale
class Program.

For proposing a better program description from Flink
side, feel free to start a new thread on dev or user
list(I would regard this as a user requirement).

Best,
tison.


Zili Chen  于2019年7月19日周五 下午5:10写道：

> Hi Biao,
>
> Thanks for your reply.
>
> For the "burden" part, inside PackagedProgram and ClusterClient
> we currently contains branches handling whether the mainClass of
> user job jar is subclass of Program. Any effort under client api
> enhancement should be compatible with such codepaths unless we
> drop it.
>
> The class Program is only active in batch codepath, while the
> so-called interactive mode, i.e., executing the main method of
> user's main class, is using widely and valid in both batch and
> streaming codepath.
>
> Previously we have drop flink-storm directly and I'm afraid this
> Program class is much less known and used than flink-storm.
> Keeping compatible with such class is an unnecessary overhead.
>
> However, the enhancement of client api is still under discussion
> so it would be ok to deprecate at first and propose removal when
> necessary.
>
> Either drop it or deprecate it prevents users from being confused
> how to use this stale class. But given the community has dropped
> stale flink-storm, flink-libarary/python|ml, I'd like to propose
> drop this class if in the survey we receiving no usage report.
>
> Best,
> tison.
>
>
> Flavio Pompermaier  于2019年7月19日周五 下午4:21写道：
>
>> In my experience a basic "official" (but optional) program description
>> would be very useful indeed (in order to ease the integration with other
>> frameworks).
>>
>> Of course it should be extended and integrated with the REST services and
>> the Web UI (when defined) in order to be useful..
>> It ease to show to the user what a job does and which parameters it
>> requires (optional or mandatory) and with a proper help description.
>> Indeed, when we write a Flink job we implement the following interface:
>>
>> public interface FlinkJob {
>>   String getDescription();
>>   List getParameters();
>>  boolean isStreamingOrBatch();
>> }
>>
>> public class ClusterJobParameter {
>>
>>   private String paramName;
>>   private String paramType = "string";
>>   private String paramDesc;
>>   private String paramDefaultValue;
>>   private Set choices;
>>   private boolean mandatory;
>> }
>>
>> This really helps to launch a Flink job by a frontend (if the rest
>> services
>> returns back those infos).
>>
>> Best,
>> Flavio
>>
>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu  wrote:
>>
>> > Hi Zili,
>> >
>> > Thank you for bring us this discussion.
>> >
>> > My gut feeling is +1 for dropping it.
>> > Usually it costs some time to deprecate a public (actually it's
>> > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
>> first.
>> > Then it might be abandoned it in some later version.
>> >
>> > I'm not sure how big the burden is to make it compatible with the
>> enhanced
>> > client API. If it's a critical blocker, I support dropping it radically
>> in
>> > 1.10. Of course a survey is necessary. And the result of survey is
>> > acceptable.
>> >
>> >
>> >
>> > Zili Chen  于2019年7月19日周五 下午1:44写道：
>> >
>> > > Hi devs,
>> > >
>> > > Participating the thread "Flink client api enhancement"[1], I just
>> notice
>> > > that inside submission codepath of Flink we always has a branch
>> dealing
>> > > with the case that main class of user program is a subclass of
>> > > o.a.f.api.common.Program, which is defined as
>> > >
>> > > @PublicEvolving
>> > > public interface Program {
>> > >   Plan getPhan(String... args);
>> > > }
>> > >
>> > > This class, as user-facing interface, asks user to implement #getPlan
>> > > which return a almost Flink internal class. FLINK-10862[2] pointed out
>> > > this confusion.
>> > >
>> > > Since our codebase contains quite a bit code handling this stale
>> class,
>> > > and also it obstructs the effort enhancing Flink cilent api,
>> > > I'd like to propose dropping it. Or we can start a survey on user list
>> > > to see if there is any user depending on this class.
>> > >
>> > > best,
>> > > tison.
>> > >
>> > > [1]
>> > >
>> > >
>> >
>> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
>> > > [2] https://issues.apache.org/jira/browse/FLINK-10862
>> > >
>> >
>>
>

Re: [DISCUSS] Drop stale class Program

[Flavio Pompermaier]

+1 to remove directly the Program class (I think nobody use it and it's not
supported at all by REST services and UI).
Moreover it requires a lot of transitive dependencies while it should be a
very simple thing..
+1 to add this discussion to "Flink client api enhancement"

On Fri, Jul 19, 2019 at 11:14 AM Biao Liu  wrote:

> To Flavio, good point for the integration suggestion.
>
> I think it should be considered in the "Flink client api enhancement"
> discussion. But the outdated API should be deprecated somehow.
>
> Flavio Pompermaier  于2019年7月19日周五 下午4:21写道：
>
> > In my experience a basic "official" (but optional) program description
> > would be very useful indeed (in order to ease the integration with other
> > frameworks).
> >
> > Of course it should be extended and integrated with the REST services and
> > the Web UI (when defined) in order to be useful..
> > It ease to show to the user what a job does and which parameters it
> > requires (optional or mandatory) and with a proper help description.
> > Indeed, when we write a Flink job we implement the following interface:
> >
> > public interface FlinkJob {
> >   String getDescription();
> >   List getParameters();
> >  boolean isStreamingOrBatch();
> > }
> >
> > public class ClusterJobParameter {
> >
> >   private String paramName;
> >   private String paramType = "string";
> >   private String paramDesc;
> >   private String paramDefaultValue;
> >   private Set choices;
> >   private boolean mandatory;
> > }
> >
> > This really helps to launch a Flink job by a frontend (if the rest
> services
> > returns back those infos).
> >
> > Best,
> > Flavio
> >
> > On Fri, Jul 19, 2019 at 9:57 AM Biao Liu  wrote:
> >
> > > Hi Zili,
> > >
> > > Thank you for bring us this discussion.
> > >
> > > My gut feeling is +1 for dropping it.
> > > Usually it costs some time to deprecate a public (actually it's
> > > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
> first.
> > > Then it might be abandoned it in some later version.
> > >
> > > I'm not sure how big the burden is to make it compatible with the
> > enhanced
> > > client API. If it's a critical blocker, I support dropping it radically
> > in
> > > 1.10. Of course a survey is necessary. And the result of survey is
> > > acceptable.
> > >
> > >
> > >
> > > Zili Chen  于2019年7月19日周五 下午1:44写道：
> > >
> > > > Hi devs,
> > > >
> > > > Participating the thread "Flink client api enhancement"[1], I just
> > notice
> > > > that inside submission codepath of Flink we always has a branch
> dealing
> > > > with the case that main class of user program is a subclass of
> > > > o.a.f.api.common.Program, which is defined as
> > > >
> > > > @PublicEvolving
> > > > public interface Program {
> > > >   Plan getPhan(String... args);
> > > > }
> > > >
> > > > This class, as user-facing interface, asks user to implement #getPlan
> > > > which return a almost Flink internal class. FLINK-10862[2] pointed
> out
> > > > this confusion.
> > > >
> > > > Since our codebase contains quite a bit code handling this stale
> class,
> > > > and also it obstructs the effort enhancing Flink cilent api,
> > > > I'd like to propose dropping it. Or we can start a survey on user
> list
> > > > to see if there is any user depending on this class.
> > > >
> > > > best,
> > > > tison.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> > > >
> > >
> >
>

Request for permission as a contributor

[蒋正贵]

HiGuys,

I want to contribute to ApacheFlink.Would you please give me the permission as 
a contributor?
My JIRA ID is zhenggui.


best regards!

Re: Request for permission as a contributor

[Biao Liu]

Hi,

There is no need for contribution permission anymore, see [1].

1.
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-JIRA-permissions-changed-Only-committers-can-assign-somebody-to-a-ticket-td30695.html

蒋正贵  于2019年7月19日周五 下午5:21写道：

> HiGuys,
>
> I want to contribute to ApacheFlink.Would you please give me the
> permission as a contributor?
> My JIRA ID is zhenggui.
>
>
> best regards!

Re: Request for permission as a contributor

[Zili Chen]

Hi,

As an addition, you can checkout
https://flink.apache.org/contributing/contribute-code.html
to see the contribution workflow.

Best,
tison.


Biao Liu  于2019年7月19日周五 下午5:32写道：

> Hi,
>
> There is no need for contribution permission anymore, see [1].
>
> 1.
>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-JIRA-permissions-changed-Only-committers-can-assign-somebody-to-a-ticket-td30695.html
>
> 蒋正贵  于2019年7月19日周五 下午5:21写道：
>
> > HiGuys,
> >
> > I want to contribute to ApacheFlink.Would you please give me the
> > permission as a contributor?
> > My JIRA ID is zhenggui.
> >
> >
> > best regards!
>

Re: flink-mapr-fs failed in travis

[Jark Wu]

Great! Thanks Chesnay for the quick fixing.


On Fri, 19 Jul 2019 at 16:40, Chesnay Schepler  wrote:

> I think I found the issue; I forgot to update travis_controller.sh .
>
> On 19/07/2019 10:02, Chesnay Schepler wrote:
> > Ah, I added it to the common options in the travis_manv_watchdog.sh .
> >
> > On 19/07/2019 09:58, Chesnay Schepler wrote:
> >> I did modify the .travis.yml do activate the unsafe-mapr-repo
> >> profile; did I modified the wrong profile?...
> >>
> >>
> >> On 19/07/2019 07:57, Jark Wu wrote:
> >>> It seems that it is introduced by this commit:
> >>>
> https://github.com/apache/flink/commit/5c36c650e6520d92191ce2da33f7dcae774319f6
> >>>
> >>> Hi @Chesnay Schepler  , do we need to add
> >>> "-Punsafe-mapr-repo" to the ".travis.yml"?
> >>>
> >>> Best,
> >>> Jark
> >>>
> >>> On Fri, 19 Jul 2019 at 10:58, JingsongLee
> >>> 
> >>> wrote:
> >>>
>  Hi everyone:
> 
>  flink-mapr-fs failed in travis, and I retried many times, and also
>  failed.
>  Anyone has idea about this?
> 
>  01:32:54.755 [ERROR] Failed to execute goal on project flink-mapr-fs:
>  Could not resolve dependencies for project
>  org.apache.flink:flink-mapr-fs:jar:1.10-SNAPSHOT: Failed to collect
>  dependencies at com.mapr.hadoop:maprfs:jar:5.2.1-mapr: Failed to read
>  artifact descriptor for com.mapr.hadoop:maprfs:jar:5.2.1-mapr:
>  Could not
>  transfer artifact com.mapr.hadoop:maprfs:pom:5.2.1-mapr from/to
>  mapr-releases (https://repository.mapr.com/maven/):
>  sun.security.validator.ValidatorException: PKIX path building failed:
>  sun.security.provider.certpath.SunCertPathBuilderException: unable
>  to find
>  valid certification path to requested target -> [Help 1]
> 
>  https://api.travis-ci.org/v3/job/560790299/log.txt
> 
>  Best, Jingsong Lee
> >>
> >>
> >>
> >
> >
>
>

Re: [ANNOUNCE] JIRA permissions changed: Only committers can assign somebody to a ticket

[Robert Metzger]

I thought about this, but decided against it. I would hope that everybody
who's involved with the development in a way that they assign issues to
themselves is also active on this list.
Users who just report a bug should not just assign themselves.
If you disagree with me, feel free to forward the email to the user@ list.
It is okay for me if you do that :)

On Thu, Jul 18, 2019 at 9:47 PM Bowen Li  wrote:

> shall we announce in user ML too? Users who are used to assign tickets to
> themselves should also be aware of this change
>
> On Thu, Jul 18, 2019 at 3:06 AM Robert Metzger 
> wrote:
>
> > Hi all,
> >
> > The permissions for the FLINK Jira project have been changed [1], to
> *only
> > allow committers and PMC members to assign somebody to a Jira ticket.*
> >
> > Anybody with a Jira account can be assigned to a ticket. There is no need
> > for "Contributor" permissions.
> >
> > This has been discussed in this mailing list thread [2]. More information
> > on the contribution process is available on the Flink website [3].
> > The goal of this change is to ensure that discussions happen in the JIRA
> > ticket, before implementation work has started.
> > I'm encouraging all committers to monitor the Jira tickets created in
> > "their" components, drive discussions to a consensus and then assign
> > somebody to the ticket (indicating that this change has been agreed upon
> > and that somebody will review and merge it).
> >
> > Best,
> > Robert
> >
> >
> >
> > [1] https://issues.apache.org/jira/browse/INFRA-18644
> > [2]
> >
> >
> https://lists.apache.org/thread.html/b39e01d636cffa74c85b2f7405a25ec63a38d47eb6e0133d22873478@%3Cdev.flink.apache.org%3E
> > [3] https://flink.apache.org/contributing/contribute-code.html
> >
>

[jira] [Created] (FLINK-13338) Add sql conformance config interface in TableConfig

[Danny Chan (JIRA)]

Danny Chan created FLINK-13338:
--

 Summary: Add sql conformance config interface in TableConfig
 Key: FLINK-13338
 URL: https://issues.apache.org/jira/browse/FLINK-13338
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / API
Affects Versions: 1.9.0, 1.10.0
Reporter: Danny Chan
 Fix For: 1.9.0


Now the TableConfig has only interface to config the SqlParser config which is 
very broad and hard to use for user, we should at least supply an interface to 
config the sql conformance.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Re: [DISCUSS] A more restrictive JIRA workflow

[Robert Metzger]

Hi all!

Chesnay wrote:

> We haven't wiped the set of contributors yet. Not sure if there's an
> easy way to remove the permissions for all of them; someone from the PMC
> may have to bite the bullet and click 600 times in a row :)


I don't think there's an easy way for us.
People with "Contributor" permissions have permissions such as "Closing"
and "Editing" (including "Transition", "Logging work", etc.) issues.
I would propose to either drastically reduce the number of people with
Contributor permissions, leaving just those in, who are helping to keep our
Jira in a clean shape (and this decision would be solely done at the
discretion of the PMC deleting people from the contributor group)
Another option would be to ask Infra to remove all people with Contributor
permissions from Flink (deleting and re-adding the group or so :) )
Moving forward, the PMC would maintain a list of contributors who are
helping to keep Jira clean and well-organized.

Tison wrote:

> IIRC someone in this thread once mentioned that
> "Don't allow contributors to set a blocker priority."


Timo wrote this when he kicked off this thread. Sadly, this permission is
not provided by Jira out of the box:
https://community.atlassian.com/t5/Jira-questions/How-to-restrict-the-setting-of-certain-priorities-to-certain/qaq-p/193230
We will have to manually monitor tickets with "Blocker" status.


Fair enough, we might rephrase as
> "Pull requests belonging to unassigned Jira tickets or not authored by
> assignee will ..."


That's a good point. If nobody disagrees, would you like to open PR for
changing it?

For the action "what should we do", iirc someone in this thread proposed
> closing it via flinkbot. Or leaving without review and merge could be ok
> before we implement automatic reply/reaction.


I proposed to automatically close. I believe that this might be too harsh.
I've implemented already an extension to Flinkbot which shows warnings for
a pull request. I just haven't found the time to test the extension and put
it in production.



On Fri, Jul 19, 2019 at 5:51 AM Zili Chen  wrote:

> Fair enough, we might rephrase as
>
> "Pull requests belonging to unassigned Jira tickets or not authored by
> assignee will ..."
>
> For the action "what should we do", iirc someone in this thread proposed
> closing it via flinkbot. Or leaving without review and merge could be ok
> before we implement automatic reply/reaction.
>
> Best,
> tison.
>
>
> Zili Chen  于2019年7月19日周五 上午11:44写道：
>
>> @Jack
>>
>> From https://flink.apache.org/contributing/contribute-code.html the
>> community
>> claims
>>
>> - Only start working on the implementation if there is consensus on the
>> approach (e.g. you are assigned to the ticket)
>>
>> and accurately answer your question,
>>
>> - Pull requests belonging to unassigned Jira tickets will not be reviewed
>> or merged by the community.
>>
>> Best,
>> tison.
>>
>>
>> Jark Wu  于2019年7月19日周五 上午11:28写道：
>>
>>> A quick question, what should we do if a developer creates a JIRA issue
>>> and
>>> then create a pull request at once without assigning?
>>>
>>>
>>> Regards,
>>> Jark
>>>
>>> On Thu, 18 Jul 2019 at 18:50, Zili Chen  wrote:
>>>
>>> > Checking the result, as a discovery, I found that one can
>>> > still file a JIRA with "blocker" priority.
>>> >
>>> > IIRC someone in this thread once mentioned that
>>> > "Don't allow contributors to set a blocker priority."
>>> >
>>> > Chesnay,
>>> >
>>> > Thanks for the clarification.
>>> >
>>> >
>>> > Best,
>>> > tison.
>>> >
>>> >
>>> > Chesnay Schepler  于2019年7月18日周四 下午6:40写道：
>>> >
>>> > > We haven't wiped the set of contributors yet. Not sure if there's an
>>> > > easy way to remove the permissions for all of them; someone from the
>>> PMC
>>> > > may have to bite the bullet and click 600 times in a row :)
>>> > >
>>> > > On 18/07/2019 12:32, Zili Chen wrote:
>>> > > > Robert,
>>> > > >
>>> > > > Thanks for your effort. Rejecting contributor permission request
>>> > > > with a nice message and pointing them to the announcement sounds
>>> > > > reasonable. Just to be clear, we now have no person with
>>> contributor
>>> > > > role, right?
>>> > > >
>>> > > > Chesnay,
>>> > > >
>>> > > > https://flink.apache.org/contributing/contribute-code.html has
>>> been
>>> > > > updated and mentions that "Only committers can assign a Jira
>>> ticket."
>>> > > >
>>> > > > I think the corresponding update has been done.
>>> > > >
>>> > > > Best,
>>> > > > tison.
>>> > > >
>>> > > >
>>> > > > Chesnay Schepler  于2019年7月18日周四 下午6:25写道：
>>> > > >
>>> > > >> Do our contribution guidelines contain anything that should be
>>> > updated?
>>> > > >>
>>> > > >> On 18/07/2019 12:24, Chesnay Schepler wrote:
>>> > > >>> Sounds good to me.
>>> > > >>>
>>> > > >>> On 18/07/2019 12:07, Robert Metzger wrote:
>>> > >  Infra has finally changed the permissions. I just announced the
>>> > >  change in a
>>> > >  separate email [1].
>>> > > 
>>> > >  One thing I wanted to discuss here is, 

Re: Issue running basic example locally

[Andres Angel]

that was the issue , thanks so much man!!
AU

On Thu, Jul 18, 2019 at 8:56 PM Caizhi Weng  wrote:

> Hi Andres,
>
> `provided` of flink-streaming-java seems suspicious, can you
> remove it and see what happens?
>
> Andres Angel  于2019年7月19日周五 上午3:10写道：
>
> > Hello everyone,
> >
> > I'm using IntelliJ in a ubuntu environment with java 1.8 to run my Flink
> > framworks. My goal is consume few kinesis stream services and build a
> sort
> > of pipeline. Initially I have build my pom.xml file attached and a test
> > java class attached too.
> >
> > However , once I run my class from the IDE and getting this error:
> >
> > Error: A JNI error has occurred, please check your installation and try
> > again
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > org/apache/flink/streaming/api/functions/source/SourceFunction
> > at java.lang.Class.getDeclaredMethods0(Native Method)
> > at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
> > at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
> > at java.lang.Class.getMethod0(Class.java:3018)
> > at java.lang.Class.getMethod(Class.java:1784)
> > at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
> > at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.flink.streaming.api.functions.source.SourceFunction
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> > ... 7 more
> >
> > I think I dont need to install or consifure any flink service before run
> > my job via IDE , please if someone could give me a hand with this issue I
> > would appreciate it.
> >
> > thanks
> >
> >
> >
>

Re: [DISCUSS] Flink project bylaws

[Robert Metzger]

Hi all,
I agree with Aljoscha that trying to reflect the current status in the
bylaws, and then implementing changes one by one is a very involved task.
Unless there's somebody who's really eager to drive this, I would stick to
Becket's initiative to come up with Bylaws for Flink, even if this means
some changes.

The cross-review requirement is the last big open point in this discussion.
It seems that a there is a slight tendency in the discussion that this is
not feasible given the current pull request review situation.
For the sake of bringing this discussion to a conclusion, I'm fine with
leaving this requirement out. As we are currently adding more committers to
the project, we might be able to revisit this discussion in 3 - 6 months.


On Thu, Jul 18, 2019 at 4:30 AM jincheng sun 
wrote:

> Hi Becket,
>
> Thanks for the proposal.
>
> Regarding the vote of FLIP, preferably at least includes a PMC vote.
> Because FLIP is usually a big change or affects the user's interface
> changes. What do you think? (I leave the comment in the wiki.)
>
> Best,
> Jincheng
>
> Dawid Wysakowicz  于2019年7月17日周三 下午9:12写道：
>
> > Hi all,
> >
> > Sorry for joining late. I just wanted to say that I really like the
> > proposed bylaws!
> >
> > One comment, I also have the same concerns as expressed by few people
> > before that the "committer +1" on code change might be hard to achieve
> > currently. On the other hand I would say this would be beneficial for
> > the quality/uniformity of our codebase and knowledge sharing.
> >
> > I was just wondering what should be the next step for this? I think it
> > would make sense to already use those bylaws and put them to PMC vote.
> >
> > Best,
> >
> > Dawid
> >
> > On 12/07/2019 13:35, Piotr Nowojski wrote:
> > > Hi Aljoscha and Becket
> > >
> > > Right, 3 days for FLIP voting is fine I think.
> > >
> > >>> I’m missing this stated somewhere clearly. If we are stating that a
> > single
> > >>> committers +1 is good enough for code review, with 0 hours delay (de
> > facto
> > >>> the current state), we should also write down that this is subject to
> > the
> > >>> best judgement of the committer to respect the components expertise
> and
> > >>> ongoing development plans (also the de facto current state).
> > >> Adding the statement would help clarify the intention, but it may be a
> > >> little difficult to enforce and follow..
> > > I would be fine with that, it’s a soft/vague rule anyway, intended to
> be
> > used with your “best judgemenet". I would like to just avoid a situation
> > when someone violates current uncodified state and refers to the bylaws
> > which are saying merging with any committer +1 is always fine (like mine
> +1
> > for flink-python or flink-ml).
> > >
> > > Piotrek
> > >
> > >> On 12 Jul 2019, at 11:29, Aljoscha Krettek 
> wrote:
> > >>
> > >> @Piotr regarding the 3 days voting on the FLIP. This is just about the
> > voting, before that there needs to be the discussion thread. If three
> days
> > have passed on a vote and there is consensus (i.e. 3 committers/PMCs have
> > voted +1) that seems a high enough bar for me. So far, we have rarely see
> > any FLIPs pass that formal bar.
> > >>
> > >> According to the recent META-FLIP thread, we want to use "lazy
> > majority" for the FLIP voting process. I think that should be changed
> from
> > “consensus” in the proposed bylaws.
> > >>
> > >> Regarding the current state: do we have a current state that we all
> > agree on? I have the feeling that if we try to come up with something
> that
> > reflects the common state, according to PMCs/commiters, that might take a
> > very long time. In that case I think it’s better to adopt something that
> we
> > all like, rather than trying to capture how we do it now.
> > >>
> > >> Aljoscha
> > >>
> > >>> On 12. Jul 2019, at 11:07, Piotr Nowojski 
> wrote:
> > >>>
> > >>> Hi,
> > >>>
> > >>> Thanks for the proposal. Generally speaking +1 from my side to the
> > general idea and most of the content. I also see merit to the Chesney's
> > proposal to start from the current state. I think either would be fine
> for
> > me.
> > >>>
> > >>> Couple of comments:
> > >>>
> > >>> 1.
> > >>>
> > >>> I also think that requiring +1 from another committer would slow down
> > and put even more strain on the current reviewing bottleneck that we are
> > having. Even if the change clear and simple, context switch cost is quite
> > high, and that’s just one less PR that the second “cross” committer could
> > have reviewed somewhere else in that time. Besides, current setup that we
> > have (with no cross +1 from another committer required) works quite well
> > and I do not feel that’s causing troubles. On the other hand reviewing
> > bottleneck is.
> > >>>
> > >>> 2.
> > >>>
> >  I think a committer should know when to ask another committer for
> > feedback or not.
> > >>> I’m missing this stated somewhere clearly. If we are stating that a
> > single committers +1 is good enough for code 

Re: [DISCUSS] Flink project bylaws

[Becket Qin]

Hi Jincheng,

Thanks for the comments. I replied on the wiki page. Just a brief summary,
the current bylaws do require some of the FLIPs to get PMC approval if
their impact is big enough. But it leaves majority of the technical
decisions to the committers who are supposed to be responsible for making
such decisions.

Re: Robert,

I agree we can simply remove the requirement of +1 from a non-author
committer and revisit it in a bit. After all, it does not make sense to
have a bylaw that we cannot afford. I have just updated the bylaws wiki.

Thanks,

Jiangjie (Becket) Qin

On Fri, Jul 19, 2019 at 11:17 PM Robert Metzger  wrote:

> Hi all,
> I agree with Aljoscha that trying to reflect the current status in the
> bylaws, and then implementing changes one by one is a very involved task.
> Unless there's somebody who's really eager to drive this, I would stick to
> Becket's initiative to come up with Bylaws for Flink, even if this means
> some changes.
>
> The cross-review requirement is the last big open point in this discussion.
> It seems that a there is a slight tendency in the discussion that this is
> not feasible given the current pull request review situation.
> For the sake of bringing this discussion to a conclusion, I'm fine with
> leaving this requirement out. As we are currently adding more committers to
> the project, we might be able to revisit this discussion in 3 - 6 months.
>
>
> On Thu, Jul 18, 2019 at 4:30 AM jincheng sun 
> wrote:
>
> > Hi Becket,
> >
> > Thanks for the proposal.
> >
> > Regarding the vote of FLIP, preferably at least includes a PMC vote.
> > Because FLIP is usually a big change or affects the user's interface
> > changes. What do you think? (I leave the comment in the wiki.)
> >
> > Best,
> > Jincheng
> >
> > Dawid Wysakowicz  于2019年7月17日周三 下午9:12写道：
> >
> > > Hi all,
> > >
> > > Sorry for joining late. I just wanted to say that I really like the
> > > proposed bylaws!
> > >
> > > One comment, I also have the same concerns as expressed by few people
> > > before that the "committer +1" on code change might be hard to achieve
> > > currently. On the other hand I would say this would be beneficial for
> > > the quality/uniformity of our codebase and knowledge sharing.
> > >
> > > I was just wondering what should be the next step for this? I think it
> > > would make sense to already use those bylaws and put them to PMC vote.
> > >
> > > Best,
> > >
> > > Dawid
> > >
> > > On 12/07/2019 13:35, Piotr Nowojski wrote:
> > > > Hi Aljoscha and Becket
> > > >
> > > > Right, 3 days for FLIP voting is fine I think.
> > > >
> > > >>> I’m missing this stated somewhere clearly. If we are stating that a
> > > single
> > > >>> committers +1 is good enough for code review, with 0 hours delay
> (de
> > > facto
> > > >>> the current state), we should also write down that this is subject
> to
> > > the
> > > >>> best judgement of the committer to respect the components expertise
> > and
> > > >>> ongoing development plans (also the de facto current state).
> > > >> Adding the statement would help clarify the intention, but it may
> be a
> > > >> little difficult to enforce and follow..
> > > > I would be fine with that, it’s a soft/vague rule anyway, intended to
> > be
> > > used with your “best judgemenet". I would like to just avoid a
> situation
> > > when someone violates current uncodified state and refers to the bylaws
> > > which are saying merging with any committer +1 is always fine (like
> mine
> > +1
> > > for flink-python or flink-ml).
> > > >
> > > > Piotrek
> > > >
> > > >> On 12 Jul 2019, at 11:29, Aljoscha Krettek 
> > wrote:
> > > >>
> > > >> @Piotr regarding the 3 days voting on the FLIP. This is just about
> the
> > > voting, before that there needs to be the discussion thread. If three
> > days
> > > have passed on a vote and there is consensus (i.e. 3 committers/PMCs
> have
> > > voted +1) that seems a high enough bar for me. So far, we have rarely
> see
> > > any FLIPs pass that formal bar.
> > > >>
> > > >> According to the recent META-FLIP thread, we want to use "lazy
> > > majority" for the FLIP voting process. I think that should be changed
> > from
> > > “consensus” in the proposed bylaws.
> > > >>
> > > >> Regarding the current state: do we have a current state that we all
> > > agree on? I have the feeling that if we try to come up with something
> > that
> > > reflects the common state, according to PMCs/commiters, that might
> take a
> > > very long time. In that case I think it’s better to adopt something
> that
> > we
> > > all like, rather than trying to capture how we do it now.
> > > >>
> > > >> Aljoscha
> > > >>
> > > >>> On 12. Jul 2019, at 11:07, Piotr Nowojski 
> > wrote:
> > > >>>
> > > >>> Hi,
> > > >>>
> > > >>> Thanks for the proposal. Generally speaking +1 from my side to the
> > > general idea and most of the content. I also see merit to the Chesney's
> > > proposal to start from the current state. I think either would be fine
> > for
> > > me.
>

Re: [DISCUSS] Create a Flink ecosystem website

[Marta Paes Moreira]

Hey, Robert.

I will keep an eye on the overall progress and get started on the blog post
to make the community announcement. Are there (mid-term) plans to
translate/localize this website as well? It might be a point worth
mentioning in the blogpost.

Hats off to you and Daryl — this turned out amazing!

Marta

On Thu, Jul 18, 2019 at 10:57 AM Congxian Qiu 
wrote:

> Robert and Daryl, thanks for the great work, I tried the website and filed
> some issues on Github.
> Best,
> Congxian
>
>
> Robert Metzger  于2019年7月17日周三 下午11:28写道：
>
>> Hey all,
>>
>> Daryl and I have great news to share. We are about to finish adding the
>> basic features to the ecosystem page.
>> We are at a stage where it is ready to be reviewed and made public.
>>
>> You can either check out a development instance of the ecosystem page
>> here: https://flink-ecosystem-demo.flink-resources.org/
>> Or you run it locally, with the instructions from the README.md:
>> https://github.com/sorahn/flink-ecosystem
>>
>> Please report all issues you find here:
>> https://github.com/sorahn/flink-ecosystem/issues or in this thread.
>>
>> The next steps in this project are the following:
>> - We fix all issues reported through this testing
>> - We set up the site on the INFRA resources Becket has secured [1], do
>> some further testing (including email notifications) and pre-fill the page
>> with some packages.
>> - We set up a packages.flink.apache.org or flink.apache.org/packages
>> domain
>> - We announce the packages through a short blog post
>>
>> Happy testing!
>>
>> Best,
>> Robert
>>
>> [1] https://issues.apache.org/jira/browse/INFRA-18010
>>
>>
>> On Thu, Apr 25, 2019 at 6:23 AM Becket Qin  wrote:
>>
>>> Thanks for the update, Robert. Looking forward to the website. If there
>>> is already a list of software we need to run the website, we can ask Apache
>>> infra team to prepare the VM for us, as that may also take some time.
>>>
>>> On Wed, Apr 24, 2019 at 11:57 PM Robert Metzger 
>>> wrote:
>>>
 Hey all,

 quick update on this project: The frontend and backend code have been
 put together into this repository:
 https://github.com/sorahn/flink-ecosystem
 We also just agreed on an API specification, and will now work on
 finishing the backend.

 It will probably take a few more weeks for this to finish, but we are
 making progress :)

 Best,
 Robert


 On Mon, Apr 15, 2019 at 11:18 AM Robert Metzger 
 wrote:

> Hey Daryl,
>
> thanks a lot for posting a link to this first prototype on the mailing
> list! I really like it!
>
> Becket: Our plan forward is that Congxian is implementing the backend
> for the website. He has already started with the work, but needs at least
> one more week.
>
>
> [Re-sending this email because the first one was blocked on dev@f.a.o]
>
>
> On Mon, Apr 15, 2019 at 7:59 AM Becket Qin 
> wrote:
>
>> Hi Daryl,
>>
>> Thanks a lot for the update. The site looks awesome! This is a great
>> progress. I really like the conciseness of GUI.
>>
>> One minor suggestion is that for the same library, there might be
>> multiple versions compatible with different Flink versions. It would be
>> good to show that somewhere in the project page as it seems important to
>> the users.
>>
>> BTW, will you share the plan to move forward? Would additional hands
>> help?
>>
>> Thanks,
>>
>> Jiangjie (Becket) Qin
>>
>> On Sat, Apr 13, 2019 at 7:10 PM Daryl Roberts 
>> wrote:
>>
>>> > Shall we add a guide page to show people how to publish their
>>> projects to the website? The exact rules can be discussed and drafted 
>>> in a
>>> separate email thread IMO
>>>
>>> This is a good idea. (Both the guise, and separate thread), I think
>>> once there is an actual packed in place we’ll be in a lot better 
>>> position
>>> to discuss this.
>>>
>>> > The "Log in with Github" link doesn't seem to work yet. Will it
>>> only allow login for admins and publishers, or for everyone?
>>>
>>> Correct, all the oauth stuff requires a real server. We are
>>> currently just faking everything.
>>>
>>> I will add a mock-login page (username/password that just accepts
>>> anything and displays whatever username you type in) so we can see the
>>> add-comment field and add-packages page once they exist.
>>>
>>>
>>>
>>>

YARN parallelism vs Config

[Dominik Wosiński]

Hey,

I was wondering about the relation between the parallelism set by YARN in
Yarn properties file. Currently, as far as I know there is only one
execution of `writeYarnPropertiesFIle` method and it sets the parallelism
in the YARN properties to the number of workers * number of slots per
worker. But doesn't the `flink-conf.yaml` take the precedence in resolving
the configuration ? I am trying to understand the reasoning between always
setting the Yarn properties to the max available slots and whether this
will be used at all, since there is a default value  in flink config for
paralellism.

Thanks in advance,
Best Regards,
Dom.

[jira] [Created] (FLINK-13339) Add an implementation of pipeline's api

[Xu Yang (JIRA)]

Xu Yang created FLINK-13339:
---

 Summary: Add an implementation of pipeline's api
 Key: FLINK-13339
 URL: https://issues.apache.org/jira/browse/FLINK-13339
 Project: Flink
  Issue Type: Sub-task
Reporter: Xu Yang


* Add an implement PipelineStage, Estimator, Transformer, Model.
 * Add MLSession to _hold the execution environment and others session shared 
variable._
 * Add AlgoOperator for the implementation of algorithms.
 * Add BatchOperator and StreamOperator based on AlgoOperator
 * Add TableSourceBatchOp and TableSourceStreamOp



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Created] (FLINK-13340) Add more Kafka topic option of flink-connector-kafka

[DuBin (JIRA)]

DuBin created FLINK-13340:
-

 Summary: Add more Kafka topic option of flink-connector-kafka
 Key: FLINK-13340
 URL: https://issues.apache.org/jira/browse/FLINK-13340
 Project: Flink
  Issue Type: Improvement
  Components: Connectors / Kafka
Affects Versions: 1.8.1
Reporter: DuBin


Currently, only 'topic' option implemented in the Kafka Connector Descriptor, 
we can only use it like :
{code:java}
val env = StreamExecutionEnvironment.getExecutionEnvironment
val tableEnv = StreamTableEnvironment.create(env)

tableEnv
  .connect(
new Kafka()
  .version("0.11")
  .topic("test-flink-1")
  .startFromEarliest()
  .property("zookeeper.connect", "localhost:2181")
  .property("bootstrap.servers", "localhost:9092"))
  .withFormat(
new Json()
  .deriveSchema()
  )
  .withSchema(
new Schema()
  .field("name", Types.STRING)
  .field("age", Types.STRING)
  ){code}
but we cannot consume multiple topics or a topic regex pattern. 

Here is my thoughts:
{code:java}
  .topic("test-flink-1") 
  //.topics("test-flink-1,test-flink-2") or topics(List topics)
  //.subscriptionPattern("test-flink-.*") or 
subscriptionPattern(Pattern pattern)
{code}
I already implement the code on my local env with help of the 
FlinkKafkaConsumer, and it works.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Re: [DISCUSS] Create a Flink ecosystem website

[Becket Qin]

The website is awesome! I really like its conciseness and yet fairly useful
information and functionalities. I cannot think of much to improve at the
moment. Just one thought, do we need an "others" category, just in case a
package does not fit into any of the current given categories?

Thanks Robert and Daryl for the great effort. Looking forward to seeing
this get published soon!!

I agree with Marta that

Jiangjie (Becket) Qin

On Sat, Jul 20, 2019 at 1:34 AM Marta Paes Moreira 
wrote:

> Hey, Robert.
>
> I will keep an eye on the overall progress and get started on the blog post
> to make the community announcement. Are there (mid-term) plans to
> translate/localize this website as well? It might be a point worth
> mentioning in the blogpost.
>
> Hats off to you and Daryl — this turned out amazing!
>
> Marta
>
> On Thu, Jul 18, 2019 at 10:57 AM Congxian Qiu 
> wrote:
>
> > Robert and Daryl, thanks for the great work, I tried the website and
> filed
> > some issues on Github.
> > Best,
> > Congxian
> >
> >
> > Robert Metzger  于2019年7月17日周三 下午11:28写道：
> >
> >> Hey all,
> >>
> >> Daryl and I have great news to share. We are about to finish adding the
> >> basic features to the ecosystem page.
> >> We are at a stage where it is ready to be reviewed and made public.
> >>
> >> You can either check out a development instance of the ecosystem page
> >> here: https://flink-ecosystem-demo.flink-resources.org/
> >> Or you run it locally, with the instructions from the README.md:
> >> https://github.com/sorahn/flink-ecosystem
> >>
> >> Please report all issues you find here:
> >> https://github.com/sorahn/flink-ecosystem/issues or in this thread.
> >>
> >> The next steps in this project are the following:
> >> - We fix all issues reported through this testing
> >> - We set up the site on the INFRA resources Becket has secured [1], do
> >> some further testing (including email notifications) and pre-fill the
> page
> >> with some packages.
> >> - We set up a packages.flink.apache.org or flink.apache.org/packages
> >> domain
> >> - We announce the packages through a short blog post
> >>
> >> Happy testing!
> >>
> >> Best,
> >> Robert
> >>
> >> [1] https://issues.apache.org/jira/browse/INFRA-18010
> >>
> >>
> >> On Thu, Apr 25, 2019 at 6:23 AM Becket Qin 
> wrote:
> >>
> >>> Thanks for the update, Robert. Looking forward to the website. If there
> >>> is already a list of software we need to run the website, we can ask
> Apache
> >>> infra team to prepare the VM for us, as that may also take some time.
> >>>
> >>> On Wed, Apr 24, 2019 at 11:57 PM Robert Metzger 
> >>> wrote:
> >>>
>  Hey all,
> 
>  quick update on this project: The frontend and backend code have been
>  put together into this repository:
>  https://github.com/sorahn/flink-ecosystem
>  We also just agreed on an API specification, and will now work on
>  finishing the backend.
> 
>  It will probably take a few more weeks for this to finish, but we are
>  making progress :)
> 
>  Best,
>  Robert
> 
> 
>  On Mon, Apr 15, 2019 at 11:18 AM Robert Metzger 
>  wrote:
> 
> > Hey Daryl,
> >
> > thanks a lot for posting a link to this first prototype on the
> mailing
> > list! I really like it!
> >
> > Becket: Our plan forward is that Congxian is implementing the backend
> > for the website. He has already started with the work, but needs at
> least
> > one more week.
> >
> >
> > [Re-sending this email because the first one was blocked on dev@f.a.o
> ]
> >
> >
> > On Mon, Apr 15, 2019 at 7:59 AM Becket Qin 
> > wrote:
> >
> >> Hi Daryl,
> >>
> >> Thanks a lot for the update. The site looks awesome! This is a great
> >> progress. I really like the conciseness of GUI.
> >>
> >> One minor suggestion is that for the same library, there might be
> >> multiple versions compatible with different Flink versions. It
> would be
> >> good to show that somewhere in the project page as it seems
> important to
> >> the users.
> >>
> >> BTW, will you share the plan to move forward? Would additional hands
> >> help?
> >>
> >> Thanks,
> >>
> >> Jiangjie (Becket) Qin
> >>
> >> On Sat, Apr 13, 2019 at 7:10 PM Daryl Roberts 
> >> wrote:
> >>
> >>> > Shall we add a guide page to show people how to publish their
> >>> projects to the website? The exact rules can be discussed and
> drafted in a
> >>> separate email thread IMO
> >>>
> >>> This is a good idea. (Both the guise, and separate thread), I think
> >>> once there is an actual packed in place we’ll be in a lot better
> position
> >>> to discuss this.
> >>>
> >>> > The "Log in with Github" link doesn't seem to work yet. Will it
> >>> only allow login for admins and publishers, or for everyone?
> >>>
> >>> Correct, all the oauth stuff requires a real server. We are
> >>

Re: [DISCUSS] Create a Flink ecosystem website

[Becket Qin]

[Sorry for the incomplete message. Clicked send by mistake...]

I agree with Marta that it might be good to have multi-language support as
a mid-term goal.

Jiangjie (Becket) Qin

On Sat, Jul 20, 2019 at 11:22 AM Becket Qin  wrote:

> The website is awesome! I really like its conciseness and yet fairly
> useful information and functionalities. I cannot think of much to improve
> at the moment. Just one thought, do we need an "others" category, just in
> case a package does not fit into any of the current given categories?
>
> Thanks Robert and Daryl for the great effort. Looking forward to seeing
> this get published soon!!
>
> I agree with Marta that
>
> Jiangjie (Becket) Qin
>
> On Sat, Jul 20, 2019 at 1:34 AM Marta Paes Moreira 
> wrote:
>
>> Hey, Robert.
>>
>> I will keep an eye on the overall progress and get started on the blog
>> post
>> to make the community announcement. Are there (mid-term) plans to
>> translate/localize this website as well? It might be a point worth
>> mentioning in the blogpost.
>>
>> Hats off to you and Daryl — this turned out amazing!
>>
>> Marta
>>
>> On Thu, Jul 18, 2019 at 10:57 AM Congxian Qiu 
>> wrote:
>>
>> > Robert and Daryl, thanks for the great work, I tried the website and
>> filed
>> > some issues on Github.
>> > Best,
>> > Congxian
>> >
>> >
>> > Robert Metzger  于2019年7月17日周三 下午11:28写道：
>> >
>> >> Hey all,
>> >>
>> >> Daryl and I have great news to share. We are about to finish adding the
>> >> basic features to the ecosystem page.
>> >> We are at a stage where it is ready to be reviewed and made public.
>> >>
>> >> You can either check out a development instance of the ecosystem page
>> >> here: https://flink-ecosystem-demo.flink-resources.org/
>> >> Or you run it locally, with the instructions from the README.md:
>> >> https://github.com/sorahn/flink-ecosystem
>> >>
>> >> Please report all issues you find here:
>> >> https://github.com/sorahn/flink-ecosystem/issues or in this thread.
>> >>
>> >> The next steps in this project are the following:
>> >> - We fix all issues reported through this testing
>> >> - We set up the site on the INFRA resources Becket has secured [1], do
>> >> some further testing (including email notifications) and pre-fill the
>> page
>> >> with some packages.
>> >> - We set up a packages.flink.apache.org or flink.apache.org/packages
>> >> domain
>> >> - We announce the packages through a short blog post
>> >>
>> >> Happy testing!
>> >>
>> >> Best,
>> >> Robert
>> >>
>> >> [1] https://issues.apache.org/jira/browse/INFRA-18010
>> >>
>> >>
>> >> On Thu, Apr 25, 2019 at 6:23 AM Becket Qin 
>> wrote:
>> >>
>> >>> Thanks for the update, Robert. Looking forward to the website. If
>> there
>> >>> is already a list of software we need to run the website, we can ask
>> Apache
>> >>> infra team to prepare the VM for us, as that may also take some time.
>> >>>
>> >>> On Wed, Apr 24, 2019 at 11:57 PM Robert Metzger 
>> >>> wrote:
>> >>>
>>  Hey all,
>> 
>>  quick update on this project: The frontend and backend code have been
>>  put together into this repository:
>>  https://github.com/sorahn/flink-ecosystem
>>  We also just agreed on an API specification, and will now work on
>>  finishing the backend.
>> 
>>  It will probably take a few more weeks for this to finish, but we are
>>  making progress :)
>> 
>>  Best,
>>  Robert
>> 
>> 
>>  On Mon, Apr 15, 2019 at 11:18 AM Robert Metzger > >
>>  wrote:
>> 
>> > Hey Daryl,
>> >
>> > thanks a lot for posting a link to this first prototype on the
>> mailing
>> > list! I really like it!
>> >
>> > Becket: Our plan forward is that Congxian is implementing the
>> backend
>> > for the website. He has already started with the work, but needs at
>> least
>> > one more week.
>> >
>> >
>> > [Re-sending this email because the first one was blocked on
>> dev@f.a.o]
>> >
>> >
>> > On Mon, Apr 15, 2019 at 7:59 AM Becket Qin 
>> > wrote:
>> >
>> >> Hi Daryl,
>> >>
>> >> Thanks a lot for the update. The site looks awesome! This is a
>> great
>> >> progress. I really like the conciseness of GUI.
>> >>
>> >> One minor suggestion is that for the same library, there might be
>> >> multiple versions compatible with different Flink versions. It
>> would be
>> >> good to show that somewhere in the project page as it seems
>> important to
>> >> the users.
>> >>
>> >> BTW, will you share the plan to move forward? Would additional
>> hands
>> >> help?
>> >>
>> >> Thanks,
>> >>
>> >> Jiangjie (Becket) Qin
>> >>
>> >> On Sat, Apr 13, 2019 at 7:10 PM Daryl Roberts > >
>> >> wrote:
>> >>
>> >>> > Shall we add a guide page to show people how to publish their
>> >>> projects to the website? The exact rules can be discussed and
>> drafted in a
>> >>> separate email thread IMO
>> >>>
>> >>> This is a goo

[DISCUSS] Setup a bui...@flink.apache.org mailing list for travis builds

[Jark Wu]

Hi all,

As far as I know, currently, email notifications of Travis builds for
master branch are sent to the commit author when a build was just broken or
still is broken. And there is no email notifications for CRON builds.

Recently, we are suffering from compile errors for scala-2.12 and java-9
which are only ran in CRON jobs. So I'm figuring out a way to get
notifications of CRON builds (or all builds) to quick fix compile errors
and failed cron tests.

After reaching out to @Chesnay Schepler  (thanks for
the helping), I know that we are using a Slack channel to receive all
failed build notifications. However, IMO, email notification might be a
better way than Slack channel to encourage more people to pay attention on
the builds.

So I'm here to propose to setup a bui...@flink.apache.org mailing list for
receiving build notifications. I also find that Beam has such mailing list
too[1]. After we have such a mailing list, we can integrate it to travis
according to the travis doc[2].

What do you think? Do we need a formal vote for this?

Best and thanks,
Jark

[1]: https://beam.apache.org/community/contact-us/
[2]:
https://docs.travis-ci.com/user/notifications/#configuring-email-notifications

Re: [DISCUSS] A more restrictive JIRA workflow

[kai wang]

There are many reasons for jira, and one of the most important reasons is
that flink varies greatly from version to version. However, I agree that
you restrict some rights of developers, including the right to assign and
edit.

Robert Metzger  于2019年7月19日周五 下午10:58写道：

> Hi all!
>
> Chesnay wrote:
>
> > We haven't wiped the set of contributors yet. Not sure if there's an
> > easy way to remove the permissions for all of them; someone from the PMC
> > may have to bite the bullet and click 600 times in a row :)
>
>
> I don't think there's an easy way for us.
> People with "Contributor" permissions have permissions such as "Closing"
> and "Editing" (including "Transition", "Logging work", etc.) issues.
> I would propose to either drastically reduce the number of people with
> Contributor permissions, leaving just those in, who are helping to keep our
> Jira in a clean shape (and this decision would be solely done at the
> discretion of the PMC deleting people from the contributor group)
> Another option would be to ask Infra to remove all people with Contributor
> permissions from Flink (deleting and re-adding the group or so :) )
> Moving forward, the PMC would maintain a list of contributors who are
> helping to keep Jira clean and well-organized.
>
> Tison wrote:
>
> > IIRC someone in this thread once mentioned that
> > "Don't allow contributors to set a blocker priority."
>
>
> Timo wrote this when he kicked off this thread. Sadly, this permission is
> not provided by Jira out of the box:
>
> https://community.atlassian.com/t5/Jira-questions/How-to-restrict-the-setting-of-certain-priorities-to-certain/qaq-p/193230
> We will have to manually monitor tickets with "Blocker" status.
>
>
> Fair enough, we might rephrase as
> > "Pull requests belonging to unassigned Jira tickets or not authored by
> > assignee will ..."
>
>
> That's a good point. If nobody disagrees, would you like to open PR for
> changing it?
>
> For the action "what should we do", iirc someone in this thread proposed
> > closing it via flinkbot. Or leaving without review and merge could be ok
> > before we implement automatic reply/reaction.
>
>
> I proposed to automatically close. I believe that this might be too harsh.
> I've implemented already an extension to Flinkbot which shows warnings for
> a pull request. I just haven't found the time to test the extension and put
> it in production.
>
>
>
> On Fri, Jul 19, 2019 at 5:51 AM Zili Chen  wrote:
>
> > Fair enough, we might rephrase as
> >
> > "Pull requests belonging to unassigned Jira tickets or not authored by
> > assignee will ..."
> >
> > For the action "what should we do", iirc someone in this thread proposed
> > closing it via flinkbot. Or leaving without review and merge could be ok
> > before we implement automatic reply/reaction.
> >
> > Best,
> > tison.
> >
> >
> > Zili Chen  于2019年7月19日周五 上午11:44写道：
> >
> >> @Jack
> >>
> >> From https://flink.apache.org/contributing/contribute-code.html the
> >> community
> >> claims
> >>
> >> - Only start working on the implementation if there is consensus on the
> >> approach (e.g. you are assigned to the ticket)
> >>
> >> and accurately answer your question,
> >>
> >> - Pull requests belonging to unassigned Jira tickets will not be
> reviewed
> >> or merged by the community.
> >>
> >> Best,
> >> tison.
> >>
> >>
> >> Jark Wu  于2019年7月19日周五 上午11:28写道：
> >>
> >>> A quick question, what should we do if a developer creates a JIRA issue
> >>> and
> >>> then create a pull request at once without assigning?
> >>>
> >>>
> >>> Regards,
> >>> Jark
> >>>
> >>> On Thu, 18 Jul 2019 at 18:50, Zili Chen  wrote:
> >>>
> >>> > Checking the result, as a discovery, I found that one can
> >>> > still file a JIRA with "blocker" priority.
> >>> >
> >>> > IIRC someone in this thread once mentioned that
> >>> > "Don't allow contributors to set a blocker priority."
> >>> >
> >>> > Chesnay,
> >>> >
> >>> > Thanks for the clarification.
> >>> >
> >>> >
> >>> > Best,
> >>> > tison.
> >>> >
> >>> >
> >>> > Chesnay Schepler  于2019年7月18日周四 下午6:40写道：
> >>> >
> >>> > > We haven't wiped the set of contributors yet. Not sure if there's
> an
> >>> > > easy way to remove the permissions for all of them; someone from
> the
> >>> PMC
> >>> > > may have to bite the bullet and click 600 times in a row :)
> >>> > >
> >>> > > On 18/07/2019 12:32, Zili Chen wrote:
> >>> > > > Robert,
> >>> > > >
> >>> > > > Thanks for your effort. Rejecting contributor permission request
> >>> > > > with a nice message and pointing them to the announcement sounds
> >>> > > > reasonable. Just to be clear, we now have no person with
> >>> contributor
> >>> > > > role, right?
> >>> > > >
> >>> > > > Chesnay,
> >>> > > >
> >>> > > > https://flink.apache.org/contributing/contribute-code.html has
> >>> been
> >>> > > > updated and mentions that "Only committers can assign a Jira
> >>> ticket."
> >>> > > >
> >>> > > > I think the corresponding update has been done.
> >>> > > >
> >>> > > > Best,
> >>> > > > 

Re: [DISCUSS] Setup a bui...@flink.apache.org mailing list for travis builds

[Xu Forward]

+1 ,Thanks jark for the proposal.
Best
Forward

Jark Wu  于2019年7月20日周六 下午12:14写道：

> Hi all,
>
> As far as I know, currently, email notifications of Travis builds for
> master branch are sent to the commit author when a build was just broken or
> still is broken. And there is no email notifications for CRON builds.
>
> Recently, we are suffering from compile errors for scala-2.12 and java-9
> which are only ran in CRON jobs. So I'm figuring out a way to get
> notifications of CRON builds (or all builds) to quick fix compile errors
> and failed cron tests.
>
> After reaching out to @Chesnay Schepler  (thanks for
> the helping), I know that we are using a Slack channel to receive all
> failed build notifications. However, IMO, email notification might be a
> better way than Slack channel to encourage more people to pay attention on
> the builds.
>
> So I'm here to propose to setup a bui...@flink.apache.org mailing list for
> receiving build notifications. I also find that Beam has such mailing list
> too[1]. After we have such a mailing list, we can integrate it to travis
> according to the travis doc[2].
>
> What do you think? Do we need a formal vote for this?
>
> Best and thanks,
> Jark
>
> [1]: https://beam.apache.org/community/contact-us/
> [2]:
>
> https://docs.travis-ci.com/user/notifications/#configuring-email-notifications
>
> <
> https://docs.travis-ci.com/user/notifications/#configuring-email-notifications
> >
>
> <
> https://docs.travis-ci.com/user/notifications/#configuring-email-notifications
> >
>