Re: Gradle migration fixit: April 3

2018-03-29 Thread Romain Manni-Bucau
Yep - sorry if it was unclear. I know linux distro often do it (never understood why though). Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | Lin

Re: Gradle migration fixit: April 3

2018-03-29 Thread Reuven Lax
On Thu, Mar 29, 2018 at 10:28 PM Romain Manni-Bucau wrote: > It was more about the drop of poms (same case as you for dataflow). > Ah - you're worried that some external users are building directly from the poms rather than using the published artifact. I think this is a valid concern, and I ag

Re: Gradle migration fixit: April 3

2018-03-29 Thread Romain Manni-Bucau
It was more about the drop of poms (same case as you for dataflow). On that there is a missing but highly important task: gradle to mvn descriptors. All the one I saw were corrupted poms so we must take care of that as part of the release work (I can work on it on the 3rd if you want). Le 29 mars

Re: Adding a StepMetadataRegistry for Python SDK

2018-03-29 Thread Robert Bradshaw
If I understand correctly, this is something runner-specific that would live solely on the runner side (i.e. over the Fn API we'd still have a single name for operations rather than pushing this complexity into that protocol as well which I'd really like to avoid, right?) If that's the case, then i

Re: [PROPOSAL] Python 3 support

2018-03-29 Thread Robert Bradshaw
Thank you, Robbie, for your offer to help with contribution here. I read over your doc and the one thing I'd like to add is that this work is very parallelizable, but if we have enough people looking at it we'll want some way to coordinate so as to not overlap work (or just waste time discovering w

Re: Common model for runners

2018-03-29 Thread Robert Bradshaw
This is all present in the master branch, the WIP nature is that as we build atop it, we may discover that changes to the proto representation are necessary. On Sat, Mar 24, 2018 at 6:26 AM zlgonzalez wrote: > Thanks Robert. I'll take a look at this for now. > > What WIP branch is involved with

Re: Gradle migration fixit: April 3

2018-03-29 Thread Reuven Lax
I don't mind notifying users@, but this does seem more interesting for dev@. We will continue to publish Maven artifacts from our Gradle build, so users are still free to use either Maven or Gradle. That being said, if this is interesting to users@ we can notify them as well. Reuven On Thu, Mar

Re: Gradle migration fixit: April 3

2018-03-29 Thread Romain Manni-Bucau
Le 29 mars 2018 21:20, "Reuven Lax" a écrit : On Thu, Mar 29, 2018 at 12:17 PM Romain Manni-Bucau wrote: > > > Le 29 mars 2018 20:35, "Reuven Lax" a écrit : > > 1. As Luke already mentioned, we should first have a subgoal of the Gradle > jenkins jobs being equivalent to the Maven jobs. Hopef

Re: Gradle migration fixit: April 3

2018-03-29 Thread Reuven Lax
On Thu, Mar 29, 2018 at 12:17 PM Romain Manni-Bucau wrote: > > > Le 29 mars 2018 20:35, "Reuven Lax" a écrit : > > 1. As Luke already mentioned, we should first have a subgoal of the Gradle > jenkins jobs being equivalent to the Maven jobs. Hopefully toward the end > of the day, we'll make this

Re: Gradle migration fixit: April 3

2018-03-29 Thread Romain Manni-Bucau
Le 29 mars 2018 20:35, "Reuven Lax" a écrit : 1. As Luke already mentioned, we should first have a subgoal of the Gradle jenkins jobs being equivalent to the Maven jobs. Hopefully toward the end of the day, we'll make this change. 2. Let's see how much progress we make on the third. There is a s

Re: Gradle migration fixit: April 3

2018-03-29 Thread Reuven Lax
1. As Luke already mentioned, we should first have a subgoal of the Gradle jenkins jobs being equivalent to the Maven jobs. Hopefully toward the end of the day, we'll make this change. 2. Let's see how much progress we make on the third. There is a side problem we have here at Google - we have an

Re: Gradle migration fixit: April 3

2018-03-29 Thread Romain Manni-Bucau
Le 29 mars 2018 18:54, "Robert Bradshaw" a écrit : On Thu, Mar 29, 2018 at 9:45 AM Lukasz Cwik wrote: > > On Thu, Mar 29, 2018 at 5:20 AM Romain Manni-Bucau > wrote: > >> Hi Reuven, a few questions: >> >> 1. any inputs on how we can work on the jenkins part? Do we test it live >> wiht "fake" P

Re: Gradle migration fixit: April 3

2018-03-29 Thread Lukasz Cwik
That is correct. All jenkins job changes will need to be serialized so active communication in Slack would be a good way for people to know that they are running the seed job. On Thu, Mar 29, 2018 at 9:54 AM Robert Bradshaw wrote: > On Thu, Mar 29, 2018 at 9:45 AM Lukasz Cwik wrote: > >> >> On

Re: Gradle migration fixit: April 3

2018-03-29 Thread Romain Manni-Bucau
Le 29 mars 2018 18:44, "Lukasz Cwik" a écrit : On Thu, Mar 29, 2018 at 5:20 AM Romain Manni-Bucau wrote: > Hi Reuven, a few questions: > > 1. any inputs on how we can work on the jenkins part? Do we test it live > wiht "fake" PRs? > The jenkins jobs are configured via this seed job ( https:/

Re: Gradle migration fixit: April 3

2018-03-29 Thread Robert Bradshaw
On Thu, Mar 29, 2018 at 9:45 AM Lukasz Cwik wrote: > > On Thu, Mar 29, 2018 at 5:20 AM Romain Manni-Bucau > wrote: > >> Hi Reuven, a few questions: >> >> 1. any inputs on how we can work on the jenkins part? Do we test it live >> wiht "fake" PRs? >> > > The jenkins jobs are configured via this s

Re: Gradle migration fixit: April 3

2018-03-29 Thread Lukasz Cwik
On Thu, Mar 29, 2018 at 5:20 AM Romain Manni-Bucau wrote: > Hi Reuven, a few questions: > > 1. any inputs on how we can work on the jenkins part? Do we test it live > wiht "fake" PRs? > The jenkins jobs are configured via this seed job ( https://github.com/apache/beam/blob/master/.test-infra/jen

Re: [PROPOSITION] schedule some sanity tests on a daily basis

2018-03-29 Thread Etienne Chauchot
Hi all, As discussed here I proposed a PR (https://github.com/apache/beam/pull/4976) to schedule nexmark runs as post commit tests. the post commits run * NM on direct runner in batch mode * NM on direct runner in streaming mode * NM on flink runner in batch mode * NM on flink runner in stream

Re: Gradle migration fixit: April 3

2018-03-29 Thread Romain Manni-Bucau
Hi Reuven, a few questions: 1. any inputs on how we can work on the jenkins part? Do we test it live wiht "fake" PRs? 2. What's the rational to not start by deleting the poms? Sounds like it will be a day working on gradle and on the 4th we'll be back on maven Romain Manni-Bucau @rmannibucau

Build failed in Jenkins: beam_Release_NightlySnapshot #728

2018-03-29 Thread Apache Jenkins Server
See Changes: [mairbek] [BEAM-3932] Adds handling of array null values to MutationSizeEstimator [lcwik] [BEAM-3326] Abstract away closing the inbound receiver, waiting for the [aromanenko.dev] [BEAM-

Re: Build breaks on examples on jenkins with dataflow runner

2018-03-29 Thread Etienne Chauchot
Thanks Yes, we changed the API. I think we need to rebuild dataflow worker like we did there (1) for example [1] https://issues.apache.org/jira/browse/BEAM-2012 Etienne Le mercredi 28 mars 2018 à 20:32 +, Alan Myrvold a écrit : > Thomas logged this issue as  > https://issues.apache.org/jira/pr