Caching success/failure of a test is legitimate useful behavior. That said, I'm not sure how it interacts with Idea.
Kenn On Sun, Apr 15, 2018 at 12:23 AM Romain Manni-Bucau <[email protected]> wrote: > If you dont the diff is empty and gradle runs nothing, no? Saw it with > gradle 4.6 > > Le 15 avr. 2018 00:49, "Kenneth Knowles" <[email protected]> a écrit : > >> You shouldn't do :module:cleanTest. If that is necessary that's a major >> bug in the build. >> >> Kenn >> >> On Fri, Apr 13, 2018 at 11:46 PM Romain Manni-Bucau < >> [email protected]> wrote: >> >>> There is a fake module xxx_test which should have the right classpath >>> but since idea compilation is messed up you will still have to run >>> :module:cleanTest :module:test --tests org...MyTest.myMethod, even with >>> idea which leads to the same latency as the command line :( >>> >>> Le 13 avr. 2018 22:23, "Eugene Kirpichov" <[email protected]> a >>> écrit : >>> >>>> While we're talking about running tests in IntelliJ with Gradle... >>>> Anybody got advice on how to run a single NeedsRunner test in >>>> sdks-java-core, say, ParDoTest? With Maven, I used to just run the test in >>>> IntelliJ and specify "runners-direct-java" as the classpath; with Gradle, >>>> the best I got is to manually run the direct runner's needsRunnerTests task >>>> with specifying --tests=..., but it takes a long time, and IntelliJ treats >>>> that as just a gradle task run, not as a test run. >>>> >>>> On Fri, Apr 13, 2018 at 11:14 AM Reuven Lax <[email protected]> wrote: >>>> >>>>> Is there a Jira for this 3 second delay? Also you're initial complaint >>>>> was not about the 3 second delay, so it wasn't clear that's what you were >>>>> complaining about. >>>>> >>>>> Reuven >>>>> >>>>> On Fri, Apr 13, 2018 at 4:42 AM Romain Manni-Bucau < >>>>> [email protected]> wrote: >>>>> >>>>>> When you launch a test with gradle runner it launches gradle which >>>>>> makes loose 3s on a very fast computer and more on a slower (6 on my >>>>>> personal one which is already fast but not as much as my work one). We >>>>>> are >>>>>> 5 to see that regression at least. So there is a reason to not use the >>>>>> gradle runner if possible cause when you work and need to debug you are >>>>>> just stucked (that is why i switched back to mvn after 15mn, i was >>>>>> loosing >>>>>> to much time). >>>>>> >>>>>> Switching back to native idea test run would fix it but tests just >>>>>> dont work this way for me whatever setup i do :( - missing resources IIRC >>>>>> in idea out dir. >>>>>> >>>>>> Le 13 avr. 2018 00:07, "Reuven Lax" <[email protected]> a écrit : >>>>>> >>>>>> I also don't quite understand what your question is, and it appears >>>>>> like Dan spent considerable time trying to reproduce your issue. For the >>>>>> record, I have had no issues running tests via Gradle in IntelliJ for the >>>>>> past few weeks. >>>>>> >>>>>> Reuven >>>>>> >>>>>> On Thu, Apr 12, 2018 at 9:47 PM Daniel Oliveira < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Sorry Romain, I'm not quite sure what you're asking. Can you clarify? >>>>>>> >>>>>>> On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Well you are the only one to not have the drawbacks to use it so >>>>>>>> maybe dont do it? I know Luke is in holidays but anyone else with the >>>>>>>> knowledge of why we nees that noise compared to idea native >>>>>>>> tooling/flow? >>>>>>>> >>>>>>>> Le 12 avr. 2018 20:16, "Daniel Oliveira" <[email protected]> >>>>>>>> a écrit : >>>>>>>> >>>>>>>>> Ah, I did not. Thanks Romain. >>>>>>>>> >>>>>>>>> I tried it again, restarting in between, and still had no >>>>>>>>> differences. Since it seems like there's no reason not to use "Gradle >>>>>>>>> Test >>>>>>>>> Runner", I'll mention it in the contributor's guide. >>>>>>>>> >>>>>>>>> On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> @Daniel: did you restart in between? Otherwise it does nothing. >>>>>>>>>> One launches JunitCoreRunner from idea and the other a gradle >>>>>>>>>> command. >>>>>>>>>> >>>>>>>>>> Le 12 avr. 2018 19:24, "Daniel Oliveira" <[email protected]> >>>>>>>>>> a écrit : >>>>>>>>>> >>>>>>>>>>> I think it depends on what exactly switching to "Gradle Test >>>>>>>>>>> Runner" from "Platform Test Runner" does. I tried it out on my >>>>>>>>>>> machine and >>>>>>>>>>> they seem to act identically to each other. The IntelliJ >>>>>>>>>>> documentation says >>>>>>>>>>> it determines what API to use to run the tests >>>>>>>>>>> <https://www.jetbrains.com/help/idea/runner.html>, so maybe >>>>>>>>>>> it's usefulness depends on the user's machine, in which case a note >>>>>>>>>>> about >>>>>>>>>>> that would be useful. Something like: "If your IDE has trouble >>>>>>>>>>> running >>>>>>>>>>> tests via IDEA shortcuts, try the following steps: [...]" >>>>>>>>>>> >>>>>>>>>>> On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Daniel, actually I did run it with default IDEA JUnit test >>>>>>>>>>>> runner. Then, in “Settings > Build, Execution, Deployment > Build >>>>>>>>>>>> Tools > >>>>>>>>>>>> Gradle > Runner" I selected “Gradle Test Runner” in “Run tests >>>>>>>>>>>> using” >>>>>>>>>>>> selectbox and it works ok when I run my tests with IDEA shortcuts. >>>>>>>>>>>> So, >>>>>>>>>>>> probably, we should add this details on >>>>>>>>>>>> https://beam.apache.org/contribute/intellij/ too. >>>>>>>>>>>> What do you think? >>>>>>>>>>>> >>>>>>>>>>>> WBR, >>>>>>>>>>>> Alexey >>>>>>>>>>>> >>>>>>>>>>>> On 11 Apr 2018, at 21:17, Daniel Oliveira < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Alexey, are you referring to tests run with "./gradlew >>>>>>>>>>>> :beam-runners-direct-java:needsRunnerTests"? That command works >>>>>>>>>>>> fine for me >>>>>>>>>>>> in both versions of IDEA, but I believe the same tests fail if you >>>>>>>>>>>> run them >>>>>>>>>>>> directly through "./gradlew test". >>>>>>>>>>>> >>>>>>>>>>>> However, I am having issues with a bunch of validatesRunner >>>>>>>>>>>> tests, mostly be caused by >>>>>>>>>>>> :beam-runners-google-cloud-dataflow-java:validatesRunner. Not sure >>>>>>>>>>>> if it's >>>>>>>>>>>> because of a code change or a gradle config, I'll keep looking >>>>>>>>>>>> into it. >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I got tests running rrconfiguring gradle (which was setup for >>>>>>>>>>>>> another project but seems beam didnt like it) but latency is >>>>>>>>>>>>> still "high" >>>>>>>>>>>>> using gradle runner for tests (like Etienne said ~3s on an i7 >>>>>>>>>>>>> with 16G vs a >>>>>>>>>>>>> few ms with default idea test runner, would be great to solve >>>>>>>>>>>>> that). >>>>>>>>>>>>> >>>>>>>>>>>>> I also find the integration quite fishy cause configurations >>>>>>>>>>>>> are customs so idea is kind of forced to propose your modukes 3 >>>>>>>>>>>>> times at >>>>>>>>>>>>> least when you select the classpath (x_test being generally the >>>>>>>>>>>>> working >>>>>>>>>>>>> one). >>>>>>>>>>>>> >>>>>>>>>>>>> Also the false positive you get if you forget a cleanX is a >>>>>>>>>>>>> bit annoying, maybe we should force a clean for test or at least >>>>>>>>>>>>> when there >>>>>>>>>>>>> is a --tests to avoid gradle to not run it cause there is no diff. >>>>>>>>>>>>> >>>>>>>>>>>>> So it works but dev productivity is reduced a lot and it >>>>>>>>>>>>> became slow enough to think if you should do a contribution or >>>>>>>>>>>>> not - at >>>>>>>>>>>>> least for me. >>>>>>>>>>>>> >>>>>>>>>>>>> Le 11 avr. 2018 19:37, "Alexey Romanenko" < >>>>>>>>>>>>> [email protected]> a écrit : >>>>>>>>>>>>> >>>>>>>>>>>>>> I’ve managed to import a project as it’s described in >>>>>>>>>>>>>> documentation (starting from empty project) using Idea 2018 and >>>>>>>>>>>>>> run unit >>>>>>>>>>>>>> tests successfully. >>>>>>>>>>>>>> For some reasons, tests, that use DirectRunner to run a >>>>>>>>>>>>>> pipeline, were failed. >>>>>>>>>>>>>> >>>>>>>>>>>>>> WBR, >>>>>>>>>>>>>> Alexey >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 11 Apr 2018, at 19:01, Daniel Oliveira < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi everyone, I was the one who initially wrote the PR with >>>>>>>>>>>>>> Idea instructions >>>>>>>>>>>>>> <https://github.com/apache/beam-site/pull/414>. I was using >>>>>>>>>>>>>> 2017.3 as well while writing it so all the instructions were >>>>>>>>>>>>>> tested on that >>>>>>>>>>>>>> version. I'll try testing the instructions on 2018 to see if I >>>>>>>>>>>>>> can >>>>>>>>>>>>>> reproduce the issues people are having. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik <[email protected]> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I use 2017.3 and it has been reliable for me. I haven't >>>>>>>>>>>>>>> tried 2018 yet. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Any of you using the idea 2018? the import works for me but >>>>>>>>>>>>>>>> then it is >>>>>>>>>>>>>>>> not as smooth as it seems for you. I'm just trying to see >>>>>>>>>>>>>>>> if it is a >>>>>>>>>>>>>>>> procedure thing or a version issue. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Romain Manni-Bucau >>>>>>>>>>>>>>>> @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles <[email protected] >>>>>>>>>>>>>>>> >: >>>>>>>>>>>>>>>> > The only reason I did "empty project then add a module" >>>>>>>>>>>>>>>> procedure was to get >>>>>>>>>>>>>>>> > all the IntelliJ files outside the source tree. IIRC >>>>>>>>>>>>>>>> directly creating from >>>>>>>>>>>>>>>> > existing sources didn't give the necessary configuration >>>>>>>>>>>>>>>> options. If you >>>>>>>>>>>>>>>> > don't care about being able to `git clean` then you can >>>>>>>>>>>>>>>> do the shorter >>>>>>>>>>>>>>>> > version. Also the particular UI for creation might have >>>>>>>>>>>>>>>> improved since I >>>>>>>>>>>>>>>> > tried it. I'll do it again. >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > On the pom, since it is only generated for machine >>>>>>>>>>>>>>>> reading, why care if the >>>>>>>>>>>>>>>> > parent is inlined? I actually prefer to avoid coupling >>>>>>>>>>>>>>>> with things that you >>>>>>>>>>>>>>>> > have to go and look up. Using inheritance is OK for human >>>>>>>>>>>>>>>> edited poms >>>>>>>>>>>>>>>> > (actually IMO it is still a mistake) but it doesn't >>>>>>>>>>>>>>>> change the semantics of >>>>>>>>>>>>>>>> > a shipped pom if they are all immutable, which they >>>>>>>>>>>>>>>> should be. It is better >>>>>>>>>>>>>>>> > to have all the info right there. Is there an actually >>>>>>>>>>>>>>>> effective difference? >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > Kenn >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot < >>>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> Hi all, >>>>>>>>>>>>>>>> >> I just tested gradle environment from a fresh source >>>>>>>>>>>>>>>> clone with this >>>>>>>>>>>>>>>> >> procedure with just a tiny change: I used "new project >>>>>>>>>>>>>>>> from existing >>>>>>>>>>>>>>>> >> sources" rather than create empty project and then add >>>>>>>>>>>>>>>> module. >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> It works fine and junit runs from intellij also work. >>>>>>>>>>>>>>>> with gradle we pay >>>>>>>>>>>>>>>> >> a 2s delay (on my machine) before running the actual >>>>>>>>>>>>>>>> test for each run. This >>>>>>>>>>>>>>>> >> delay seems quite constant no matter the module: I have >>>>>>>>>>>>>>>> run all the tests at >>>>>>>>>>>>>>>> >> once in runner-core and a single test in another module >>>>>>>>>>>>>>>> with a similar >>>>>>>>>>>>>>>> >> delay. >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> I also tested a debug session from intellij and it runs >>>>>>>>>>>>>>>> fine also. >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> I'll do some more testing and keep you posted. >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> I still have some concerns about the potential >>>>>>>>>>>>>>>> difficulty for new >>>>>>>>>>>>>>>> >> contributors to have to learn gradle in addition to Beam >>>>>>>>>>>>>>>> itself comparing to >>>>>>>>>>>>>>>> >> maven which is more mainstream for java developers. That >>>>>>>>>>>>>>>> could be >>>>>>>>>>>>>>>> >> discouraging for ex for part-time contributors >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> Etienne >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> Le mardi 10 avril 2018 à 14:38 +0000, Lukasz Cwik a >>>>>>>>>>>>>>>> écrit : >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> beam-site PR/414 updates the instructions for using >>>>>>>>>>>>>>>> Intellij and how to >>>>>>>>>>>>>>>> >> import a module: >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> 1. Create an empty IntelliJ project outside of the Beam >>>>>>>>>>>>>>>> source tree. >>>>>>>>>>>>>>>> >> 2. Under Project Structure > Project, select a Project >>>>>>>>>>>>>>>> SDK. >>>>>>>>>>>>>>>> >> 3. Under Project Structure > Modules, click the + sign >>>>>>>>>>>>>>>> to add a module and >>>>>>>>>>>>>>>> >> select "Import Module". >>>>>>>>>>>>>>>> >> 1. Select the directory containing the Beam source >>>>>>>>>>>>>>>> tree. >>>>>>>>>>>>>>>> >> 2. Tick the "Import module from external model" >>>>>>>>>>>>>>>> button and select >>>>>>>>>>>>>>>> >> Gradle >>>>>>>>>>>>>>>> >> from the list. >>>>>>>>>>>>>>>> >> 3. Tick the following boxes. >>>>>>>>>>>>>>>> >> * Use auto-import >>>>>>>>>>>>>>>> >> * Create separate module per source set >>>>>>>>>>>>>>>> >> * Store generated project files externally >>>>>>>>>>>>>>>> >> * Use default gradle wrapper >>>>>>>>>>>>>>>> >> 4. Delegate build actions to Gradle by going to Settings >>>>>>>>>>>>>>>> > Build, >>>>>>>>>>>>>>>> >> Execution, >>>>>>>>>>>>>>>> >> Deployment > Build Tools > Gradle and checking >>>>>>>>>>>>>>>> "Delegate IDE build/run >>>>>>>>>>>>>>>> >> actions to gradle". >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré < >>>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>> >> wrote: >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> That's a very important issue for contribution. Up to >>>>>>>>>>>>>>>> now, I used Maven >>>>>>>>>>>>>>>> >> for setup IntelliJ (and it works just fine). If we >>>>>>>>>>>>>>>> remove the pom.xml, >>>>>>>>>>>>>>>> >> we have to support Eclipse and IntelliJ "smoothly". >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> Let me try in IntelliJ. >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> Regards >>>>>>>>>>>>>>>> >> JB >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> On 10/04/2018 15:21, Romain Manni-Bucau wrote: >>>>>>>>>>>>>>>> >> > You dont have issue due to the build setup with that >>>>>>>>>>>>>>>> option. I get: >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> >> > avr. 10, 2018 3:20:10 PM >>>>>>>>>>>>>>>> >> > org.apache.beam.runners.direct.DirectTransformExecutor >>>>>>>>>>>>>>>> run >>>>>>>>>>>>>>>> >> > GRAVE: Error occurred within >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a >>>>>>>>>>>>>>>> >> > com.google.common.util.concurrent.ExecutionError: >>>>>>>>>>>>>>>> >> > java.lang.NoClassDefFoundError: >>>>>>>>>>>>>>>> net/bytebuddy/NamingStrategy >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> >> > ? >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> >> > Romain Manni-Bucau >>>>>>>>>>>>>>>> >> > @rmannibucau | Blog | Old Blog | Github | LinkedIn | >>>>>>>>>>>>>>>> Book >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> >> > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik < >>>>>>>>>>>>>>>> [email protected]>: >>>>>>>>>>>>>>>> >> >> I have found that the simplest setup is to delegate >>>>>>>>>>>>>>>> the build/test >>>>>>>>>>>>>>>> >> >> actions >>>>>>>>>>>>>>>> >> >> to Gradle. This allows you to run unit tests very >>>>>>>>>>>>>>>> easily and since its >>>>>>>>>>>>>>>> >> >> in >>>>>>>>>>>>>>>> >> >> the same manner that Gradle would have, you know that >>>>>>>>>>>>>>>> if its passing it >>>>>>>>>>>>>>>> >> >> will >>>>>>>>>>>>>>>> >> >> pass on the command line and on Jenkins. Here is one >>>>>>>>>>>>>>>> site that >>>>>>>>>>>>>>>> >> >> discusses how >>>>>>>>>>>>>>>> >> >> to set this up: >>>>>>>>>>>>>>>> >> >> >>>>>>>>>>>>>>>> >> >> >>>>>>>>>>>>>>>> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html >>>>>>>>>>>>>>>> >> >> >>>>>>>>>>>>>>>> >> >> >>>>>>>>>>>>>>>> >> >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau >>>>>>>>>>>>>>>> >> >> <[email protected]> >>>>>>>>>>>>>>>> >> >> wrote: >>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>>>> >> >>> What's the plan to make idea supporting gradle on >>>>>>>>>>>>>>>> beam project? Do we >>>>>>>>>>>>>>>> >> >>> import the workaround mentionned in >>>>>>>>>>>>>>>> >> >>> https://youtrack.jetbrains.com/issue/IDEA-175172? >>>>>>>>>>>>>>>> >> >>> For the ones who didn't see this issue in action: >>>>>>>>>>>>>>>> idea will compile in >>>>>>>>>>>>>>>> >> >>> out/ instead of build/ and you will just miss all >>>>>>>>>>>>>>>> the resources you >>>>>>>>>>>>>>>> >> >>> need like some SPI registration which are used by >>>>>>>>>>>>>>>> all our registrar => >>>>>>>>>>>>>>>> >> >>> no way to run tests in idea without hacking the >>>>>>>>>>>>>>>> configuration quite >>>>>>>>>>>>>>>> >> >>> deeply :( >>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>>>> >> >>> Romain Manni-Bucau >>>>>>>>>>>>>>>> >> >>> @rmannibucau | Blog | Old Blog | Github | LinkedIn >>>>>>>>>>>>>>>> | Book >>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>>>> >> >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot < >>>>>>>>>>>>>>>> [email protected]>: >>>>>>>>>>>>>>>> >> >>>> As a gradle beginner, I could not agree more ! >>>>>>>>>>>>>>>> >> >>>> +1 >>>>>>>>>>>>>>>> >> >>>> Etienne >>>>>>>>>>>>>>>> >> >>>> Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste >>>>>>>>>>>>>>>> Onofré a écrit : >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Hi all, >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> I did multiple gradle build since last week and I >>>>>>>>>>>>>>>> would like to share >>>>>>>>>>>>>>>> >> >>>> one of my concern: it's about the communities. >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> If I think our users won't see any change for them >>>>>>>>>>>>>>>> due to Gradle >>>>>>>>>>>>>>>> >> >>>> build >>>>>>>>>>>>>>>> >> >>>> (I think that most of our users will still use >>>>>>>>>>>>>>>> Maven with artifacts >>>>>>>>>>>>>>>> >> >>>> provided by Gradle), I'm more concerned by the dev >>>>>>>>>>>>>>>> community and the >>>>>>>>>>>>>>>> >> >>>> contribution. >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Maven is well known and straight forward for a >>>>>>>>>>>>>>>> large part of >>>>>>>>>>>>>>>> >> >>>> potential >>>>>>>>>>>>>>>> >> >>>> contributors. I think we have to keep in mind that >>>>>>>>>>>>>>>> we still have to >>>>>>>>>>>>>>>> >> >>>> grow >>>>>>>>>>>>>>>> >> >>>> up our contributors community. >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Today, maybe I'm wrong, but I have the feeling that >>>>>>>>>>>>>>>> gradle build is >>>>>>>>>>>>>>>> >> >>>> not >>>>>>>>>>>>>>>> >> >>>> straight forward (build.gradle includes >>>>>>>>>>>>>>>> build_rules.gradle, gathering >>>>>>>>>>>>>>>> >> >>>> all taks all together). >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> I would like to add a task in the gradle >>>>>>>>>>>>>>>> "migration" process: >>>>>>>>>>>>>>>> >> >>>> simplify >>>>>>>>>>>>>>>> >> >>>> the gradle structure and files, and document this. >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> I know we already have a Jira about the >>>>>>>>>>>>>>>> documentation part, but I >>>>>>>>>>>>>>>> >> >>>> would >>>>>>>>>>>>>>>> >> >>>> like to "polish" and use a clean structure for the >>>>>>>>>>>>>>>> Gradle resources. >>>>>>>>>>>>>>>> >> >>>> As >>>>>>>>>>>>>>>> >> >>>> already quickly discussed, I think that having one >>>>>>>>>>>>>>>> gradle file per >>>>>>>>>>>>>>>> >> >>>> tasks >>>>>>>>>>>>>>>> >> >>>> in the .gradle directory would be helpful. >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> The goal is really to simplify the contribution. >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Do you agree if I add a Jira about "Gradle polish" ? >>>>>>>>>>>>>>>> >> >>>> Thoughts ? >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Regards >>>>>>>>>>>>>>>> >> >>>> JB >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> On 07/04/2018 04:52, Scott Wegner wrote: >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Here's an end-of-day update on migration work: >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> * Snapshot unsigned dailies and signed release >>>>>>>>>>>>>>>> builds are working >>>>>>>>>>>>>>>> >> >>>> (!!). >>>>>>>>>>>>>>>> >> >>>> PR/5048 [1] merges changes from Luke's branch >>>>>>>>>>>>>>>> >> >>>> * python precommit failing... will investigate >>>>>>>>>>>>>>>> python precommit >>>>>>>>>>>>>>>> >> >>>> Monday >>>>>>>>>>>>>>>> >> >>>> * All Precommits are gradle only >>>>>>>>>>>>>>>> >> >>>> * All Postcommits except performance tests and >>>>>>>>>>>>>>>> Java_JDK_Versions_Test >>>>>>>>>>>>>>>> >> >>>> use gradle (after PR/5047 [2] merged) >>>>>>>>>>>>>>>> >> >>>> * Nightly snapshot release using gradle is ready; >>>>>>>>>>>>>>>> needs PR/5048 to be >>>>>>>>>>>>>>>> >> >>>> merged before switching >>>>>>>>>>>>>>>> >> >>>> * ValidatesRunner_Spark failing consistently; >>>>>>>>>>>>>>>> investigating >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Thanks for another productive day of hacking. I'll >>>>>>>>>>>>>>>> pick up again on >>>>>>>>>>>>>>>> >> >>>> Monday. >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> [1] https://github.com/apache/beam/pull/5048 >>>>>>>>>>>>>>>> >> >>>> [2] https://github.com/apache/beam/pull/5047 >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau >>>>>>>>>>>>>>>> >> >>>> <[email protected] <mailto: >>>>>>>>>>>>>>>> [email protected]>> wrote: >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Why building a zip per runner which its stack >>>>>>>>>>>>>>>> and just pointing >>>>>>>>>>>>>>>> >> >>>> out >>>>>>>>>>>>>>>> >> >>>> on that zip and let beam lazy load the runner: >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> --runner=LazyRunner --lazyRunnerDir=... >>>>>>>>>>>>>>>> --lazyRunnerOptions=... >>>>>>>>>>>>>>>> >> >>>> (or >>>>>>>>>>>>>>>> >> >>>> the fromSystemProperties() if it gets merged a >>>>>>>>>>>>>>>> day ;)) >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Le 6 avr. 2018 20:21, "Kenneth Knowles" < >>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>> >> >>>> <mailto:[email protected]>> a écrit : >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> I'm working on finding a solution for >>>>>>>>>>>>>>>> launching the Nexmark >>>>>>>>>>>>>>>> >> >>>> suite with each runner. This doesn't have >>>>>>>>>>>>>>>> to be done via >>>>>>>>>>>>>>>> >> >>>> Gradle, >>>>>>>>>>>>>>>> >> >>>> but we anyhow need built artifacts that >>>>>>>>>>>>>>>> don't require user >>>>>>>>>>>>>>>> >> >>>> classpath intervention. >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> It looks to me like the examples are also >>>>>>>>>>>>>>>> missing this - >>>>>>>>>>>>>>>> >> >>>> they >>>>>>>>>>>>>>>> >> >>>> have separate configuration e.g. >>>>>>>>>>>>>>>> sparkRunnerPreCommit but >>>>>>>>>>>>>>>> >> >>>> that >>>>>>>>>>>>>>>> >> >>>> is overspecified compared to a free-form >>>>>>>>>>>>>>>> launching of a >>>>>>>>>>>>>>>> >> >>>> main() >>>>>>>>>>>>>>>> >> >>>> program with a runner profile. >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik >>>>>>>>>>>>>>>> >> >>>> <[email protected] >>>>>>>>>>>>>>>> >> >>>> <mailto:[email protected]>> wrote: >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Romain, are you talking about the >>>>>>>>>>>>>>>> profiles that exist as >>>>>>>>>>>>>>>> >> >>>> part of the archetype examples? >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> If so, then those still exist and >>>>>>>>>>>>>>>> haven't been changed. >>>>>>>>>>>>>>>> >> >>>> If >>>>>>>>>>>>>>>> >> >>>> not, can you provide a link to the >>>>>>>>>>>>>>>> profile in a pom file >>>>>>>>>>>>>>>> >> >>>> to >>>>>>>>>>>>>>>> >> >>>> be clearer? >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> On Fri, Apr 6, 2018 at 12:40 PM Romain >>>>>>>>>>>>>>>> Manni-Bucau >>>>>>>>>>>>>>>> >> >>>> <[email protected] <mailto: >>>>>>>>>>>>>>>> [email protected]>> >>>>>>>>>>>>>>>> >> >>>> wrote: >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Hi Scott, >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> is it right that 2 doesn't handle >>>>>>>>>>>>>>>> the hierachy >>>>>>>>>>>>>>>> >> >>>> anymore >>>>>>>>>>>>>>>> >> >>>> and that it doesn't handle >>>>>>>>>>>>>>>> profiles for runners as >>>>>>>>>>>>>>>> >> >>>> it is >>>>>>>>>>>>>>>> >> >>>> currently with maven? >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Romain Manni-Bucau >>>>>>>>>>>>>>>> >> >>>> @rmannibucau < >>>>>>>>>>>>>>>> https://twitter.com/rmannibucau> | >>>>>>>>>>>>>>>> >> >>>> Blog >>>>>>>>>>>>>>>> >> >>>> <https://rmannibucau.metawerx.net/> >>>>>>>>>>>>>>>> | Old Blog >>>>>>>>>>>>>>>> >> >>>> <http://rmannibucau.wordpress.com> >>>>>>>>>>>>>>>> | Github >>>>>>>>>>>>>>>> >> >>>> <https://github.com/rmannibucau> >>>>>>>>>>>>>>>> | LinkedIn >>>>>>>>>>>>>>>> >> >>>> < >>>>>>>>>>>>>>>> https://www.linkedin.com/in/rmannibucau> | Book >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> < >>>>>>>>>>>>>>>> https://www.packtpub.com/application-development/java-ee-8-high-performance >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> 2018-04-06 18:32 GMT+02:00 Scott >>>>>>>>>>>>>>>> Wegner >>>>>>>>>>>>>>>> >> >>>> <[email protected] <mailto: >>>>>>>>>>>>>>>> [email protected]>>: >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> I wanted to start a thread to >>>>>>>>>>>>>>>> summarize the >>>>>>>>>>>>>>>> >> >>>> current >>>>>>>>>>>>>>>> >> >>>> state of Gradle migration. >>>>>>>>>>>>>>>> We've made lots of >>>>>>>>>>>>>>>> >> >>>> good >>>>>>>>>>>>>>>> >> >>>> progress so far this week. >>>>>>>>>>>>>>>> Here's the status >>>>>>>>>>>>>>>> >> >>>> from >>>>>>>>>>>>>>>> >> >>>> what I can tell-- please add >>>>>>>>>>>>>>>> or correct anything >>>>>>>>>>>>>>>> >> >>>> I >>>>>>>>>>>>>>>> >> >>>> missed: >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> * Release artifacts can be >>>>>>>>>>>>>>>> built and published >>>>>>>>>>>>>>>> >> >>>> for >>>>>>>>>>>>>>>> >> >>>> Snapshot and officlal releases >>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>> >> >>>> * Gradle-generated releases >>>>>>>>>>>>>>>> have been validated >>>>>>>>>>>>>>>> >> >>>> with >>>>>>>>>>>>>>>> >> >>>> the the Apache Beam archetype >>>>>>>>>>>>>>>> generation >>>>>>>>>>>>>>>> >> >>>> quickstart; >>>>>>>>>>>>>>>> >> >>>> still needs additional >>>>>>>>>>>>>>>> validation. >>>>>>>>>>>>>>>> >> >>>> * Generated release pom files >>>>>>>>>>>>>>>> have correct >>>>>>>>>>>>>>>> >> >>>> project >>>>>>>>>>>>>>>> >> >>>> metadata [2] >>>>>>>>>>>>>>>> >> >>>> * The python pre-commits are >>>>>>>>>>>>>>>> now working in >>>>>>>>>>>>>>>> >> >>>> Gradle >>>>>>>>>>>>>>>> >> >>>> [3] >>>>>>>>>>>>>>>> >> >>>> * Ismaël has started a >>>>>>>>>>>>>>>> collaborative doc of >>>>>>>>>>>>>>>> >> >>>> Gradle >>>>>>>>>>>>>>>> >> >>>> tips [4] as we all learn the >>>>>>>>>>>>>>>> new system-- please >>>>>>>>>>>>>>>> >> >>>> add >>>>>>>>>>>>>>>> >> >>>> your own. This will eventually >>>>>>>>>>>>>>>> feed into >>>>>>>>>>>>>>>> >> >>>> official >>>>>>>>>>>>>>>> >> >>>> documentation on the website. >>>>>>>>>>>>>>>> >> >>>> * Łukasz Gajowy is working on >>>>>>>>>>>>>>>> migrating >>>>>>>>>>>>>>>> >> >>>> performance >>>>>>>>>>>>>>>> >> >>>> testing framework [5] >>>>>>>>>>>>>>>> >> >>>> * Daniel is working on >>>>>>>>>>>>>>>> updating documentation to >>>>>>>>>>>>>>>> >> >>>> refer to Gradle instead of >>>>>>>>>>>>>>>> maven >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> If I missed anything, please >>>>>>>>>>>>>>>> add it to this >>>>>>>>>>>>>>>> >> >>>> thread. >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> The general roadmap we're >>>>>>>>>>>>>>>> working towards is: >>>>>>>>>>>>>>>> >> >>>> (a) Publish release artifacts >>>>>>>>>>>>>>>> with Gradle >>>>>>>>>>>>>>>> >> >>>> (SNAPSHOT >>>>>>>>>>>>>>>> >> >>>> and signed releases) >>>>>>>>>>>>>>>> >> >>>> (b) Postcommits migrated to >>>>>>>>>>>>>>>> Gradle >>>>>>>>>>>>>>>> >> >>>> (c) Migrate documentation from >>>>>>>>>>>>>>>> maven to Gradle >>>>>>>>>>>>>>>> >> >>>> (d) Migrate perfkit suites to >>>>>>>>>>>>>>>> use Gradle >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> For those of you that are >>>>>>>>>>>>>>>> hacking: thanks for >>>>>>>>>>>>>>>> >> >>>> your >>>>>>>>>>>>>>>> >> >>>> help so far! Progress is being >>>>>>>>>>>>>>>> roughly tracked >>>>>>>>>>>>>>>> >> >>>> on >>>>>>>>>>>>>>>> >> >>>> the Kanban [6]; please make >>>>>>>>>>>>>>>> sure the issues >>>>>>>>>>>>>>>> >> >>>> assigned >>>>>>>>>>>>>>>> >> >>>> to you are up-to-date. Many of >>>>>>>>>>>>>>>> the changes are >>>>>>>>>>>>>>>> >> >>>> staged on lukecwik's local >>>>>>>>>>>>>>>> branch [7]; we'll >>>>>>>>>>>>>>>> >> >>>> work on >>>>>>>>>>>>>>>> >> >>>> merging them back soon. >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> [1] >>>>>>>>>>>>>>>> >> >>>> https://github.com/lukecwik/incubator-beam/pull/7 >>>>>>>>>>>>>>>> >> >>>> [2] >>>>>>>>>>>>>>>> >> >>>> https://github.com/lukecwik/incubator-beam/pull/3 >>>>>>>>>>>>>>>> >> >>>> [3] >>>>>>>>>>>>>>>> https://github.com/apache/beam/pull/5032 >>>>>>>>>>>>>>>> >> >>>> [4] >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit >>>>>>>>>>>>>>>> >> >>>> [5] >>>>>>>>>>>>>>>> https://github.com/apache/beam/pull/5003 >>>>>>>>>>>>>>>> >> >>>> [6] >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242 >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> [7] >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> https://github.com/lukecwik/incubator-beam/tree/gradle >>>>>>>>>>>>>>>> >> >>>> -- >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> Got feedback? >>>>>>>>>>>>>>>> http://go/swegner-feedback >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>
