There is a fake module xxx_test which should have the right classpath but since idea compilation is messed up you will still have to run :module:cleanTest :module:test --tests org...MyTest.myMethod, even with idea which leads to the same latency as the command line :(
Le 13 avr. 2018 22:23, "Eugene Kirpichov" <[email protected]> a écrit : > While we're talking about running tests in IntelliJ with Gradle... > Anybody got advice on how to run a single NeedsRunner test in > sdks-java-core, say, ParDoTest? With Maven, I used to just run the test in > IntelliJ and specify "runners-direct-java" as the classpath; with Gradle, > the best I got is to manually run the direct runner's needsRunnerTests task > with specifying --tests=..., but it takes a long time, and IntelliJ treats > that as just a gradle task run, not as a test run. > > On Fri, Apr 13, 2018 at 11:14 AM Reuven Lax <[email protected]> wrote: > >> Is there a Jira for this 3 second delay? Also you're initial complaint >> was not about the 3 second delay, so it wasn't clear that's what you were >> complaining about. >> >> Reuven >> >> On Fri, Apr 13, 2018 at 4:42 AM Romain Manni-Bucau <[email protected]> >> wrote: >> >>> When you launch a test with gradle runner it launches gradle which makes >>> loose 3s on a very fast computer and more on a slower (6 on my personal one >>> which is already fast but not as much as my work one). We are 5 to see that >>> regression at least. So there is a reason to not use the gradle runner if >>> possible cause when you work and need to debug you are just stucked (that >>> is why i switched back to mvn after 15mn, i was loosing to much time). >>> >>> Switching back to native idea test run would fix it but tests just dont >>> work this way for me whatever setup i do :( - missing resources IIRC in >>> idea out dir. >>> >>> Le 13 avr. 2018 00:07, "Reuven Lax" <[email protected]> a écrit : >>> >>> I also don't quite understand what your question is, and it appears like >>> Dan spent considerable time trying to reproduce your issue. For the record, >>> I have had no issues running tests via Gradle in IntelliJ for the past few >>> weeks. >>> >>> Reuven >>> >>> On Thu, Apr 12, 2018 at 9:47 PM Daniel Oliveira <[email protected]> >>> wrote: >>> >>>> Sorry Romain, I'm not quite sure what you're asking. Can you clarify? >>>> >>>> On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau < >>>> [email protected]> wrote: >>>> >>>>> Well you are the only one to not have the drawbacks to use it so maybe >>>>> dont do it? I know Luke is in holidays but anyone else with the knowledge >>>>> of why we nees that noise compared to idea native tooling/flow? >>>>> >>>>> Le 12 avr. 2018 20:16, "Daniel Oliveira" <[email protected]> a >>>>> écrit : >>>>> >>>>>> Ah, I did not. Thanks Romain. >>>>>> >>>>>> I tried it again, restarting in between, and still had no >>>>>> differences. Since it seems like there's no reason not to use "Gradle >>>>>> Test >>>>>> Runner", I'll mention it in the contributor's guide. >>>>>> >>>>>> On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> @Daniel: did you restart in between? Otherwise it does nothing. One >>>>>>> launches JunitCoreRunner from idea and the other a gradle command. >>>>>>> >>>>>>> Le 12 avr. 2018 19:24, "Daniel Oliveira" <[email protected]> a >>>>>>> écrit : >>>>>>> >>>>>>>> I think it depends on what exactly switching to "Gradle Test >>>>>>>> Runner" from "Platform Test Runner" does. I tried it out on my machine >>>>>>>> and >>>>>>>> they seem to act identically to each other. The IntelliJ documentation >>>>>>>> says >>>>>>>> it determines what API to use to run the tests >>>>>>>> <https://www.jetbrains.com/help/idea/runner.html>, so maybe it's >>>>>>>> usefulness depends on the user's machine, in which case a note about >>>>>>>> that >>>>>>>> would be useful. Something like: "If your IDE has trouble running >>>>>>>> tests via >>>>>>>> IDEA shortcuts, try the following steps: [...]" >>>>>>>> >>>>>>>> On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Daniel, actually I did run it with default IDEA JUnit test runner. >>>>>>>>> Then, in “Settings > Build, Execution, Deployment > Build Tools > >>>>>>>>> Gradle > >>>>>>>>> Runner" I selected “Gradle Test Runner” in “Run tests using” >>>>>>>>> selectbox and >>>>>>>>> it works ok when I run my tests with IDEA shortcuts. So, probably, we >>>>>>>>> should add this details on https://beam.apache.org/ >>>>>>>>> contribute/intellij/ too. >>>>>>>>> What do you think? >>>>>>>>> >>>>>>>>> WBR, >>>>>>>>> Alexey >>>>>>>>> >>>>>>>>> On 11 Apr 2018, at 21:17, Daniel Oliveira <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Alexey, are you referring to tests run with "./gradlew >>>>>>>>> :beam-runners-direct-java:needsRunnerTests"? That command works >>>>>>>>> fine for me in both versions of IDEA, but I believe the same tests >>>>>>>>> fail if >>>>>>>>> you run them directly through "./gradlew test". >>>>>>>>> >>>>>>>>> However, I am having issues with a bunch of validatesRunner tests, >>>>>>>>> mostly be caused by >>>>>>>>> :beam-runners-google-cloud-dataflow-java:validatesRunner. >>>>>>>>> Not sure if it's because of a code change or a gradle config, I'll >>>>>>>>> keep >>>>>>>>> looking into it. >>>>>>>>> >>>>>>>>> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> I got tests running rrconfiguring gradle (which was setup for >>>>>>>>>> another project but seems beam didnt like it) but latency is still >>>>>>>>>> "high" >>>>>>>>>> using gradle runner for tests (like Etienne said ~3s on an i7 with >>>>>>>>>> 16G vs a >>>>>>>>>> few ms with default idea test runner, would be great to solve that). >>>>>>>>>> >>>>>>>>>> I also find the integration quite fishy cause configurations are >>>>>>>>>> customs so idea is kind of forced to propose your modukes 3 times at >>>>>>>>>> least >>>>>>>>>> when you select the classpath (x_test being generally the working >>>>>>>>>> one). >>>>>>>>>> >>>>>>>>>> Also the false positive you get if you forget a cleanX is a bit >>>>>>>>>> annoying, maybe we should force a clean for test or at least when >>>>>>>>>> there is >>>>>>>>>> a --tests to avoid gradle to not run it cause there is no diff. >>>>>>>>>> >>>>>>>>>> So it works but dev productivity is reduced a lot and it became >>>>>>>>>> slow enough to think if you should do a contribution or not - at >>>>>>>>>> least for >>>>>>>>>> me. >>>>>>>>>> >>>>>>>>>> Le 11 avr. 2018 19:37, "Alexey Romanenko" < >>>>>>>>>> [email protected]> a écrit : >>>>>>>>>> >>>>>>>>>>> I’ve managed to import a project as it’s described in >>>>>>>>>>> documentation (starting from empty project) using Idea 2018 and run >>>>>>>>>>> unit >>>>>>>>>>> tests successfully. >>>>>>>>>>> For some reasons, tests, that use DirectRunner to run a >>>>>>>>>>> pipeline, were failed. >>>>>>>>>>> >>>>>>>>>>> WBR, >>>>>>>>>>> Alexey >>>>>>>>>>> >>>>>>>>>>> On 11 Apr 2018, at 19:01, Daniel Oliveira < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi everyone, I was the one who initially wrote the PR with Idea >>>>>>>>>>> instructions <https://github.com/apache/beam-site/pull/414>. I >>>>>>>>>>> was using 2017.3 as well while writing it so all the instructions >>>>>>>>>>> were >>>>>>>>>>> tested on that version. I'll try testing the instructions on 2018 >>>>>>>>>>> to see if >>>>>>>>>>> I can reproduce the issues people are having. >>>>>>>>>>> >>>>>>>>>>> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> I use 2017.3 and it has been reliable for me. I haven't tried >>>>>>>>>>>> 2018 yet. >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Any of you using the idea 2018? the import works for me but >>>>>>>>>>>>> then it is >>>>>>>>>>>>> not as smooth as it seems for you. I'm just trying to see if >>>>>>>>>>>>> it is a >>>>>>>>>>>>> procedure thing or a version issue. >>>>>>>>>>>>> >>>>>>>>>>>>> Romain Manni-Bucau >>>>>>>>>>>>> @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles <[email protected]>: >>>>>>>>>>>>> > The only reason I did "empty project then add a module" >>>>>>>>>>>>> procedure was to get >>>>>>>>>>>>> > all the IntelliJ files outside the source tree. IIRC >>>>>>>>>>>>> directly creating from >>>>>>>>>>>>> > existing sources didn't give the necessary configuration >>>>>>>>>>>>> options. If you >>>>>>>>>>>>> > don't care about being able to `git clean` then you can do >>>>>>>>>>>>> the shorter >>>>>>>>>>>>> > version. Also the particular UI for creation might have >>>>>>>>>>>>> improved since I >>>>>>>>>>>>> > tried it. I'll do it again. >>>>>>>>>>>>> > >>>>>>>>>>>>> > On the pom, since it is only generated for machine reading, >>>>>>>>>>>>> why care if the >>>>>>>>>>>>> > parent is inlined? I actually prefer to avoid coupling with >>>>>>>>>>>>> things that you >>>>>>>>>>>>> > have to go and look up. Using inheritance is OK for human >>>>>>>>>>>>> edited poms >>>>>>>>>>>>> > (actually IMO it is still a mistake) but it doesn't change >>>>>>>>>>>>> the semantics of >>>>>>>>>>>>> > a shipped pom if they are all immutable, which they should >>>>>>>>>>>>> be. It is better >>>>>>>>>>>>> > to have all the info right there. Is there an actually >>>>>>>>>>>>> effective difference? >>>>>>>>>>>>> > >>>>>>>>>>>>> > Kenn >>>>>>>>>>>>> > >>>>>>>>>>>>> > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot < >>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>> > wrote: >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> Hi all, >>>>>>>>>>>>> >> I just tested gradle environment from a fresh source clone >>>>>>>>>>>>> with this >>>>>>>>>>>>> >> procedure with just a tiny change: I used "new project from >>>>>>>>>>>>> existing >>>>>>>>>>>>> >> sources" rather than create empty project and then add >>>>>>>>>>>>> module. >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> It works fine and junit runs from intellij also work. with >>>>>>>>>>>>> gradle we pay >>>>>>>>>>>>> >> a 2s delay (on my machine) before running the actual test >>>>>>>>>>>>> for each run. This >>>>>>>>>>>>> >> delay seems quite constant no matter the module: I have run >>>>>>>>>>>>> all the tests at >>>>>>>>>>>>> >> once in runner-core and a single test in another module >>>>>>>>>>>>> with a similar >>>>>>>>>>>>> >> delay. >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> I also tested a debug session from intellij and it runs >>>>>>>>>>>>> fine also. >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> I'll do some more testing and keep you posted. >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> I still have some concerns about the potential difficulty >>>>>>>>>>>>> for new >>>>>>>>>>>>> >> contributors to have to learn gradle in addition to Beam >>>>>>>>>>>>> itself comparing to >>>>>>>>>>>>> >> maven which is more mainstream for java developers. That >>>>>>>>>>>>> could be >>>>>>>>>>>>> >> discouraging for ex for part-time contributors >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> Etienne >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> Le mardi 10 avril 2018 à 14:38 +0000, Lukasz Cwik a écrit : >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> beam-site PR/414 updates the instructions for using >>>>>>>>>>>>> Intellij and how to >>>>>>>>>>>>> >> import a module: >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> 1. Create an empty IntelliJ project outside of the Beam >>>>>>>>>>>>> source tree. >>>>>>>>>>>>> >> 2. Under Project Structure > Project, select a Project SDK. >>>>>>>>>>>>> >> 3. Under Project Structure > Modules, click the + sign to >>>>>>>>>>>>> add a module and >>>>>>>>>>>>> >> select "Import Module". >>>>>>>>>>>>> >> 1. Select the directory containing the Beam source tree. >>>>>>>>>>>>> >> 2. Tick the "Import module from external model" button >>>>>>>>>>>>> and select >>>>>>>>>>>>> >> Gradle >>>>>>>>>>>>> >> from the list. >>>>>>>>>>>>> >> 3. Tick the following boxes. >>>>>>>>>>>>> >> * Use auto-import >>>>>>>>>>>>> >> * Create separate module per source set >>>>>>>>>>>>> >> * Store generated project files externally >>>>>>>>>>>>> >> * Use default gradle wrapper >>>>>>>>>>>>> >> 4. Delegate build actions to Gradle by going to Settings > >>>>>>>>>>>>> Build, >>>>>>>>>>>>> >> Execution, >>>>>>>>>>>>> >> Deployment > Build Tools > Gradle and checking "Delegate >>>>>>>>>>>>> IDE build/run >>>>>>>>>>>>> >> actions to gradle". >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré < >>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>> >> wrote: >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> That's a very important issue for contribution. Up to now, >>>>>>>>>>>>> I used Maven >>>>>>>>>>>>> >> for setup IntelliJ (and it works just fine). If we remove >>>>>>>>>>>>> the pom.xml, >>>>>>>>>>>>> >> we have to support Eclipse and IntelliJ "smoothly". >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> Let me try in IntelliJ. >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> Regards >>>>>>>>>>>>> >> JB >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> On 10/04/2018 15:21, Romain Manni-Bucau wrote: >>>>>>>>>>>>> >> > You dont have issue due to the build setup with that >>>>>>>>>>>>> option. I get: >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > avr. 10, 2018 3:20:10 PM >>>>>>>>>>>>> >> > org.apache.beam.runners.direct.DirectTransformExecutor >>>>>>>>>>>>> run >>>>>>>>>>>>> >> > GRAVE: Error occurred within >>>>>>>>>>>>> >> > org.apache.beam.runners.direct.DirectTransformExecutor@ >>>>>>>>>>>>> 66761b7a >>>>>>>>>>>>> >> > com.google.common.util.concurrent.ExecutionError: >>>>>>>>>>>>> >> > java.lang.NoClassDefFoundError: >>>>>>>>>>>>> net/bytebuddy/NamingStrategy >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > ? >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > Romain Manni-Bucau >>>>>>>>>>>>> >> > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik <[email protected] >>>>>>>>>>>>> >: >>>>>>>>>>>>> >> >> I have found that the simplest setup is to delegate the >>>>>>>>>>>>> build/test >>>>>>>>>>>>> >> >> actions >>>>>>>>>>>>> >> >> to Gradle. This allows you to run unit tests very easily >>>>>>>>>>>>> and since its >>>>>>>>>>>>> >> >> in >>>>>>>>>>>>> >> >> the same manner that Gradle would have, you know that if >>>>>>>>>>>>> its passing it >>>>>>>>>>>>> >> >> will >>>>>>>>>>>>> >> >> pass on the command line and on Jenkins. Here is one >>>>>>>>>>>>> site that >>>>>>>>>>>>> >> >> discusses how >>>>>>>>>>>>> >> >> to set this up: >>>>>>>>>>>>> >> >> >>>>>>>>>>>>> >> >> http://mrhaki.blogspot.com/2016/11/gradle-goodness- >>>>>>>>>>>>> delegate-build-and-run.html >>>>>>>>>>>>> >> >> >>>>>>>>>>>>> >> >> >>>>>>>>>>>>> >> >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau >>>>>>>>>>>>> >> >> <[email protected]> >>>>>>>>>>>>> >> >> wrote: >>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> >> >>> What's the plan to make idea supporting gradle on beam >>>>>>>>>>>>> project? Do we >>>>>>>>>>>>> >> >>> import the workaround mentionned in >>>>>>>>>>>>> >> >>> https://youtrack.jetbrains.com/issue/IDEA-175172? >>>>>>>>>>>>> >> >>> For the ones who didn't see this issue in action: idea >>>>>>>>>>>>> will compile in >>>>>>>>>>>>> >> >>> out/ instead of build/ and you will just miss all the >>>>>>>>>>>>> resources you >>>>>>>>>>>>> >> >>> need like some SPI registration which are used by all >>>>>>>>>>>>> our registrar => >>>>>>>>>>>>> >> >>> no way to run tests in idea without hacking the >>>>>>>>>>>>> configuration quite >>>>>>>>>>>>> >> >>> deeply :( >>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> >> >>> Romain Manni-Bucau >>>>>>>>>>>>> >> >>> @rmannibucau | Blog | Old Blog | Github | LinkedIn | >>>>>>>>>>>>> Book >>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> >> >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot < >>>>>>>>>>>>> [email protected]>: >>>>>>>>>>>>> >> >>>> As a gradle beginner, I could not agree more ! >>>>>>>>>>>>> >> >>>> +1 >>>>>>>>>>>>> >> >>>> Etienne >>>>>>>>>>>>> >> >>>> Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste >>>>>>>>>>>>> Onofré a écrit : >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Hi all, >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> I did multiple gradle build since last week and I >>>>>>>>>>>>> would like to share >>>>>>>>>>>>> >> >>>> one of my concern: it's about the communities. >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> If I think our users won't see any change for them due >>>>>>>>>>>>> to Gradle >>>>>>>>>>>>> >> >>>> build >>>>>>>>>>>>> >> >>>> (I think that most of our users will still use Maven >>>>>>>>>>>>> with artifacts >>>>>>>>>>>>> >> >>>> provided by Gradle), I'm more concerned by the dev >>>>>>>>>>>>> community and the >>>>>>>>>>>>> >> >>>> contribution. >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Maven is well known and straight forward for a large >>>>>>>>>>>>> part of >>>>>>>>>>>>> >> >>>> potential >>>>>>>>>>>>> >> >>>> contributors. I think we have to keep in mind that we >>>>>>>>>>>>> still have to >>>>>>>>>>>>> >> >>>> grow >>>>>>>>>>>>> >> >>>> up our contributors community. >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Today, maybe I'm wrong, but I have the feeling that >>>>>>>>>>>>> gradle build is >>>>>>>>>>>>> >> >>>> not >>>>>>>>>>>>> >> >>>> straight forward (build.gradle includes >>>>>>>>>>>>> build_rules.gradle, gathering >>>>>>>>>>>>> >> >>>> all taks all together). >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> I would like to add a task in the gradle "migration" >>>>>>>>>>>>> process: >>>>>>>>>>>>> >> >>>> simplify >>>>>>>>>>>>> >> >>>> the gradle structure and files, and document this. >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> I know we already have a Jira about the documentation >>>>>>>>>>>>> part, but I >>>>>>>>>>>>> >> >>>> would >>>>>>>>>>>>> >> >>>> like to "polish" and use a clean structure for the >>>>>>>>>>>>> Gradle resources. >>>>>>>>>>>>> >> >>>> As >>>>>>>>>>>>> >> >>>> already quickly discussed, I think that having one >>>>>>>>>>>>> gradle file per >>>>>>>>>>>>> >> >>>> tasks >>>>>>>>>>>>> >> >>>> in the .gradle directory would be helpful. >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> The goal is really to simplify the contribution. >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Do you agree if I add a Jira about "Gradle polish" ? >>>>>>>>>>>>> >> >>>> Thoughts ? >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Regards >>>>>>>>>>>>> >> >>>> JB >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> On 07/04/2018 04:52, Scott Wegner wrote: >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Here's an end-of-day update on migration work: >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> * Snapshot unsigned dailies and signed release builds >>>>>>>>>>>>> are working >>>>>>>>>>>>> >> >>>> (!!). >>>>>>>>>>>>> >> >>>> PR/5048 [1] merges changes from Luke's branch >>>>>>>>>>>>> >> >>>> * python precommit failing... will investigate >>>>>>>>>>>>> python precommit >>>>>>>>>>>>> >> >>>> Monday >>>>>>>>>>>>> >> >>>> * All Precommits are gradle only >>>>>>>>>>>>> >> >>>> * All Postcommits except performance tests and >>>>>>>>>>>>> Java_JDK_Versions_Test >>>>>>>>>>>>> >> >>>> use gradle (after PR/5047 [2] merged) >>>>>>>>>>>>> >> >>>> * Nightly snapshot release using gradle is ready; >>>>>>>>>>>>> needs PR/5048 to be >>>>>>>>>>>>> >> >>>> merged before switching >>>>>>>>>>>>> >> >>>> * ValidatesRunner_Spark failing consistently; >>>>>>>>>>>>> investigating >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Thanks for another productive day of hacking. I'll >>>>>>>>>>>>> pick up again on >>>>>>>>>>>>> >> >>>> Monday. >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> [1] https://github.com/apache/beam/pull/5048 >>>>>>>>>>>>> >> >>>> [2] https://github.com/apache/beam/pull/5047 >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau >>>>>>>>>>>>> >> >>>> <[email protected] <mailto:[email protected]>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Why building a zip per runner which its stack and >>>>>>>>>>>>> just pointing >>>>>>>>>>>>> >> >>>> out >>>>>>>>>>>>> >> >>>> on that zip and let beam lazy load the runner: >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> --runner=LazyRunner --lazyRunnerDir=... >>>>>>>>>>>>> --lazyRunnerOptions=... >>>>>>>>>>>>> >> >>>> (or >>>>>>>>>>>>> >> >>>> the fromSystemProperties() if it gets merged a >>>>>>>>>>>>> day ;)) >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Le 6 avr. 2018 20:21, "Kenneth Knowles" < >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> >> >>>> <mailto:[email protected]>> a écrit : >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> I'm working on finding a solution for >>>>>>>>>>>>> launching the Nexmark >>>>>>>>>>>>> >> >>>> suite with each runner. This doesn't have to >>>>>>>>>>>>> be done via >>>>>>>>>>>>> >> >>>> Gradle, >>>>>>>>>>>>> >> >>>> but we anyhow need built artifacts that don't >>>>>>>>>>>>> require user >>>>>>>>>>>>> >> >>>> classpath intervention. >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> It looks to me like the examples are also >>>>>>>>>>>>> missing this - >>>>>>>>>>>>> >> >>>> they >>>>>>>>>>>>> >> >>>> have separate configuration e.g. >>>>>>>>>>>>> sparkRunnerPreCommit but >>>>>>>>>>>>> >> >>>> that >>>>>>>>>>>>> >> >>>> is overspecified compared to a free-form >>>>>>>>>>>>> launching of a >>>>>>>>>>>>> >> >>>> main() >>>>>>>>>>>>> >> >>>> program with a runner profile. >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik >>>>>>>>>>>>> >> >>>> <[email protected] >>>>>>>>>>>>> >> >>>> <mailto:[email protected]>> wrote: >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Romain, are you talking about the >>>>>>>>>>>>> profiles that exist as >>>>>>>>>>>>> >> >>>> part of the archetype examples? >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> If so, then those still exist and haven't >>>>>>>>>>>>> been changed. >>>>>>>>>>>>> >> >>>> If >>>>>>>>>>>>> >> >>>> not, can you provide a link to the >>>>>>>>>>>>> profile in a pom file >>>>>>>>>>>>> >> >>>> to >>>>>>>>>>>>> >> >>>> be clearer? >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> On Fri, Apr 6, 2018 at 12:40 PM Romain >>>>>>>>>>>>> Manni-Bucau >>>>>>>>>>>>> >> >>>> <[email protected] <mailto: >>>>>>>>>>>>> [email protected]>> >>>>>>>>>>>>> >> >>>> wrote: >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Hi Scott, >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> is it right that 2 doesn't handle the >>>>>>>>>>>>> hierachy >>>>>>>>>>>>> >> >>>> anymore >>>>>>>>>>>>> >> >>>> and that it doesn't handle profiles >>>>>>>>>>>>> for runners as >>>>>>>>>>>>> >> >>>> it is >>>>>>>>>>>>> >> >>>> currently with maven? >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Romain Manni-Bucau >>>>>>>>>>>>> >> >>>> @rmannibucau <https://twitter.com/ >>>>>>>>>>>>> rmannibucau> | >>>>>>>>>>>>> >> >>>> Blog >>>>>>>>>>>>> >> >>>> <https://rmannibucau.metawerx.net/> >>>>>>>>>>>>> | Old Blog >>>>>>>>>>>>> >> >>>> <http://rmannibucau.wordpress.com> | >>>>>>>>>>>>> Github >>>>>>>>>>>>> >> >>>> <https://github.com/rmannibucau> | >>>>>>>>>>>>> LinkedIn >>>>>>>>>>>>> >> >>>> <https://www.linkedin.com/in/ >>>>>>>>>>>>> rmannibucau> | Book >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> <https://www.packtpub.com/ >>>>>>>>>>>>> application-development/java-ee-8-high-performance> >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> 2018-04-06 18:32 GMT+02:00 Scott >>>>>>>>>>>>> Wegner >>>>>>>>>>>>> >> >>>> <[email protected] <mailto: >>>>>>>>>>>>> [email protected]>>: >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> I wanted to start a thread to >>>>>>>>>>>>> summarize the >>>>>>>>>>>>> >> >>>> current >>>>>>>>>>>>> >> >>>> state of Gradle migration. We've >>>>>>>>>>>>> made lots of >>>>>>>>>>>>> >> >>>> good >>>>>>>>>>>>> >> >>>> progress so far this week. Here's >>>>>>>>>>>>> the status >>>>>>>>>>>>> >> >>>> from >>>>>>>>>>>>> >> >>>> what I can tell-- please add or >>>>>>>>>>>>> correct anything >>>>>>>>>>>>> >> >>>> I >>>>>>>>>>>>> >> >>>> missed: >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> * Release artifacts can be built >>>>>>>>>>>>> and published >>>>>>>>>>>>> >> >>>> for >>>>>>>>>>>>> >> >>>> Snapshot and officlal releases [1] >>>>>>>>>>>>> >> >>>> * Gradle-generated releases have >>>>>>>>>>>>> been validated >>>>>>>>>>>>> >> >>>> with >>>>>>>>>>>>> >> >>>> the the Apache Beam archetype >>>>>>>>>>>>> generation >>>>>>>>>>>>> >> >>>> quickstart; >>>>>>>>>>>>> >> >>>> still needs additional validation. >>>>>>>>>>>>> >> >>>> * Generated release pom files >>>>>>>>>>>>> have correct >>>>>>>>>>>>> >> >>>> project >>>>>>>>>>>>> >> >>>> metadata [2] >>>>>>>>>>>>> >> >>>> * The python pre-commits are now >>>>>>>>>>>>> working in >>>>>>>>>>>>> >> >>>> Gradle >>>>>>>>>>>>> >> >>>> [3] >>>>>>>>>>>>> >> >>>> * Ismaël has started a >>>>>>>>>>>>> collaborative doc of >>>>>>>>>>>>> >> >>>> Gradle >>>>>>>>>>>>> >> >>>> tips [4] as we all learn the new >>>>>>>>>>>>> system-- please >>>>>>>>>>>>> >> >>>> add >>>>>>>>>>>>> >> >>>> your own. This will eventually >>>>>>>>>>>>> feed into >>>>>>>>>>>>> >> >>>> official >>>>>>>>>>>>> >> >>>> documentation on the website. >>>>>>>>>>>>> >> >>>> * Łukasz Gajowy is working on >>>>>>>>>>>>> migrating >>>>>>>>>>>>> >> >>>> performance >>>>>>>>>>>>> >> >>>> testing framework [5] >>>>>>>>>>>>> >> >>>> * Daniel is working on updating >>>>>>>>>>>>> documentation to >>>>>>>>>>>>> >> >>>> refer to Gradle instead of maven >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> If I missed anything, please add >>>>>>>>>>>>> it to this >>>>>>>>>>>>> >> >>>> thread. >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> The general roadmap we're working >>>>>>>>>>>>> towards is: >>>>>>>>>>>>> >> >>>> (a) Publish release artifacts >>>>>>>>>>>>> with Gradle >>>>>>>>>>>>> >> >>>> (SNAPSHOT >>>>>>>>>>>>> >> >>>> and signed releases) >>>>>>>>>>>>> >> >>>> (b) Postcommits migrated to Gradle >>>>>>>>>>>>> >> >>>> (c) Migrate documentation from >>>>>>>>>>>>> maven to Gradle >>>>>>>>>>>>> >> >>>> (d) Migrate perfkit suites to use >>>>>>>>>>>>> Gradle >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> For those of you that are >>>>>>>>>>>>> hacking: thanks for >>>>>>>>>>>>> >> >>>> your >>>>>>>>>>>>> >> >>>> help so far! Progress is being >>>>>>>>>>>>> roughly tracked >>>>>>>>>>>>> >> >>>> on >>>>>>>>>>>>> >> >>>> the Kanban [6]; please make sure >>>>>>>>>>>>> the issues >>>>>>>>>>>>> >> >>>> assigned >>>>>>>>>>>>> >> >>>> to you are up-to-date. Many of >>>>>>>>>>>>> the changes are >>>>>>>>>>>>> >> >>>> staged on lukecwik's local branch >>>>>>>>>>>>> [7]; we'll >>>>>>>>>>>>> >> >>>> work on >>>>>>>>>>>>> >> >>>> merging them back soon. >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> [1] >>>>>>>>>>>>> >> >>>> https://github.com/lukecwik/incubator-beam/pull/7 >>>>>>>>>>>>> >> >>>> [2] >>>>>>>>>>>>> >> >>>> https://github.com/lukecwik/incubator-beam/pull/3 >>>>>>>>>>>>> >> >>>> [3] https://github.com/apache/ >>>>>>>>>>>>> beam/pull/5032 >>>>>>>>>>>>> >> >>>> [4] >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> https://docs.google.com/document/d/ >>>>>>>>>>>>> 1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit >>>>>>>>>>>>> >> >>>> [5] https://github.com/apache/ >>>>>>>>>>>>> beam/pull/5003 >>>>>>>>>>>>> >> >>>> [6] >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> https://issues.apache.org/jira/secure/RapidBoard.jspa? >>>>>>>>>>>>> rapidView=242 >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> [7] >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> https://github.com/lukecwik/incubator-beam/tree/gradle >>>>>>>>>>>>> >> >>>> -- >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> Got feedback? >>>>>>>>>>>>> http://go/swegner-feedback >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>> >>>>>>>>>>>>> >> >>>>>>>>>>>>> > >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>
