Re: [ANNOUNCE] New committer announcement: Mark Liu

gt;>>>> >>>>>>>>>>>>>>> Congratulations, Mark! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for your contributions, in particular for your >>>>>>>>>>>>>>> efforts to parallelize test execution for Python SDK and >>>>>>>>>>>>>>> increase the speed >>>>>>>>>>>>>>> of Python precommit checks. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sun, Mar 24, 2019 at 9:40 PM Kenneth Knowles < >>>>>>>>>>>>>>> k...@apache.org> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please join me and the rest of the Beam PMC in welcoming a >>>>>>>>>>>>>>>> new committer: Mark Liu. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Mark has been contributing to Beam since late 2016! He has >>>>>>>>>>>>>>>> proposed 100+ pull requests. Mark was instrumental in >>>>>>>>>>>>>>>> expanding test and >>>>>>>>>>>>>>>> infrastructure coverage, especially for Python. In >>>>>>>>>>>>>>>> consideration of Mark's contributions, the Beam PMC trusts >>>>>>>>>>>>>>>> Mark with the >>>>>>>>>>>>>>>> responsibilities of a Beam committer [1]. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thank you, Mark, for your contributions. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Kenn >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> [1] https://beam.apache.org/contribute/become-a-committer/ >>>>>>>>>>>>>>>> #an-apache-beam-committer >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> *Aizhamal Nurmamat kyzy* >>>>>>>>>> >>>>>>>>>> Open Source Program Manager >>>>>>>>>> >>>>>>>>>> 646-355-9740 Mobile >>>>>>>>>> >>>>>>>>>> 601 North 34th Street, Seattle, WA 98103 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Cheers, >>>>>>>> Gleb >>>>>>>> >>>>>>> >> >> -- >> >> Ruoyun Huang >> >> -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: Add code quality checks to pre-commits.

No bugs/Vulnerabilities added. >> > >> > As another part of this proposal, I want to >> suggest the >> > use of SonarQube for tracking code statistics >> and as >> > agent for enforcing code quality thresholds. It >> is >> > Apache provided tool that has integration with >> Jenkins >> > or Gradle via plugins. >> > >> > I believe some reporting to SonarQube was >> configured for >> > mvn builds of some of Beam sub-projects, but >> was lost >> > during migration to gradle. >> > >> > I was looking for other options, but so far >> found only >> > general configs to gradle builds that will fail >> build if >> > code coverage for project is too low. Such >> approach will >> > force us to backfill tests for all existing >> code that >> > can be tedious and demand learning of all >> legacy code >> > that might not be part of current work. >> > >> > I suggest to discuss and come to conclusion on >> two >> > points in this tread: >> > 1. Do we want to add code quality checks to our >> > pre-commit jobs and require them to pass before >> PR is >> > merged? >> > >> > Suggested: Add code quality checks listed >> above at >> > first, adjust them as we see fit in the >> future. >> > >> > 2. What tools do we want to utilize for >> analyzing code >> > quality? >> > >> > Under discussion. Suggested: SonarQube, but >> will >> > depend on functionality level we want to >> achieve. >> > >> > >> > Regards, >> > --Mikhail >> > >> > >> > >> > -- >> > >> > >> > >> > >> > Got feedback? tinyurl.com/swegner-feedback >> > <https://tinyurl.com/swegner-feedback> >> > >> > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: [RFC] I made a new tabbed Beam view in Jenkins

Oh, a good fact! Thanks for the info. :) On Tue, Dec 18, 2018 at 1:48 PM Alan Myrvold wrote: > The _Cron variants of pre-commits are run post-commit, to make it easier > to tell if the pre-commit is flaky or broken. > > On Tue, Dec 18, 2018 at 1:40 PM Jason Kuster > wrote:

Re: [RFC] I made a new tabbed Beam view in Jenkins

YT about making this the top-level Beam view? (vs >>>>> https://builds.apache.org/view/A-D/view/Beam/) >>>>> >>>>> After that, maybe we could clean the categories so they fit into the >>>>> tabs more easily with fewer regexes (to make sure things don't get >>>>> missed). >>>>> I have read also that if you use / instead of _ as a separator in a name >>>>> then Jenkins will display jobs as nested in folders automatically. Not >>>>> sure >>>>> it actually results in a better view; haven't tried it. >>>>> >>>>> Kenn >>>>> >>>> >>> >>> -- >>> >>> >>> >>> >>> Got feedback? tinyurl.com/swegner-feedback >>> >> -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: [GitHub] aaltay commented on issue #1: Add .gitmodules file

t; On Tue, Aug 14, 2018 at 1:15 AM Jason Kuster > wrote: > >> Looks like these are getting sent to dev. Is there a per-project config >> we need to tweak or a ticket we need to file in order to get these sent to >> commits@ instead? >> On Mon, Aug 13, 2018 at 6:17 PM Git

Re: [GitHub] aaltay commented on issue #1: Add .gitmodules file

the specific comment. > > For queries about this service, please contact Infrastructure at: > us...@infra.apache.org > > > With regards, > Apache Git Services > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

FYI: Jenkins Upgrade

mplement a global retention window which would affect us at that point. Best, Jason -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: Automatically create JIRA tickets for failing post-commit tests

t;>> My assumption is that we need someone to triage new issues. >>>>>>>>> >>>>>>>>> Ideally, any contributor, who sees failing test, should check >>>>>>>>> unassigned tickets and either do triage, or assign them to s

Re: [PROPOSAL] Prepare Beam 2.6.0 release

cherry-picked into the >> > release branch. >> > - As soon as tests look good, and blockers have been addressed, I will >> > perform the other release tasks. >> > >> > Does that seem reasonable to the community? >> > >> > Best >> &

Re: [ANN] Apache Beam 2.5.0 has been released!

ng generic to all runners > > > You can take a look on the following blog post and Release Notes for > > details: > > > https://beam.apache.org/blog/2018/06/26/beam-2.5.0.html > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&ver

Re: Is something wrong with our Jenkins? Waiting 2+ hours for tests to start running.

> multi-hour (!) delays in running tests. >>>>>>>> >>>>>>>> For example: https://github.com/apache/beam/pull/5545 >>>>>>>> >>>>>>>> I triggered the Dataflow ValidatesRunner tests. The check showed up >>>>>>>> as yellow with no "Details" link, which generally means it had not yet >>>>>>>> been >>>>>>>> scheduled on Jenkins. It stayed like this for two hours, never even >>>>>>>> starting to run. Apparently there were a ton of idle Jenkins executors >>>>>>>> at >>>>>>>> the time, so it's not that all our Jenkins executors were busy. As of >>>>>>>> now, >>>>>>>> it still has not started running. >>>>>>>> >>>>>>>> Does anybody have any idea what's going on here? >>>>>>>> >>>>>>>> Reuven >>>>>>>> >>>>>>> -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: Precommits broken?

; wrote: >>>> >>>>> Looks like statuses got posted between me writing this email and >>>>> sending it. Still wondering why the python and go jobs appear to be >>>>> missing? >>>>> >>>>> Andrew >>>>

Re: Design Proposal: Beam-Site Automation Reliability

ary permissions and expertise to >>>>>> resolve them. Overall, the reliability of Beam-site automation is >>>>>> impeding >>>>>> productivity for Beam-site development. >>>>>> >>>>>> At this point I'm seeking feedback on a few possible solutions: >>>>>> >>>>>> 1. Invest in improvements to Mergebot reliability. Make stability >>>>>> tweaks for various failure modes, distribute Mergebot expertise and >>>>>> operations permissions to more committers. >>>>>> 2. Deprecate Mergebot and revert to manual process. With the current >>>>>> unreliability, some committers choose to forego merge automation anyway. >>>>>> 3. Generate HTML only during publishing. This seems to be newly >>>>>> supported by the Apache GitPubSub workflow. This would eliminate most or >>>>>> all of the automation that Mergebot is responsible for. >>>>>> >>>>>> Feel free to add comments in the doc. >>>>>> >>>>>> Thanks, >>>>>> Scott >>>>>> >>>>>> >>>>>> >>>>>> Got feedback? tinyurl.com/swegner-feedback >>>>>> >>>>> >>>> -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: [VOTE] Code Review Process

. We believe that changing this process will improve code review >>>>>> throughput, reduce committer load, and engage more of the community in >>>>>> the >>>>>> code review process. >>>>>> >>>>>> Please vote: >>>>>> [ ] +1: Accept the above proposal to change the Beam code >>>>>> review/merge policy >>>>>> [ ] -1: Leave the Code Review policy unchanged >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Thomas >>>>>> >>>>>> [1] >>>>>> https://lists.apache.org/thread.html/7c1fde3884fbefacc252b6d4b434f9a9c2cf024f381654aa3e47df18@%3Cdev.beam.apache.org%3E >>>>>> >>>>> >>>> >>> -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: [VOTE] Use probot/stale to automatically manage stale pull requests

://lists.apache.org/thread.html/bda552ea7073ca165aaf47034610afafe22d589e386525023d33609e@%3Cdev.beam.apache.org%3E >> [3] https://github.com/probot/stale >> [4] https://issues.apache.org/jira/browse/BEAM-4423 >> [5] https://issues.apache.org/jira/browse/INFRA-16589 >> [6] https://github.com/apache/beam/pull/5532 >> > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: [ANNOUNCEMENT] New committers, May 2018 edition!

in welcoming the following >>> > contributors as our newest committers. They have significantly >>> > contributed to the project in different ways, and we look forward to >>> > many more contributions in the future. >>> > >>> >

Re: [VOTE] Go SDK

s an official part of Beam > [ ] -1, Do not approve (please provide specific comments) > > Thanks, > The Gophers of Apache Beam > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: I'm back and ready to help grow our community!

olks around, I'd like to use this >> opportunity to re-introduce myself. I'm Gris Cuevas and I work at Google. >> Now that I'm back, I'll continue to work on supporting our community in two >> main streams: Contribution Experience & Events, Meetups, and Conf

Re: Apache Beam - jenkins question

uld now have all the permissions needed. (You should, > however, evaluate whether this is a good place for it. Executors > themselves, for example, might be an alternative.) > > On Fri, Apr 27, 2018 at 7:42 PM, Jason Kuster > wrote: > >> See >> https://github.com/ap

Re: Apache Beam - jenkins question

See https://github.com/apache/beam/blob/master/.test-infra/jenkins/common_job_properties.groovy#L119 for an example of this being done in practice to add the coveralls repo token as an environment variable. On Fri, Apr 27, 2018 at 12:41 PM Jason Kuster wrote: > Hi Kamil, Davor, > > I t

Re: Apache Beam - jenkins question

> > I sent you a message on asf slack, wasn't sure how can I reach you. > > Anyway are you able to add secret (environment variable) to jenkins. ?? > Or point me to a person that would be able to do that ? > > Kind Regards > Kamil Szewczyk > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: [Go SDK] Proposal: Set up a Vanity Import Path

gt; >>>> >> >> >>>> Currently users need to import paths like: >> >> >>>> >> >> >>>> import "github.com/apache/beam/sdks/go/pkg/beam/io/textio" >> >> >>>> >> >> >>>> to get at SDK packages. If we implement this proposal, they would >> >> >>>> look >> >> >>>> like: >> >> >>>> >> >> >>>> import "beam.apache.org/sdks/go/pkg/beam/io/textio" >> >> >>>> >> >> >>>> which are a bit shorter, a bit more stable, and a bit nicer, with >> the >> >> >>>> benefits outlined above. >> >> >>>> >> >> >>>> I wrote a doc with details which is at >> >> >>>> https://s.apache.org/go-beam-vanity-import >> >> >>>> (Thanks you Thomas for short linking it for me.) >> >> >>>> >> >> >>>> The doc should answer most of your questions, but please let me >> know >> >> >>>> if >> >> >>>> you have others either here, or in a doc comment. >> >> >>>> >> >> >>>> If there's consensus to do so, it would be better it's done sooner >> >> >>>> rather than after folks begin depending on it. We wouldn't want to >> >> >>>> have >> >> >>>> fragmented examples. >> >> >>>> >> >> >>>> Robert Burke >> >> >>>> (One of the Gopher Googlers who have been quietly lurking on the >> >> >>>> list, >> >> >>>> and submitting the occasional PR for the Go SDK. I look forward to >> >> >>>> working >> >> >>>> with you all!) >> >> >>> >> >> >>> >> >> > >> > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Node Errors ("Backing Channel... is Disconnected.")

rallelism somehow to mitigate the issue. In the meantime, if you see this issue, feel free to add the failing build as a comment to the bug. Best, Jason -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: Inconsistent jenkins workers

t reproduce the failure >>>> locally >>>> and the remote logs (e.g. >>>> >>>> https://builds.apache.org/job/beam_PreCommit_Python_GradleBuild/471/console >>>> ) don't seem to be very enlightening either. This leads to a couple of >>>> questions: >>>> >>>> * How are our jenkins beam workers configured, and why aren't they the >>>> same? >>>> * How does one go about debugging failures like this? >>>> >>>> Before too much effort is invested, how far are we from using >>>> containers to >>>> manage the build environments? >>>> >>> -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: Rebuilding beam before running Performance Test - question.

ormality question", but does anyone object that we > rebuild beam like this in each Performance Tests Jenkins job? > > [1] > https://beam.apache.org/documentation/io/testing/#implementing-integration-tests > > Best regards, > Łukasz > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: Performance tests status and anomaly detection proposal

maven-based, we responded to ongoing Gradle migration >>>>> and >>>>> created PR <https://github.com/apache/beam/pull/5003> that allows running >>>>> them via Gradle.All of those tests are executed on daily basis on Apache >>>>> Jenkins <https://builds.apache.org/> and their results are published to >>>>> individual BigQuery tables. There is also a dashboard on which tests >>>>> results may be viewed and >>>>> compared:https://apache-beam-testing.appspot.com/explore?dashboard=5755685136498688 >>>>> <https://apache-beam-testing.appspot.com/explore?dashboard=5755685136498688>As >>>>> we have some amount of tests already, we’re currently working on a tool >>>>> that will analyze the results and search for anomalies, so devs are >>>>> notified if degraded performance is observed. You can find proposal >>>>> document >>>>> here:https://docs.google.com/document/d/1Cb7XVmqe__nA_WCrriAifL-3WCzbZzV4Am5W_SkQLeA >>>>> <https://docs.google.com/document/d/1Cb7XVmqe__nA_WCrriAifL-3WCzbZzV4Am5W_SkQLeA>We >>>>> welcome you to share your thoughts on performance tests in general as well >>>>> as proposed solution for anomaly detection.Best,Dariusz Aniszewski* >>>>> >>>>> >>>> -- >> Got feedback? go/pabloem-feedback >> <https://goto.google.com/pabloem-feedback> >> > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: Beam7 Outage

Almost all builds there break. > > On Thu, Apr 12, 2018, 6:41 PM Jason Kuster wrote: > >> Hi all, >> >> The Jenkins Beam7 executor gave up the ghost some time in the last couple >> of days. I've been on the line with Infra yesterday and today getting it >>

Beam7 Outage

r patience. Best, Jason -- ------- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: slack @the-asf?

ub >>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn >>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book >>>>>>>>>> <https://www.packtpub.com/application-development/java-ee-8-high-performanc

Re: Jenkins slowness

bout an hour to > run, not including time waiting in the queue. > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow See something? Say something. go/jasonkuster-feedback

Re: Gradle status

gt;>> >>>>>>>>> Well, I'd be for a strong switch otherwise PR will keep using >>>>>>>>> maven, jenkins will not test the code and at the end we fail to >>>>>>>>> deliver >>>>>>>>> something consistent. So whatever tool is selected I'm tempted to say >>>>>>>>> drop >>>>>>>>> other build files and jenkins hooks references. >>>>>>>>> >>>>>>>>> What about doing it after 2.4 vote? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Mar 7, 2018 at 2:24 AM, Romain Manni-Bucau < >>>>>>>>> rmannibu...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Up, >>>>>>>>>> >>>>>>>>>> We discussed to have a strong switch to gradle or rollback to >>>>>>>>>> maven around april to not be blocked by the build tool. I noticed >>>>>>>>>> gradle >>>>>>>>>> build rarely passes on PR and kind of blurry our vision - not sure >>>>>>>>>> why >>>>>>>>>> exactly. Also, PR don't always contain the gradle updates - generally >>>>>>>>>> dependencies+plugins are added in pom.xml AFAIK, so it seems the >>>>>>>>>> adoption >>>>>>>>>> is very slow to not say rejected. >>>>>>>>>> >>>>>>>>>> What do we do about that? When do we drop the double build >>>>>>>>>> maintenance - whatever is picked? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Romain Manni-Bucau >>>>>>>>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog >>>>>>>>>> <https://rmannibucau.metawerx.net/> | Old Blog >>>>>>>>>> <http://rmannibucau.wordpress.com> | Github >>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn >>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book >>>>>>>>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >>>>>>>>>> >>>>>>>>>> 2018-01-12 6:30 GMT+01:00 Romain Manni-Bucau < >>>>>>>>>> rmannibu...@gmail.com>: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Le 11 janv. 2018 23:13, "Kenneth Knowles" a >>>>>>>>>>> écrit : >>>>>>>>>>> >>>>>>>>>>> On Thu, Jan 11, 2018 at 8:43 AM, Romain Manni-Bucau < >>>>>>>>>>> rmannibu...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> 2. gradle build doesn't use the same output directory than >>>>>>>>>>>> maven so it is not really smooth to have both and have to maintain >>>>>>>>>>>> both >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I also have an opinion on this. It is useful and reasonable to >>>>>>>>>>> be able to build even when the source is on a read-only filesystem. >>>>>>>>>>> Maven's >>>>>>>>>>> defaults are undesirable and require workarounds. We shouldn't >>>>>>>>>>> mimic the >>>>>>>>>>> behavior, but actually should set gradle up to build to a directory >>>>>>>>>>> outside >>>>>>>>>>> the source tree always, if we can. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hmm, which is something you can do with maven as well so not >>>>>>>>>>> sure I get it. >>>>>>>>>>> >>>>>>>>>>> Also note the thread is no more about the technical points but >>>>>>>>>>> more the sources maintenance and consistency. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Kenn >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>> >> -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: A personal update

>> >>>>>>> transition >> >>>>>>> in my >> >>>>>>> professional life. As things have settled, I’m happy to again be >> >>>>>>> working among >> >>>>>>> all of you, as we

Re: [PROPOSAL] Beam Go SDK feature branch

he design are in any way final -- > there are many open questions and we absolutely welcome ideas and > contributions. Please let us know if you have any comments or objections > (or would like to help!). > > Thanks, > Henning Rohde, Bill Neubauer, and Robert Burke > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: [VOTE] Use Gradle for Apache Beam developmental processes

Docker Support: Rudimentary (via docker plugin) >> ASF Release Automation: ? >> Jenkins Support: Mature >> Configuration Language: Groovy >> Multiple Java Versions: Yes >> Static Analysis Tools: Some >> ASF Release Audit Tool (RAT): Rudimentary (plugin just calls Apache Maven >> ANT plugin) >> IntelliJ Integration: Mature >> Eclipse Integration: Mature >> Extensibility: Mature >> Number of GitHub Projects Using It: 122k >> Continuous build daemon: Mature >> Incremental build support: Mature >> Intra-module dependencies: Mature (via configurations) >> >> >> -- >> Wesley Tanakahttps://wtanaka.com/ >> >> -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: [VOTE] Migrate to gitbox

> > > >> [ ] -1, Do not migrate (please provide specific comments) > > > > > >> > > > > > >> The vote will be open for at least 36 hours. It is adopted by > > > majority > > > > > >> approval, with at least 3 PMC affirmative votes. > > > > > >> > > > > > >> Thanks, > > > > > >> Regards > > > > > >> JB > > > > > >> -- > > > > > >> Jean-Baptiste Onofré > > > > > >> jbono...@apache.org > > > > > >> http://blog.nanthrax.net > > > > > >> Talend - http://www.talend.com > > > > > > > > > > > > > > > > > > > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: beam-site issues with Jenkins and MergeBot

n't know how to test or deploy it. > > On Thu, Aug 10, 2017 at 3:32 PM Jason Kuster > wrote: > >> Investigating mergebot outage currently. Apologies for the downtime. >> >> On Wed, Aug 9, 2017 at 9:55 PM, Eugene Kirpichov >> wrote: >> >>&g

Re: [ANNOUNCEMENT] New PMC members, August 2017 edition!

the Spark runner, Aviem has worked to > > improve > > > > how the project operates, leading discussions on inclusiveness and > > > > openness. > > > > > > > > Congratulations to both! Welcome! > > > > > > > > Davor > > > > > > > > > -- > > Thanks, > > > > Jesse > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: [ANNOUNCEMENT] New committers, August 2017 edition!

y shown > > > himself to be an insightful code reviewer, significantly impacting the > > > project’s code quality and ensuring the success of the new major > > component. > > > > > > * Manu Zhang > > > Manu initiated and developed a runner for the Apache Gearpump > > (incubating) > > > engine, and has driven it to the point of merging to the master branch. > > In > > > this effort, he accumulated 65 commits (7,812 ++ / 4,882 --) and > extended > > > the project to the new user community. > > > > > > Congratulations to all five! Welcome! > > > > > > Davor > > > > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: beam-site issues with Jenkins and MergeBot

However, Mergebot appears to still be not working. > https://github.com/apache/beam-site/pull/283 fixes the dead link and it > passes the Jenkins precommit tests, but my "@asfgit merge" appears to have > done nothing. I'm gonna have to merge things manually for now. > >

Re: is it ok to have a dicussion without subscribe the list

Mon, Aug 7, 2017 at 4:20 PM, derek wrote: > is it ok to have a dicussion without subscribe the list? > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Jenkins Downtime 0100 UTC 19 July

Downtime begins in ~1.5h so Infra can try to resolve some ongoing issues. -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Fwd: jenkins emergency restart notice - Tues 00:00 UTC

address problems related to ‘too many open files’ and to attempt to correct a problem with some windows agents. I will be stopping new builds so that the queue can empty over the next 2 hours. -Chris -- --- Jason Kuster Apache Beam / Google Cloud Dataflow signature.asc Description: PGP

Groovy Expertise?

Hey all, If anyone has experience working with Groovy, I would love for someone to take a look at https://github.com/apache/beam/pull/3545, which does a decent amount of work with the Jenkins Job DSL in preparation for some precommit changes. Thanks, Jason -- --- Jason Kuster Apache Beam

Re: MergeBot is here!

he > > (common) case of an advanced user having fixup commits in the review, > > but being able to merge without waiting for her to manually squash > > them after the LGTM. > > > > OK, yea, I think it is fair to allow the bot to try to reorder if it goes > cleanly. So sounds good to me. > > Kenn > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Jenkins Upgrade Sunday

the cross-JDK test. More info can be found in the below link. https://lists.apache.org/thread.html/1185e30193ff764870e9a7f58818e0d6a9af3a50b6e3ebd62476244c@%3Cbuilds.apache.org%3E Best, Jason -- ------- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: MergeBot is here!

(quick update re #2 above): ~4 minutes after I reopened the ticket, it's fixed. https://github.com/apache/infrastructure-puppet/commit/709944291da5e8aea711cb8578f0594deb45e222 updates the website to the correct address. Infra is once again the best. On Mon, Jul 10, 2017 at 12:38 PM, Jason K

Re: MergeBot is here!

ason and the other who made this possible, this is great! > > > Ismaël > > > > > > ps. I’m eager to see this included too for the beam project. > > > > > > On Sat, Jul 8, 2017 at 7:28 AM, tarush grover > > > wrote: > > >> This is really

MergeBot is here!

to fruition. Additionally, this could not have happened without the extensive work Davor put in to keep things moving along. Thank you Davor. Looking forward to hearing your comments and feedback. Thanks. Jason -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: Making it easier to run IO ITs

t; > ITs. We use pkb to control kubernetes and capture our benchmark times. > This > > means you'll need to install pkb if you'd like to use this nicer > > experience, however, devs will never have to use pkb if they don't want > to, > > nor is making changes in pkb required when you want to add a new IO IT. > You > > can always spin up the data store yourself, and invoke the integration > test > > directly. > > > > Drafts of these changes can be seen at [0] and [1] - however, I don't > > expect most folks will care about these changes other than "how do I > invoke > > this?", so let me know if you have comments about how this is invoked. > > > > S > > > > [0] pom changes hooking up the call to pkb - > > https://github.com/ssisk/beam/commit/eec7cb5b71330761e71850e8e6f65f > > 34249641b0 > > [1] pkb changes enabling kubernetes spin up- > > https://github.com/ssisk/PerfKitBenchmarker/commits/kubernetes_create > > (last > > 2 changes) > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: Jenkins Executor Issue

>> will try to communicate back here with relevant status updates. >> >> Best, >> >> Jason >> >> > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Jenkins Executor Issue

Hi all, There appears to be an issue ongoing where our Jenkins jobs are not being scheduled onto our executors. I've filed https://issues.apache.org/jira/browse/INFRA-14476 to track this issue, and will try to communicate back here with relevant status updates. Best, Jason -- ---

Precommit Jenkins Linkage Broken

/jira/browse/INFRA-14247 -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: First stable release completed!

tware Foundation press release: > > >> http://globenewswire.com/news-release/2017/05/17/986839/0/ > > >> en/The-Apache-Software-Foundation-Announces-Apache-Beam-v2-0-0.html > > >> > > >> Beam blog: > > >> https://beam.apache.org/blog/2017/05/17/beam-fi

Re: [VOTE] First stable release: release candidate #4

+1 -- validated several common cases on spark yarn and dataflow runner. Exciting milestone for the project! On May 12, 2017 21:48, "Davor Bonaci" wrote: Hi everyone -- After going through several release candidates, setting and validating acceptance criteria, running a hackathon, and polishing

Re: First stable release: Acceptance criteria

t; > I'm focusing on: > > > > > > * user reported bugs (Avro, TextIO, MongoDb) > > > * the actual Apache Release criteria (licensing, dependencies, etc.) > > > > > > On Thu, May 11, 2017 at 3:04 PM, Lukasz Cwik > > > >

Re: First stable release: Acceptance criteria

on Spark, doh On Thu, May 11, 2017 at 5:45 PM, Jason Kuster wrote: > Just validated a decently-sized wordcount on a YARN cluster successfully. > > On Thu, May 11, 2017 at 3:51 PM, Kenneth Knowles > wrote: > >> I gave the archetype-based quickstart a try on as many runners

Re: First stable release: Acceptance criteria

on the thread about that specific > > release > > > candidate. > > > > > > Thanks! > > > > > > Davor > > > > > > [1] > > > https://lists.apache.org/thread.html/37caa5a94cec1405638410857f489d > > 7cf7fa12bbe3c36e9925b2d6e2@%3Cdev.beam.apache.org%3E > > > [2] > > > https://docs.google.com/document/d/1XwojJ4Mj3wSlnBO1YlBs51P8kuGyg > > YRj2lrNrqmAUvo/ > > > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: Slack Invites

t now, people have to email in (causing extra dev@/user@ emails). > I > > > did > > > a quick search and found this <https://github.com/rauchg/slackin> so > > > people > > > can invite themselves. > > > > > > Thanks, > > > > > > Jesse > > > -- > > > Thanks, > > > > > > Jesse > > > > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Congratulations Davor!

://blogs.apache.org/foundation/entry/the-apache-sof tware-foundation-welcomes P.S. I dug through the list to make sure I wasn't missing any other Beam community members; if I have, my sincerest apologies and please recognize them on this or a new thread. -- --- Jason Kuster Apache Beam / G

Re: [JENKINS] - Removal of top level views

ojects’ may be a benefit at top level if used correctly. > ‘PreCommit Builds’ is not one owned by any one project, Im unsure whether > this could stay top level and be a benefit. > > I’ll be moving the Infra views shortly. > > If, after a week, there are still projects at the top lev

Re: An Update on Jenkins

perin wrote: > > If not, feel free to reply to this thread > > ... not. :) :( > > On Tue, Apr 25, 2017 at 9:58 PM, Jean-Baptiste Onofré > wrote: > > > Thanks for the update ! > > > > Regards > > JB > > > > On Apr 26, 2017,

An Update on Jenkins

everything starting back up again. They expect that waiting a couple of hours should solve the problem, so hopefully by tomorrow things will be back to normal. If not, feel free to reply to this thread, and I'll try to keep things up to date with status. Best, Jason -- --- Jason Kuster Apach

Re: Community hackathon

This could be a self-organized event, and coordinated via the Slack > > > >> channel. For example, we (as a community and participants > > > ) can try out the > > > >> project in various ways -- quickstart, examples, different runners, > > > >> different platforms -- immediately fixing issues as we > > > run into them. It > > > >> could last, say, 24 hours, with people from different time zones > > > >> participating at the time of their choosing. > > > >> > > > >> Thoughts? > > > >> > > > >> Davor > > > > > > > > -- > > > > Jean-Baptiste Onofré > > > > jbono...@apache.org > > > > http://blog.nanthrax.net > > > > Talend - http://www.talend.com > > > > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: Build failed in Jenkins: beam_SeedJob #214

ERROR: script not yet approved for use > > > On Tue, Apr 18, 2017 at 10:16 AM, Jason Kuster < > jasonkus...@google.com.invalid> wrote: > > > I'm looking into this currently as well; that's one of the mitigations > I'm > > considering too but I

Re: Build failed in Jenkins: beam_SeedJob #214

tImpl.java:146) > > > > at hudson.remoting.UserRequest.perform(UserRequest.java:153) > > > > at hudson.remoting.UserRequest.perform(UserRequest.java:50) > > > > at hudson.remoting.Request$2.run(Request.java:336) > > > > at hudson.remoting.InterceptingExecutorService$1.call( > > > > InterceptingExecutorService.java:68) > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > > > > ThreadPoolExecutor.java:1145) > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > > > > ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:745) > > > > at ..remote call to beam4(Native Method) > > > > at hudson.remoting.Channel.attachCallSiteStackTrace( > > > > Channel.java:1537) > > > > at hudson.remoting.UserResponse. > retrieve(UserRequest.java:253) > > > > at hudson.remoting.Channel.call(Channel.java:822) > > > > at org.jenkinsci.plugins.gitclient.RemoteGitImpl$ > > > > CommandInvocationHandler.execute(RemoteGitImpl.java:146) > > > > at sun.reflect.GeneratedMethodAccessor862.invoke(Unknown > > Source) > > > > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > > > > DelegatingMethodAccessorImpl.java:43) > > > > at java.lang.reflect.Method.invoke(Method.java:498) > > > > at org.jenkinsci.plugins.gitclient.RemoteGitImpl$ > > > > CommandInvocationHandler.invoke(RemoteGitImpl.java:132) > > > > at com.sun.proxy.$Proxy103.execute(Unknown Source) > > > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:804) > > > > ... 11 more > > > > ERROR: null > > > > Retrying after 10 seconds > > > > > git rev-parse --is-inside-work-tree # timeout=10 > > > > Fetching changes from the remote Git repository > > > > > git config remote.origin.url https://github.com/apache/beam.git # > > > > timeout=10 > > > > Pruning obsolete local branches > > > > Fetching upstream changes from https://github.com/apache/beam.git > > > > > git --version # timeout=10 > > > > > git fetch --tags --progress https://github.com/apache/beam.git > > > > +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/ > > > origin/pr/* > > > > --prune > > > > > git rev-parse origin/master^{commit} # timeout=10 > > > > Checking out Revision 034bcb4e2a2ee59c1a9bf16690547765064327e2 > > > > (origin/master) > > > > > git config core.sparsecheckout # timeout=10 > > > > > git checkout -f 034bcb4e2a2ee59c1a9bf16690547765064327e2 > > > > > git rev-list c52ce7c4bd952d943bccb8acff53b36b40c35428 # > timeout=10 > > > > Cleaning workspace > > > > > git rev-parse --verify HEAD # timeout=10 > > > > Resetting working tree > > > > > git reset --hard # timeout=10 > > > > > git clean -fdx # timeout=10 > > > > [EnvInject] - Executing scripts and injecting environment variables > > after > > > > the SCM step. > > > > [EnvInject] - Injecting as environment variables the properties > content > > > > SPARK_LOCAL_IP=127.0.0.1 > > > > > > > > [EnvInject] - Variables injected successfully. > > > > Processing DSL script job_beam_PerformanceTests_Dataflow.groovy > > > > ERROR: startup failed: > > > > <https://builds.apache.org/job/beam_SeedJob/ws/.test- > > > > infra/jenkins/job_beam_PerformanceTests_Dataflow.groovy>: 19: unable > > to > > > > resolve class common_job_properties > > > > @ line 19, column 1. > > > >import common_job_properties > > > >^ > > > > > > > > 1 error > > > > > > > > > > > > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: [DISCUSS] Change "RunnableOnService" To A More Intuitive Name

gt; > > > > > > > > >> > > > The idea of two categories with a common superclass > > > > actually > > > > > > > has a > > > > > > > > >> > > pitfall: > > > > > > > > >> > > > what if a test is put in the superclass category, > when > > > it > > > > > does > > > > > > > not > > > > > > > > >> > have a > > > > > > > > >> > > > clear meaning? And also, I don't have any good ideas > > for > > > > > > names. > > > > > > > > >> > > > > > > > > > > > >> > > > So I think just replacing RunnableOnService with > > > > RunnerTest > > > > > to > > > > > > > > make > > > > > > > > >> > clear > > > > > > > > >> > > > that it is there just to test the runner is good. We > > > might > > > > > > also > > > > > > > > want > > > > > > > > >> > > > RunnerIntegrationTest extends NeedsRunner to use in > > the > > > IO > > > > > > > > modules. > > > > > > > > >> > > > > > > > > > > > >> > > > See also Thomas's doc on capability matrix testing* > > > which > > > > is > > > > > > > > aimed at > > > > > > > > >> > > case > > > > > > > > >> > > > 2. Those tests should all have a category from the > > doc, > > > > or a > > > > > > new > > > > > > > > one > > > > > > > > >> > > added. > > > > > > > > >> > > > > > > > > > > > >> > > > * > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > https://docs.google.com/document/d/ > 1fICxq32t9yWn9qXhmT07xpclHeHX2 > > > > > > > > >> > VlUyVtpi2WzzGM/edit > > > > > > > > >> > > > > > > > > > > > >> > > > Kenn > > > > > > > > >> > > > > > > > > > > > >> > > > On Wed, Nov 9, 2016 at 12:20 PM, Jean-Baptiste > Onofré > > < > > > > > > > > >> j...@nanthrax.net > > > > > > > > >> > > > > > > > > > > >> > > > wrote: > > > > > > > > >> > > > > > > > > > > > >> > > > > Hi Mark, > > > > > > > > >> > > > > > > > > > > > > >> > > > > Generally speaking, I agree. > > > > > > > > >> > > > > > > > > > > > > >> > > > > As RunnableOnService extends NeedsRunner, > > > > @TestsWithRunner > > > > > > or > > > > > > > > >> > > > @RunOnRunner > > > > > > > > >> > > > > sound clearer. > > > > > > > > >> > > > > > > > > > > > > >> > > > > Regards > > > > > > > > >> > > > > JB > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > On 11/09/2016 09:00 PM, Mark Liu wrote: > > > > > > > > >> > > > > > > > > > > > > >> > > > >> Hi all, > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> I'm working on building RunnableOnService in > Python > > > > SDK. > > > > > > > After > > > > > > > > >> > having > > > > > > > > >> > > > >> discussions with folks, "RunnableOnService" looks > > > like > > > > > not > > > > > > a > > > > > > > > very > > > > > > > > >> > > > >> intuitive > > > > > > > > >> > > > >> name for those unit tests that require runners > and > > > > build > > > > > > > > >> lightweight > > > > > > > > >> > > > >> pipelines to test specific components. > Especially, > > > they > > > > > > don't > > > > > > > > have > > > > > > > > >> > to > > > > > > > > >> > > > run > > > > > > > > >> > > > >> on a service. > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> So I want to raise this idea to the community and > > see > > > > if > > > > > > > anyone > > > > > > > > >> have > > > > > > > > >> > > > >> similar thoughts. Maybe we can come up with a > name > > > this > > > > > is > > > > > > > > tight > > > > > > > > >> to > > > > > > > > >> > > > >> runner. > > > > > > > > >> > > > >> Currently, I have two names in my head: > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> - TestsWithRunners > > > > > > > > >> > > > >> - RunnerExecutable > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> Any thoughts? > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> Thanks, > > > > > > > > >> > > > >> Mark > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > > -- > > > > > > > > >> > > > > Jean-Baptiste Onofré > > > > > > > > >> > > > > jbono...@apache.org > > > > > > > > >> > > > > http://blog.nanthrax.net > > > > > > > > >> > > > > Talend - http://www.talend.com > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: [DISCUSS] Change "RunnableOnService" To A More Intuitive Name

ICxq32t9yWn9qXhmT07xpclHeHX2 > > > > >> > VlUyVtpi2WzzGM/edit > > > > >> > > > > > > > >> > > > Kenn > > > > >> > > > > > > > >> > > > On Wed, Nov 9, 2016 at 12:20 PM, Jean-Baptiste Onofré < > > > > >> j...@nanthrax.net > > > > >> > > > > > > >> > > > wrote: > > > > >> > > > > > > > >> > > > > Hi Mark, > > > > >> > > > > > > > > >> > > > > Generally speaking, I agree. > > > > >> > > > > > > > > >> > > > > As RunnableOnService extends NeedsRunner, @TestsWithRunner > > or > > > > >> > > > @RunOnRunner > > > > >> > > > > sound clearer. > > > > >> > > > > > > > > >> > > > > Regards > > > > >> > > > > JB > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > On 11/09/2016 09:00 PM, Mark Liu wrote: > > > > >> > > > > > > > > >> > > > >> Hi all, > > > > >> > > > >> > > > > >> > > > >> I'm working on building RunnableOnService in Python SDK. > > > After > > > > >> > having > > > > >> > > > >> discussions with folks, "RunnableOnService" looks like > not > > a > > > > very > > > > >> > > > >> intuitive > > > > >> > > > >> name for those unit tests that require runners and build > > > > >> lightweight > > > > >> > > > >> pipelines to test specific components. Especially, they > > don't > > > > have > > > > >> > to > > > > >> > > > run > > > > >> > > > >> on a service. > > > > >> > > > >> > > > > >> > > > >> So I want to raise this idea to the community and see if > > > anyone > > > > >> have > > > > >> > > > >> similar thoughts. Maybe we can come up with a name this > is > > > > tight > > > > >> to > > > > >> > > > >> runner. > > > > >> > > > >> Currently, I have two names in my head: > > > > >> > > > >> > > > > >> > > > >> - TestsWithRunners > > > > >> > > > >> - RunnerExecutable > > > > >> > > > >> > > > > >> > > > >> Any thoughts? > > > > >> > > > >> > > > > >> > > > >> Thanks, > > > > >> > > > >> Mark > > > > >> > > > >> > > > > >> > > > >> > > > > >> > > > > -- > > > > >> > > > > Jean-Baptiste Onofré > > > > >> > > > > jbono...@apache.org > > > > >> > > > > http://blog.nanthrax.net > > > > >> > > > > Talend - http://www.talend.com > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

First IO IT Running!

[3] https://docs.google.com/document/d/1PsjGPSN6FuorEEPrKEP3u3m16tyOzph5FnL2DhaRDz0/edit?ts=58a78e73 -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: [ANNOUNCEMENT] New committers, March 2017 edition!

aphs/contributors?from= > > 2016-02-01&to=2017-03-17&type=c > > [2] > > https://github.com/apache/beam/blob/v0.6.0/sdks/python/ > > apache_beam/io/iobase.py#L70 > > [3] > > https://github.com/apache/beam/blob/v0.6.0/sdks/python/ > > apache_beam/io/iobase.py#L561 > > [4] https://s.apache.org/splittable-do-fn > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: Performance Testing Next Steps

t; Thanks, > Ismaël > > On Thu, Mar 2, 2017 at 11:49 PM, Amit Sela wrote: > > Looks great, and I'll be sure to follow this. Ping me if I can assist in > > any way! > > > > On Fri, Mar 3, 2017 at 12:09 AM Ahmet Altay > > wrote: > > > >>

Re: Jenkins build became unstable: beam_SeedJob #165

Manual build, ignore. On Tue, Mar 14, 2017 at 10:37 PM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See <https://builds.apache.org/job/beam_SeedJob/165/display/redirect> > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: Performance Testing Next Steps

PM, Ahmet Altay wrote: > Thank you Jason, this is great. > > Which one of these issues fall into the land of sdk-py? > > Ahmet > > On Thu, Mar 2, 2017 at 12:34 PM, Jason Kuster < > jasonkus...@google.com.invalid> wrote: > > > Glad to hear the excitement. :) >

Re: Performance Testing Next Steps

anything to add, but I'm > pretty excited to follow this project. Have the JIRAs been filed since you > shared the doc? > > On Wed, Feb 22, 2017 at 10:38 AM, Jason Kuster < > jasonkus...@google.com.invalid> wrote: > > > Hey all, just wanted to pop this up agai

Re: Performance Testing Next Steps

Hey all, just wanted to pop this up again for people -- if anyone has thoughts on performance testing please feel welcome to chime in. :) On Fri, Feb 17, 2017 at 4:03 PM, Jason Kuster wrote: > Hi all, > > I've written up a doc on next steps for getting performance testing up and

Performance Testing Next Steps

all aspects of Beam, which would be really exciting. Looking forward to your thoughts. https://docs.google.com/document/d/1PsjGPSN6FuorEEPrKEP3u3m16tyOzph5FnL2DhaRDz0/edit?ts=58a78e73 Best, Jason -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: Build failed in Jenkins: beam_SeedJob #115

PoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > at ..remote call to beam1(Native Method) > at hudson.remoting.Channel.attachCallSiteStackTrace( > Channel.java:1416) > at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) > at hudson.remoting.Channel.call(Channel.java:781) > at org.jenkinsci.plugins.gitclient.RemoteGitImpl$ > CommandInvocationHandler.execute(RemoteGitImpl.java:145) > at sun.reflect.GeneratedMethodAccessor637.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.jenkinsci.plugins.gitclient.RemoteGitImpl$ > CommandInvocationHandler.invoke(RemoteGitImpl.java:131) > at com.sun.proxy.$Proxy93.execute(Unknown Source) > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:797) > ... 11 more > ERROR: null > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: Build failed in Jenkins: beam_SeedJob #113

gt; [EnvInject] - Variables injected successfully. > Processing DSL script job_beam_PostCommit_Java_MavenInstall.groovy > ERROR: (common_job_properties.groovy, line 33) No signature of method: > static common_job_properties.setTopLevelJobProperties() is applicable for > argument types

Re: We've hit 2000 PRs!

wrote: > > > > > Hey folks! > > > > > > Just wanted to send out a note -- we've hit 1000 PRs in GitHub as of > > > Saturday! That's a tremendous amount of work for the 7 months since > PR#1. > > > > > > I bet we hit 2000 in much fewer than 7 months ;) > > > > > > Dan > > > > > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: Jenkins build became unstable: beam_PostCommit_Java_MavenInstall #2510

Looks like a flake in WindowedWordCount -- listed change seems unrelated. On Thu, Feb 2, 2017 at 11:14 AM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See <https://builds.apache.org/job/beam_PostCommit_Java_ > MavenInstall/2510/changes> > > -- ---

Re: Jenkins build became unstable: beam_PostCommit_Java_RunnableOnService_Apex #363

> wrote: > See <https://builds.apache.org/job/beam_PostCommit_Java_ > RunnableOnService_Apex/363/changes> > > -- --- Jason Kuster Apache Beam / Google Cloud Dataflow

Re: [ANNOUNCEMENT] New committers, January 2017 edition!

redesigning IOChannelFactory / FileSystem APIs (in > > progress), which would extend Beam’s portability to many additional file > > systems and cloud providers. > > > > Congratulations to all three! Welcome! > > > > Davor > > > -- --- Jason Kuster Apache Beam (Incubating) / Google Cloud Dataflow

Re: Better developer instructions for using Maven?

t; >>>>>>>> Dan > >>>>>>>> > >>>>>>>> > >>>>>>>> They say to run mvn verify before commits. > >>>>>>>>> > >>>>>>>>> On Thu, Jan 5, 2017 at 9:25 AM Dan Halperin > >>>>>>>>> > >>>>>>>> >>>>>>> > >>>>>>>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> Several folks seem to have been confused after BEAM-246, where > >>>>>>>>>> > >>>>>>>>> we > >>>> > >>>>> moved > >>>>>>> > >>>>>>>> the > >>>>>>>>> > >>>>>>>>>> "slow things" into the release profile. I've started a > >>>>>>>>>> > >>>>>>>>> discussion > >>>> > >>>>> with > >>>>>>> > >>>>>>>> https://github.com/apache/beam/pull/1740 to see if there are > >>>>>>>>>> > >>>>>>>>> things > >>>>> > >>>>>> we > >>>>>>> > >>>>>>>> can > >>>>>>>>> > >>>>>>>>>> do to fill these gaps. > >>>>>>>>>> > >>>>>>>>>> Would love folks to chime in with opinions. > >>>>>>>>>> > >>>>>>>>>> Dan > >>>>>>>>>> > >>>>>>>>>> On Wed, Jan 4, 2017 at 1:34 PM, Jesse Anderson < > >>>>>>>>>> > >>>>>>>>> je...@smokinghand.com> > >>>>>>> > >>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> @Eugene, yes that failed on the checkstyle. > >>>>>>>>>>> > >>>>>>>>>>> On Wed, Jan 4, 2017 at 1:27 PM Eugene Kirpichov > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Try just -Prelease. > >>>>>>>>>>>> On Wed, Jan 4, 2017 at 1:21 PM Jesse Anderson < > >>>>>>>>>>>> > >>>>>>>>>>> je...@smokinghand.com > >>>>>>>> > >>>>>>>>> > >>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> Fails because I don't have a secret key. > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Wed, Jan 4, 2017 at 1:03 PM Jean-Baptiste Onofré < > >>>>>>>>>>>>> > >>>>>>>>>>>> j...@nanthrax.net > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Hi Jesse, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Could you try the same with: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> mvn verify -Prelease,apache-release > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> ? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Regards > >>>>>>>>>>>>>> JB > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On 01/04/2017 09:53 PM, Jesse Anderson wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> For some reason, running "mvn verify" isn't running > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> checkstyle > >>>>>>>> > >>>>>>>>> on > >>>>>>>>> > >>>>>>>>>> everything. I had checkstyle errors in > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> beam-sdks-java-core > >>>>> > >>>>>> that > >>>>>>>> > >>>>>>>>> weren't > >>>>>>>>>>>> > >>>>>>>>>>>>> being found. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I thought this was due to the extra parameters. I > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> reran > >>>> > >>>>> with > >>>>>>> > >>>>>>>> the > >>>>>>>>> > >>>>>>>>>> plain > >>>>>>>>>>>> > >>>>>>>>>>>>> "mvn > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> verify" and it still didn't find them. From the > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> output, > >>>> > >>>>> it > >>>>> > >>>>>> doesn't > >>>>>>>>>> > >>>>>>>>>>> look > >>>>>>>>>>>> > >>>>>>>>>>>>> like they're being run at all. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Jesse > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> -- > >>>>>>>>>>>>>> Jean-Baptiste Onofré > >>>>>>>>>>>>>> jbono...@apache.org > >>>>>>>>>>>>>> http://blog.nanthrax.net > >>>>>>>>>>>>>> Talend - http://www.talend.com > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>>> > >>> > >> > > -- > > Jean-Baptiste Onofré > > jbono...@apache.org > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > > -- --- Jason Kuster Apache Beam (Incubating) / Google Cloud Dataflow

Precommit Build Fixed

), so please feel free to kick Jenkins off again on your PRs. Best, Jason -- --- Jason Kuster Apache Beam (Incubating) / Google Cloud Dataflow

Re: Build failed in Jenkins: beam_PostCommit_Java_RunnableOnService_Dataflow #2046

dk.util.ReshuffleTest > > Results : > > Tests run: 199, Failures: 0, Errors: 0, Skipped: 3 > > [JENKINS] Recording test results > 2017-01-18T07:12:26.996 [INFO] > 2017-01-18T07:12:26.996 [INFO] --- maven-dependency-plugin:2.10:analyze-only > (default) @ beam-runners-google-cloud-dataflow-java --- > 2017-01-18T07:12:27.054 [INFO] No dependency problems found > [JENKINS] Archiving disabled > [JENKINS] Archiving disabled > [JENKINS] Archiving disabled > [JENKINS] Archiving disabled > [JENKINS] Archiving disabled > [JENKINS] Archiving disabled > [JENKINS] Archiving disabled > 2017-01-18T07:12:29.809 [INFO] -- > -- > 2017-01-18T07:12:29.809 [INFO] Reactor Summary: > 2017-01-18T07:12:29.810 [INFO] > 2017-01-18T07:12:29.810 [INFO] Apache Beam :: Parent > .. SUCCESS [ 21.063 s] > 2017-01-18T07:12:29.810 [INFO] Apache Beam :: SDKs > SUCCESS [ 0.519 s] > 2017-01-18T07:12:29.810 [INFO] Apache Beam :: SDKs :: Java > SUCCESS [ 0.454 s] > 2017-01-18T07:12:29.810 [INFO] Apache Beam :: SDKs :: Java :: Core > SUCCESS [02:09 min] > 2017-01-18T07:12:29.810 [INFO] Apache Beam :: Runners > . SUCCESS [ 0.600 s] > 2017-01-18T07:12:29.810 [INFO] Apache Beam :: Runners :: Google Cloud > Dataflow SUCCESS [ 01:08 h] > 2017-01-18T07:12:29.810 [INFO] -- > -- > 2017-01-18T07:12:29.810 [INFO] BUILD SUCCESS > 2017-01-18T07:12:29.810 [INFO] -- > -- > 2017-01-18T07:12:29.810 [INFO] Total time: 01:11 h > 2017-01-18T07:12:29.810 [INFO] Finished at: 2017-01-18T07:12:29+00:00 > 2017-01-18T07:12:30.215 [INFO] Final Memory: 90M/3226M > 2017-01-18T07:12:30.215 [INFO] -- > -- > Waiting for Jenkins to finish collecting data > channel stopped > -- --- Jason Kuster Apache Beam (Incubating) / Google Cloud Dataflow

Re: Build failed in Jenkins: beam_PostCommit_Java_MavenInstall #2365

:498) > at org.codehaus.plexus.classworlds.launcher.Launcher. > launchStandard(Launcher.java:330) > at org.codehaus.plexus.classworlds.launcher.Launcher. > launch(Launcher.java:238) > at jenkins.maven3.agent.Maven32Main.launch(Maven32Main.java:186) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at hudson.maven.Maven3Builder.call(Maven3Builder.java:136) > at hudson.maven.Maven3Builder.call(Maven3Builder.java:71) > at hudson.remoting.UserRequest.perform(UserRequest.java:153) > at hudson.remoting.UserRequest.perform(UserRequest.java:50) > at hudson.remoting.Request$2.run(Request.java:332) > at hudson.remoting.InterceptingExecutorService$1.call( > InterceptingExecutorService.java:68) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > 2017-01-17T22:05:40.361 [ERROR] > 2017-01-17T22:05:40.361 [ERROR] Re-run Maven using the -X switch to enable > full debug logging. > 2017-01-17T22:05:40.361 [ERROR] > 2017-01-17T22:05:40.361 [ERROR] For more information about the errors and > possible solutions, please read the following articles: > 2017-01-17T22:05:40.361 [ERROR] [Help 1] http://cwiki.apache.org/ > confluence/display/MAVEN/MojoNotFoundException > channel stopped > -- --- Jason Kuster Apache Beam (Incubating) / Google Cloud Dataflow

Re: Jenkins build became unstable: beam_PostCommit_Java_MavenInstall #2341

2:11 PM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See <https://builds.apache.org/job/beam_PostCommit_Java_ > MavenInstall/2341/changes> > > -- --- Jason Kuster Apache Beam (Incubating) / Google Cloud Dataflow

Re: Some Thoughts on IO Integration Tests

and timeout b/c the port isn't > correct. > > > > I also don't think that having per-IT options is a good idea. If 2 > > different ITs can share a data store, we should be able to pass them the > > same config - that also makes me strongly favor an option where

Re: Jenkins build became unstable: beam_Release_NightlySnapshot #293

er < jenk...@builds.apache.org> wrote: > See <https://builds.apache.org/job/beam_Release_ > NightlySnapshot/293/changes> > > -- --- Jason Kuster Apache Beam (Incubating) / Google Cloud Dataflow

Some Thoughts on IO Integration Tests

spective of the Jenkins executor, it can essentially just turn all ITs on and run them as is. * Downsides * Hardest for individual developers to run an entire suite of ITs -- since defaults are configured for running in CI environment, they will likely fail when running on the user's machi

Re: Jenkins build became unstable: beam_SeedJob_Main #74

Manual build -- ignore. On Tue, Jan 10, 2017 at 11:28 AM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See <https://builds.apache.org/job/beam_SeedJob_Main/74/> > > -- --- Jason Kuster Apache Beam (Incubating) / Google Cloud Dataflow

Re: Graduation!

>> Regards, > > > > >>>>>> Jacky > > > > >>>>>> > > > > >>>>>>> 在 2017年1月10日，下午7:11，Sergio Fernández 写道： > > > > >>>>>>> > > > > >>>>>&

Re: Jenkins build became unstable: beam_PostCommit_Java_RunnableOnService_Apex #119

Error is Expected: <1970-01-01T00:00:00.000Z> but: was <2016-12-27T17:53:23.629Z> 2016-12-27T17:53:23.629Z is the time the test was run. Looks like it could be an actual issue? On Tue, Dec 27, 2016 at 11:11 AM, Jason Kuster wrote: > Sorry, this is for #117. Moving there

Re: Jenkins build became unstable: beam_PostCommit_Java_RunnableOnService_Apex #117

ache.org/job/beam_PostCommit_Java_ > RunnableOnService_Apex/117/> > > -- --- Jason Kuster Apache Beam (Incubating) / Google Cloud Dataflow

Re: Jenkins build became unstable: beam_PostCommit_Java_RunnableOnService_Apex #119

Sorry, this is for #117. Moving there, investigating #119. On Tue, Dec 27, 2016 at 11:10 AM, Jason Kuster wrote: > Pipeline succeeded when it should have failed (PAssert for a > containsinanyorder on (1, 2, 3, 4) and (2, 1, 3, 4, 7)). Is there a known > issue for this? > > On Tu

Re: Jenkins build became unstable: beam_PostCommit_Java_RunnableOnService_Apex #119

ache.org/job/beam_PostCommit_Java_ > RunnableOnService_Apex/119/changes> > > -- --- Jason Kuster Apache Beam (Incubating) / Google Cloud Dataflow