Re: Apache-hosted CI (was Re: Build error in master branch)

Julian Hyde wrote: A*contributor* doesn’t need to run checkstyle every build. But a*committer* has a special role in Apache as a gatekeeper. I think a committer should run checkstyle before every commit to master. I don’t think it is an undue burden to ask a committer to run checkstyle and

Re: Apache-hosted CI (was Re: Build error in master branch)

As you say, it’s a matter of pragmatics what standards we set for what goes into master. In my opinion, we should check things which are binary. We can’t check non-deterministic tests, and we can’t check for performance. Those will show up over time. We can check for compliance with automated c

Re: Apache-hosted CI (was Re: Build error in master branch)

>especially considering the benefit to all of having the master branch always >at a particular standard. That is of no discussion. You are 100% right here. However, that statement is a bit sparse. What I mean is "a standard" could be to "test javadoc in CI only". Or even "skip checkstyle for def

Re: Apache-hosted CI (was Re: Build error in master branch)

Vladimir, We’re not going to be able to have a civilized discussion if you don’t read what I write. I said that I would not put an “undue burden” on committers, and you chose to ignore that. Let’s consider what is a reasonable burden, especially considering the benefit to all of having the ma

Re: Apache-hosted CI (was Re: Build error in master branch)

>Is the water in the river safe to drink? You need a test for that (== CI). No tests -- no safety. Frankly speaking, I would rather go though "send PR, wait CI, then use that commit" route instead of "mvn test in my dev environment, then commit". At the end of the day, my git working copy is nev

Re: Apache-hosted CI (was Re: Build error in master branch)

Vladimir Sitnikov wrote: We can get as fancy as we'd like :) I suppose job tuning require some grants. What is the process to get that? Vladimir See my earlier comment, Julian should be the one to assign your Karma. You either have an account already (from a PMC committership) or you sign

Re: Apache-hosted CI (was Re: Build error in master branch)

Is the water in the river safe to drink? Each committer is ensuring that it is safe to drink. We can change the definition of “safe” so we don’t put an undue burden on the committer. But I would be inclined to put more burden on the committer so that each developer knows that the master branch

Re: Apache-hosted CI (was Re: Build error in master branch)

So, I see this as two things. What I did last night is just a regular CI build that runs periodically to make sure the repository is good. We will also want to have a PreCommit job which is a TBD for me to hook up (this is where Yetus will come into play) which will run some changeset through r

Re: Apache-hosted CI (was Re: Build error in master branch)

> If there are even checkstyle errors, they should be fixed before the commit. Of course they should. However, I am not sure if it wise to ask "full javadoc build" before each commit. It takes enormous time (especially, if jdk sources are included to properly inherit javadocs from java.* classes).

Re: Apache-hosted CI (was Re: Build error in master branch)

Yes. I see this build as the basic minimum, so we want it to pass 100% of the time. By the way, that means that committers should run through this process BEFORE they commit. If there are even checkstyle errors, they should be fixed before the commit. > On Feb 9, 2016, at 11:36 AM, Josh Elser

Re: Apache-hosted CI (was Re: Build error in master branch)

>We can get as fancy as we'd like :) I suppose job tuning require some grants. What is the process to get that? Vladimir

Re: Apache-hosted CI (was Re: Build error in master branch)

Are we good with Jenkins notifications coming to dev@calcite for now? Julian Hyde wrote: Anyway, don’t worry too much about why the test is failing. We’ll figure out it. The task is complete when you’ve added a link to the web page. And thanks again. Julian On Feb 9, 2016, at 11:12 AM, Josh

Re: Apache-hosted CI (was Re: Build error in master branch)

Vladimir Sitnikov wrote: Cool. I wonder: 1) Can we use Docker? I think that is the easiest way to launch "real databases" in there. Well, docker management is no way easy, however if you have other options, just propose. I do not think it is wise to "install DB from scratch for each CI run". 2)

Re: Apache-hosted CI (was Re: Build error in master branch)

Cool. I wonder: 1) Can we use Docker? I think that is the easiest way to launch "real databases" in there. Well, docker management is no way easy, however if you have other options, just propose. I do not think it is wise to "install DB from scratch for each CI run". 2) Are there startup penalties

Re: Apache-hosted CI (was Re: Build error in master branch)

Anyway, don’t worry too much about why the test is failing. We’ll figure out it. The task is complete when you’ve added a link to the web page. And thanks again. Julian > On Feb 9, 2016, at 11:12 AM, Josh Elser wrote: > > I'm not sure, actually. The first run passed, I haven't looked closely

Re: Apache-hosted CI (was Re: Build error in master branch)

I'm not sure, actually. The first run passed, I haven't looked closely enough to guess why it fails. Will try to take a look tonight. For those who don't already have Jenkins accounts, I think Julian (as VP) should have the karma to give you the necessary permission to update/tweak the Jenkins

Re: Apache-hosted CI (was Re: Build error in master branch)

Fantastic! We’ve needed CI for a long time. I think the only thing left is to link to that site from somewhere on our web site, maybe http://calcite.apache.org/develop/ . I’ve assigned https://issues.apache.org/jira/browse/CALCITE-623 to you. Can you mark it

Re: Apache-hosted CI (was Re: Build error in master branch)

Better late than never? https://builds.apache.org/view/A-D/view/Calcite/ Here are two jobs that will run hourly (if changes are present), compile, run tests, and build javadocs. As an added perk, if the build was successful, it will deploy the snapshot to the Apache snapshots repository (hoor

Re: Apache-hosted CI (was Re: Build error in master branch)

Ugh, sorry, I've been otherwise swamped and haven't looked into this at all. https://blogs.apache.org/infra/entry/github_pull_request_builds_now covers the infra post on what they recently providing for building/testing pull requests. In general, we have access to a Jenkins instance which we

Re: Apache-hosted CI (was Re: Build error in master branch)

Any success with CI? I just came across https://travis-ci.org/apache/jackrabbit-oak, and they somehow managed to enable Travis for an Apache project. I've no idea what opportunities Apache's CI provides, however having some sort (e.g. Travis) of "github PR testing" would be good. Vladimir

Re: Apache-hosted CI (was Re: Build error in master branch)

#1 and #4 are easily achieved. We could easily run the failsafe-plugin runnable ITs, but I'm not sure about automating Vladimir's VM for the rest of #2. I'll have to see what Windows support actually exists within the ASF for #3. Let me see about getting the ball rolling.. Julian Hyde wrote:

Re: Apache-hosted CI (was Re: Build error in master branch)

Here are the tasks that I see, at varying levels of ambition. 1. Most basic would be to have Jenkins running a build and core tests after each commit, just so we’re sure the basics work. 2. More advanced than that would be to run the integration tests. Those require a running VM set up by Vladi

Apache-hosted CI (was Re: Build error in master branch)

There are actually a number of options. 1. We can enable some jobs that will build Calcite on commit via Jenkins (https://builds.apache.org). 2. We can enable a PreCommit job which should build every Patch/PullRequest that is attached to JIRA or Github via Apache Yetus (https://yetus.apache.