Re: Scalastyle improvements / large code reformatting

2014-10-13 Thread Erik Erlandson
- Original Message - > I'm also against these huge reformattings. They slow down development and > backporting for trivial reasons. Let's not do that at this point, the style > of the current code is quite consistent and we have plenty of other things > to worry about. Instead, what you c

Re: Scalastyle improvements / large code reformatting

2014-10-13 Thread Nicholas Chammas
On Mon, Oct 13, 2014 at 11:57 AM, Patrick Wendell wrote: > That would even work for imports as well, > you'd just have a thing where if anyone modified some imports they > would have to fix all the imports in that file. It's at least worth a > try. > OK, that sounds like a fair compromise. I've

Re: Scalastyle improvements / large code reformatting

2014-10-13 Thread Marcelo Vanzin
Another option is to add new style rules that trigger too many errors as warnings, and slowly clean them up. This means that reviewers will be burdened with manually enforcing the rules for a while, and we need to remember to turn them to errors once some threshold is reached. (The Hadoop build ha

Re: Scalastyle improvements / large code reformatting

2014-10-13 Thread Patrick Wendell
Hey Nick, I think the best solution is really to find a way to only apply certain rules to code modified after a certain date. I also don't think it would be that hard to implement because git can output per-line information about modification times. So you'd just run the scalastyle rules and then

Re: Scalastyle improvements / large code reformatting

2014-10-13 Thread Nicholas Chammas
The arguments against large scale refactorings make sense. Doing them, if at all, during QA cycles or around releases sounds like a promising idea. Coupled with that, would it be useful to implement new rules outside of these potential windows for refactoring in such a way that they report on styl

Re: Scalastyle improvements / large code reformatting

2014-10-12 Thread Matei Zaharia
I'm also against these huge reformattings. They slow down development and backporting for trivial reasons. Let's not do that at this point, the style of the current code is quite consistent and we have plenty of other things to worry about. Instead, what you can do is as you edit a file when you

Re: Scalastyle improvements / large code reformatting

2014-10-12 Thread Patrick Wendell
Another big problem with these patches are that they make it almost impossible to backport changes to older branches cleanly (there becomes like 100% chance of a merge conflict). One proposal is to do this: 1. We only consider new style rules at the end of a release cycle, when there is the smalle

Re: Scalastyle improvements / large code reformatting

2014-10-12 Thread Reynold Xin
I actually think we should just take the bite and follow through with the reformatting. Many rules are simply not possible to enforce only on deltas (e.g. import ordering). That said, maybe there are better windows to do this, e.g. during the QA period. On Sun, Oct 12, 2014 at 9:37 PM, Josh Rosen

Scalastyle improvements / large code reformatting

2014-10-12 Thread Josh Rosen
There are a number of open pull requests that aim to extend Spark’s automated style checks (see https://issues.apache.org/jira/browse/SPARK-3849 for an umbrella JIRA).  These fixes are mostly good, but I have some concerns about merging these patches.  Several of these patches make large reforma