Re: [build system] speeding up maven build building only changed modules compared to master branch

2019-01-26 Thread Sean Owen
Sounds interesting; would it be able to handle R and Python modules built by this project ? The home grown solution here does I think and that is helpful. On Sat, Jan 26, 2019, 6:57 AM vaclavkosar I think it would be good idea to use gitflow-incremental-builder maven > plugin for Spark builds. It

Re: [PySpark] Revisiting PySpark type annotations

2019-01-26 Thread zero323
As already pointed out by Nicholas, there is no Python 2 conflict here. Moreover, despite the fact that I used Python 3 specific feature, Python 2 users can benefit from the annotations as well in some circumstances (already mentioned MyPy is one option, PyCharm another, maybe some other tools as w

[build system] speeding up maven build building only changed modules compared to master branch

2019-01-26 Thread vaclavkosar
I think it would be good idea to use gitflow-incremental-builder maven plugin for Spark builds. It saves resources by building only modules that are impacted by changes compared to git master branch via gitflow-incremental-builder maven plugin. For example if there is only a change introduced i