Re: [Spark SQL]: SQL, Python, Scala and R API Consistency

2021-01-30 Thread Kent Yao
Hi, MrPowersI'm also interested in this idea.I started https://github.com/yaooqinn/spark-func-extras a few month agoOn 2021/01/30 15:45:30, Matthew Powers wrote: Maciej - I like the idea of a separate library to provide easy access to> functions that the maintainers don't want to m

Re: [Spark SQL]: SQL, Python, Scala and R API Consistency

2021-01-30 Thread Matthew Powers
Maciej - I like the idea of a separate library to provide easy access to functions that the maintainers don't want to merge into Spark core. I've seen this model work well in other open source communities. The Rails Active Support library provides the Ruby community with core functionality like b

How to contribute the code

2021-01-30 Thread hammerCS
I have learned Spark for one year .And now, I want to get the hang of spark by contributing code .. But the code is so massive , I don't know how to do. -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ -

Re: [Spark SQL]: SQL, Python, Scala and R API Consistency

2021-01-30 Thread Maciej
Just thinking out loud ‒ if there is community need for providing language bindings for less popular SQL functions, could these live outside main project or even outside the ASF?  As long as expressions are already implemented, bindings are trivial after all. If could also allow usage of more scal

Re: Hyperparameter Optimization via Randomization

2021-01-30 Thread Sean Owen
I was thinking ParamGridBuilder would have to change to accommodate a continuous range of values, and that's not hard, though other code wouldn't understand that type of value, like the existing simple grid builder. It's all possible just wondering if simply randomly sampling the grid is enough. Th

Re: Hyperparameter Optimization via Randomization

2021-01-30 Thread Phillip Henry
Hi, Sean. Perhaps I don't understand. As I see it, ParamGridBuilder builds an Array[ParamMap]. What I am proposing is a new class that also builds an Array[ParamMap] via its build() method, so there would be no "change in the APIs". This new class would, of course, have methods that defined the se