I don't know of anyone working on that. Yes I think it could be useful. I think it might be easiest to implement by simply having some parameter to the grid search process that says what fraction of all possible combinations you want to randomly test.
On Fri, Jan 29, 2021 at 5:52 AM Phillip Henry <londonjava...@gmail.com> wrote: > Hi, > > I have no work at the moment so I was wondering if anybody would be > interested in me contributing code that generates an Array[ParamMap] for > random hyperparameters? > > Apparently, this technique can find a hyperparameter in the top 5% of > parameter space in fewer than 60 iterations with 95% confidence [1]. > > I notice that the Spark code base has only the brute force > ParamGridBuilder unless I am missing something. > > Hyperparameter optimization is an area of interest to me but I don't want > to re-invent the wheel. So, if this work is already underway or there are > libraries out there to do it please let me know and I'll shut up :) > > Regards, > > Phillip > > [1] > https://www.oreilly.com/library/view/evaluating-machine-learning/9781492048756/ch04.html >
