Hi, If you wish to explore the behaviour of CLOP on a particular problem, I recommend the "test_swig.py" script in the programs/clop/script/artificial directory.
You have to edit this script to select the experiment you wish to run. First, select your problem. You can do this by uncommenting one (only) of the "p = " lines at the beginning of the script. For the Log function, uncomment "p = CPLog1D()" and comment out the other ones. Then, you can select the optimization method. The default is normal quadratic CLOP. You can also use cubic CLOP, CEM, or BAST if you wish. Finally, in the "main program" section you can select what you wish to do with this problem and method: - eb.threads will run many replications of the optimization experiment in parallel, and will produce some statistics - eb.gnuplot will run the optimization once and plot the samples. You can select the seed. - eb.multi_gnuplot will plot the optimization for many seeds in succession - eb.tikz: probably produces something similar to eb.gnuplot in tikz format for incorporation into LaTeX documents Rémi On 21 mars 2013, at 09:19, Chin-Chang Yang wrote: > Thank you, Remi and Olivier. > > I implemented the test functions in my project, and tested the performance of > my optimizers, which reached the regret to 1e-3 with 1e7 samples in the > 1-dimensional LOG function. In the CLOP paper, it reports that CLOP can reach > the regret to 1e-5 with 1e7 samples. It is very interesting that how CLOP can > perform so well. I am going to trace the CLOP codes by gdb, but I cannot find > the script that can reproduce the experimental results in Fig.4 (a). I guess > the script is located in programs/clop/script/artificial but I am not sure > which script can reproduce the experiments. > > I will be glad if anyone can give me some hints. > > Best regards, > Chin-Chang Yang, 2013/3/21 > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Remi Coulom > Sent: Wednesday, March 6, 2013 5:37 PM > To: [email protected] > Subject: Re: [Computer-go] CLOP: Confident Local Optimization for Noisy > Black-Box Parameter Tuning > > Yes. f(x) is not the output. The output is either 0 or 1, and f(x) is the > probability of 1. > > Rémi > > On 6 mars 2013, at 09:04, Chin-Chang Yang wrote: > >> Thank you, Olivier. >> >> Let the observable function value be o(x). It can be defined as: >> >> o(x) = 1, with probability f(x); >> o(x) = 0, with probability (1 - f(x)). >> >> where f(x) = 1 / (1 + e(-r(x))) has been defined in the paper. Also, we can >> see that the expected value is f(x). >> Did I get this correct? >> >> Best regards, >> Chin-Chang Yang, 2013/03/06 >> 2013/3/6 Olivier Teytaud <[email protected]> It's a Bernoulli noise. >> define f (x) = 1/ (1 + e(−r(x)) ) >> and the objective function at x is 1 with probability f(x). >> So the expected value at x is f(x), but the values you get are noisy. >> >> Best regards, >> Olivier >> >> Since the functions are not noise-free, they should be defined in terms of >> some noise. I really need the definition of the noise for comparison between >> CLOP and other optimizers. >> >> I have downloaded the source codes, but I cannot find the codes related to >> the noise currently. >> >> >> _______________________________________________ >> Computer-go mailing list >> [email protected] >> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go >> >> >> >> -- >> Chin-Chang Yang _______________________________________________ >> Computer-go mailing list >> [email protected] >> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go > > _______________________________________________ > Computer-go mailing list > [email protected] > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go > > _______________________________________________ > Computer-go mailing list > [email protected] > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go _______________________________________________ Computer-go mailing list [email protected] http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
