The order of magnitude matches my parameter numbers. (My attempt to reproduce a simplified version of this is currently evolving at https://github.com/pasky/michi/tree/nnet but the code is a mess right now.)
On Thu, Oct 19, 2017 at 07:23:31AM -0400, Álvaro Begué wrote: > This is a quick check of my understanding of the network architecture. > Let's count the number of parameters in the model: > * convolutional block: (17*9+1)*256 + 2*256 > [ 17 = number of input channels > 9 = size of the 3x3 convolution window > 1 = bias (I am not sure this is needed if you are going to do batch > normalization immediately after) > 256 = number of output channels > 2 = mean and standard deviation of the output of the batch normalization > 256 = number of channels in the batch normalization ] > * residual block: (256*9+1)*256 + 2*256 + (256*9+1)*256 + 2*256 > * policy head: (256*1+1)*2 + 2*2 + (2*361+1)*362 > * value head: (256*1+1)*1 + 2*1 + (1*361+1)*256 + (256+1)*1 > > Summing it all up, I get 22,837,864 parameters for the 20-block network and > 46,461,544 parameters for the 40-block network. > > Does this seem correct? > > Álvaro. > > > > On Thu, Oct 19, 2017 at 6:17 AM, Petr Baudis <pa...@ucw.cz> wrote: > > > On Wed, Oct 18, 2017 at 04:29:47PM -0700, David Doshay wrote: > > > I saw my first AlphaGo Zero joke today: > > > > > > After a few more months of self-play the games might look like this: > > > > > > AlphaGo Zero Black - move 1 > > > AlphaGo Zero White - resigns > > > > ...which is exactly what my quick attempt to reproduce AlphaGo Zero > > yesterday converged to overnight. ;-) But I'm afraid it's because of > > a bug, not wisdom... > > > > Petr Baudis > > _______________________________________________ > > Computer-go mailing list > > Computer-go@computer-go.org > > http://computer-go.org/mailman/listinfo/computer-go > > > _______________________________________________ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go -- Petr Baudis, Rossum Run before you walk! Fly before you crawl! Keep moving forward! If we fail, I'd rather fail really hugely. -- Moist von Lipwig _______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
