Thank you for your clarification. At the end I figured out, it was a silly programming error, now it just works fine. Thank you again for your explanation.
2015-05-11 22:31 GMT+02:00 Stephan Ewen <se...@apache.org>: > Can you share the respective part of the code, then we can have a look? > > In general, a DataSet is re-broadcasted in each iteration, if it depends > on the IterativeDataSet. > You have to re-grab it from the RuntimeContext in each superstep. If you > call getBroadcastSet() in the open() method, then it will get executed in > each superstep (open gets called at the beginning of each superstep). > > Greetings, > Stephan > > > On Mon, May 11, 2015 at 6:36 PM, Ventura Del Monte < > venturadelmo...@gmail.com> wrote: > >> Hello, >> >> I am writing a Flink application whose purpose is to train neural >> networks. I have implemented a version of SDG based on mini batches using >> the IterativeDataSet class. In a nutshell, my code groups the input >> features in a set called mini batch (i have as many mini batches as the >> parallelism degree), then it calculates a gradient over each mini batch in >> a map tasks and then those gradients are merged in a reduce task. >> Unfortunately it seems the trained param is not broadcasted at the end of >> each epoch and it looks like that it starts the next epoch using the >> initial parameter. I know it is probably a programming error which I cannot >> see at the moment, but I was wondering if there were some guidelines for >> broadcasting an object in the correct way. >> Is it asking too much if you could have a look at my code, please? Thank >> you in advance. >> >> Regards, >> Ventura >> > >
