Hi Till, I have created the JIRA: https://issues.apache.org/jira/browse/FLINK-4205
Thank you, Do On Tue, Jul 12, 2016 at 6:05 PM, Till Rohrmann <trohrm...@apache.org> wrote: > Stratified sampling would also be beneficial for the DataSet API. I think > it would be best if this method is also added to DataSetUtils or made > available via the flink-contrib module. Furthermore, I think that it would > be easiest if you created the JIRA for this feature, because you know what > you want to add. For that you have to register at > https://issues.apache.org/jira, if you haven't done this, and then we can > add you as a contributor. Based on the JIRA description we can > discuss possible implementations then. > > Cheers, > Till > > On Tue, Jul 12, 2016 at 12:11 PM, Paris Carbone <par...@kth.se> wrote: > > > Hey Do, > > > > I think that more sophisticated samplers could make a better fit in the > ML > > library and not in the core API but I am not very familiar with the > > milestones there. > > Maybe the maintainers of the batch ML library could check if sampling > > techniques could be useful there I guess. > > > > Paris > > > > > On 11 Jul 2016, at 16:15, Le Quoc Do <lequo...@gmail.com> wrote: > > > > > > Hi all, > > > > > > Thank you all for your answers. > > > By the way, I also recognized that Flink doesn't support "stratified > > > sampling" function (only simple random sampling) for DataSet. > > > It would be nice if someone can create a Jira for it, and assign the > task > > > to me so that I can work for it. > > > > > > Thank you, > > > Do > > > > > > On Mon, Jul 11, 2016 at 11:44 AM, Vasiliki Kalavri < > > > vasilikikala...@gmail.com> wrote: > > > > > >> Hi Do, > > >> > > >> Paris and Martha worked on sampling techniques for data streams on > Flink > > >> last year. If you want to implement your own samplers, you might find > > >> Martha's master thesis helpful [1]. > > >> > > >> -Vasia. > > >> > > >> [1]: http://kth.diva-portal.org/smash/get/diva2:910695/FULLTEXT01.pdf > > >> > > >> On 11 July 2016 at 11:31, Kostas Kloudas <k.klou...@data-artisans.com > > > > >> wrote: > > >> > > >>> Hi Do, > > >>> > > >>> In DataStream you can always implement your own > > >>> sampling function, hopefully without too much effort. > > >>> > > >>> Adding such functionality it to the API could be a good idea. > > >>> But given that in sampling there is no “one-size-fits-all” > > >>> solution (as not every use case needs random sampling and not > > >>> all random samplers fit to all workloads), I am not sure if we > > >>> should start adding different sampling operators. > > >>> > > >>> Thanks, > > >>> Kostas > > >>> > > >>>> On Jul 9, 2016, at 5:43 PM, Greg Hogan <c...@greghogan.com> wrote: > > >>>> > > >>>> Hi Do, > > >>>> > > >>>> DataSet provides a stable @Public interface. DataSetUtils is marked > > >>>> @PublicEvolving which is intended for public use, has stable > behavior, > > >>> but > > >>>> method signatures may change. It's also good to limit DataSet to > > common > > >>>> methods whereas the utility methods tend to be used for specific > > >>>> applications. > > >>>> > > >>>> I don't have the pulse of streaming but this sounds like a useful > > >> feature > > >>>> that could be added. > > >>>> > > >>>> Greg > > >>>> > > >>>> On Sat, Jul 9, 2016 at 10:47 AM, Le Quoc Do <lequo...@gmail.com> > > >> wrote: > > >>>> > > >>>>> Hi all, > > >>>>> > > >>>>> I'm working on approximate computing using sampling techniques. I > > >>>>> recognized that Flink supports the sample function for Dataset > > >>>>> (org/apache/flink/api/java/utils/DataSetUtils.java). I'm just > > >> wondering > > >>> why > > >>>>> you didn't merge the function to > > >> org/apache/flink/api/java/DataSet.java > > >>>>> since the sample function works as a transformation operator? > > >>>>> > > >>>>> The second question is that are you planning to support the sample > > >>>>> function for DataStream (within windows) since I did not see it in > > >>>>> DataStream code ? > > >>>>> > > >>>>> Thank you, > > >>>>> Do > > >>>>> > > >>> > > >>> > > >> > > > > >
