Re: Best way to join with inequalities (historical data)

2015-05-04 Thread Stephan Ewen
tthias J. Sax [mailto:mj...@informatik.hu-berlin.de] > Envoyé : lundi 4 mai 2015 11:52 > À : user@flink.apache.org > Objet : Re: Best way to join with inequalities (historical data) > > Hi, > > there is no other system support to express this join. > > However, you coul

RE: Best way to join with inequalities (historical data)

2015-05-04 Thread LINZ, Arnaud
.org Objet : Re: Best way to join with inequalities (historical data) Hi, there is no other system support to express this join. However, you could perform some "hand wired" optimization by partitioning your input data into distinct intervals. It might be tricky though. Especially, if the

Re: Best way to join with inequalities (historical data)

2015-05-04 Thread Matthias J. Sax
Hi, there is no other system support to express this join. However, you could perform some "hand wired" optimization by partitioning your input data into distinct intervals. It might be tricky though. Especially, if the time-ranges in your "range-key" dataset are overlapping everywhere (-> data r

Best way to join with inequalities (historical data)

2015-05-04 Thread LINZ, Arnaud
Hello, I was wondering how to join large data sets on inequalities. Let say I have a data set whose “keys” are two timestamps (start time & end time of validity) and value is a label : final DataSet> historical = …; I also have events, with an event name and a timestamp : final