Hi,
yes, I think exposing a simple form of namespaces as String would be good.
By the way, do we also need access to state with namespaces then?
@Gyula: Please go ahead and open the Jira issue.
Cheers,
Aljoscha
On Sat, 29 Oct 2016 at 17:28 Gyula Fóra <gyula.f...@gmail.com> wrote:
> Thanks for the feedback guys,
>
> I think exposing the namespace in a simplified form in the user facing API
> is I think a very good idea, that already let's the users implement
> practically anything they want. Maybe doing it as a simple string as Jamie
> suggested would be a nice way to do it and that would serve as a label or
> timer-id for the user. Thinking of it as a label/id is probably a much
> simpler concept than the "namespace".
>
> Should we open a JIRA for this? Judging from the internal timer service
> this should be a fairly straight forward extension as Aljoscha pointed out.
>
> Gyula
>
> Jamie Grier <ja...@data-artisans.com> ezt írta (időpont: 2016. okt. 29.,
> Szo, 15:37):
>
> > Hi guys,
> >
> > Good points, Gyula. I think it would be much easier on a user if there
> > could be multiple timers in flight per key. I prefer the second
> approach,
> > though, where a user associates some bit of metadata with the timer and
> we
> > pass it back to them in the onTimer() callback, otherwise they are forced
> > to maintain this state themselves.
> >
> > It looks to me like somehow exposing the namespaces, even if it's simpler
> > and just a string, is the way to go.
> >
> > I'm really excited by this guys! I think the TimelyFlatMap and
> > TimelyCoFlatMap are going to get a LOT of use. This is gonna make a lot
> of
> > people happy.
> >
> > -Jamie
> >
> >
> > On Fri, Oct 28, 2016 at 1:58 PM, Aljoscha Krettek <aljos...@apache.org>
> > wrote:
> >
> > > Hi Gyula,
> > > if you look at the internal API you'll notice that it is pretty much
> like
> > > your second proposal. Just for reference, the interface is roughly
> this:
> > >
> > > public interface InternalTimerService<N> {
> > > long currentProcessingTime();
> > > long currentWatermark();
> > > void registerProcessingTimeTimer(N namespace, long time);
> > > void deleteProcessingTimeTimer(N namespace, long time);
> > > void registerEventTimeTimer(N namespace, long time);
> > > void deleteEventTimeTimer(N namespace, long time);
> > > }
> > >
> > > that namespace bit can be anything for which you can provide a
> > > TypeSerializer.
> > >
> > > IMHO, this goes back a bit to the discussion about adding a low level
> > > operator-like interface that allows pretty much anything a custom
> > operator
> > > can do but with a defined, stable interface. The internal operator
> > > interface is somewhat in flux, still, so I wouldn't recommend people to
> > use
> > > it directly.
> > >
> > > The only thing missing, really, from TimelyFlatMap is access to
> > namespaces
> > > for state and timers. With that, you could implement even the
> > > WindowOperator as a TimelyFlatMap since I worked on abstracting
> > everything
> > > that it uses away behind interfaces that any operator can use. The last
> > > pice, the generic timer API went in last, of course. :-)
> > >
> > > Cheers,
> > > Aljoscha
> > >
> > > On Fri, 28 Oct 2016 at 16:55 Gyula Fóra <gyf...@apache.org> wrote:
> > >
> > > > Hello,
> > > >
> > > > I was thinking about the methods provided by the timely functions and
> > the
> > > > timerservice and I am wondering if it makes sense to change them a
> > little
> > > > so they can cover a wider set of use case. Maybe I missed something
> > > > completely obvious so please shoot me down in that case :)
> > > >
> > > > Currently the user gets a TimerService to register timers that will
> in
> > > the
> > > > future call the onTimer method. It is not completely obvious to me
> how
> > > > would I implement a function that needs to trigger two types of
> > callbacks
> > > > in the future. If I get only one onTimer method I should be able to
> > pass
> > > in
> > > > some sort of parameter or flag so I can branch in my onTimer
> > > > implementation.
> > > >
> > > > As parameters are not supported I am left with states that are scoped
> > to
> > > > the keys which is also pretty useless if I want to trigger different
> > > timed
> > > > actions for the same keys.
> > > >
> > > > I know this is quite tricky but I see some alternative options:
> > > > - The register timer method returns a unique (per key) timer id, so
> we
> > > can
> > > > associate state with this id to fetch info about the timer
> registered.
> > > (We
> > > > could also remove timers with this id and maybe add methods to remove
> > all
> > > > for the current key)
> > > > - Allow the users to pass a custom parameter when registering the
> > > > callback, and the parameter would be passed to the onTimer method
> > > > - Allow users to pass custom callback functions when registering the
> > > > timers, but this would mean we have to support some sort of context
> for
> > > > accessing the state (like the window context before)
> > > > - We could go for an annotation based API like in beam but thats
> > > probably
> > > > not good to mix in the current ones
> > > >
> > > > I personally prefer the first one.
> > > >
> > > > What do you think?
> > > >
> > > > Regards,
> > > > Gyula
> > > >
> > >
> >
> >
> >
> > --
> >
> > Jamie Grier
> > data Artisans, Director of Applications Engineering
> > @jamiegrier <https://twitter.com/jamiegrier>
> > ja...@data-artisans.com
> >
>
