Thanks for the feedback guys,
I think exposing the namespace in a simplified form in the user facing API
is I think a very good idea, that already let's the users implement
practically anything they want. Maybe doing it as a simple string as Jamie
suggested would be a nice way to do it and that would serve as a label or
timer-id for the user. Thinking of it as a label/id is probably a much
simpler concept than the "namespace".
Should we open a JIRA for this? Judging from the internal timer service
this should be a fairly straight forward extension as Aljoscha pointed out.
Gyula
Jamie Grier <ja...@data-artisans.com> ezt írta (időpont: 2016. okt. 29.,
Szo, 15:37):
> Hi guys,
>
> Good points, Gyula. I think it would be much easier on a user if there
> could be multiple timers in flight per key. I prefer the second approach,
> though, where a user associates some bit of metadata with the timer and we
> pass it back to them in the onTimer() callback, otherwise they are forced
> to maintain this state themselves.
>
> It looks to me like somehow exposing the namespaces, even if it's simpler
> and just a string, is the way to go.
>
> I'm really excited by this guys! I think the TimelyFlatMap and
> TimelyCoFlatMap are going to get a LOT of use. This is gonna make a lot of
> people happy.
>
> -Jamie
>
>
> On Fri, Oct 28, 2016 at 1:58 PM, Aljoscha Krettek <aljos...@apache.org>
> wrote:
>
> > Hi Gyula,
> > if you look at the internal API you'll notice that it is pretty much like
> > your second proposal. Just for reference, the interface is roughly this:
> >
> > public interface InternalTimerService<N> {
> > long currentProcessingTime();
> > long currentWatermark();
> > void registerProcessingTimeTimer(N namespace, long time);
> > void deleteProcessingTimeTimer(N namespace, long time);
> > void registerEventTimeTimer(N namespace, long time);
> > void deleteEventTimeTimer(N namespace, long time);
> > }
> >
> > that namespace bit can be anything for which you can provide a
> > TypeSerializer.
> >
> > IMHO, this goes back a bit to the discussion about adding a low level
> > operator-like interface that allows pretty much anything a custom
> operator
> > can do but with a defined, stable interface. The internal operator
> > interface is somewhat in flux, still, so I wouldn't recommend people to
> use
> > it directly.
> >
> > The only thing missing, really, from TimelyFlatMap is access to
> namespaces
> > for state and timers. With that, you could implement even the
> > WindowOperator as a TimelyFlatMap since I worked on abstracting
> everything
> > that it uses away behind interfaces that any operator can use. The last
> > pice, the generic timer API went in last, of course. :-)
> >
> > Cheers,
> > Aljoscha
> >
> > On Fri, 28 Oct 2016 at 16:55 Gyula Fóra <gyf...@apache.org> wrote:
> >
> > > Hello,
> > >
> > > I was thinking about the methods provided by the timely functions and
> the
> > > timerservice and I am wondering if it makes sense to change them a
> little
> > > so they can cover a wider set of use case. Maybe I missed something
> > > completely obvious so please shoot me down in that case :)
> > >
> > > Currently the user gets a TimerService to register timers that will in
> > the
> > > future call the onTimer method. It is not completely obvious to me how
> > > would I implement a function that needs to trigger two types of
> callbacks
> > > in the future. If I get only one onTimer method I should be able to
> pass
> > in
> > > some sort of parameter or flag so I can branch in my onTimer
> > > implementation.
> > >
> > > As parameters are not supported I am left with states that are scoped
> to
> > > the keys which is also pretty useless if I want to trigger different
> > timed
> > > actions for the same keys.
> > >
> > > I know this is quite tricky but I see some alternative options:
> > > - The register timer method returns a unique (per key) timer id, so we
> > can
> > > associate state with this id to fetch info about the timer registered.
> > (We
> > > could also remove timers with this id and maybe add methods to remove
> all
> > > for the current key)
> > > - Allow the users to pass a custom parameter when registering the
> > > callback, and the parameter would be passed to the onTimer method
> > > - Allow users to pass custom callback functions when registering the
> > > timers, but this would mean we have to support some sort of context for
> > > accessing the state (like the window context before)
> > > - We could go for an annotation based API like in beam but thats
> > probably
> > > not good to mix in the current ones
> > >
> > > I personally prefer the first one.
> > >
> > > What do you think?
> > >
> > > Regards,
> > > Gyula
> > >
> >
>
>
>
> --
>
> Jamie Grier
> data Artisans, Director of Applications Engineering
> @jamiegrier <https://twitter.com/jamiegrier>
> ja...@data-artisans.com
>
