Thanks a lot Guozhang. I will try and let you know.
Really appreciate all the help. This community has been amazing.
Thanks
On Tue, Feb 23, 2021 at 5:48 PM Guozhang Wang wrote:
> Sorry I was not very clear before: by "WindowStore" I meant implementing
> your own customized store based on a kvS
Sorry I was not very clear before: by "WindowStore" I meant implementing
your own customized store based on a kvStore where the key is a combo
. Note you put timestamp first then key in your
serialization format, so that you can range-fetch with just the prefix on
timestamp then. In fact `WindowSto
Thanks Guozhang.
I don't see the remove method in window stores. Am I missing something? It
would be very nice to implement the optimization you had mentioned.
Thanks
On Tue, Feb 23, 2021 at 11:11 AM Guozhang Wang wrote:
> I see. In that case I think your design with a KVstore plus a book-keep
I see. In that case I think your design with a KVstore plus a book-keeping
window store would work better. One minor optimization you can try though,
is that instead of trying to check if the TTL has changed or not when
expiring from the window store, you can try to delete from the window store
whe
Thanks Liam & Guozhang.
First of all, we use PAPI in our entire topology and we would like to
retain it that way rather than combining with DSL. Secondly, even I was
more leaning towards session store but the problem I found with session
store is I cannot get all the expired sessions without keys
Hello Navneeth,
I would agree with Liam that a session store seems a good fit for your
case. But note that session stores would not expire a session themselves
and it is still the processor node's job to find those already expired
sessions and emit results / delete. You can take a look at
the KStr
Hmmm, thanks Navneeth,
I feel like a session store set to an inactivity period of 10 minutes,
suppressed until session window closed, combined with a GlobalKTable would
be how I'd start to approach this in the DSL, with the below topology. I
have no idea if my ASCII art below will survive email fo
Hi Liam,
The use case is stream all data and send it to storage after processing.
Also when the user is inactive for a 10 min period then send a special
event that marks the user as inactive. I'm trying to implement the special
event here.
Thanks
On Tue, Feb 16, 2021 at 1:18 AM Liam Clarke-Hutc
Hey Navneeth,
So to understand your problem better - do you only want to stream users
active within 10 minutes to storage?
Cheers,
Liam
On Tue, Feb 16, 2021 at 9:50 AM Navneeth Krishnan
wrote:
> It’s just for emitting to data storage. There is no join here.
>
> Thanks
>
> On Mon, Feb 15, 2021
It’s just for emitting to data storage. There is no join here.
Thanks
On Mon, Feb 15, 2021 at 1:42 AM Liam Clarke-Hutchinson <
liam.cla...@adscale.co.nz> wrote:
> Hi Navneeth,
>
> What is the purpose of holding these user records? Is it to join against
> other streams, or emit to data storage?
>
Hi Navneeth,
What is the purpose of holding these user records? Is it to join against
other streams, or emit to data storage?
Cheers,
Liam Clarke-Hutchinson
On Mon, 15 Feb. 2021, 9:08 pm Navneeth Krishnan,
wrote:
> Hi All,
>
> I have a question about how I can use window stores to achieve t
11 matches
Mail list logo
