Hi Guozhang,

Thanks for your input. That would be a great solution if it wasn't that
more than one client could be connected via WebSocket. Hence, a single
client can only tell what they've consumed and the consumer is shared for
everyone connected.

Is there any other way I can go about this problem? Really would appreciate
any input. :)

2015-05-20 18:37 GMT+02:00 Guozhang Wang <wangg...@gmail.com>:

> Hi Kevin,
>
> The current high-level Scala consumer does not have a rewind function but
> if the other client is able to notify you through WebSocket periodically as
> long as it is not disconnected, then what you can do is let your Java app
> to buffer messages even after pushing them to the WebSocket, and only flush
> them and commit back to Kafka after you got notified from the WebSocket
> that they have been successfully consumed.
>
> Guozhang
>
> On Wed, May 20, 2015 at 5:48 AM, Kevin Sjöberg <kevin.sjob...@fortnox.se>
> wrote:
>
> > Thanks for the link, Sharninder. I'm not entirely sure this is what I'm
> > looking for. I do not know whether or not the client actually received
> the
> > messages or not.
> >
> > The Java-application consumes the messages from Kafka and then push them
> to
> > Pusher (https://pusher.com/). The client, connected to the Pusher
> > WebSocket-server (i.e., not provided by me), then get the messages I sent
> > them. The scenario that could happen is that the client loses the
> > connection to the WebSocket server and may miss messages that were pushed
> > during the timeout. In this case, they want to tell me that they lost the
> > connection and start to get messages from a given point in time again.
> The
> > only way they interact with me is connecting to the WebSocket server
> again
> > with an optional message that I can get notified about. They do not
> connect
> > to my Java-application directly.
> >
> > 2015-05-20 14:24 GMT+02:00 Sharninder <sharnin...@gmail.com>:
> >
> > > > I have a backend service written in PHP. This service pushes messages
> > to
> > > > Apache Kafka (over the topic "posts") when posts are created, read
> and
> > > > removed. I also have a backend service written in Java. This service
> > > > consumes messages from Apache Kafka (for the "posts" topic) and push
> > them
> > > > out over WebSockets to whoever is listening. This works out great.
> > > >
> > > > The problem I have is as follows. Say, for some reason, the WebSocket
> > > > client timeout and because of this he or she does not get some of the
> > > > messages. The messages are still being consumed but the client isn't
> > > there
> > > > to acknowledge them. Now, the client want "rewind" and get all the
> > > messages
> > > > that they missed may have missed. This is where I'm stuck.
> > > >
> > > > Ideally, the client would provide a timestamp for when they last got
> a
> > > > message and I'd then use that timestamp to get messages from that
> point
> > > and
> > > > onwards. I'm not sure if this is even possible, it's just my
> thoughts.
> > > >
> > > >
> > > Something like this was discussed a few weeks back on the list. But,
> > > essentially what you need to do is turn off auto commit and only commit
> > > when you're sure the message has been acknowledged by the next stage.
> > >
> > > I think the link is this ->
> > >
> > >
> >
> http://ingest.tips/2014/10/12/kafka-high-level-consumer-frequently-missing-pieces/
> > >
> > > Regards,
> > > Sharninder
> > >
> >
> >
> >
> > --
> > Kevin Sjöberg
> >
> > --
> > Du skapar din faktura, vi skickar den och ser till att kunden betalar. Vi
> > sköter allt, hela vägen tills fakturan är betald!
> > Läs om Nox Finans här
> > <
> >
> https://www.fortnox.se/nox/?utm_source=signature&utm_medium=email&utm_campaign=noxfinans&utm_term=fakturahantering
> > >
> > .
> >
>
>
>
> --
> -- Guozhang
>



-- 
Kevin Sjöberg

-- 
Du skapar din faktura, vi skickar den och ser till att kunden betalar. Vi 
sköter allt, hela vägen tills fakturan är betald! 
Läs om Nox Finans här 
<https://www.fortnox.se/nox/?utm_source=signature&utm_medium=email&utm_campaign=noxfinans&utm_term=fakturahantering>
.

Reply via email to