But i thought trident method ensures you read the message only once. while in traditional kafkaSpout reads the message until it is processed successfully. Am I missing something here? Help clarify this. Thanks.
-- Kushan Maskey 817.403.7500 M. Miller & Associates <http://mmillerassociates.com/> [email protected] On Tue, Dec 2, 2014 at 4:24 PM, Parth Brahmbhatt < [email protected]> wrote: > The message is always replayed from the Spout, you will have to add dedup > mechanism in B1 if you want to avoid duplicate processing. > > Implementing trident still won’t help you for the use case you are trying > to solve. If you want intermediate bolts to dedup, you have to write the > dedup logic. > > Thanks > Parth > > On Dec 2, 2014, at 2:19 PM, Kushan Maskey < > [email protected]> wrote: > > I have a main bolt B1 listening to spout S1. Once processing is done on B1 > it emits message to B2, B3, B4 ad B5. After B5 process the data, it then > emits this message to B6, B7and B8. In the process if B3 fails to process > the data, then does the failed acknowledgement gets acked all the way back > to the spout? Or if B3 fails, the message will just get re-emitted from > bolt B1? > > In this case the message it going to get read twice and the data is > duplicated. How do I prevent this to happen. We are using KafkaSpout that > came in with storm.0.9.2. Is implementing trident method gonna help resolve > this issue? Thanks. > > -- > Kushan Maskey > 817.403.7500 > M. Miller & Associates <http://mmillerassociates.com/> > [email protected] > > > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
