So let's assume if we are reading data from a topic named A and performing aggregation on 4 hour window and a different pipeline which does same aggregation for 6 hour window, so in this case we are reading same data couple of times and processing it for different intervals, can we do it using just a single read.
Thanks and Regards , Surendra Lalwani On Tue, May 17, 2022 at 1:19 PM Caizhi Weng <tsreape...@gmail.com> wrote: > Hi! > > I wanted to explore if we can avoid reading multiple times and read only >> once >> > > Could you elaborate more on the reason you need this? If both of your > queries need full data from that topic you'll have to read the whole topic > many times. If each query only need to consume a specific portion of that > topic I would suggest splitting it into multiple topics. > > > Surendra Lalwani <surendra.lalw...@swiggy.in> 于2022年5月17日周二 15:44写道： > >> Hi Team, >> >> We have various SQL queries where we are querying the same kafka topic >> but both of the queries are completely different. I wanted to explore if we >> can avoid reading multiple times and read only once and perform different >> processing according to the query and dump data accordingly. >> >> Thanks and Regards , >> Surendra Lalwani >> >> >> ------------------------------ >> IMPORTANT NOTICE: This e-mail, including any attachments, may contain >> confidential information and is intended only for the addressee(s) named >> above. If you are not the intended recipient(s), you should not >> disseminate, distribute, or copy this e-mail. Please notify the sender by >> reply e-mail immediately if you have received this e-mail in error and >> permanently delete all copies of the original message from your system. >> E-mail transmission cannot be guaranteed to be secure as it could be >> intercepted, corrupted, lost, destroyed, arrive late or incomplete, or >> contain viruses. Company accepts no liability for any damage or loss of >> confidential information caused by this email or due to any virus >> transmitted by this email or otherwise. > > -- IMPORTANT NOTICE: This e-mail, including any attachments, may contain confidential information and is intended only for the addressee(s) named above. If you are not the intended recipient(s), you should not disseminate, distribute, or copy this e-mail. Please notify the sender by reply e-mail immediately if you have received this e-mail in error and permanently delete all copies of the original message from your system. E-mail transmission cannot be guaranteed to be secure as it could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Company accepts no liability for any damage or loss of confidential information caused by this email or due to any virus transmitted by this email or otherwise.
