Please take a look at the hello-world example. You can implement your business logic in the process() callback.
What kind of transformation are you doing? Are you doing a group by/count style aggregation to generate the report? If so, you could use the embedded rocksdb store in Samza and potentially batch your writes to the database. How many Qps do you process at peak? Do you expect to buffer any state per message? What's the ratio of input to output messages on average? There's nothing that stops you from using JDBC and Samza. On Tue, Mar 1, 2016 at 8:58 PM, Manohar Reddy < manohar.re...@happiestminds.com> wrote: > Hello Team, > > we are part of some service based company and trying to explore the > available real time streaming technologies.so one of the first option we > are trying is Samza. > let me explain brief about my use case here: > > we are trying to build real time reporting dashboard for e-learning > domain. > To build this dash board the input is RDBMS.so if any > DML(inserts/updates/deletes) into the source RDBMS,immediately some > adapter will publish to kafka with RDBMS table name and primary keys as > JSON format. > Now Samza has to consume the kafka event and query back to source RDBMS > table to get the whole data set of RDBMS relation tables by using json > event information. > now do some transformation here as per business and load into > Target(Reporting DB) RDBMS. > more or less here we are handling with few JDBC calls through Samza and > here every day data load is small I can say max 2Gb of data but we need > real time processing ecosystem in place. > that's it brief about my usecase,so team please provide your inputs how we > can approach with samza for this requirement.is there any utility API > with Samza for JDBC calls. > > Thank you very much in Advance. > > ~~Manohar > ________________________________ > Happiest Minds Disclaimer > > This message is for the sole use of the intended recipient(s) and may > contain confidential, proprietary or legally privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. If you > are not the original intended recipient of the message, please contact the > sender by reply email and destroy all copies of the original message. > > Happiest Minds Technologies <http://www.happiestminds.com> > > ________________________________ > -- Jagadish V, Graduate Student, Department of Computer Science, Stanford University
