Dear Sirs, I'm working on a development project for my company that is looking at the possibility of live streaming transaction data from our Oracle databases into Hadoop HDFS. Doing some research I've been looking into leveraging the power of Kafka to form part of the solution and need some a pointer which I hope you can help me with.
We use Goldengate to stream data into a Kafka topic and can see this data when running to command: bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning My question is how would I get Kafka to write this data from the topic into a HDFS directory? Any help you can provide will be gratefully received. Many thanks Regards Kevin Woodbine * (CUCI) 01784 736039 * 07990 563 530 kevin.woodbi...@centrica.com<mailto:kevin.woodbi...@centrica.com> _____________________________________________________________________ The information contained in or attached to this email is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are not authorised to and must not disclose, copy, distribute, or retain this message or any part of it. It may contain information which is confidential and/or covered by legal professional or other privilege (or other rules or laws with similar effect in jurisdictions outside England and Wales). The views expressed in this email are not necessarily the views of Centrica plc, and the company, its directors, officers or employees make no representation or accept any liability for its accuracy or completeness unless expressly stated to the contrary. PH Jones is a trading name of British Gas Social Housing Limited. British Gas Social Housing Limited (company no: 01026007), British Gas Trading Limited (company no: 03078711), British Gas Services Limited (company no: 3141243), British Gas Insurance Limited (company no: 06608316), British Gas New Heating Limited (company no: 06723244), British Gas Services (Commercial) Limited (company no: 07385984) and Centrica Energy (Trading) Limited (company no: 02877397) are all wholly owned subsidiaries of Centrica plc (company no: 3033654). Each company is registered in England and Wales with a registered office at Millstream, Maidenhead Road, Windsor, Berkshire SL4 5GD. British Gas Insurance Limited is authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority. British Gas Services Limited and Centrica Energy (Trading) Limited are authorised and regulated by the Financial Conduct Authority. British Gas Trading Limited is an appointed representative of British Gas Services Limited which is authorised and regulated by the Financial Conduct Authority.
