Thanks Jan, I didn't know InputFormat and LineReader could help, though I didn't look at them closely. I was thinking about implementing a Table-Generating Function (UDTF) if there is no an already implemented solution.
Ruslan On Thu, Jun 14, 2012 at 10:03 AM, Jan Dolinár <dolik....@gmail.com> wrote: > Hi Ruslan, > > I've been in similar situation and solved it by writing a custom > InputFormat and LineReader that loads the data from MySQL in > constructor. In my case I use it just to check value ranges and > similar stuff. If you want to join the data with whats in your hdfs > files, you can do that as well, InputFormat allows you to add the > columns easily. I'm not sure how well this solution would behave for a > bigger data, but for small data (I load about 5 tables, ~100 lines > each) it works just fine. > > Best Regards, > Jan > > > > On 6/13/12, Ruslan Al-Fakikh <metarus...@gmail.com> wrote: >> Hello to everyone, >> >> I need to join hdfs data with little data taken from RDBMS. A possible >> solution is to import RDBMS data to a regular hive table using Sqoop, >> but this way I'll have to keep that imported hive table up-to-date >> which means that I will have to update it every time before joining in >> a query. >> Is there a way to load RDBMS data on the fly? Maybe a UDF which would >> take RDBMS connection properties and load the data? >> >> Thanks in advance, >> Ruslan Al-Fakikh >> -- Best Regards, Ruslan Al-Fakikh
