Re: Business Rules Engine for Hive

2018-04-16 Thread Joel D
The business rules we've here are currently embedded in hive code. They range from basic standardization using case blocks to complex multi-column validation. Thanks. On Mon, Apr 16, 2018 at 5:03 PM Jörn Franke wrote: > The question is what do your rules do? Do you need to maintain a factbase >

Re: Business Rules Engine for Hive

2018-04-16 Thread Jörn Franke
The question is what do your rules do? Do you need to maintain a factbase or do they just check data quality within certain tables? > On 16. Apr 2018, at 22:28, Joel D wrote: > > Ok. > > Rough ideas: > To keep the business logic outside code, I was thinking to give a custom UI. > > Next read

Re: Business Rules Engine for Hive

2018-04-16 Thread Joel D
Ok. Rough ideas: To keep the business logic outside code, I was thinking to give a custom UI. Next read from UI data and build UDFs using the rules defined outside the UDF. 1 UDF per data object. Not sure these are just thoughts. On Mon, Apr 16, 2018 at 1:40 PM Jörn Franke wrote: > I would n

Re: Business Rules Engine for Hive

2018-04-16 Thread Joel D
Hi Pivonka, we are more inclined towards using open source products and closer integration with Hive since we've most of our ETL in Hive. Thanks. On Mon, Apr 16, 2018 at 12:51 PM Al Pivonka wrote: > I am not the product owner an have not implemented it yet. > I would check out > http://cask.co/

Re: Business Rules Engine for Hive

2018-04-16 Thread Jörn Franke
I would not use Drools with Spark, it does not scale to the distributed setting. You could translate the rules to hive queries but this would not be exactly the same thing. > On 16. Apr 2018, at 17:59, Joel D wrote: > > Hi, > > Any suggestions on how to implement Business Rules Engine with Hi

Re: Business Rules Engine for Hive

2018-04-16 Thread Al Pivonka
I am not the product owner an have not implemented it yet. I would check out http://cask.co/products/rules-engine/ On Mon, Apr 16, 2018 at 11:59 AM, Joel D wrote: > Hi, > > Any suggestions on how to implement Business Rules Engine with Hive ETLs? > > For spark based Etl jobs, I was exploring Dro

Business Rules Engine for Hive

2018-04-16 Thread Joel D
Hi, Any suggestions on how to implement Business Rules Engine with Hive ETLs? For spark based Etl jobs, I was exploring Drools but not sure about Hive. Thanks.