Re: [Discuss] project chop up

2013-08-21 Thread Thiruvel Thirumoolan
+1 Thanks Edward. On 8/20/13 11:35 PM, "amareshwari sriramdasu" wrote: >Sounds great! Looking forward ! > > >On Tue, Aug 20, 2013 at 7:58 PM, Edward Capriolo >wrote: > >> Just an update. This is going very well: >> >> NFO] Nothing to compile - all classes are up to date >> [INFO] >>

Re: [Discuss] project chop up

2013-08-20 Thread amareshwari sriramdasu
Sounds great! Looking forward ! On Tue, Aug 20, 2013 at 7:58 PM, Edward Capriolo wrote: > Just an update. This is going very well: > > NFO] Nothing to compile - all classes are up to date > [INFO] > > [INFO] Reactor Summary

Re: [Discuss] project chop up

2013-08-20 Thread Edward Capriolo
Just an update. This is going very well: NFO] Nothing to compile - all classes are up to date [INFO] [INFO] Reactor Summary: [INFO] [INFO] Apache Hive ... SUCCESS [0.002s] [INFO] hive-shims

Re: [Discuss] project chop up

2013-08-16 Thread Xuefu Zhang
Thanks, Edward. I'm big +1 to mavenize Hive. Hive has long reached a point where it's hard to manage its build using ant. I'd like to help on this too. Thanks, Xuefu On Fri, Aug 16, 2013 at 7:31 PM, Edward Capriolo wrote: > For those interested in pitching in. > https://github.com/edwardcaprio

Re: [Discuss] project chop up

2013-08-16 Thread Edward Capriolo
For those interested in pitching in. https://github.com/edwardcapriolo/hive On Fri, Aug 16, 2013 at 11:58 AM, Edward Capriolo wrote: > Summary from hive-irc channel. Minor edits for spell check/grammar. > > The last 10 lines are a summary of the key points. > > [10:59:17] noland: et all. Do yo

Re: [Discuss] project chop up

2013-08-16 Thread Edward Capriolo
Summary from hive-irc channel. Minor edits for spell check/grammar. The last 10 lines are a summary of the key points. [10:59:17] noland: et all. Do you want to talk about hive in maven? [11:01:06] smonchi [~ ro...@host34-189-dynamic.23-79-r.retail.telecomitalia.it] has quit IRC: Quit: ... 'caus

Re: [Discuss] project chop up

2013-08-15 Thread Edward Capriolo
I have opened https://issues.apache.org/jira/browse/HIVE-5107 because I am growing tired of how long hive's build take. I have started playing with this by creating a simple multi-module project and copying stuff as I go. I have ported a minimal shims and common and I have all the tests in common

Re: [Discuss] project chop up

2013-08-07 Thread Owen O'Malley
On Wed, Aug 7, 2013 at 2:04 PM, Edward Capriolo wrote: > "Some of the hard part was that some of the test classes are in the wrong > module that references classes in a later module." > > I think the modules will have to be able to reference each other in many > cases. Serde and QL are tightly cou

Re: [Discuss] project chop up

2013-08-07 Thread Edward Capriolo
"Some of the hard part was that some of the test classes are in the wrong module that references classes in a later module." I think the modules will have to be able to reference each other in many cases. Serde and QL are tightly coupled. QL is really too large and we should find a way to cut that

Re: [Discuss] project chop up

2013-08-07 Thread Owen O'Malley
On Wed, Aug 7, 2013 at 12:55 PM, kulkarni.swar...@gmail.com < kulkarni.swar...@gmail.com> wrote: > > I'd like to propose we move towards Maven. > > Big +1 on this. Most of the major apache projects(hadoop, hbase, avro etc.) > are maven based. > A big +1 from me too. I actually took a pass at it a

Re: [Discuss] project chop up

2013-08-07 Thread Brock Noland
t; > Based on my other Apache experiences, I prefer the option #3 "Breakup > the > > > projects within our own source tree". Make multiple modules or > > > sub-projects. By default, only key modules will be built. > > > > > > Maven could be a

Re: [Discuss] project chop up

2013-08-07 Thread Edward Capriolo
> > > > > > > > > From: Edward Capriolo > > To: "dev@hive.apache.org" > > Sent: Saturday, July 27, 2013 7:03 AM > > Subject: Re: [Discuss] project chop up > > > > > > Or feel free to suggest

Re: [Discuss] project chop up

2013-08-07 Thread kulkarni.swar...@gmail.com
; > sub-projects. By default, only key modules will be built. > > > > Maven could be a possible candidate. > > > > Regards, > > Mohammad > > > > > > > > > > From: Edward Capriolo > > To: "dev@h

Re: [Discuss] project chop up

2013-08-07 Thread Brock Noland
> From: Edward Capriolo > To: "dev@hive.apache.org" > Sent: Saturday, July 27, 2013 7:03 AM > Subject: Re: [Discuss] project chop up > > > Or feel free to suggest different approach. I am used to managing software > as multi-module mave

Re: [Discuss] project chop up

2013-07-27 Thread Mohammad Islam
, only key modules will be built.  Maven could be a possible candidate. Regards, Mohammad From: Edward Capriolo To: "dev@hive.apache.org" Sent: Saturday, July 27, 2013 7:03 AM Subject: Re: [Discuss] project chop up Or feel free to suggest diff

Re: [Discuss] project chop up

2013-07-27 Thread Brock Noland
I am open to any approach to making the build and project more modular. Maven is no silver bullet but it does have some big positives. Most importantly I have been a little distressed with Ivy when changing library versions as it caused hundreds of tests to fail for no good reason in HIVE-3632. I

Re: [Discuss] project chop up

2013-07-27 Thread Edward Capriolo
Or feel free to suggest different approach. I am used to managing software as multi-module maven projects. >From a development standpoint if I was working on beeline, it would be nice to only require some of the sub-projects to be open in my IDE to do that. Also managing everything globally is not

Re: [Discuss] project chop up

2013-07-26 Thread Brock Noland
Yeah I think the tar target should build the whole project. On Jul 26, 2013 11:05 PM, "Alan Gates" wrote: > But I assume they'd still be a part of targets like package, tar, and > binary? Making them compile and test separately and explicitly load the > core Hive jars from maven/ivy seems reason

Re: [Discuss] project chop up

2013-07-26 Thread Alan Gates
But I assume they'd still be a part of targets like package, tar, and binary? Making them compile and test separately and explicitly load the core Hive jars from maven/ivy seems reasonable. Alan. On Jul 26, 2013, at 8:40 PM, Brock Noland wrote: > Hi, > > I think thats part of it but I'd like

Re: [Discuss] project chop up

2013-07-26 Thread Brock Noland
Hi, I think thats part of it but I'd like to decouple the downstream projects even further so that the only connection is the dependency on the hive jars. Brock On Jul 26, 2013 10:10 PM, "Alan Gates" wrote: > I'm not sure how this is different from what hcat does today. It needs > Hive's jars

Re: [Discuss] project chop up

2013-07-26 Thread Alan Gates
I'm not sure how this is different from what hcat does today. It needs Hive's jars to compile, so it's one of the last things in the compile step. Would moving the other modules you note to be in the same category be enough? Did you want to also make it so that the default ant target doesn't

Re: [Discuss] project chop up

2013-07-26 Thread Edward Capriolo
My mistake on saying hcat was a fork metastore. I had a brain fart for a moment. One way we could do this is create a folder called downstream. In our release step we can execute the downstream builds and then copy the files we need back. So nothing downstream will be on the classpath of the main

Re: [Discuss] project chop up

2013-07-26 Thread Thejas Nair
+1 to the idea of making the build of core hive and other downstream components independent. bq. I was under the impression that Hcat and hive-metastore was supposed to merge up somehow. The metastore code was never forked. Hcat was just using hive-metastore and making the metadata available to

Re: [Discuss] project chop up

2013-07-26 Thread Edward Capriolo
Also i believe hcatalog web can fall into the same designation. Question , hcatalog was initily a big hive-metastore fork. I was under the impression that Hcat and hive-metastore was supposed to merge up somehow. What is the status on that? I remember that was one of the core reasons we brought it

Re: [Discuss] project chop up

2013-07-25 Thread Edward Capriolo
I prefer option 3 as well. On Fri, Jul 26, 2013 at 12:52 AM, Brock Noland wrote: > On Thu, Jul 25, 2013 at 9:48 PM, Edward Capriolo >wrote: > > > I have been developing my laptop on a duel core 2 GB Ram laptop for years > > now. With the addition of hcatalog, hive-thrift2, and some other growt

Re: [Discuss] project chop up

2013-07-25 Thread Brock Noland
On Thu, Jul 25, 2013 at 9:48 PM, Edward Capriolo wrote: > I have been developing my laptop on a duel core 2 GB Ram laptop for years > now. With the addition of hcatalog, hive-thrift2, and some other growth > trying to develop hive in a eclipse on this machine craws, especially if > 'build automati

[Discuss] project chop up

2013-07-25 Thread Edward Capriolo
I have been developing my laptop on a duel core 2 GB Ram laptop for years now. With the addition of hcatalog, hive-thrift2, and some other growth trying to develop hive in a eclipse on this machine craws, especially if 'build automatically' is turned on. As we look to add on more things this is onl