Re: Best practice for automating jobs

2013-01-10 Thread Alexander Alten-Lorenz
+1 This is the best solution to automate jobs. cheers, Alex On Jan 10, 2013, at 11:11 PM, Sean McNamara wrote: >> I want to know if there are any accepted patterns or best practices for >> this? > > http://oozie.apache.org/ > > > >> New partitions will be added regularly > > What type of

Hive Index

2013-01-10 Thread Santosh Achhra
Hello Hive Users, I have created index on table with deferred rebuild. 1) After Index is created I see two tables; a) MAIN_TABLE b) MAIN_TABLE_INDEX No when I do a explain on the table, I see that it is going for table scan and not index scan ? MAIN_TABLE *

Re: Best practice for automating jobs

2013-01-10 Thread Qiang Wang
Are you using Embedded Metastore ? Only one process can connect to this metastore at a time. 2013/1/11 Tom Brown > When I've tried to create concurrent CLI sessions, I thought the 2nd > one got an error about not being able to lock the metadata store. > > Is that error a real thing, or have I b

Re: Best practice for automating jobs

2013-01-10 Thread Tom Brown
When I've tried to create concurrent CLI sessions, I thought the 2nd one got an error about not being able to lock the metadata store. Is that error a real thing, or have I been mistaken this whole time? --Tom On Thursday, January 10, 2013, Qiang Wang wrote: > The HWI will create a cli session

Re: Best practice for automating jobs

2013-01-10 Thread Qiang Wang
The HWI will create a cli session for each query through hive libs, so several queries can run concurrently. 2013/1/11 Tom Brown > How is concurrency achieved with this solution? > > > On Thursday, January 10, 2013, Qiang Wang wrote: > >> I believe the HWI (Hive Web Interface) can give you a ha

Re: Best practice for automating jobs

2013-01-10 Thread Tom Brown
How is concurrency achieved with this solution? On Thursday, January 10, 2013, Qiang Wang wrote: > I believe the HWI (Hive Web Interface) can give you a hand. > > https://github.com/anjuke/hwi > > You can use the HWI to submit and run queries concurrently. > Partition management can be achieved b

Re: Best practice for automating jobs

2013-01-10 Thread Qiang Wang
I believe the HWI (Hive Web Interface) can give you a hand. https://github.com/anjuke/hwi You can use the HWI to submit and run queries concurrently. Partition management can be achieved by creating crontabs using the HWI. It's simple and easy to use. Hope it helps. Regards, Qiang 2013/1/11 T

Re: Best practice for automating jobs

2013-01-10 Thread Dean Wampler
If you know make and bash, have a look at Stampede for scheduling work: https://github.com/ThinkBigAnalytics/stampede (Full disclosure: I wrote it) On Thu, Jan 10, 2013 at 4:11 PM, Sean McNamara wrote: > > I want to know if there are any accepted patterns or best practices for > >this? > > htt

Re: Best practice for automating jobs

2013-01-10 Thread Sean McNamara
> I want to know if there are any accepted patterns or best practices for >this? http://oozie.apache.org/ > New partitions will be added regularly What type of partitions are you adding? Why frequently? Sean On 1/10/13 3:03 PM, "Tom Brown" wrote: >All, > >I want to automate jobs agains

Re: Stack function in Hive : how to specify multiple aliases?

2013-01-10 Thread Dean Wampler
Try "as (alias1, alias2, ...)" On Thu, Jan 10, 2013 at 3:42 AM, Mathieu Despriee wrote: > Not working either : > > SELECT stack(2,AppWeb, ByteWeb, PacketWeb, AppP2P, ByteP2P, PacketP2P) AS > App,Byte,Packet FROM testApp2; > > FAILED: SemanticException 1:76 Only a single expression in the SELECT >

unsubscribe

2013-01-10 Thread Sachin Khurana
unsubscribe American Express made the following annotations on Thu Jan 10 2013 06:03:03 ** "This message and any attachments are solely for the intended recipient and may contain confidential or privileged information

Re: Stack function in Hive : how to specify multiple aliases?

2013-01-10 Thread Nitin Pawar
I dont think you can do this directly in a query. (I can be wrong) I just asked took suggestion from a hive guru and he suggested to try lateral view https://cwiki.apache.org/Hive/languagemanual-lateralview.html On Thu, Jan 10, 2013 at 1:42 AM, Mathieu Despriee wrote: > Not working either : >

Re: Stack function in Hive : how to specify multiple aliases?

2013-01-10 Thread Mathieu Despriee
Not working either : SELECT stack(2,AppWeb, ByteWeb, PacketWeb, AppP2P, ByteP2P, PacketP2P) AS App,Byte,Packet FROM testApp2; > FAILED: SemanticException 1:76 Only a single expression in the SELECT clause is supported with UDTF's. Error encountered near token 'Byte' I tried to quote the aliases o

Re: Stack function in Hive : how to specify multiple aliases?

2013-01-10 Thread Nitin Pawar
I never ran into this kind of problem but can you try select as A,B,C On Thu, Jan 10, 2013 at 12:58 AM, Mathieu Despriee wrote: > SELECT stack(2,AppWeb, ByteWeb, PacketWeb, AppP2P, ByteP2P, PacketP2P) AS > A FROM testApp2; > > > 2013/1/10 Nitin Pawar > >> can you provide your query ? >> >> >> O

Re: Stack function in Hive : how to specify multiple aliases?

2013-01-10 Thread Mathieu Despriee
SELECT stack(2,AppWeb, ByteWeb, PacketWeb, AppP2P, ByteP2P, PacketP2P) AS A FROM testApp2; 2013/1/10 Nitin Pawar > can you provide your query ? > > > On Thu, Jan 10, 2013 at 12:39 AM, Mathieu Despriee wrote: > >> Hi folks, >> >> I want to use the stack function, described here : >> https://cwik

Re: Stack function in Hive : how to specify multiple aliases?

2013-01-10 Thread Nitin Pawar
can you provide your query ? On Thu, Jan 10, 2013 at 12:39 AM, Mathieu Despriee wrote: > Hi folks, > > I want to use the stack function, described here : > https://cwiki.apache.org/Hive/languagemanual-udf.html#LanguageManualUDF-BuiltinTableGeneratingFunctions%2528UDTF%2529 > > Hive asks me to pr

Stack function in Hive : how to specify multiple aliases?

2013-01-10 Thread Mathieu Despriee
Hi folks, I want to use the stack function, described here : https://cwiki.apache.org/Hive/languagemanual-udf.html#LanguageManualUDF-BuiltinTableGeneratingFunctions%2528UDTF%2529 Hive asks me to provide the multiple aliases for the resulting columns ("The number of aliases in the AS clause does n