Re: Chocolatey package for Windows

I've heard that there are some problems in running Pig/Hive and Hadoop itself on WIndows... On Mon, Aug 26, 2013 at 10:21 PM, Andrew Pennebaker wrote: > I love how you can "apt-get install hadoop-hive" in Ubuntu, and "brew > install hive" in Mac. Could we submit a > Chocolatey

Re: Elastic MapReduce Hive & Avro SerDe

Hi. My guess is that you can try to look it up in their docs or mailing lists (Amazon EMR). IIRC, CDH had the patch for Avro+Hive before it was included in Hive itself, so Amazon EMR can have similar patches... Ruslan On Thu, Jul 4, 2013 at 12:20 PM, Dan Filimon wrote: > Hi! > > I'm working on

Re: Locking in HIVE : How to use locking/unlocking features using hive java API ?

Hi Manish! Why do you need metadata backup? Can't you just store all the table create statements in an init file? If you care about Partitions that have been created dynamically then you can restore them from data by RECOVER PARTITIONS (if using Amazon EMR) or an analog check command for a regular

Re: PK violation during Hive add partition

Hi! Have you enabled Hive concurrency? Hive should not be accessed concurrently if the appropriate property is not enabled. Ruslan On Sat, Dec 8, 2012 at 6:01 AM, Karlen Lie wrote: > nal table, and the query below is run concurrently by multiple oo

Re: "Subject" etiquette

+1 On Thu, Nov 22, 2012 at 6:27 PM, Mohammad Tariq wrote: > +1 > > Regards, > Mohammad Tariq > > > > On Thu, Nov 22, 2012 at 7:47 PM, Dean Wampler < > dean.wamp...@thinkbiganalytics.com> wrote: > >> As a service to everyone on this list, please fill in the "Subject" field >> when you post t

Re: how to transform the date format in hive?

Hi, also take a look at Hive date functions: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions On Thu, Nov 22, 2012 at 3:59 AM, Tom Hubina wrote: > Could convert them to unix time which will give you two bigints that you > can subtract to get se

Re: hive 0.7.1 Error: Non-Partition column appears in the partition specification

Hey Pavel, Also note that the dynamic partition values are selected by ordering, not name, and taken as the last columns from the select clause. So you have to have a column for the author partition in your most outer 'select'. So your error messages are normal. In the first one you do not have a

Re: How to conver into date in hive

Probably you could hardcode those in a ternary operator or a switch/case or if/else expression. Ruslan On Tue, Oct 16, 2012 at 10:16 AM, wrote: > Hi all, > > Is there any way to convert months in names. > like. > > 1 to Jan > 2 to Feb > 3 to Mar > > and so on (here 1, 2, 3 are

Re: Multiple Hive Connection Issues

Just a note: There is also hiveserver2 which fixes connection issues, and it is included in cdh 4 On Tue, Oct 9, 2012 at 4:24 PM, nagarjuna kanamarlapudi wrote: > Hi, > > > I have a requirement of using multiple hive connections simultaneously to > run multiple queries in parallel. I use JDBC Cl

Re: On hive date functions

Hi Try the hour() function from here: https://cwiki.apache.org/Hive/languagemanual-udf.html#LanguageManualUDF-DateFunctions Ruslan On Tue, Oct 9, 2012 at 7:54 AM, Matthieu Labour wrote: > Hi > Is it possible with Hive to truncate date to a specified precision? > For example in Postgresql date_t

Re: Error on hive web interface

Hey, Are you using Cloudera's distribution? If yes - as far as I know they don't support the Hive Web Interface, recommending Hue. Best Regards On Thu, Sep 27, 2012 at 10:38 PM, Germain Tanguy wrote: > Hi > > I am a new user of Hive, I am on version 0.9.0. I try to use hive web > interface and

Re: Hive File Sizes, Merging, and Splits

Hi, Can you look up the file names of each mapper? You can do so by looking at a running task UI in the status column. Also what split property do you mean? Can you give your job's console output? Also, the best recommended way is to use a splittable format like Avro, Seq files, indexed LZO, etc.

Incomplete example on page cwiki.apache.org/Hive/compressedstorage.html

Hey guys, I spent a lot of time to figure out that when I use the example from the page cwiki.apache.org/Hive/compressedstorage.html: SET hive.exec.compress.output=true; SET io.seqfile.compression.type=BLOCK; -- NONE/RECORD/BLOCK (see below) INSERT OVERWRITE TABLE raw_sequence SELECT * FROM raw;

Re: How to overwrite a file inside hive table folder

Hi, I think you could try to drop partition and create a new partition if you table is partitioned Ruslan On Mon, Sep 17, 2012 at 8:12 AM, MiaoMiao wrote: > What do you mean by "a file"? If your HDFS dir contains several file > and you want to overwrite one, then no way you can do it with HiveQ

Re: How to use the IF conditional function in Hive scripts

Hi, I guess this goes beyond Hive scripts. You can use some kind of an external automation tool like Oozie or a wrapper sh script Ruslan On Thu, Sep 13, 2012 at 3:49 PM, Amila Maha Arachchi wrote: > Hi, > > I am trying to write a hive script which is doing some summarization. There > are two su

Re: Hive UDF intialization

>> org.apache.hadoop.hive.ql.exec.FunctionTask >> hive> create temporary function ip_2_city as "udfs.common.IP_2_GEO"; >> OK >> Time taken: 0.0080 seconds >> >> >> 1) Is there an alternate way to achieve what I expect from the command >> >> hive> create temporary function ip_2_city as >> "udfs.common.IP_2_GEO('/mnt/ravi/GeoLiteCity.dat','city')"; ? >> >> 2) What is the common practice of converting ip to City and Country in hive? >> -- >> RAVI SHETYE -- Best Regards, Ruslan Al-Fakikh

Re: Hive sort by using a single reducer

reducer). >> > According to docs "order by" clause has a limitation of 1 reducer. Does >> > sort by has same limitation? Are there any other ways of solving the above >> > requirement? >> > >> > Binesh Gummadi >> > >> > > > -- Best Regards, Ruslan Al-Fakikh

RE: Continuous log analysis requires 'dynamic' partitions, is that possible?

two 'pseudo' file tree : hdfs and hive partitions. Bertrand On Wed, Jul 25, 2012 at 10:51 AM, Bertrand Dechoux wrote: @Puneet Khatod : I found that out. And that's why I am asking here. I guess non AWS users might have the same problems and a way to solve it. @Ruslan Al-fakikh : It seems

RE: Continuous log analysis requires 'dynamic' partitions, is that possible?

If you are not using Amazon take a look at this: https://issues.apache.org/jira/browse/HIVE-874 Ruslan From: Puneet Khatod [mailto:puneet.kha...@tavant.com] Sent: Tuesday, July 24, 2012 8:32 PM To: user@hive.apache.org Subject: RE: Continuous log analysis requires 'dynamic' partitions, is

Re: Hive upload

e that imported > table gets uploaded into hdfs idr /user/hive/warehouse > I want it to be present into hive, I used this command > > sqoop import --connect jdbc:mysql://localhost:3306/Demo --username sqoop1 > --password SQOOP1 -table newone --hive-table newhive --create-hive-table > --hive-import --target-dir /user/hive/warehouse/new > > > And another thing is, > If I upload any file or table from HDFS or from Local then its uploads but > data doesn't show in Hive table, > > If I run command > Select * from NewTable; > it reflects > > Null Null NullNull > > > although the real data is > > Yogesh4Bangalore 1234 > > > Please Suggest and help > > Regards > Yogesh Kumar > > > > -- Best Regards, Ruslan Al-Fakikh

Re: Quering RDBMS table in a Hive query

ms you can place these steps in order in a shell script and just > execute the script. > > > Regards > Bejoy KS > > Sent from handheld, please excuse typos. > > -Original Message- > From: Ruslan Al-Fakikh > Date: Sat, 16 Jun 2012 04:40:36 > To: > R

Re: Quering RDBMS table in a Hive query

once the transfer was done. > > cheers! > esteban. > > -- > Cloudera, Inc. > > > > > On Fri, Jun 15, 2012 at 10:28 AM, Ruslan Al-Fakikh > wrote: >> >> Thanks Jan >> >> On Fri, Jun 15, 2012 at 4:35 PM, Jan Dolinár wrote: >> > On

Re: Quering RDBMS table in a Hive query

Thanks Jan On Fri, Jun 15, 2012 at 4:35 PM, Jan Dolinár wrote: > On 6/15/12, Ruslan Al-Fakikh wrote: >> I didn't know InputFormat and LineReader could help, though I didn't >> look at them closely. I was thinking about implementing a >> Table-Generating Function (

Re: Quering RDBMS table in a Hive query

> files, you can do that as well, InputFormat allows you to add the > columns easily. I'm not sure how well this solution would behave for a > bigger data, but for small data (I load about 5 tables, ~100 lines > each) it works just fine. > > Best Regards, > Jan > &

Quering RDBMS table in a Hive query

ng in a query. Is there a way to load RDBMS data on the fly? Maybe a UDF which would take RDBMS connection properties and load the data? Thanks in advance, Ruslan Al-Fakikh

Hadoop Russia user group

Hi everyone, I've created a group on Linkedin for Russian-speaking folks. It is about Hadoop and its components (Hive, Pig, etc). http://www.linkedin.com/groups/Hadoop-Russia-4468740?gid=4468740 Thanks, Ruslan Al-Fakikh