Re: Need help with query

your requirement in my opinion. >> Regards >> Dev >> >> On 13 Sep 2016 12:49 am, "Igor Kravzov" wrote: >> >>> Hi, >>> >>> I have a query like this one >>> >>> alter table my_table >>> add if not e

Need help with query

Hi, I have a query like this one alter table my_table add if not exists partition (mmdd=20160912) location '/mylocation/20160912'; Is it possible to make so I don't have to change date every day? Something with CURRENT_DATE;? Thanks in advance.

What can cause the exception bellow

Hi, SELECT explode(context_ngrams(sentences(lower(text)), array(null,null,null), 25)) AS x from andy_tweets where regexp_extract(lower(text), '(^.*?)(\\bclinton\\b)(.*?$)', 2) = 'clinton' and regexp_extract(lower(text), '(^.*?)(\\bpetition\\b)(.*?$)', 2) = 'petition'; The statement above causes e

Re: JsonSerDe and mapping tweet's user structure error

nfiguration+Properties#ConfigurationProperties-hive.support.sql11.reserved.keywords>=false. > > {quote} > > > > On Tue, Aug 16, 2016 at 3:23 PM, Igor Kravzov > wrote: > >> Hi, >> >> I am storing tweets in HDFS and creating extern

JsonSerDe and mapping tweet's user structure error

Hi, I am storing tweets in HDFS and creating external table on top of these tweets.. The command I am using is bellow: *create external table tweets (* * id bigint,* * in_reply_to_screen_name string,* * in_reply_to_status_id bigint,* * user struct<* * id: bigint, * * name: strin

Re: hive.aux.jars.path question

It worked. Thanks. Still not sure if it is right way to do it. On Wed, Jul 6, 2016 at 5:45 PM, Fan Xu wrote: > Yes, we need to restart hiveserver in Ambari > -- > *From:* Igor Kravzov > *Sent:* Wednesday, July 6, 2016 2:36:41 PM > *To:* user@

Re: hive.aux.jars.path question

th work, would it be a > bug? > > The only way I made it work is to add my jar to {hive_home}/lib/ > > > Thanks, > > > Fan > > -- > *From:* Igor Kravzov > *Sent:* Wednesday, July 6, 2016 2:13:07 PM > *To:* user@hive.apache.org &g

hive.aux.jars.path question

Hi guys, HDP 2.4.2 distribution I want to add elasticsearch-hadoop-2.3.2.jar connector to hive path using hive.aux.jars.path property. Added it to custom hive-site.xml Copied jar to /user/hive/aux_jars folder in HDFS. I specified path for the property as /user/hive/aux_jars/elasticsearch-hadoop

Re: JsonSerDe file format question

Found the issue. Looks like rows should be separated by new line. On Wed, Jun 8, 2016 at 2:55 PM, Igor Kravzov wrote: > I am merging multiple JSON file in a bigger one before saving it to HDFS. > So merged file looks like this > > {"id":160889136,"url":&quo

Re: JsonSerDe file format question

There are 3 files with 3 JSON records each. But when I run select * from testtable; it returns me only first row from each one of files instead of 9. On Wed, Jun 8, 2016 at 2:55 PM, Igor Kravzov wrote: > I am merging multiple JSON file in a bigger one before saving it to HDFS. > So merge

JsonSerDe file format question

I am merging multiple JSON file in a bigger one before saving it to HDFS. So merged file looks like this {"id":160889136,"url":" http://twitter.com/PatrocinarBRA/statuses/740301352052654080";, ..}{"id":160889137,"url":" http://twitter.com/tchiagoolimpio/statuses/740301352253825024 ",...}{"id":1608

Re: Why does the user need write permission on the location of external hive table?

n.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > > On 6 June 2016 at 21:00, Igor Kravzov wrote: > >> I see file

Re: Why does the user need write permission on the location of external hive table?

I see file are with extension .gz. Are these zipped? Did you try with unzipped files? Maybe in order to read the data hive needs to unzip files but does not have write permission? Just a wild guess... On Tue, May 31, 2016 at 4:20 AM, Sandeep Giri wrote: > Yes, when I run hadoop fs it gives resul

Re: Convert date in string format to timestamp in table definition

adddate; > > HTH > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://tale

Re: Convert date in string format to timestamp in table definition

gt; > > www.blablabla.compg12016-05-17T02:10:44.527 > 2016-05-16T02:08:55 YadaYada 20160604 > > > > hive> > > create view threads_test_v > > as > > select url > > ,pagetype > >,cast (concat_ws(' ',substr (adddate ,1,1

Convert date in string format to timestamp in table definition

Hi, I have 2 dates in Json file defined like this "addDate": "2016-05-17T02:10:44.527", "postDate": "2016-05-16T02:08:55", Right now I define external table based on this file like this: CREATE external TABLE threads_test (url string, pagetype string, adddate string, postdate string, postte

Re: External partitoned table based on yyyy/mm/dd HDFS structure

all > enough to be handled by operations such as > > ALTER TABLE table_name [PARTITION partition_spec] SET FILEFORMAT > file_format; > > in a reasonable time. > > > > > > Common practice is a daily partition > > ~365 days * 10 years = 3,650 partitions, which is O

Re: External partitoned table based on yyyy/mm/dd HDFS structure

└── data.txt > > │ ├── mm=02 > > │ │ └── dd=17 > > │ │ └── data.txt > > │ └── mm=03 > > │ ├── dd=04 > > │ │ └── data.txt > > │ ├── dd=05 > > │ │ └── data.txt > > │ └── dd=06 > > │

External partitoned table based on yyyy/mm/dd HDFS structure

Is it possible to create? What would be the table definition in this case?

Is it possible to use external table on top of Elasticsearch index for arbitrary FTS

I know that external table can be defined like this CREATE EXTERNAL TABLE artists ( id BIGINT, nameSTRING, links STRUCT) STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' TBLPROPERTIES('es.resource' = 'radio/artists', 'es.query' = '?q=me*'); But can I just define

Re: Test

thanks guys. On Sun, May 29, 2016 at 2:31 PM, Markovitz, Dudu wrote: > Hi J > > > > *From:* Igor Kravzov [mailto:igork.ine...@gmail.com] > *Sent:* Sunday, May 29, 2016 8:02 PM > *To:* user@hive.apache.org > *Subject:* Test > > > > Please someone reply. Not sure if subscribed properly >

Test

Please someone reply. Not sure if subscribed properly

To use a table defined on Elasticsearchindex for arbitrary full text search

down votefavorite I know that it is possible to create Hive table with predefined query like this CREATE EXTERNAL TABLE artists ( id BIGINT, nameSTRING, links