Hive : ngrams : stopword usage

hi All, I was trying to explore the usage of ngrams with a custom stopword list. Though the documentation states that it can be used, there is no example provided and the syntax is also not apparent. Any pointers are welcome Please forgive me if this is a repost/known issue and I am still ignorant

UDF development group details

hi All, Can you please redirect/connect me with Hive custom UDF development group. My intent is to create / co-develop custom UDFs for text analytics and data mining over Hive directly. -- Devopam Mittra Life and Relations are not binary

Re: Hive giving errors while running queries

Please check your yarn log. It will have details around this error . regards Dev +91 958 305 9899 > On Jul 7, 2014, at 10:46 AM, Ritesh Kumar Singh > wrote: > > hive>select COUNT(*) from movies; > Total MapReduce jobs = 1 > Launching Job 1 out of 1 > Number of reduce tasks determined at compil

Re: Drop Partition by ID

there. But what > exactly has to be deleted? I guess there are several dependencies. > > Other idea: is there a possibility in Hive to delete a partition by a > unique ID or something like that? > > Or what is needed to delete the table with the normal "alter table drop > partition" command? > > Cheers > Wolli > -- Devopam Mittra Life and Relations are not binary

Re: SELECT specific data

ny attachment to it. Murex cannot guarantee that it is > virus free and accepts no responsibility for any loss or damage arising > from its use. If you have received this e-mail in error please notify > immediately the sender and delete the original email received, any > attachments and all copies from your system. > -- Devopam Mittra Life and Relations are not binary

Re: SELECT specific data

wrote: > I am only using Hive and hadoop, nothing more. > > > > *From:* Devopam Mittra [mailto:devo...@gmail.com] > *Sent:* Wednesday, July 30, 2014 12:15 PM > > *To:* user@hive.apache.org > *Subject:* Re: SELECT specific data > > > > Are you using any tool

Re: Hive Data

You may please give dbpedia dataset a try - I am sure you won't be disappointed :) regards Dev +91 958 305 9899 > On Jul 30, 2014, at 6:05 PM, CHEBARO Abdallah > wrote: > > Till now I don’t have a file. I am willing to search online for a sample > dataset that contains at least 1 million row

Re: Tuning Triangle Joins on Hive

duce.tasks.maximum=8; > set mapred.child.java.opts=-Xmx5120m; > > The dataset we're using has 5 million nodes and 70 million edges, and most > of our time is spent on garbage collection. We have about 30 machines in > our cluster, and each machine has 45GB of RAM. Any thoughts on how we can > improve performance? Thanks in advance! > -- Devopam Mittra Life and Relations are not binary

Re: How can I know one table is a partitioned table in hive?

ich can interact with hcatalog ? > or you can do a describle table and check for partion column details in > there > > > On Thu, Jul 31, 2014 at 1:11 PM, 张甲超 wrote: > >> dear all, >> I want know that one table is a partitioned table in hive, and >> ret

Re: Hive or phoenix

support its own security layer, which is concern to > me. I need some clarification on following points > > > > 1. Is it possible to run MR as a pre-complied one rather than on > fly. If yes, then what are the step to take to fulfil this one. > > > > > > > > > > Thanks in advance, > > > > > > Regards, > > Prakash Hosalli > > > -- Devopam Mittra Life and Relations are not binary

Re: audit hive query

to view the hive > queries executed on hive data. > > > > Any help is appreciated, > sivakumar.c > -- Devopam Mittra Life and Relations are not binary

Re: audit hive query

me populated in a sql table, but not through the web browser. Can > u suggest me some idea on dis scenario. > > > Thanks and regards, > siva > On Mon, Sep 22, 2014 at 9:18 AM, Devopam Mittra wrote: > >> Please check .hivehistory settings and it might be of easy / sim

Re: Insert into hive table without overwriting data

to use it as is, or any part of it. A wrong > transmission does not break its confidentiality. If you've received it > because of a mistake or erroneous transmission, please notify the sender > and delete it from your system immediately. This communication environment > is controlled and monitored. > > B2W Digital > > > -- Devopam Mittra Life and Relations are not binary

Re: PLEASE HELP :: HOW TO DO INNER JOIN IN HIVE

hi Yogesh, Please try to leverage common table expression (WITH ...) to achieve your desired outcome regards Dev > On Oct 15, 2014, at 1:18 PM, yogesh dhari wrote: > > Hello all, > > I have a use case where I need to do inner join.. > > Like > > select A.iteam , B.Decsription, > from itea

Re: will I get conflict if I run 2 "INSERT INTO TABLE" in parallel ?

ions are run in parallel ? it's fine if > I get a blocking, but I need to guarantee correctness of data. > > if there is conflict, would "INSERT OVERWRITE PARTITION" get conflicts ? > the different processes indeed process different partitions > > thanks > Yang

Need to place an edit request for https://wiki.apache.org/hadoop/FrontPage

hi , I wanted to make some edits to the Hadoop main page. https://wiki.apache.org/hadoop/FrontPage Not able to trace the list of administrators / place a request for granting edit rights. Any help/pointers will be big help ! -- Devopam Mittra Life and Relations are not binary

Re: Need to place an edit request for https://wiki.apache.org/hadoop/FrontPage

; > which suggests that's where you request edit permission ( > common-...@hadoop.apache.org > <http://hadoop.apache.org/mailing_lists.html#Common>). > > -- Lefty > > On Fri, Oct 24, 2014 at 12:33 AM, Devopam Mittra > wrote: > >> hi , >> I wanted to

Re: [ANNOUNCE] New Hive PMC Member - Alan Gates

Congratulations Alan ! regards Dev On Tue, Oct 28, 2014 at 4:08 AM, Carl Steinbach wrote: > I am pleased to announce that Alan Gates has been elected to the Hive > Project Management Committee. Please join me in congratulating Alan! > > Thanks. > > - Carl > -- De

Unix script for identifying current active namenode in a HA cluster

can leverage the same without spending effort to code it. Do share your feedback/ fixes if you spot any. -- Devopam Mittra Life and Relations are not binary findActiveNameNode.sh Description: Bourne shell script

Re: Unix script for identifying current active namenode in a HA cluster

v 5, 2014 at 10:02 AM, Devopam Mittra wrote: > >> hi All, >> Please find attached a simple shell script to dynamically determine the >> active namenode in the HA Cluster and subsequently run the Hive job / query >> via Talend OS generated workflows. >> >> I

Re: Parsing Hive Query to get table names and column names

input = new ANTLRStringStream("SELECT x FROM >>> abc"); >>> HiveLexer lexer = new HiveLexer(input); >>> TokenStream tokens = new CommonTokenStream(lexer); >>> HiveParser parser = new HiveParser(tokens); >>> System.out.println(parser.statement()); >>> >>> *How should I proceed from here to extract the table names and column >>> names?* >>> *And, Is the way I am doing it correct?* >>> >>> Thank You. >>> Regards, >>> Ritesh >>> >>> >> >> >> -- >> Alok Kumar >> http://sharepointorange.blogspot.in/ >> > > -- Devopam Mittra Life and Relations are not binary

Re: How to do single user multiple access in hive

ple > queries simultaneously. > > Thanks > Amit > -- Devopam Mittra Life and Relations are not binary

Re: Setting job diagnostics to REDUCE capability required - error in hive

ability:8192 > > I tried to setup in hive: SET hive.exec.reducers.max=8192 > > but this doesn't change anything. What I did wrong? > > > -- Devopam Mittra Life and Relations are not binary

Re: How to do single user multiple access in hive

tegrate oozie with this. > > Thanks > Amit > > > > On Fri, Nov 7, 2014 at 7:58 PM, Devopam Mittra wrote: > >> hi Amit, >> Please try to see if Hive CLI (client) installed on the 'different' >> machines helps you achieve your goal at the minimalist e

Re: [ANNOUNCE] New Hive PMC Member - Prasad Mujumdar

+1 :) regards Devopam On Wed, Dec 10, 2014 at 3:47 AM, Carl Steinbach wrote: > I am pleased to announce that Prasad Mujumdar has been elected to the Hive > Project Management Committee. Please join me in congratulating Prasad! > > Thanks. > > - Carl > -- Devopam Mitt

Re: compare float column

; hive> select long from test_float; > select long from test_float > Status: Finished successfully > OK > -41.338276 > Time taken: 6.843 seconds, Fetched: 1 row(s) > > > Any ideas? I am using hive version 0.13. > > BR, > Patcharee > > > > > -- Devopam Mittra Life and Relations are not binary

struggling with Hive ACID - need help

oat) _FUNC_(int) _FUNC_(smallint) _FUNC_(string) _FUNC_(timestamp) _FUNC_(tinyint) _FUNC_(void) * Any pointers are welcome -- Devopam Mittra Life and Relations are not binary

Re: struggling with Hive ACID - need help

, Devopam Mittra wrote: > Env: Hive 0.14 , HDP2.2, Centos 6.6 > > I created a simple table (ref syntax below) > DROP TABLE IF EXISTS twitter_user_info; > > CREATE TABLE twitter_user_info > ( userid BIGINT COMMENT 'User Id', > username STRING COMMENT 'Us

Re: date_add statement problem

lso did > > *Date_add(‘1985/01/01’,cast(day_key as int))* > > Resulted in an error saying the column is short. > > > > Please help me with this. > > > > Thanks > > Regards, > > Yashwanth M *|* +91 8884 177 884 *|*www.mu-sigma.com *|* > > > Disclaimer: http://www.mu-sigma.com/disclaimer.html > -- Devopam Mittra Life and Relations are not binary

Hive UPDATE : query around usage

a SELECT with other table to identify / match the record. Kindly drop a pointer to the syntax manual to look it up. Happy weekend ! -- Devopam Mittra Life and Relations are not binary

Re: Hive UPDATE : query around usage

he.org/confluence/display/Hive/Hive+Transactions> > > > -- Lefty Leverenz > > > On Fri, Jan 16, 2015 at 6:09 AM, Devopam Mittra wrote: > >> hi All, >> I wanted to UPDATE multiple rows in a table by joining it with another >> table for identification of matching re

Hive Logo details

hi, We don't have the Hive logo available on Wikipedia page. Can someone please point me to the logo owner or the creative license details + location so that I can do the same for proper display. ref: https://en.wikipedia.org/wiki/Apache_Hive -- Devopam Mittra Life and Relations are not binary

Re: convert empty strings to null

best way to achieve this?? > -- Devopam Mittra Life and Relations are not binary

Re: Warning when running hive

ion > does not exist > WARN conf.HiveConf: HiveConf of name hive.semantic.analyzer.factory.impl > does not exist > WARN conf.HiveConf: HiveConf of name > hive.auto.convert.sortmerge.join.noconditionaltask does not exist > > Regards, > Philippe Kernévez > > -- Devopam Mittra Life and Relations are not binary

Re: [ANNOUNCE] New Hive PMC Members - Szehon Ho, Vikram Dixit, Jason Dere, Owen O'Malley and Prasanth Jayachandran

ase join me in congratulating the these new PMC > members! > > Thanks. > > - Carl > -- Devopam Mittra Life and Relations are not binary

Re: Unable to retrieve date

esult: Null* > > What may be the issue? > -- > With Thanks & Regards > Nagarjuna Vissarapu > 9052179339 > > -- Devopam Mittra Life and Relations are not binary

Re: Which [open-souce] SQL engine atop Hadoop?

t your experience is. >> >> On Mon, Feb 2, 2015 at 12:08 AM, Samuel Marks >> wrote: >> >>> Well what I am seeking is a Big Data database that can work with Small >>> Data also. I.e.: scaleable from one node to vast clusters; whilst >>> maintaining

Re: Which [open-souce] SQL engine atop Hadoop?

operations. It is optimized for data > warehousing or analytics¹ > <http://prestodb.io/docs/current/overview/use-cases.html>" > > Your thoughts? > > Best, > > Samuel Marks > http://linkedin.com/in/samuelmarks > On 03/02/2015 6:06 pm, "Devopam Mittra"

Re: Hive - regexp_replace function for multiple strings

gt;> the white spaces. Below is what I tried. I also tried using trim with >> regexp_replace but it wants the third argument in the regexp_replace >> function. >> >> drop table vp_hiphop; >> create table vp_hiphop asselect userid, ntext, >> regexp_replace(ntext, '(hip hop)|(rock music)') as ntext1from >> vp_nlp_protext_males; >> >> > -- Devopam Mittra Life and Relations are not binary

Re: How to query data by page in Hive?

ata a limit 1,2; > FAILED: ParseException line 1:31 missing EOF at ',' near '1' > > ------ > r7raul1...@163.com > -- Devopam Mittra Life and Relations are not binary

Re: Re: How to query data by page in Hive?

1...@163.com > > > *From:* Devopam Mittra > *Date:* 2015-02-05 18:05 > *To:* user@hive.apache.org > *Subject:* Re: How to query data by page in Hive? > You may want to use a ROW_NUMBER OR RANK / DENSE RANK in the inner query > and then select only a subset of it in the

Re: [ANNOUNCE] Apache Hive 1.0.0 Released

> Hive 1.0.0 Release Notes are available here: > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12329278&styleName=Text&projectId=12310843 > > > > > > We would like to thank the many contributors who made this release > > possible. > > > > Regards, > > > > The Apache Hive Team > -- Devopam Mittra Life and Relations are not binary

Re: Remove duplicated rows

Can you please state the hive version regards Dev +91 958 305 9899 > On Feb 17, 2015, at 11:01 PM, Philippe Kernévez wrote: > > Hi, > > I have a table (named DEDUPLICATED) that contains about 1 billions rows. > Each day I receive a new file with about 5 millions rows. About 10% of those > r

Re: Remove duplicated rows

wrote: > Hi Dev, > > I'm using hive 0.14. > > Regards, > > On Tue, Feb 17, 2015 at 8:12 PM, Devopam Mittra wrote: > >> Can you please state the hive version >> >> regards >> Dev >> +91 958 305 9899 >> >> On Feb 17, 2015, at

Re: Remove duplicated rows

mpact of the partition columns on the map reduce > job, they will probably produce a big number of files to produce those > partitions. > > > Regards, > Philippe > > > > On Wed, Feb 18, 2015 at 2:49 AM, Devopam Mittra wrote: > >> dear Philippe, >> I asked

Re: Re: How to query data by page in Hive?

numFiles'='1', > 'numRows'='0', > 'rawDataSize'='0', > 'totalSize'='1979173', > 'transient_lastDdlTime'='1421076916') > > *columns :* >movieid > > -- > r7raul1...@163.com > > &

Re: how to add partitioned by column

from source table. Hope this helps regards Dev On Tue, Mar 3, 2015 at 1:23 PM, smartzjp wrote: > I have created a table with " create table partitioned by (year int > ,mont int)" > now I want to add a day partition. how can i do it ? TKS > -- Devopam Mittra Life and Relations are not binary

need input for heavy duty string manipulation

hi there, Need help / suggestion to formulate a hive-only best feasible/optimized strategy for: - I have a table with list of reserved words - one word per row - I have a string wherein I want to search the presence of each reserve word and take a count of it .e.g: If there are 5 matches

Re: partition and bucket

ore neither Peridale Ltd, its subsidiaries nor their employees accept > any responsibility. > > > > *From:* Ashok Kumar [mailto:ashok34...@yahoo.com] > *Sent:* 10 April 2015 17:46 > *To:* user@hive.apache.org > *Subject:* partition and bucket > > > > Greeting all, > > Glad to join the user group. I am from DBA background Oracle/Sybase/MSSQL. > > I would like to understand partition and bucketing in Hive and the > difference between. > > Shall be grateful if someone explains where shall I use partition or > bucket for best practices. > > thanks > > > -- Devopam Mittra Life and Relations are not binary

Re: creating a table based on where column=x

>> CREATE TABLE derivedtable AS SELECT * FROM initialtable WHERE >> deployment_id='22’; >> >> >> >> I would just like to create a table where the deployment_id is 22 and not >> have to create the initial table. >> >> >> >> If someone can shed some light on how to do this I would appreciate it. >> The only example I have seen seem to follow the above? >> >> >> >> >> >> Thanks, >> >> Gary C >> > > -- Devopam Mittra Life and Relations are not binary

Re: Updating hive metadata

ssemination, distribution or copying of this > communication is strictly prohibited. If you are not one of the named > recipient(s) or otherwise have reason to believe that you received this > message in error, please immediately notify sender by e-mail, and destroy > the original message. Thank You. > -- Devopam Mittra Life and Relations are not binary

Re: Date Functions in Hive

his can be done with existing Hive 14.0 functionality > or it will require custom UDF. > > Please let me know if any more information is required on the same. > > Thanks, > Saurabh > > > > > -- Devopam Mittra Life and Relations are not binary

Re: EXPORTing multiple partitions

lt; >>> brian.jelt...@digitalenvoy.net> wrote: >>> >>> > Using Hive .13, I would like to export multiple partitions of a table, >>> something conceptually like: >>> > >>> > EXPORT TABLE foo PARTITION (id=1,2,3) to ‘path’ >>> > >>> > Is there any way to accomplish this? >>> > >>> > Brian >>> >>> >> > -- Devopam Mittra Life and Relations are not binary

Re: [ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan

ny of the major advances that > have been made over the past couple of years. Please join me in > congratulating Ashutosh on his new role! > -- Devopam Mittra Life and Relations are not binary

need help to keep the Hive logo on wikipedia

logo on the respective wikipedia page(s) else the logo image will be deleted... e.g. https://it.wikipedia.org/wiki/Apache_Hive , https://en.wikipedia.org/wiki/Apache_Hive Can someone help me with this ? regards -- Devopam Mittra Life and Relations are not binary

Re: need help to keep the Hive logo on wikipedia

rk restrictions, while the permissions > demanded by Wikipedia is about Copyright. > > Copyright license is covered by the APL v2, since the source image is > checked into hive git repo under docs/images/. > > The reason Wikipedia has flagged this image is probably because it is > mark

request for editing cwiki, username: devopam

regards Dev

Re: Hive alternatives?

I agree with the suggestions presented already. You may want to check presto as an alternative as well. But please remember , Presto is an added layer on top of Hive and not an independent alternative. It simplifies your semantic layer and querying while being faster than Hive. For OLAP , I will al

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

s these days. > Please give John a warm welcome to the project! > > Ashutosh > -- Devopam Mittra Life and Relations are not binary

Re: Storing the Hive Query Results into Variable

make use of it latter stage. > > Thanks, > Mahender > -- Devopam Mittra Life and Relations are not binary

Re: Hive Support for Unicode languages

> On 04 Dec 2015, at 01:25, mahender bigdata > wrote: > > > > > > Hi Team, > > > > > > Does hive supports Hive Unicode like UTF-8,UTF-16 and UTF-32. I would > like to see different language supported in hive table. Is there any serde > which can show ex

Re: Fastest way to get the row count

an running Select count(1) or > Explain > commands, is there better way to get row count of affected row. (previous > step row count) > > Mahender > -- Devopam Mittra Life and Relations are not binary

Re: Is it ok to build an entire ETL/ELT data flow using HIVE queries?

tored in HIVE. We have transactional and reference data. We need >> to build a small warehouse. >> >> Need suggestion on alternatives too. >> >> Regards, >> Rams > > -- Devopam Mittra Life and Relations are not binary

Re: count(*) not allowed in order by

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+WindowingAndAnalytics This should help you , try rank/ dense rank as appropriate and mold it to best use for yourself Regards Dev On Mar 7, 2016 10:35 PM, "Awhan Patnaik" wrote: > I have to take the first 25 IDs ranked by count(*).

Re: What is the best way to store IPv6 address in Hive?

My best bet will be string data type itself with partitioning to aid partial search. Please do consider the fact that ipv6 address is more complicated than ipv4 in terms of searching . Regards Dev On 28 Jun 2016 9:35 pm, "Igor Kuzmenko" wrote: > Currently I'm using ORC transactional tables, and

Re: Need help with query

Kindly learn dynamic partition from cwiki. That will be the perfect solution to your requirement in my opinion. Regards Dev On 13 Sep 2016 12:49 am, "Igor Kravzov" wrote: > Hi, > > I have a query like this one > > alter table my_table > add if not exists partition (mmdd=20160912) location

Re: Happy Diwali to those forum members who celebrate this great festival

+1 Thanks and regards Devopam On 30 Oct 2016 9:37 pm, "Mich Talebzadeh" wrote: > Enjoy the festive season. > > Regards, > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >

Re: DateFunction

PM, Mahender Sarangam < > mahender.bigd...@outlook.com> wrote: > >> Hi, >> >> Is there any Date Function which returns Full Month Name for given time >> stamp. >> >> > -- Devopam Mittra Life and Relations are not binary

Re: VARCHAR or STRING fields in Hive

of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > -- Devopam Mittra Life and Relations are not binary

Re: Difference between join and inner join

+1 to this question. I will appreciate if someone can shed light on behind the scenes for this one. Might give us ideas for best practice. Regards Devopam On 11 Feb 2017 8:02 pm, "Divya Gehlot" wrote: > Hi , > What's the difference between" join " and "inner join" in hive ? > > Thanks , > Divya

Re: Programmatically determine version of Hive running on server

regards, > Bohdan > -- Devopam Mittra Life and Relations are not binary

Re: out of memory using Union operator and array column type

t issues so > suspect something with the actual data. > > The table schema is this: > c1string > c2bigint > c3array> > local_date string > > > I've narrowed it down and (not surprisingly) the 3rd column seems to be the > cause of the issue, If I remove that the union works again just fine. > > Anyone has similar experiences? Perhaps any pointers on how to tackle this? > > Kind regards, > > Patrick > > > -- Devopam Mittra Life and Relations are not binary

Re: out of memory using Union operator and array column type

ery good question, Yes that does give the same problem. > > Op ma 11 mrt. 2019 om 16:28 schreef Devopam Mittra : > >> Can you please try doing SELECT DISTINCT * FROM DELTA into a physical >> table first ? >> regards >> Dev >> >> >> On Mon,

Re: out of memory using Union operator and array column type

ee if that gets anywhere. Other pointers are much appreciated. > > Thanks for the help! > > > > > > Op ma 11 mrt. 2019 om 19:35 schreef Devopam Mittra : > >> hi Patrick, >> If it sounds worth trying please do the same: >> >> 1. Create physical ta

Re: Converting Hive Column from Varchar to String

The table has data in it perhaps that is beyond ASCII. Easier way is to go for additional column , update with data and the drop the older one after validation of records in String type col. Regards Dev On Thu, Jul 18, 2019, 4:44 AM William Shen wrote: > Hi all, > > I assumed that it should be

Re: Subquery failing - is there an alternative?

Please try with subquery alias . Regards On Mon, Dec 9, 2019, 6:06 AM Dan Horne wrote: > Hi All > > I'm trying to run the following subquery but it returns an error that says > "cannot recognize input near 'select' 'max' '(' in expression specification" > > select id, > > first_name, > > last_n

Re: [BUG] Hive 3.1.2 ALTER TABLE statement

Hi, Could you please try the same statement with CASCADE clause. As a general rule, please avoid doing it in production directly unless that is the last option. Regards Dev On Mon, 19 Apr, 2021, 8:03 pm Vinícius Matheus Olivieri, < olivierivi...@gmail.com> wrote: > Hey guys! > > I am using HIVE

Re: [ANNOUNCE] Apache Hive 4.1.0 Released

customary +1 to the congratulatory note ! warm regards Devopam Mittra tel://+919583059899 On Mon, Aug 4, 2025 at 11:10 AM Butao Zhang wrote: > Thank you to all contributors for making the Hive 4.1.0 release possible. > I would especially thank Denys and Zhihua for their invaluable assi