;ve missed
it and it is documented somewhere, or maybe someone knows from their
experience?
If it's an atomic operation, is there any difference whether the table is
external or a managed one?
Thank you,
Mark
Could anyone point me a bit into the right direction as to where such code is
supposed to be written?
With kind regards,
Mark
Hey,
I was looking for some info and found that nice article, just read it here
<http://spardepengi.paidmarketing.tips/aepysl>
Wishes, Mark Markovic
Hi guys,
who knows why class HiveConf has no HADOOPCONF enum type in hive common jar
now?
I write code using hive-common-1.2.1.jar HiveConf class to access HDFS(HA
namenode), and I get an error below. I realized my code didn't
config HADOOPCONF so it can't connect to HDFS, but there is no HADOOPCO
-19 8:12 GMT+08:00 Alan Gates :
> This looks to me like a Hadoop issue rather than Hive. It appears that
> you cannot connect to HDFS. Have you tried connecting to HDFS outside of
> Hive/HCatalog?
>
> Alan.
>
> > On May 18, 2016, at 04:24, Mark Memory wrote:
> >
&g
hello guys, sorry to bother you.
I'm using hcatalog to write hive tables, but I don't know how to do with
namenode HA
my code was copied from
https://github.com/apache/hive/blob/master/hcatalog/core/src/test/java/org/apache/hive/hcatalog/data/TestReaderWriter.java
*below is my config:*
hiveC
Actually, I found a UDF that does this very nicely!
https://github.com/petrabarus/HiveUDFs/blob/master/src/main/java/net/petrabarus/hiveudfs/LongToIP.java
Share and enjoy!
--
Mark E. Sunderlin
Data Architect // Data Solutions
P: 703-265-6935 // C: 540-327-6222 // 22000 AOL Way, Dulles, VA
it in
the "dot" format.
raw_formatdot_format166136010699.6.87.234162645867796.241.202.53
--
Mark E. Sunderlin
Data Architect // Data Solutions
P: 703-265-6935 // C: 540-327-6222 // 22000 AOL Way, Dulles, VA 20166
AIM: MESunderlin // Slack: @Mark.Sunderlin
very much appreciated
Thanks in advance
Mark Wildig
Sent from my iPhone
join because you're using the partitioning step
in order to enforce the join predicate instead of actually evaluating it.
This could be much more efficient in some cases, so I was wondering if Hive
can do it and if so, how can I trigger it?
Cheers,
Mark
On Mon, Jun 9, 2014 at 10:34 AM,
d the associated rows are rare, it's very inefficient.
So, can hive perform the join the first way I described? If so, any idea
how I trigger it? Or do I have to write my own MR job?
Cheers,
Mark
A single result set that looks like:
Sequence_numint
User_id string
User-attributes map /* A map of keyid=valueid for each struct from the
original array */
Can this be done in Hive? Is inline() the key to doing this?
--
Mark E. Sunderlin
Data Architect // AOL Platforms
P: 703-265-6
The size(map) function is defined as follows: size(Map)Returns the
number of elements in the map type
What if I want the total size of the map for that row? This doesn't work:
select length(MAP);
How can I get the total size of a map column in either bytes or characters?
---
M
Hive Metatool experts,
Trying to use the hive "metatool" to update the locations for all of our tables
after a configuration change did not do this automatically. The tool gets a
heap error even though we upped the JVM to 4 GB. Does anyone know of a
workaround?
Thanks,
Mark
n Mon, Oct 14, 2013 at 12:18 PM, Mark Grover wrote:
> I wasn't able to start webhcat server. It seems to be related to file
> permissions.
>
> FWIW, it's not a regression (the same problem existed in Hive 0.11).
> Having said that, it makes webhcat pretty unusable out of the
I wasn't able to start webhcat server. It seems to be related to file
permissions.
FWIW, it's not a regression (the same problem existed in Hive 0.11). Having
said that, it makes webhcat pretty unusable out of the box. I
created HIVE-5534 to track this.
On Sun, Oct 13, 2013 at 4:37 PM, Carl Stei
Hmm.. No replies on this one? Is no one use Hue? :-) That would be
interesting to know .. if not Hue, how are others exposing Hive to "end users?"
without given them a direct login to a node on the cluster?
---
Mark E. Sunderlin
Data Architect | AOL NETWORKS BDM
P: 703-265-6935
lines of output, so it cannot be a number of rows issue. Is
there a limit to the width of the data you can download? A limit on the number
of columns?
Anyone seen this before? Does anyone know a fix or a work around?
---
Mark E. Sunderlin
Data Architect | AOL NETWORKS BDM
P: 703-265-6935
Stephen is right. Here is some context:
Looking at the UDF source code (
https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFFromUnixTime.java#L121),
it's using SimpleDateFormat where the format string follows the convention
at
http://docs.oracle.com/javase/1.4.2
Hi Omkar,
What Serde does the table use? Given that you're able to index on leg_id it
doesn't sound quite the same as the issue I hit. Is there any partitioning
of the table?
-Mark
On Wed, Jul 31, 2013 at 10:39 PM, Omkar Joshi
wrote:
> I'm facing issues while building an
;
> On Tue, Jul 30, 2013 at 1:40 PM, Mark Wagner wrote:
>
>> My id is mwagner. Thanks!
>>
>>
>> On Tue, Jul 30, 2013 at 1:36 PM, Ashutosh Chauhan
>> wrote:
>>
>>> Mark,
>>>
>>> Do you have an account on hive cwiki. Whats your id
My id is mwagner. Thanks!
On Tue, Jul 30, 2013 at 1:36 PM, Ashutosh Chauhan wrote:
> Mark,
>
> Do you have an account on hive cwiki. Whats your id ?
>
> Thanks,
> Ashutosh
>
>
> On Tue, Jul 30, 2013 at 1:06 PM, Mark Wagner wrote:
>
>> Hi all,
>>
>>
Hi all,
Would someone with the right permissions grant me write access to the Hive
wiki? I'd like to update some information on the Avro Serde.
Thanks,
Mark
?&icid=mainmoiz%dl' LIKE ('%?icid=main%') THEN 'T'
ELSE 'F'
END
AS case1,
CASE
WHEN '?&icid=mainmoiz%dl' LIKE ('%?icid=main%dl%') THEN 'T'
ELSE 'F'
END
We are using Hive 0.80.
---
Mark E. Sunderlin
Solutions Architect |AOL Networks BDM
P: 703-265-6935 |C: 540-327-6222 | AIM: MESunderlin
22000 AOL Way | Dulles, VA | 20166
From: Stephen Sprague [mailto:sprag...@gmail.com]
Sent: Wednesday, June 12, 2013 1:00 PM
To: user@hive.apache.org
Subject
wo or more '%' wild cards in a Hive like expression a
known issue?
---
Mark E. Sunderlin
Solutions Architect |AOL NETWORKS BDM
P: 703-265-6935 |C: 540-327-6222 | AIM: MESunderlin
22000 AOL Way, Dulles, VA 20166
Avro SerDe
transparently. This JIRA is tracking imporved union support for Hive, but
it's not being actively worked on:
https://issues.apache.org/jira/browse/HIVE-2390.
Thanks,
Mark
On Thu, May 23, 2013 at 11:45 AM, Scott Carey wrote:
> The Hive mailing list would have more info on the Avr
Glad to hear, Steffan. Do you mind creating a JIRA for this at
https://issues.apache.org/jira?
Thanks in advance!
Mark
On Mon, Apr 8, 2013 at 6:38 AM, LUTTER, Steffen wrote:
> Hi Mark,
>
> ** **
>
> Correct, I just did some tests and the cast is the way to go. While f
Glad to hear!
On Fri, Apr 5, 2013 at 3:02 PM, Sadananda Hegde wrote:
> Thanks, Mark.
>
> I found the problem. For some reason, Hive is not able to write Avro
> output file when the schema has a complex field with NULL option. It read
> without any problem; but cannot write with
Steffan,
One thing that may be different is that equal can cast operands to make
equals work but that may not be true for IN. FWIW, this is me just
speculating, I haven't looked at the code just yet.
Perhaps, you could explicit casting to get around this?
On Fri, Apr 5, 2013 at 7:36 AM, LUTTER, S
Can you please check your Jobtracker logs? The is a generic error related
to grabbing the Task Attempt Log URL, the real error is in JT logs.
On Wed, Apr 3, 2013 at 7:17 PM, Sadananda Hegde wrote:
> Hi Dean,
>
> I tried inserting a bucketed hive table from a non-bucketed table using
> insert ove
Trying to make sure I understand this correctly. All databases and tables are
stored in hive.metastore.warehouse.dir but the actual metadata for the database
and tables (columns, types, partitions, etc) are stored in the hive database
(ie.. mysql)?
Is that correct?
itters can grant you permissions to edit the wiki.
Thanks again!
Mark
On Mon, Mar 25, 2013 at 9:36 AM, chenchun wrote:
> Hi mark,
> I read the the wiki of PhabricatorCodeReview
> https://cwiki.apache.org/confluence/display/Hive/PhabricatorCodeReview and
> send a review https://reviews.
rectory name:
> /path/to/hive -f /path/to/addlogpartition.hql -d YEAR=$YEAR -d MON=$MONTH -d
> DAY=$DAY
>
>
> (Of course, all the /path/to will be different...)
>
> So, be careful of how how "03" vs. "3" is handled in both the ALTER TABLE
> statement
code.google.com/p/snappy/
>
>
> Note that for example - gzip format is not splittable..so huge gzip files
> without splits are not recommended as input to maps
>
> Hope this helps
>
> sanjay
>
>
> On 3/29/13 10:19 AM, "Mark" wrote:
>
>> We have e
Thanks
Does this mean I need to create a partition for each day manually? There is no
way to have infer that from my directory structure?
On Mar 29, 2013, at 10:32 AM, Dean Wampler
wrote:
>
>
> On Fri, Mar 29, 2013 at 12:19 PM, Mark wrote:
> We have existing log data in direct
We have existing log data in directories in the format of YEAR/MONTH/DAY.
- How can we create a table over this table without hive modifying and/or
moving it?
- How can we tell Hive to partition this data so it knows about each day of
logs?
- Does hive out of the box work with reading compresse
Hey Dean,
I am not a power user of the sampling feature but my understanding was that
sampling in Hive only works on bucketed tables. I am happy to be corrected
though.
Mark
On Wed, Mar 20, 2013 at 12:20 PM, Dean Wampler <
dean.wamp...@thinkbiganalytics.com> wrote:
> Mark,
>
>
Then you will be able to be sample through it.
If the predicate is on partition column, you may be able to get around the
intermediate table requirement but in general, as far as I know,
intermediate bucketed table might be the only choice.
Mark
On Wed, Mar 20, 2013 at 9:56 AM, Robert Li wrote
For the sake of readers, I just saw another manifestation of this problem.
The issue was that mapreduce.framework.name in mapred-site.xml was set
to "yarn" instead of "classic" when the intent was to use MR1.
Hope that helps,
Mark
On Mon, Mar 4, 2013 at 8:26 AM, Mark Grover
Hi Chenchun,
Can you please:
1. Send a separate email to this same mailing list to add you as a
contributor to the JIRA? I don't have credentials to do so and seems
like other people are not watching this thread:-)
2. Upload this patch to the JIRA?
Thanks!
Mark
On Sat, Mar 16, 2013 at 4:
Someone will have to add you to the project contributors on JIRA.
Could you please share your JIRA account ID and email address?
Mark
On Thu, Mar 14, 2013 at 9:58 AM, chenchun wrote:
> Cool, I'd like to steal it from you. I'd like to get a few days to solve it
> but how I can
ke to contribute a
patch (along with updated tests), please feel free to steal it from
me.
Mark
On Thu, Mar 14, 2013 at 12:44 AM, chenchun wrote:
> Hi,
> I have a question about HIVE-3675 NaN does not work correctly for round(n).
> The issue did solve the NaN problem, but the hql below result
Created https://issues.apache.org/jira/browse/HIVE-4144
On Fri, Mar 8, 2013 at 5:25 AM, Dean Wampler
wrote:
> It's odd that there is no such command. The trick Ramki mentioned is the
> only one I know of. Two points about it, though:
>
> 1. It only works on Hive v0.8+.
> 2. I've seen a few cases
Suresh,
By default, the partition column name has to be appear in HDFS
directory structure.
e.g.
/user/hive/warehouse//=/data1.txt
/user/hive/warehouse//=/data2.txt
On Thu, Mar 7, 2013 at 7:20 AM, Suresh Krishnappa
wrote:
> Hi All,
> I have the following directory structure in hdfs
>
> /test/a/
or.java:471)
>> at
>> org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
>> at
>> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
>> ... 9 more
>> Caused by: java.lang.IllegalArgumentException: Timestamp
I typically change my query to query from a limited version of the whole table.
Change
select really_expensive_select_clause
from
really_big_table
where
something=something
group by something=something
to
select really_expensive_select_clause
from
(
select
*
from
really_big_table
limit 100
)t
w
derby metastore
and use MySQL or PostgreSQL for metastore instead. Googling should
give you some nice articles on how to do that.
Mark
On Tue, Mar 5, 2013 at 3:48 AM, Sai Sai wrote:
> Hello
>
> I have noticed when i execute the following command from hive shell in diff
> folders it beh
Hi Dilip,
Are you able to run this query successfully?
select d_date_sk, d_date_id, d_date, d_month_seq, d_week_seq,
d_quarter_seq, d_dow, d_moy, d_dom, d_qoy, d_fy_year,
d_fy_quarter_seq, d_fy_week_seq, d_day_name, d_quarter_name,
d_holiday, d_weekend, d_following_holiday, d_first_dom, d_last_dom
Congrats, Kevin. Keep up the good work!
On Mon, Mar 4, 2013 at 11:54 AM, Carl Steinbach wrote:
> On behalf of the Apache Hive PMC I am pleased to welcome
> Kevin Wilfong as a member of the Apache Hive PMC.
>
> Please join me in congratulating Kevin on his new role!
>
> Thanks.
>
> Carl
Sorry for the delayed response, Marcin, I was away.
This error in the Gist means that the jar containing the class in
question is not on your classpath.
Exception in thread "Thread-39" java.lang.NoClassDefFoundError:
org/apache/hadoop/mapreduce/util/HostUtil
at
org.apache.hadoop.hive.shim
s original problem
because it doesn't take ranges.
I created HIVE-4100 for improving regex_replace UDF.
Mark
On Fri, Mar 1, 2013 at 9:31 AM, Dean Wampler
wrote:
> Anyone know if translate takes ranges, like some implementations? e.g.,
>
> translate ('[a-z]', '[A-Z]
Chunky,
There may be another way to do this but to get tab separated output, I
usually create an external table that's tab separated and insert
overwrite into that table.
For limiting the number of records in the output, you can use the
limit clause in your query.
Mark
On Tue, Feb 19, 20
that fixes it?
Also, if you could provide me any information (your config, setup,
which queries fail, etc.) that could help me in reproducing the
problem on hive trunk would be greatly appreciated!
Mark
On Tue, Feb 19, 2013 at 1:08 AM, Marcin Cylke wrote:
>
> On 14/02/13 08:27, Ma
://issues.apache.org/jira/browse/HIVE-4003 which is assigned to me, I
will try to get this in sooner than later.
In the meanwhile, a workaround is to set yarn.resourcemanager.address
property to some value other than "local" so Hive queries can run on your
MR2 cluster.
Mark
On Wed, Feb 13, 20
mple with sample data and expected output, that might be helpful.
Mark
On Mon, Feb 11, 2013 at 7:34 PM, John Omernik wrote:
> I am trying to do a union, group by, and multi insert all at once. I know
> this convoluted but I what I am trying to do is avoid having to scan
> through the origina
and see if that fits your use case?
Mark
On Mon, Feb 11, 2013 at 2:59 PM, Parag Sarda wrote:
> Hello Hive Users,
>
> I am writing a program in java which is bundled as JAR and executed using
> hadoop jar command. I would like to access hive metadata (read partitions
> informations) i
Hi Abhishek,
The code looks incomplete.
See the comment at
https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/UDAF.java#L22
Those are all the methods your UDAF class needs to implement but you seem
to be missing them.
Mark
On Sat, Feb 9, 2013 at 11:08 PM
on github somewhere so it's easier
to analyze?
Mark
On Fri, Feb 8, 2013 at 2:05 PM, Abhishek Bhattacharya wrote:
> Hi,
>
> I have implemented a simple UDAF for top-n-percent as follows:
> import java.util.ArrayList;
> import java.util.Collections;
>
> import org.apach
Suresh,
Take a look at this:
https://issues.apache.org/jira/browse/HIVE-3231
On Thu, Feb 7, 2013 at 11:46 AM, Krishnappa, Suresh <
suresh.krishna...@rsa.com> wrote:
> Hi All,
>
> I have created a partitioned HIVE external table as follows
>
> ** **
>
> create external table test_part (key
Thanks for the update, Marc. Could you please create a Hive JIRA for this?
Thanks!
Harsh, it's most likely the case with your JIRA as well that the error
message is misleading. Thanks for reporting it!
Mark
On Thu, Jan 31, 2013 at 4:58 PM, Marc Limotte wrote:
> Thanks, Harsh. M
Marc,
It seemed to work on my installation of Hive 0.9. Is this a table or a view
you are trying to drop?
Mind sharing the create table definition?
Mark
On Thu, Jan 31, 2013 at 10:42 AM, Marc Limotte wrote:
> Hi,
>
> I'm testing with Hive 0.9.0 (in Amazon EC2, but NOT emr). I&#x
, you'll have to look
into the documentation to see if there is an alter table partition
(partition spec)... command that will let you alter metadata about the
partition.
If I am not mistaken, alter table doesn't touch your existing columns, just
modifies the partitions going forward.
Ma
ns:
1. Download the patch, rebuild hive and use it
2. Find a hacky way to recover your partitions when they are empty and
populate them later.
Sorry for the inconvenience.
Mark
On Tue, Jan 29, 2013 at 5:09 PM, Sadananda Hegde wrote:
> Thanks Mark,
>
> Recover partition feature will sa
Measure 1: +1
Measure 2: +1
On Tue, Jan 29, 2013 at 4:21 PM, Zhenxiao Luo wrote:
>
> Measure 1: +1
> Measure 2: +1
>
>
> Thanks,
> Zhenxiao
>
>
Sadananda,
See if this helps:
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Recoverpartitions
On Mon, Jan 28, 2013 at 8:05 PM, Sadananda Hegde wrote:
> Hello,
>
> My hive table is partitioned by year, month and day. I have defined it as
> external table. Th
Hi John,
Thanks for reporting this.
Can you please take a look at the Lateral View issues here:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20HIVE%20AND%20text%20~%20%22lateral%20view%22
and create a new JIRA with the details if one previously doesn't exist?
Thanks again,
Mar
Thanks for the update, Viral.
One way I can think of is to drop the metadata for the entire table and
recreate the table with the updated regex and recover partitions from HDFS.
This, of course, works if your table is external.
Mark
On Fri, Jan 25, 2013 at 12:21 PM, Viral Bajaria wrote:
> W
Viral,
To me, I think the only option seems to be here to alter all the existing
partitions.
I would be interested in knowing what you ended up finding/doing.
Thanks,
Mark
On Sun, Jan 20, 2013 at 3:58 PM, Viral Bajaria wrote:
> Hi,
>
> I have a hive table which has pre-defined schema
uot;data_table" has around 5k fields, all doubles.
> As for the "age_mean" table, here it is:
>
> hive> desc age_mean;
> OK
> id string
> name string
> age_mean double
> Time taken: 0.127 seconds
>
> Does this help?
>
> Thanks!
> Fernando
>
&
Fernando,
Could you share your table definitions as well please?
On Tue, Jan 15, 2013 at 10:31 AM, Fernando Andrés Doglio Turissini <
fernando.dog...@globant.com> wrote:
> Hello everyone, I'm struggling with an exception I'm getting on a
> particular query that's driving me crazy!
>
> Here is the
Brad is correct, there is a JIRA about this already:
https://issues.apache.org/jira/browse/HIVE-3822
Sorry for the inconvenience.
Mark
On Fri, Jan 4, 2013 at 8:25 AM, Brad Cavanagh wrote:
> Try multiplying your values by 1000, then running the conversions. I bet
> they expect millis
Souvik,
I am not sure I understand, you may want to play around with distinct
keyword if you want to join on unique values.
In any case, it looks like you've got it figured out already, so it's all
good:-)
Happy trails!
Mark
On Sat, Dec 29, 2012 at 2:19 PM, Souvik Banerjee
wrote:
>
);
Mark
On Thu, Dec 27, 2012 at 9:26 PM, Souvik Banerjee
wrote:
> Thanks a lot Mark for your attention.
> But I think I cannot go for INNER join, the reason behind the fact being
> that I want all rows of Table3 irrespective of there is any row
> corresponding to that ID in Table
rted merge join or
sorted merge bucketed joins depending on your use case.
Mark
On Thu, Dec 27, 2012 at 3:16 PM, Souvik Banerjee
wrote:
> Hi,
>
> I am struggling with a problem described below.
> Any help how to resolve this problem is highly appreciated.
>
> I have got few tables
About #2, doesn't sound normal to me. Did you figure that out or still
running into it?
Mark
On Thu, Dec 20, 2012 at 5:01 PM, java8964 java8964 wrote:
> Hi, I have 2 questions related to the hive behavior when using 'add jar'.
>
> I am testing the implementing of my o
reboot?
Mark
On Wed, Dec 19, 2012 at 11:21 PM, Neil Guo wrote:
> another log,
>
> cd $HADOOP_LOG_DIR && tail -n 0 -f *.log |tee log_merge.log
>
>
> ==> hadoop-neil-datanode-Ct-bj108.log <==
>
> ==> hadoop-neil-jobtracker-Ct-bj108.log <==
>
> ==>
Hi Cheng,
It would really help if you could simplify your example so it's easy
to reproduce by one of us.
Also, can you share the value of hive.optimize.ppd property by doing
hive> set hive.optimize.ppd;
Does inverting the property (making it false, if it's true) change anything?
On Thu, Nov 15,
like make additions to those UDFs, you are
welcome to do so by creating a JIRA and posting a patch. UDFs are an
easy and excellent way to contribute back to the Hive community.
Thanks!
Mark
On Wed, Dec 19, 2012 at 8:52 AM, java8964 java8964 wrote:
> Hi, I have a question related to the X
And clients may be on mac, which takes '\r' as line delimiter
>
> So I think '\r' should also be replaced with space in HiveHistory.log, so
> that HiveHistory.parseHiveHistory could be consistent with HiveHistory.log
> and allow clients from mac
>
> Thanks!
will take another look.
Thanks!
Mark
On Mon, Dec 17, 2012 at 5:48 AM, Qiang Wang wrote:
> anybody has an idea about this ?
>
> https://issues.apache.org/jira/browse/HIVE-3810
>
>
>
> 2012/12/16 Qiang Wang
>>
>> glad to receive your reply!
>>
>> her
;s code to see what's going on.
Mark
On Mon, Dec 17, 2012 at 2:19 AM, Mohit Chaudhary01
wrote:
>
> I am using serde in hive to store data into hive table from xml file.
>
> Whenever I retrieve data using command select * from table it give all
> records from table.
>
&g
Krishna,
I usually put it in my home directory and that works. Did you try that?
HIVE-2911, adds another location to where it can be picked up from. If your
present version supports .hiverc (which is most likely the case), home
directory should work as well.
Mark
On Mon, Dec 17, 2012 at 5:44 AM
sure. I am not sure of any such
restriction for #1. For #3 your best bet is to look at the ODBC driver
code.
Mark
On Mon, Dec 17, 2012 at 11:48 AM, Omer, Farah wrote:
> Hi,
>
> Does anyone know of the SQL limits for Hive? Particularly the following:
>
>
> 1. The maximum n
Sekine,
You may want to go to the jobtracker web interface and look at the
failed task logs for more info.
Mark
On Sun, Dec 16, 2012 at 1:36 PM, Sékine Coulibaly wrote:
> Hi there,
>
> I loaded data from the movielens database into hive, into a u_data table. I
> wish I could cou
No, .hiverc only works for CLI.
UDFs are tricky. The only way I can think of is to add them to the
function registry
(https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java)
and recompile Hive.
On Mon, Dec 10, 2012 at 8:01 AM, John Omernik wrot
Fernando,
It is more likely related to your SerDe and the underlying data not
matching up to it and than being related to the table being external
on S3.
Mark
On Tue, Dec 11, 2012 at 6:05 AM, Fernando Andrés Doglio Turissini
wrote:
> Long subject, I know.. let me explain a bit more about
which the
last I saw was 0.8*), it would be interesting how Shark's Hive 0.9 is
going to play around with EMR's version of Hive. FWIW, commands like
"ALTER TABLE RECOVER PARTITIONS" are only available in EMR Hive.
Keep us posted!
Mark
On Mon, Dec 10, 2012 at 1:46 PM, Erik Th
date?), also make sure you
don't suffer from the small file problem:
http://www.cloudera.com/blog/2009/02/the-small-files-problem/
Good luck!
Mark
On Wed, Dec 12, 2012 at 11:36 PM, Nitin Pawar wrote:
> 6GB size is nothing. We have done it with few TB of data in hive.
> Error you are seeing is o
This is just my understanding, the full proof answer would lie in
checking out the explain plans and the Semantic Analyzer code.
And for completeness, there is a conditional task (starting Hive 0.7)
that will convert your joins automatically to map joins where
applicable. This can be enabled by enab
is
to this discussion but I will have to look further to comment more.
In the meanwhile take a look at the UDF code and see if your personal Java
code on Linux is equivalent to the Hive UDF code.
Keep us posted!
Mark
On Fri, Dec 7, 2012 at 1:27 PM, Periya.Data wrote:
> Hi Hive Users,
>
lost (or gained) minutes due to day
light savings time. If that's important, it will be best to convert precise
timestamps to UTC and then subtract them.
Mark
On Wed, Dec 5, 2012 at 12:33 PM, Babe Ruth wrote:
> Hello All,
>
> I would like to find the total amount of minutes between t
ive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTranslate.java#L172
Good luck!
Mark
On Tue, Dec 4, 2012 at 6:58 AM, Søren wrote:
> Thanks. Did you mean I should handle null in my udf or my serde?
>
> I did try to check for null inside the code in my udf, bu
arounds go, you could try having each table in a sub-query
and have an extra virtual column ("1 as one") and join on that virtual
column.
Mark
On Mon, Dec 3, 2012 at 9:26 AM, Periya.Data wrote:
> Hi Hive users,
> I have Hive CDH - 0.7.1. I want to know if I can do cross-j
Yong,
In my opinion, #2 wouldn't be that hard to do.
Of course, upgrading to the new API (albeit by updating Hive) would the
right thing to do long term.
Mark
On Mon, Dec 3, 2012 at 7:42 AM, java8964 java8964 wrote:
> Hi,
>
> Our company current is using CDH3 release, which c
Hey David,
Sure thing. Play around with that property's value, see if that makes any
difference.
Also, if you could search to see if a file with a name like *hive_skew_join_
**bigkeys* exists on HDFS. Perhaps, it's looking at a different path. If
so, we can figure out how to fix that.
Hi David,
It seems like Hive is unable to find the skewed keys on HDFS.
Did you set *hive.skewjoin.key property? If so, to what value?*
Mark
On Fri, Nov 30, 2012 at 2:10 AM, David Morel wrote:
> Hi,
>
> I am trying to solve the "last reducer hangs because of GC because of
> t
So, to summarize, a single table may work, if the following 3 conditions
are met.
1. New columns get added to the very end of the existing columns
2. No existing columns are deleted
3. You are ok with NULL showing up as value for old records that don't have
a given new column.
I tested the above wi
Adding user@hive.apache.org
Ben,
That's great to hear. It would be awesome if you'd like to contribute this
back to Hive so others in the community could use it too. Let us know what
you think!
Mark
On Wed, Nov 28, 2012 at 8:05 PM, ben wrote:
> Mark,
>
> Just wanted to let
1 - 100 of 304 matches
Mail list logo
