Hi Mich,
No, I am not using HBase as a metastore now, but I am eager for it to
become production ready and released in CDH and HDP.
Concerning locks, I think HBase would do fine because it is ACID at the row
level. It only appends data on HDFS, but
it works by keeping regions in RAM, plus a write
Well the auto compactor fails every timeand it has been going on
for couple of days. As "day" is a partition, I have to assume, it
happens with all of them.
Both manual/auto compactor fails.
On Wed, Oct 26, 2016 at 12:52 AM, Eugene Koifman
wrote:
> does this happen for 1 specific partition o
Which of your tables are are transactional? Can you provide the DDL?
I don’t think “File does not exist” error is causing your queries to fail.
It’s an INFO level msg.
There should be some other error.
Eugene
From: satyajit vegesna
mailto:satyajit.apas...@gmail.com>>
Reply-To: "user@hive.ap
Look at this local metastore architecture:
[cid:87afe275-ff1d-4dc3-88ed-b2081163c512]
If I set different database name in javax.jdo.option.ConnectionURL, say,
"jdbc:mysql://x/hivemeta_1?createDatabaseIfNotExist=true&characterEncoding=UTF-8"
and
"jdbc:mysql://x/hivemeta_2?createDataba
HI All,
i am using hive 2.1.0 , hadoop 2.7.2 , but when i try running queries like
simple insert,
set mapreduce.job.queuename=default;set
hive.exec.dynamic.partition=true;set
hive.exec.dynamic.partition.mode=nonstrict;set
hive.exec.max.dynamic.partitions.pernode=400;set
hive.exec.max.dynamic.par
Hi Furcy,
Having used Hbase for part of Batch layer in Lambda Architecture I have
come to conclusion that it is a very good product despite the fact that
because of its cryptic nature it is not much loved or appreciated. However,
it may be useful to have a Hive metastore skin on top of Hbase table
does this happen for 1 specific partition or all of them?
On 10/25/16, 12:47 AM, "aft" wrote:
>Hi,
>
>Table created with this :
>
>$hive>create table syslog_staged (id string, facility string,
>sender string, severity string, tstamp string, service string, msg
>string) partitioned by (hostna
Hi Huang,
Hive metastore is a component of the "Hive database". See:
https://cwiki.apache.org/confluence/display/Hive/Design
The metastore uses traditional RDBMS to store "the structure information of
the various tables and partitions in the warehouse". The
javax.jdo.option.ConnectionURL and the
Hi,
To use hive metastore service, I must set `javax.jdo.option.ConnectionURL`,
`javax.jdo.option.ConnectionDriverName` and `hive.metastore.uris` in
hive-site.xml, like this:
javax.jdo.option.ConnectionURL
jdbc:mysql://x/hivemeta?createDatabaseIfNotExist=true&characterEncodi
Hi Mich,
I mostly agree with you, but I would comment on the part about using HBase
as a maintenance free core product:
I would say that most medium company using Hadoop rely on Hortonworks or
Cloudera, that both provides a pre-packaged HBase installation. It would
probably make sense for them to
Hi ,
I have configured mysql metastore. issues is on s3 supporting jar.
On Tue, Oct 25, 2016 at 4:43 PM, dv akhil wrote:
> Hi,
>which metastore are you using for hive? . Have you copied the jar
> containing the JDBC driver for your metadata db into hive's lib dir?
>
>
> down voteaccepted
Hi,
which metastore are you using for hive? . Have you copied the jar
containing the JDBC driver for your metadata db into hive's lib dir?
down voteaccepted
On Tue, Oct 25, 2016 at 4:29 PM, Rajendra Bhat wrote:
> hive> list jars;
> /opt/apache-hive-2.0.1-bin/lib/hadoop-aws-2.7.1.jar
> /op
hive> list jars;
/opt/apache-hive-2.0.1-bin/lib/hadoop-aws-2.7.1.jar
/opt/apache-hive-2.0.1-bin/lib/aws-java-sdk-1.7.4.jar
hive>
On Tue, Oct 25, 2016 at 4:27 PM, Rajendra Bhat wrote:
> yes, I have added the jar on hive prompt.
>
> On Tue, Oct 25, 2016 at 4:25 PM, aft wrote:
>
>> On Tue, Oct 25
yes, I have added the jar on hive prompt.
On Tue, Oct 25, 2016 at 4:25 PM, aft wrote:
> On Tue, Oct 25, 2016 at 6:11 AM, Rajendra Bhat
> wrote:
> > Hi,
> >
> > i am getting below error on create extrnal table. I have copied
> > hadoop-aws-2.7.1.jar and aws-java-sdk-1.7.4.jar to hive_home/lib fo
On Tue, Oct 25, 2016 at 6:11 AM, Rajendra Bhat wrote:
> Hi,
>
> i am getting below error on create extrnal table. I have copied
> hadoop-aws-2.7.1.jar and aws-java-sdk-1.7.4.jar to hive_home/lib folder.
> please let me know where should need to place supporting jar..
>
>
> hive> create external ta
Hi,
i am getting below error on create extrnal table. I have
copied hadoop-aws-2.7.1.jar and aws-java-sdk-1.7.4.jar to hive_home/lib
folder. please let me know where should need to place supporting jar..
hive> create external table kv (key int, value string) location
's3a://myntra-datasciences/
I have setup the meta store. I need to create hive external table, table
data stored in S3 with parquet format.
As i am not installed hive server for execute create command. How can I
possible execute to create table commands..?
On Tue, Oct 25, 2016 at 1:41 PM, Rajendra Bhat wrote:
> I need ex
I need external table. data should refer from S3.
On Tue, Oct 25, 2016 at 1:00 PM, Damien Carol
wrote:
> You could use CTAS in presto
>
> 2016-10-25 9:09 GMT+02:00 Rajendra Bhat :
>
>> Hi Team,
>>
>> I have configured only meta store and started the meta store service,
>> hwich i ma used on pres
I don't understand what you mean by Hive installing your schema at startup.
If you have created an empty database/schema in postgres (I am not familiar
with it), you can run the relevant script in directory
$HIVE_HOME/scripts/metastore/upgrade/postgres
That will create all the associated tables
> Could someone provide me with a code snippet (preferably Java) that installs
> the schema (through datanucleus) on my empty metastore (postgres)
I wish it was that simple, but do not leave it to the Hive startup to create it
- create it explicitly with schematool
https://cwiki.apache.org/conf
Sorry, no help from me I'm afraid, but I have a similar question:
Could someone provide me with a code snippet (preferably Java) that
installs the schema (through datanucleus) on my empty metastore (postgres).
I don't want Hive to install the schema at startup, but rather do it
explicitly myself.
Hi,
Table created with this :
$hive>create table syslog_staged (id string, facility string,
sender string, severity string, tstamp string, service string, msg
string) partitioned by (hostname string, year string, month string,
day string) clustered by (id) into 20 buckets stored as orc
tblpr
You could use CTAS in presto
2016-10-25 9:09 GMT+02:00 Rajendra Bhat :
> Hi Team,
>
> I have configured only meta store and started the meta store service,
> hwich i ma used on presto.
>
> I need create table on metastore.. how can i able create that.. as i am
> not started hive server service, b
Hi Team,
I have configured only meta store and started the meta store service, hwich
i ma used on presto.
I need create table on metastore.. how can i able create that.. as i am not
started hive server service, bcoz hadoop not installed on my syatem.
--
Thanks and
Regards
Rajendra Bhat
reply
On Tue, Oct 25, 2016 at 12:19 PM, wrote:
> Hi! This is the ezmlm program. I'm managing the
> user@hive.apache.org mailing list.
>
> To confirm that you would like
>
>rajhalk...@gmail.com
>
> added to the user mailing list, please send
> a short reply to this address:
>
>user-sc.147
25 matches
Mail list logo
