For the most part, I think just using the string type is sufficient.
Replied Message
| From | liubin_w...@yeah.net |
| Date | 01/16/2025 15:01 |
| To | user@hive.apache.org |
| Cc | |
| Subject | which one is better to use in hive when storage date like text type
,string or varchar
them in this
table, the varchar for definite length field, and the string for indefinite
length field.
Thanks for your help!
table, the varchar for definite length field, and the string for indefinite
length field.
Thanks for your help!
Thank you everyone for your help! Owen, we're on an old version of hive
(1.1.0-cdh5.9.2).
On Thu, Jul 18, 2019 at 9:38 AM Owen O'Malley
wrote:
> ORC files expect UTF-8, which is a superset of ascii, in strings, char,
> and varchar. The only place that I know that will cause tr
ORC files expect UTF-8, which is a superset of ascii, in strings, char, and
varchar. The only place that I know that will cause trouble if you put
non-utf-8 data in strings is the statistics. The API for getting the
min/max will convert to Java strings.
But back to your original point, the schema
should be compatible to convert column type varchar to
> string, however, after running ALTER TABLE table CHANGE col col STRING, I
> encounter the following error when querying the column from hive:
>
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.q
Which version of Hive are you on? The recent versions (hive >= 2.3) should
support schema evolution in the ORC reader.
.. Owen
On Wed, Jul 17, 2019 at 11:07 PM Jörn Franke wrote:
> You have to create a new table with this column as varchar and do a select
> insert from the old table.
You have to create a new table with this column as varchar and do a select
insert from the old table.
> Am 18.07.2019 um 01:14 schrieb William Shen :
>
> Hi all,
>
> I assumed that it should be compatible to convert column type varchar to
> string, however, after running
Wed, Jul 17, 2019 at 4:14 PM William Shen
wrote:
> Hi all,
>
> I assumed that it should be compatible to convert column type varchar to
> string, however, after running ALTER TABLE table CHANGE col col STRING, I
> encounter the following error when querying the column from hive:
Hi all,
I assumed that it should be compatible to convert column type varchar to
string, however, after running ALTER TABLE table CHANGE col col STRING, I
encounter the following error when querying the column from hive:
Failed with exception
> Sounds like VARCHAR and CHAR types were created for Hive to have ANSI SQL
> Compliance. Otherwise they seem to be practically the same as String types.
They are relatively identical in storage, except both are slower on the CPU in
actual use (CHAR has additional padding code in the ho
Sounds like VARCHAR and CHAR types were created for Hive to have ANSI SQL
Compliance. Otherwise they seem to be practically the same as String types.
HTH
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.
Thanks Elliot for the insight.
Another issue that Spark does not support "CHAR" types. It supports
VARCHAR. Often one uses Spark as well on these tables.
This should not really matter. I tend to define CHA(N) to be VARCHAR(N) as
the assumption is that the table ingested into Parq
Internally it looks as though Hive simply represents CHAR/VARCHAR values
using a Java String and so I would not expect a significant change in
execution performance. The Hive JIRA suggests that these types were added
to 'support for more SQL-compliant behavior, such as SQL string compa
thanks both.
String has a max length of 2GB so in a MapReduce with a 128MB block size we
are talking about 16 blocks. With VARCHAR(30) we are talking about 1 block.
I have not really experimented with this, however, I assume a table of 100k
rows with VARCHAR columns will have a smaller footprint
table. For example if a table created in Hive as Parquet I will
use VARCHAR(30) for column that has been defined as VARCHAR(30) as source.
If a column is defined as TEXT in RDBMS table I use STRING in Hive with a
max size of 2GB I believe.
My view is that it is more efficient storage wise to have
Few things that might have an effect:
1. Compression (better if you are in VARCHAR with finite length, instead of
a STRING)
2. Multicharset support (like NVARCHAR)
3. LOBs from RDBMS world are more suitable to be typecast to STRING for
pure text data (not images e.g.)
regards
Devopam
On Mon
Coming from DBMS background I tend to treat the columns in Hive similar to
an RDBMS table. For example if a table created in Hive as Parquet I will
use VARCHAR(30) for column that has been defined as VARCHAR(30) as source.
If a column is defined as TEXT in RDBMS table I use STRING in Hive with a
I created https://issues.apache.org/jira/browse/HIVE-13119
Title: "java.io.IOException: java.io.IOException: ORC does not support type
conversion from CHAR to VARCHAR (due to Schema Evolution HIVE-11981)?"
Thanks for reporting the issue.
New version Hive 2
a simple Job fails with
2016-02-22T18:23:29,008 ERROR [main]: mr.MapredLocalTask
(MapredLocalTask.java:executeInProcess(381)) - Hive Runtime Error: Map
local work failed
JAVA.IO.IOEXCEPTION: JAVA.IO.IOEXCEPTION: ORC DOES NOT SUPPORT TYPE
CONVERSION FROM CHAR TO VARCHAR
@hive.apache.org
Subject: Re: Read error : Varchar cannot be cast to string
Hi,
I don¹t think I have the privileges to move an ATLAS JIRA into HIVE.
And the pre-commit tests for HIVE do not run for ATLAS issues.
If you have access to the JIRA admin for that project, please move that
issue over
vansh Srivastava"
wrote:
>Gopal,
>
>I have raised the bug (ATLAS-27).
>
>Thanks,
>Devansh
>
>
>From: Devansh Srivastava
>Sent: Thursday, June 18, 2015 4:05 PM
>To: user@hive.apache.org
>Subject: Re: Read error : Varchar can
Gopal,
I have raised the bug (ATLAS-27).
Thanks,
Devansh
From: Devansh Srivastava
Sent: Thursday, June 18, 2015 4:05 PM
To: user@hive.apache.org
Subject: Re: Read error : Varchar cannot be cast to string
Hi Gopal,
Yes, one of the partition column is
Hi Gopal,
Yes, one of the partition column is having VARCHAR as datatype.
My target table has structure like this :--
CREATE EXTERNAL TABLE test_table(
dob string COMMENT '',
version_nbr int COMMENT '',
record_status string COMMENT '',
creati
ion column the one marked as a varchar?
Can you write a small test-case and post a bug about this?
I can take a look at this, looks like a simple missed call to toString().
Cheers,
Gopal
Hi,
I have one table with VARCHAR and CHAR datatypes. While reading data through
hive, I am getting below error :--
Diagnostic Messages for this Task:
Error: java.io.IOException: java.io.IOException: java.lang.RuntimeException:
java.lang.ClassCastException
Varchar was only added to Hive in 0.12, before that there was only string if
you wanted to deal with string types. Varchar will enforce the max character
length, truncating the string value if necessary. We've tried to make it as
compatible with string as possible.
One thing about varchar
>
> I am a newbie to hive. I am trying to query an oracle table in hive. The
> data is in hadoop and i have created similar varchar columns in my external
> table. Would like to know what are the limitations of varchar over string.
> And why people prefer string over varc
I am a newbie to hive. I am trying to query an oracle table in hive. The
data is in hadoop and i have created similar varchar columns in my external
table. Would like to know what are the limitations of varchar over string.
And why people prefer string over varchar.
Thanks,
Gayathri
What version of Hive?
Do you have some sample SQL?
On Aug 26, 2014, at 1:20 PM, upd r wrote:
> Hi,
>
> I have a created a table with fields defined as varchar(length). Is it
> correct to insert data in to the table casting the fields as VARCHAR(length).
>
> I am getting t
Hi,
I have a created a table with fields defined as varchar(length). Is it
correct to insert data in to the table casting the fields as
VARCHAR(length).
I am getting this error.
Error occurred executing hive query: OK FAILED: SemanticException Generate
Map Join Task Error: Class cannot be
Hi,
I have a created a table with fields defined as varchar(length). Is it
correct to insert data in to the table casting the fields as
VARCHAR(length).
I am getting this error.
Error occurred executing hive query: OK FAILED: SemanticException Generate
Map Join Task Error: Class cannot be
; Hive 0.12.0.
>
> When hive.optimize.index.filter is set to true, queries against ORC tables
> with VARCHAR fields fail with the following errors:
>
> Error: java.io.IOException: java.lang.reflect.In
All,
I am running Hadoop 2.2.0 & Hive 0.12.0.
When hive.optimize.index.filter is set to true, queries against ORC tables
with VARCHAR fields fail with the following errors:
Error: java.io.IOException: java.lang.reflect.InvocationTargetExceptio
34 matches
Mail list logo
