[
https://issues.apache.org/jira/browse/HIVE-1364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123532#comment-14123532
]
David Beveridge commented on HIVE-1364:
---------------------------------------
I think this should be re-opened.
The current limit is at 4000. This is proving to be insufficient for our own
big-data work, while using nested/hierarchical JSON files (some nodes have 16K
of structs within them).
If possible, a more generous limit should be set--our minimum would be 6500 as
of today. We modified it to be 32000 (via MySQL) and it appears works well so
far.
Cheers!
> Increase the maximum length of various metastore fields, and remove TYPE_NAME
> from COLUMNS primary key
> ------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1364
> URL: https://issues.apache.org/jira/browse/HIVE-1364
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.5.0
> Reporter: Carl Steinbach
> Assignee: Carl Steinbach
> Fix For: 0.6.0
>
> Attachments: HIVE-1364.2.patch.txt,
> HIVE-1364.3.backport-060.patch.txt, HIVE-1364.3.patch.txt,
> HIVE-1364.4.backport-060.patch.txt, HIVE-1364.4.patch.txt, HIVE-1364.patch
>
>
> The value component of a SERDEPROPERTIES key/value pair is currently limited
> to a maximum length of 767 characters. I believe that the motivation for
> limiting the length to
> 767 characters is that this value is the maximum allowed length of an index in
> a MySQL database running on the InnoDB engine:
> http://bugs.mysql.com/bug.php?id=13315
> * The Metastore OR mapping currently limits many fields (including
> SERDEPROPERTIES.PARAM_VALUE) to a maximum length of 767 characters despite
> the fact that these fields are not indexed.
> * The maximum length of a VARCHAR value in MySQL 5.0.3 and later is 65,535.
> * We can expect many users to hit the 767 character limit on
> SERDEPROPERTIES.PARAM_VALUE when using the hbase.columns.mapping
> serdeproperty to map a table that has many columns.
> I propose increasing the maximum allowed length of
> SERDEPROPERTIES.PARAM_VALUE to 8192.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
