[
https://issues.apache.org/jira/browse/HIVE-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2607:
------------------------------
Attachment: HIVE-2607.D543.1.patch
kevinwilfong requested code review of "HIVE-2607 [jira] Add caching to
json_tuple".
Reviewers: JIRA
I added an object to serve as a cache to the implementation of json_tuple.
It is copied from get_json_object, I wasn't sure how people would feel about
introducinc a dependency between the two functions.
Whenever a string is not found in the MRU cache, it is parsed into a JSON
object, and added to the cache.
TEST PLAN
I had a specific case, where a query which should have performed better in
terms of CPU usage using json_tuple rather than get_json_object, but it was
actually performing much worse because the number of unique JSON objects was
much smaller than the number of rows. After this change json_tuple performed
significantly better than get_json_object. I did not see a significant
performance degradation in other cases.
I also ran the unit tests.
REVISION DETAIL
https://reviews.facebook.net/D543
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFJSONTuple.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1197/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> Add caching to json_tuple
> -------------------------
>
> Key: HIVE-2607
> URL: https://issues.apache.org/jira/browse/HIVE-2607
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2607.D543.1.patch
>
>
> get_json_object uses a variety of caches to improve its performance.
> json_tuple could benefit from having a similar cache from JSON string to
> JSONObject.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
