[jira] [Created] (HIVE-13315) Option to reuse existing restored HBase snapshots

2016-03-18 Thread Liyin Tang (JIRA)
Liyin Tang created HIVE-13315: - Summary: Option to reuse existing restored HBase snapshots Key: HIVE-13315 URL: https://issues.apache.org/jira/browse/HIVE-13315 Project: Hive Issue Type

[jira] [Commented] (HIVE-2095) auto convert map join bug

2011-04-07 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017208#comment-13017208 ] Liyin Tang commented on HIVE-2095: -- it looks good to me. Thanks Yongqiang > auto

[jira] [Commented] (HIVE-2095) auto convert map join bug

2011-04-07 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016871#comment-13016871 ] Liyin Tang commented on HIVE-2095: -- I will take a look > auto convert map j

[jira] [Commented] (HIVE-1966) mapjoin operator should not load hashtable for each new inputfile if the hashtable to be loaded is already there.

2011-03-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13010510#comment-13010510 ] Liyin Tang commented on HIVE-1966: -- +1 > mapjoin operator should not load hashta

[jira] [Commented] (HIVE-1965) Auto convert mapjoin should not throw exception if the top operator is union operator.

2011-03-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13010508#comment-13010508 ] Liyin Tang commented on HIVE-1965: -- +1 > Auto convert mapjoin should not throw ex

Re: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-12-27 Thread Liyin Tang
ed? Did I understand it right? > Thanks! > Alicia > > On Mon, Dec 27, 2010 at 12:05 AM, Liyin Tang wrote: > > > Hi, > > If multiple tables join on different join keys, it will be separated into > > multiple MapRed Tasks. > > Also the threshold of the sma

Re: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-12-27 Thread Liyin Tang
and map join them. What if t1 and t2 are not co-located with join keys? > What > if t1 and t2 together are bigger than 25M? > > Thanks in advance! > On Thu, Dec 23, 2010 at 7:16 PM, Liyin Tang wrote: > > > Hi, > > How large is t1 and t2 ? > > if both of t1 and t2 i

Re: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-12-23 Thread Liyin Tang
Hi, How large is t1 and t2 ? if both of t1 and t2 is larger than 25M (a default threshold), the query processor will do the common join. Thanks Liyin On 23 December 2010 18:50, wrote: > Hi, > > I set hive.auto.convert.join=true and run the following query: > > select t1.foo, count(t2.bar) from

[jira] Updated: (HIVE-1845) Some attributes in the Eclipse template file is deprecated

2010-12-09 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1845: - Status: Patch Available (was: Open) > Some attributes in the Eclipse template file is depreca

[jira] Updated: (HIVE-1845) Some attributes in the Eclipse template file is deprecated

2010-12-09 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1845: - Attachment: hive-1845-1.patch Update all the eclipse template files. > Some attributes in the Ecli

[jira] Created: (HIVE-1845) Some attributes in the Eclipse template file is deprecated

2010-12-09 Thread Liyin Tang (JIRA)
: Liyin Tang Assignee: Liyin Tang In the eclipse template file, it will reference this jar file, which is deprecated. /@PROJECT@/build/metastore/hive-mod...@hive_version@.jar So the correct one should be: /@PROJECT@/build/metastore/hive-metasto...@hive_version@.jar Just update all

[jira] Updated: (HIVE-1842) Add the local flag to all the map red tasks, if the query is running locally.

2010-12-08 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1842: - Attachment: hive-1842-1.patch Add the local flag to all the map red tasks, if the query is running

[jira] Updated: (HIVE-1842) Add the local flag to all the map red tasks, if the query is running locally.

2010-12-08 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1842: - Status: Patch Available (was: Open) Add the local flag to all the map red tasks, if the query is running

[jira] Created: (HIVE-1842) Add the local flag to all the map red tasks, if the query is running locally.

2010-12-08 Thread Liyin Tang (JIRA)
Issue Type: Sub-task Components: Query Processor Affects Versions: 0.4.1 Reporter: Liyin Tang Assignee: Liyin Tang -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1830) mappers in group followed by joins may die OOM

2010-12-08 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1830: - Attachment: hive-1830-5.patch 1) Remove the debug statements 2) Add the memory threshold to group by desc

[jira] Updated: (HIVE-1830) mappers in group followed by joins may die OOM

2010-12-07 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1830: - Attachment: hive-1830-4.patch 1) Add more descriptions in the config file 2) Set the memory usage of

[jira] Updated: (HIVE-1830) mappers in group followed by joins may die OOM

2010-12-06 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1830: - Attachment: hive-1830-3.patch Carefully measure the memory usage of map side group by. Flush frequently

[jira] Updated: (HIVE-1830) mappers in group followed by joins may die OOM

2010-12-06 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1830: - Attachment: hive-1830-2.patch Add a new test: auto_join26.q > mappers in group followed by joins

[jira] Updated: (HIVE-1830) mappers in group followed by joins may die OOM

2010-12-06 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1830: - Attachment: hive-1830-1.patch > mappers in group followed by joins may die

[jira] Updated: (HIVE-1827) Audit how many queries will be run in the local mode

2010-12-06 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1827: - Attachment: hive-1827-1.patch Add a new attribute isLocalMode in Task. > Audit how many queries will

[jira] Commented: (HIVE-1832) Dynamically allocate and measure memory usage when a map join op followed by a group by op

2010-12-06 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12967268#action_12967268 ] Liyin Tang commented on HIVE-1832: -- Duplicate of Hive-1830 > Dynamically alloc

[jira] Created: (HIVE-1832) Dynamically allocate and measure memory usage when a map join op followed by a group by op

2010-12-06 Thread Liyin Tang (JIRA)
Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Liyin Tang Assignee: Liyin Tang Right now, if a map join operator followed by a map-side group by, this map reduce task will be memory intensive task. Memory usage should be

[jira] Created: (HIVE-1827) Audit how many queries will be run in the local mode

2010-12-03 Thread Liyin Tang (JIRA)
Tang Assignee: Liyin Tang Hive can run query in local mode. It would be nice to track and audit how many queries will be run in the local mode. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Assigned: (HIVE-1700) Optimiza JDBM to make mapjoin faster

2010-12-01 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang reassigned HIVE-1700: Assignee: Liyin Tang > Optimiza JDBM to make mapjoin fas

[jira] Resolved: (HIVE-1700) Optimiza JDBM to make mapjoin faster

2010-12-01 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang resolved HIVE-1700. -- Resolution: Won't Fix Release Note: The JDBM component has been removed from Hive. No ne

[jira] Updated: (HIVE-1811) Show the time the local task takes

2010-11-24 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1811: - Attachment: hive-1811-1.patch The original showTime code has potential bug if the local task takes more

[jira] Updated: (HIVE-1804) Mapjoin will fail if there are no files associating with the join tables

2010-11-24 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1804: - Attachment: hive-1804-3.patch Since there are some other patches committed recently, I regenerate the

[jira] Created: (HIVE-1811) Show the time the local task takes

2010-11-24 Thread Liyin Tang (JIRA)
: 0.7.0 Reporter: Liyin Tang Assignee: Liyin Tang Fix For: 0.7.0 After the local tasks finished, show the how much time it takes -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1792) track the joins which are being converted to map-join automatically

2010-11-24 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1792: - Attachment: hive-1792-4.patch 1) Remove unrelated change from this patch 2) Set the backup tag in the

[jira] Commented: (HIVE-1792) track the joins which are being converted to map-join automatically

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935234#action_12935234 ] Liyin Tang commented on HIVE-1792: -- There will be 2 cases to run the common join. On

[jira] Updated: (HIVE-1792) track the joins which are being converted to map-join automatically

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1792: - Attachment: hive-1792-3.patch Since Hive-1785 has been committed, I generate the diff again. So this diff

[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935186#action_12935186 ] Liyin Tang commented on HIVE-1785: -- Thanks John's review and I have created a

[jira] Updated: (HIVE-1810) a followup patch for changing the description of hive.exec.pre/post.hooks in conf/hive-default.xml

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1810: - Status: Patch Available (was: Open) Patch is available > a followup patch for changing the descript

[jira] Updated: (HIVE-1810) a followup patch for changing the description of hive.exec.pre/post.hooks in conf/hive-default.xml

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1810: - Attachment: hive-1810-1.patch change the hive-default.xml. So new pre/post hook should implements the

[jira] Created: (HIVE-1810) a followup patch for changing the description of hive.exec.pre/post.hooks in conf/hive-default.xml

2010-11-23 Thread Liyin Tang (JIRA)
-1810 Project: Hive Issue Type: Sub-task Reporter: Liyin Tang Assignee: Liyin Tang a followup patch for changing the description of hive.exec.pre/post.hooks in conf/hive-default.xml -- This message is automatically generated by JIRA. - You can reply

[jira] Commented: (HIVE-1792) track the joins which are being converted to map-join automatically

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935171#action_12935171 ] Liyin Tang commented on HIVE-1792: -- Still need this change to tag on all the join t

[jira] Updated: (HIVE-1792) track the joins which are being converted to map-join automatically

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1792: - Attachment: hive-1792-2.patch > track the joins which are being converted to map-join automatica

[jira] Assigned: (HIVE-1808) but in auto_join25.q

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang reassigned HIVE-1808: Assignee: Liyin Tang > but in auto_join25.q > > >

[jira] Updated: (HIVE-1808) bug in auto_join25.q

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1808: - Summary: bug in auto_join25.q (was: but in auto_join25.q) > bug in auto_join2

[jira] Updated: (HIVE-1808) but in auto_join25.q

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1808: - Attachment: hive-1808-1.patch The bug fixed in this patch > but in auto_join2

[jira] Updated: (HIVE-1792) track the joins which are being converted to map-join automatically

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1792: - Attachment: (was: hive-1792-2.patch) > track the joins which are being converted to map-j

[jira] Updated: (HIVE-1792) track the joins which are being converted to map-join automatically

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1792: - Attachment: hive-1792-2.patch The previous patch includes the fix of another jira(Hive-1808) Now I

[jira] Created: (HIVE-1808) but in auto_join25.q

2010-11-23 Thread Liyin Tang (JIRA)
but in auto_join25.q Key: HIVE-1808 URL: https://issues.apache.org/jira/browse/HIVE-1808 Project: Hive Issue Type: Bug Reporter: Liyin Tang In this test case, there are 2 SET statements: set

[jira] Updated: (HIVE-1792) track the joins which are being converted to map-join automatically

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1792: - Attachment: hive-1792-1.patch Add a new hook: MapJoinCounterHook, which will measure how many joins

[jira] Updated: (HIVE-1804) Mapjoin will fail if there are no files associating with the join tables

2010-11-23 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1804: - Attachment: hive-1804-2.patch Remove all the debug print statements. Please review > Mapjoin will f

[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

2010-11-22 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1785: - Attachment: hive-1785_6.patch Thanks for the careful review and sorry to submit the wrong patch before

[jira] Updated: (HIVE-1804) Mapjoin will fail if there are no files associating with the join tables

2010-11-22 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1804: - Attachment: hive-1804-1.patch If the partition desc is empty, then create an empty hashtable file for it

[jira] Updated: (HIVE-1804) Mapjoin will fail if there are no files associating with the join tables

2010-11-22 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1804: - Status: Patch Available (was: Open) If the parition desc is empty, just create a empty hash table file

[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

2010-11-22 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1785: - Status: Patch Available (was: Open) > change Pre/Post Query Hooks to take in 1 parameter: HookCont

[jira] Updated: (HIVE-1797) Compressed the hashtable dump file before put into distributed cache

2010-11-22 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1797: - Status: Patch Available (was: Open) > Compressed the hashtable dump file before put into distribu

[jira] Created: (HIVE-1804) Mapjoin will fail if there are no files associating with the join tables

2010-11-22 Thread Liyin Tang (JIRA)
: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Liyin Tang Assignee: Liyin Tang Fix For: 0.7.0 If there are some empty tables without any file associated, the map join will fail. -- This message is automatically generated by

[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

2010-11-22 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1785: - Attachment: hive_1758_5.patch 1) make the old interface be deprecated 2) let the existing Prehook and

[jira] Updated: (HIVE-1797) Compressed the hashtable dump file before put into distributed cache

2010-11-20 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1797: - Attachment: hive-1797_3.patch In this patch, all the hashtable dumped files will be compressed and

[jira] Updated: (HIVE-1797) Compressed the hashtable dump file before put into distributed cache

2010-11-20 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1797: - Attachment: hive-1797_2.patch In this patch, all the hashtable dumped files will be compressed and

[jira] Updated: (HIVE-1797) Compressed the hashtable dump file before put into distributed cache

2010-11-20 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1797: - Attachment: (was: hive-1797_2.patch) > Compressed the hashtable dump file before put into distribu

[jira] Created: (HIVE-1798) Clear empty files in Hive

2010-11-17 Thread Liyin Tang (JIRA)
Clear empty files in Hive -- Key: HIVE-1798 URL: https://issues.apache.org/jira/browse/HIVE-1798 Project: Hive Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang There are 4

[jira] Updated: (HIVE-1797) Compressed the hashtable dump file before put into distributed cache

2010-11-17 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1797: - Attachment: hive-1797.patch Compress the hashtable dumped file by gzip before adding to distributed cache

[jira] Created: (HIVE-1797) Compressed the hashtable dump file before put into distributed cache

2010-11-17 Thread Liyin Tang (JIRA)
: Improvement Components: Query Processor Affects Versions: 0.7.0 Reporter: Liyin Tang Assignee: Liyin Tang Clearly, the size of small table is the performance bottleneck for map join. Because the size of the small table will affect the memory usage and dumped

[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

2010-11-17 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1785: - Attachment: hive-1785_4.patch In this patch, I add the Hook interface over Pre/PostExecute and

[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

2010-11-17 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933096#action_12933096 ] Liyin Tang commented on HIVE-1785: -- How about adding one more layer over Pre/PostExe

[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

2010-11-17 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1785: - Attachment: hive-1785_3.patch In order to be compatible, we check whether the hook implements the

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive-1642_11.patch When the local task runs out of memory, do NOT print any thing out and

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive-1642_10.patch After discussing, we think the function: replaceWithConditionalTask is not

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive-1642_9.patch some minor changes in ConditionalResolverCommonJoin.java > Convert j

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive-1642_7.patch In Task.java public void replaceWithConditionalTask(ConditionalTask cndTsk

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive-1642_6.patch Remove the getBackupTask interface from all the Conditional Resolver

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: (was: hive-1642_5.patch) > Convert join queries to map-join based on size of table/

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive-1642_5.patch Add more detailed description on configuration xml file Revert the

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive-1642_5.patch Add more descriptions to the configuration files. Revert the DriverContext

[jira] Commented: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932623#action_12932623 ] Liyin Tang commented on HIVE-1785: -- I generate the diff based on the Hive-1642. Pl

[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1785: - Attachment: hive_1785_2.patch Thanks for John's comments. Now in the Driver.java: for (PostExecut

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive_1642_4.patch This patch formats the output of local task. > Convert join queries to

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-16 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive_1642_2.patch Thanks for the comments. I have updated the patch according to the review

[jira] Commented: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-15 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932132#action_12932132 ] Liyin Tang commented on HIVE-1642: -- There are 2 kinds of backup. 1) task level 2) br

[jira] Commented: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-14 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931995#action_12931995 ] Liyin Tang commented on HIVE-1642: -- Thanks for reviewing. 1. I will add these parame

[jira] Created: (HIVE-1792) track the joins which are being converted to map-join automatically

2010-11-14 Thread Liyin Tang (JIRA)
Feature Components: Query Processor Affects Versions: 0.7.0 Reporter: Liyin Tang Assignee: Liyin Tang Fix For: 0.7.0 We should be able to track how many queries (join) got converted to map-join -- This message is automatically generated by JIRA. - You

[jira] Updated: (HIVE-1785) change Pre/Post Query Hooks to take in 1 parameter: HookContext

2010-11-14 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1785: - Attachment: hive_1785_1.patch In this patch I have changed the interface of pre hoook and post hook. So

[jira] Resolved: (HIVE-1688) In the MapJoinOperator, the code uses tag as alias, which is not always true

2010-11-13 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang resolved HIVE-1688. -- Resolution: Fixed Fix Version/s: 0.7.0 > In the MapJoinOperator, the code uses tag as al

[jira] Commented: (HIVE-1688) In the MapJoinOperator, the code uses tag as alias, which is not always true

2010-11-13 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931710#action_12931710 ] Liyin Tang commented on HIVE-1688: -- This bug has been fixed in Hive-1641 earlier and

[jira] Commented: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-13 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931705#action_12931705 ] Liyin Tang commented on HIVE-1642: -- In the case: A left outer join B right outer join

[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-12 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive_1642_1.patch > Convert join queries to map-join based on size of table/

[jira] Commented: (HIVE-1642) Convert join queries to map-join based on size of table/row

2010-11-12 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931599#action_12931599 ] Liyin Tang commented on HIVE-1642: -- I just finished converting common join into map

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

2010-11-11 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Attachment: hive-1754_9.patch > Remove JDBM component from Map J

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

2010-11-10 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Attachment: hive-1754_7.patch Change the code style according to the reviewer comments > Remove J

[jira] Resolved: (HIVE-1647) Incorrect initialization of thread local variable inside IOContext ( implementation is not threadsafe )

2010-11-09 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang resolved HIVE-1647. -- Resolution: Fixed Release Note: This problem is fixed in Hive-1754 > Incorrect initialization

[jira] Resolved: (HIVE-1775) Assertation on inputObjInspectors.length in Groupy operator

2010-11-09 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang resolved HIVE-1775. -- Resolution: Fixed Release Note: This bug is fixed in Hive-1754 > Assertation

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

2010-11-09 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Attachment: hive-1754_5.patch This patch clears the conflicts of test output file completely. > Rem

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

2010-11-09 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Attachment: hive-1754_4.patch Resolved all the output conflicts in this patch > Remove JDBM compon

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

2010-11-08 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Attachment: Hive-1754_3.patch Fix a bug when join value is null Also fix the hive-1775 > Remove J

[jira] Commented: (HIVE-1775) Assertation on inputObjInspectors.length in Groupy operator

2010-11-08 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929869#action_12929869 ] Liyin Tang commented on HIVE-1775: -- resolved in patch hive-1754 > Asserta

[jira] Commented: (HIVE-1775) Assertation on inputObjInspectors.length in Groupy operator

2010-11-07 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929480#action_12929480 ] Liyin Tang commented on HIVE-1775: -- Yes. I can just comment out this asser

[jira] Created: (HIVE-1775) Assertation on inputObjInspectors.length in Groupy operator

2010-11-07 Thread Liyin Tang (JIRA)
: Query Processor Affects Versions: 0.6.0, 0.7.0 Reporter: Liyin Tang Assignee: Liyin Tang Fix For: 0.7.0 In the Groupby Operator: Line 188: assert (inputObjInspectors.length == 1); But this assertion may not necessary true -- This message is automatically

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

2010-11-04 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Attachment: Hive-1754_2.patch Remove JDBM from Hive completely > Remove JDBM component from Map J

[jira] Commented: (HIVE-1754) Remove JDBM component from Map Join

2010-11-04 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12928262#action_12928262 ] Liyin Tang commented on HIVE-1754: -- This patch has some potential bugs. I will fi

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

2010-11-01 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Attachment: Hive-1754.patch > Remove JDBM component from Map J

[jira] Updated: (HIVE-1754) Remove JDBM component from Map Join

2010-11-01 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1754: - Status: Patch Available (was: Open) This patch modifies the following things 1) Remove the JDBM from

[jira] Resolved: (HIVE-1702) optimize JDBM to make mapjoin faster

2010-11-01 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang resolved HIVE-1702. -- Resolution: Won't Fix Release Note: JDBM will be removed from Hive > optimize JDBM

[jira] Resolved: (HIVE-1733) Make the bucket size of JDBM configurable

2010-11-01 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang resolved HIVE-1733. -- Resolution: Not A Problem Release Note: The JDBM will be removed from Hive > Make the bucket s

[jira] Updated: (HIVE-1756) failures in fatal.q in TestNegativeCliDriver

2010-10-28 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1756: - Attachment: Hive-1756.patch remove fatal.q > failures in fatal.q in TestNegativeCliDri

[jira] Updated: (HIVE-1757) test cleanup for Hive-1641

2010-10-27 Thread Liyin Tang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1757: - Attachment: Hive-1757.patch Remove some unnecessary print out statements > test cleanup for Hive-1

  1   2   >