[
https://issues.apache.org/jira/browse/HIVE-21469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799134#comment-16799134
]
Hive QA commented on HIVE-21469:
--------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
26s{color} | {color:blue} ql in master has 2255 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 1 new + 49 unchanged - 34
fixed = 50 total (was 83) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
43s{color} | {color:red} ql generated 1 new + 2252 unchanged - 3 fixed = 2253
total (was 2255) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Write to static field
org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManager.metrics
from instance method
org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManager.setContext(HiveLockManagerCtx)
At ZooKeeperHiveLockManager.java:from instance method
org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManager.setContext(HiveLockManagerCtx)
At ZooKeeperHiveLockManager.java:[line 124] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16635/dev-support/hive-personality.sh
|
| git revision | master / 2fa22bf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16635/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16635/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16635/yetus.txt |
| Powered by | Apache Yetus http://yetus.apache.org |
This message was automatically generated.
> Review of ZooKeeperHiveLockManager
> ----------------------------------
>
> Key: HIVE-21469
> URL: https://issues.apache.org/jira/browse/HIVE-21469
> Project: Hive
> Issue Type: Improvement
> Components: Locking
> Affects Versions: 4.0.0, 3.2.0
> Reporter: David Mollitor
> Assignee: David Mollitor
> Priority: Major
> Attachments: HIVE-21469.1.patch, HIVE-21469.2.patch,
> HIVE-21469.3.patch, HIVE-21469.4.patch, HIVE-21469.5.patch
>
>
> A lot of sins in this class to resolve:
> {code:java}
> @Override
> public void setContext(HiveLockManagerCtx ctx) throws LockException {
> try {
> curatorFramework = CuratorFrameworkSingleton.getInstance(conf);
> parent = conf.getVar(HiveConf.ConfVars.HIVE_ZOOKEEPER_NAMESPACE);
> try{
> curatorFramework.create().withMode(CreateMode.PERSISTENT).forPath("/"
> + parent, new byte[0]);
> } catch (Exception e) {
> // ignore if the parent already exists
> if (!(e instanceof KeeperException) || ((KeeperException)e).code() !=
> KeeperException.Code.NODEEXISTS) {
> LOG.warn("Unexpected ZK exception when creating parent node /" +
> parent, e);
> }
> }
> {code}
> Every time a new session is created and this {{setContext}} method is called,
> it attempts to create the root node. I have seen that, even though the root
> node exists, an create node action is written to the ZK logs. Check first if
> the node exists before trying to create it.
> {code:java}
> try {
> curatorFramework.delete().forPath(zLock.getPath());
> } catch (InterruptedException ie) {
> curatorFramework.delete().forPath(zLock.getPath());
> }
> {code}
> There has historically been a quite a few bugs regarding leaked locks. The
> Driver will signal the session {{Thread}} by performing an interrupt. That
> interrupt can happen any time and it can kill a create/delete action within
> the ZK framework. We can see one example of workaround for this. If the ZK
> action is interrupted, simply do it again. Well, what if it's interrupted
> yet again? The lock will be leaked. Also, when the {{InterruptedException}}
> is caught in the try block, the thread's interrupted flag is cleared. The
> flag is not reset in this code and therefore we lose the fact that this
> thread has been interrupted. This flag should be preserved so that other
> code paths will know that it's time to exit.
> {code:java}
> if (tryNum > 1) {
> Thread.sleep(sleepTime);
> }
> unlockPrimitive(hiveLock, parent, curatorFramework);
> break;
> } catch (Exception e) {
> if (tryNum >= numRetriesForUnLock) {
> String name = ((ZooKeeperHiveLock)hiveLock).getPath();
> throw new LockException("Node " + name + " can not be deleted after
> " + numRetriesForUnLock + " attempts.",
> e);
> }
> }
> {code}
> ... related... the sleep here may be interrupted, but we still need to delete
> the lock (again, for fear of leaking it). This sleep should be
> uninterruptible. If we need to get the lock deleted, and there's a problem,
> interrupting the sleep will cause the code to eventually exit and locks will
> be leaked.
> It also requires a bunch more TLC.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
