[
https://issues.apache.org/jira/browse/HBASE-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123927#comment-16123927
]
Hudson commented on HBASE-18551:
--------------------------------
FAILURE: Integrated in Jenkins build HBase-2.0 #311 (See
[https://builds.apache.org/job/HBase-2.0/311/])
HBASE-18551 [AMv2] UnassignProcedure and crashed regionservers (stack: rev
5940f4224c0ce0c01e98cdb28f74c6e227c918e3)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashException.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableNamespaceManager.java
* (edit)
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
> [AMv2] UnassignProcedure and crashed regionservers
> --------------------------------------------------
>
> Key: HBASE-18551
> URL: https://issues.apache.org/jira/browse/HBASE-18551
> Project: HBase
> Issue Type: Bug
> Components: amv2
> Reporter: stack
> Assignee: stack
> Fix For: 2.0.0
>
> Attachments: HBASE-18551.master.001.patch,
> HBASE-18551.master.002.patch, HBASE-18551.master.003.patch
>
>
> This has been [~uagashe] and my obsession over the last few days, what should
> an UnassignProcedure do when it dispatches a CLOSE but the CLOSE fails
> because of ConnectException or SocketTimeout.
> + We used to let UnassignProcedure continue presuming the Region would be
> closed since the server is dead. BUT, if the unassign was part of a
> MoveProcedure, the unassign would proceed and the Move would then run WITHOUT
> first splitting logs. Bad.
> + So, we made it so UnassignProcedure failed; let the upper layers take care
> of the failure. See HBASE-18491 that enabled this behavior. BUT, we are since
> figuring that even if the UP completes as a failure, since it gives up the
> Region lock on completion, another procedure -- say an AssignProcedure --
> could cut in before the ServerCrashProcedure had finished and again there
> could be dataloss.
> + Now we are thinking the UP should hold on to the Region lock until we are
> signalled by a ServerCrashProcedure; only then let go of the region. The UP
> has context that is hard to pass another. Waiting on a SCP has the UP living
> on for what could be a good amount of time. It might be ok if we can suspend
> the procedure.
> There is a good sample scenario that came up doing the no-regions-on-master
> issue, HBASE-18511. When meta is not on master, TestSplitTransactionOnCluster
> is failing. It fails because though the test completes, the tests commonly
> kill a RegionServer. The teardown for the test runs before we've noticed the
> aborted RS. So, the disable of the table in the teardown prepartory to our
> deleting the test table as part of clean up, goes to unassign regions but the
> unassign fails against the aborted server.
> Good stuff.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
