[
https://issues.apache.org/jira/browse/CASSANDRA-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16044472#comment-16044472
]
Andrés de la Peña commented on CASSANDRA-10130:
-----------------------------------------------
Here is another version of the patch:
||[trunk|https://github.com/apache/cassandra/compare/trunk...adelapena:10130-trunk]|[utests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-10130-trunk-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-10130-trunk-dtest/]|
The key changes are forbidding concurrent full (re)builds of the same index
(there is no reason to want them) and making the initial mark operation of a
full index rebuild wait until there are no running partial builds:
* Partial builds are always fired during the addition of a new SSTables,
notified by {{SSTableAddedNotification}}. These use the regular counters system
and only wait if there are ongoing calls to {{markReindexingAllSSTables}}.
These calls are done by full index rebuilds and should be a very fast operation
to don't block repairs for long.
* Full builds during index creation acquire a lock to avoid full index rebuilds
of the index and call to {{markIndexingAllSSTables}}, without checking if there
are running partial builds. Since this is done before registering the index,
this is the very first and unique build for the index being created. Once the
call to {{markIndexingAllSSTables}} has finished we can register the index and
throw the build task. Once the task has finished we release the lock to avoid
full index rebuilds of the index.
* Manual full index rebuilds fail if there is any another running (re)build of
the index or if it's being created. In the case of finding a full build the
fail is instantaneously, whereas there is a small wait period if there are
running partial rebuilds. This shouldn't be a big problem given that they are
manually thrown and they can be retried if they are rejected. If they are not
rejected, the initial call to {{markReindexingAllSSTables}} will set the
involved partial builds counters to zero.
This way, manual rebuilds will be able to reset the
pending-for-rebuild-during-restart of the indexes. at the price of requiring
mutual exclusion with other rebuilds of the same index. Partial rebuilds can
always run but they can be held during the presumably fast call to
{{markReindexingAllSSTables}} of the manual full index rebuilds.
bq. Other than that, I have a concern about [flushing indexes in the future
transformation|https://github.com/apache/cassandra/compare/trunk...adelapena:10130-trunk#diff-3f2c8994c4ff8748c3faf7e70958520dR399],
which would cause such blocking activity to happen in the compaction thread, a
departure from previous behaviour and probably an unwanted one.
Right, I have modified the transformation to be run [in the
asyncExecutor|https://github.com/adelapena/cassandra/blob/ebfcb302bcc8301928c329a377f0d694976eb477/src/java/org/apache/cassandra/index/SecondaryIndexManager.java#L500].
The reason for flushing indexes in each future is that we should flush before
marking the indexes as built, specially for non {{ColumnFamilyStore}}-based
indexes, which is not in line with the previous behaviour.
What do you think? Am I missing some case? Is the [timeout of index rebuilds
waiting for partial
rebuilds|https://github.com/adelapena/cassandra/blob/ebfcb302bcc8301928c329a377f0d694976eb477/src/java/org/apache/cassandra/index/SecondaryIndexManager.java#L118]
adequate or should we use another value or fail instantly?
> Node failure during 2i update after streaming can have incomplete 2i when
> restarted
> -----------------------------------------------------------------------------------
>
> Key: CASSANDRA-10130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10130
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
> Reporter: Yuki Morishita
> Assignee: Andrés de la Peña
> Priority: Minor
>
> Since MV/2i update happens after SSTables are received, node failure during
> MV/2i update can leave received SSTables live when restarted while MV/2i are
> partially up to date.
> We can add some kind of tracking mechanism to automatically rebuild at the
> startup, or at least warn user when the node restarts.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
