[
https://issues.apache.org/jira/browse/CASSANDRA-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-13762:
------------------------------------
Status: Patch Available (was: Open)
The problem in this ticket description is reproduced by this
[dtest|https://github.com/apache/cassandra-dtest/compare/master...pauloricardomg:13762#diff-62ba429edee6a4681782f078246c9893R1185].
The fix consists in applying the optimization from CASSANDRA-13065 only if the
source node has built all the views from the streamed base table. Otherwise,
the mutations are sent through the write path as it was done before.
In order to check if a source node has built all views we need to query the
{{system_distrbuted.view_build_status}} table. Querying this table on a
bootstrapping node was [previously
throwing|https://github.com/pauloricardomg/cassandra/blob/c47cfb9f3299274f97e47610eb6329390dcacdc0/src/java/org/apache/cassandra/service/StorageProxy.java#L1619]
{{IsBootstrappingException}}, but I didn't find any good reason for this
limitation, so I lifted it
[here|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-13762#diff-71f06c193f5b5e270cf8ac695164f43aR1580].
One problem that arises when sending mutations through the write path during
bootstrap is that the views are not propagated to replicas, but [written to the
batchlog|https://github.com/apache/cassandra/blob/e18a49a2399a3fe667c3c08d7350b7528614f0a6/src/java/org/apache/cassandra/service/StorageProxy.java#L831]
so the views may not be fully available when the node finishes bootstrapping,
even though they are [marked as
built|https://github.com/apache/cassandra/blob/e18a49a2399a3fe667c3c08d7350b7528614f0a6/src/java/org/apache/cassandra/service/StorageService.java#L1537].
This is the same problem described on CASSANDRA-13629.
The fix to the problem above is to [wait for batchlog
replay|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-13762#diff-b76a607445d53f18a98c9df14323c7ddR1543]
before finishing bootstrap if views are present (we should probably reopen
CASSANDRA-13629 and backport this fix btw). One downside of this approach is
that we need to [wait for the batchlog
timeout|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-13762#diff-b76a607445d53f18a98c9df14323c7ddR1541]
to ensure all batchlogs are replayed, what should increase the bootstrap in
tens of seconds, but should be acceptable. If necessary we can try to raise
this limitation in another ticket.
I also added a dtest to check that view rebuild is skipped during streaming if
the source node has built the views
([here|https://github.com/apache/cassandra-dtest/compare/master...pauloricardomg:13762#diff-62ba429edee6a4681782f078246c9893R1189]).
Patch and tests below (CI looks good):
||trunk||dtest||
|[branch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-13762]|[branch|https://github.com/apache/cassandra-dtest/compare/master...pauloricardomg:13762]|
|[testall|https://issues.apache.org/jira/secure/attachment/12904779/trunk-13762-testall.png]|
|[dtest|https://issues.apache.org/jira/secure/attachment/12904778/trunk-13762-dtest.png]|
> Ensure views created during (or just before) range movements are properly
> built
> -------------------------------------------------------------------------------
>
> Key: CASSANDRA-13762
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13762
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Reporter: Paulo Motta
> Assignee: Paulo Motta
> Priority: Minor
> Labels: materializedviews
> Attachments: trunk-13762-dtest.png, trunk-13762-testall.png
>
>
> CASSANDRA-13065 assumes the source node has its views built to skip running
> base mutations through the write path during range movements.
> It is possible that the source node has not finished building the view, or
> that a new view is created during a range movement, in which case the view
> may be wrongly marked as built on the destination node.
> The former problem was introduced by #13065, but even before that a view
> created during a range movement may not be correctly built on the destination
> node because the view builder will be triggered before it has finished
> streaming the source data, wrongly marking the view as built on that node.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
