[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248213#comment-17248213
]
Caleb Rackliffe commented on CASSANDRA-16226:
---------------------------------------------
Thanks [~ifesdjeen].
bq. add tests for tombstones, since previously we weren't taking them into
consideration
Indeed, although there is some coverage of this logic across
{{SSTablesIteratedTest}}, {{DeleteTest}}, and {{UpgradeTest}}. It might be
useful to have a test or two around the number of SSTables hit with range
tombstones in {{SSTablesIteratedTest}}.
bq. should be "more recent than"
Fixed.
bq. should we consider renaming canRemoveRow?
I settled on {{isRowComplete()}}, which at least describes what the method
tells us, and _not_ what it tells use we can do as a result.
New commit with the changes above is
[here|https://github.com/apache/cassandra/pull/823/commits/b71d830ebbe2e5726de2c18b03179ca2b8a74023].
...and just to have everything in one place for [~mck]: [3.0
patch|https://github.com/apache/cassandra/pull/823],
[CircleCI|https://app.circleci.com/pipelines/github/maedhroz/cassandra/171/workflows/4781de11-c5e2-40b1-967c-0e96728f843b]
I'll post the 3.11 and 4.0 branches, which I expect not to deviate much (if at
all) from the 3.0 diff, once we've got a second +1.
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Reporter: Caleb Rackliffe
> Assignee: Caleb Rackliffe
> Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key
> liveness information for non-compact/CQL tables. Tombstones seem to be
> handled at a level above that anyway. (One potential problem with that is
> whether or not the distinction will continue to exist in 4.0, and dropping
> compact storage from a table doesn't magically make pk liveness information
> appear.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
