[
https://issues.apache.org/jira/browse/CASSANDRA-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reassigned CASSANDRA-1674:
-----------------------------------------
Assignee: Stu Hood
Here is one way to reproduce something similar, at least.
I start with 1 node, and put 1M rows on it. Then I add a 2nd node, then a
third, then run cleanup. So I have 25% 50% 25% of the 1M rows on those 3
machines:
{code}
$ nodetool -h localhost -p 8080 ring
Address Status State Load Token
164074424718159380631425626216484638578
127.0.0.2 Up Normal 95.39 MB
36509681143663337709904175976843745575
127.0.0.1 Up Normal 190.72 MB
121466565577718822332569289829746132598
127.0.0.3 Up Normal 95.4 MB
164074424718159380631425626216484638578
{code}
I increase the RF to 2, and run repair against .2:
{code}
Address Status State Load Token
164074424718159380631425626216484638578
127.0.0.2 Up Normal 381.41 MB
36509681143663337709904175976843745575
127.0.0.1 Up Normal 381.41 MB
121466565577718822332569289829746132598
127.0.0.3 Up Normal 190.75 MB
164074424718159380631425626216484638578
{code}
Let's call the ranges of data that .2, .1, and .3 originally had R, G, and B.
Post repair, .2 should have RB but it has RGB. Node 1 should have GB but it
also has RGB.
Cleanup puts things in the expected state:
{code}
Address Status State Load Token
164074424718159380631425626216484638578
127.0.0.2 Up Normal 190.74 MB
36509681143663337709904175976843745575
127.0.0.1 Up Normal 285.61 MB
121466565577718822332569289829746132598
127.0.0.3 Up Normal 190.75 MB
164074424718159380631425626216484638578
{code}
This is reproducible against 0.6 as well as 0.7.
> Repair using abnormally large amounts of disk space, dropping messages
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-1674
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1674
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Fix For: 0.7.1
>
>
> I'm watching a repair on a 7 node cluster. Repair was sent to one node; the
> node had 18G of data. No other node has more than 28G. The node where the
> repair initiated is now up to 261G with 53/60 AES tasks outstanding.
> I have seen repair take more space than expected on 0.6 but nothing this
> extreme.
> Other nodes in the cluster are occasionally logging
> WARN [ScheduledTasks:1] 2010-10-28 08:31:14,305 MessagingService.java (line
> 515) Dropped 7 messages in the last 1000ms
> The cluster is quiesced except for the repair. Not sure if the dropped
> messages are contributing the the disk space (b/c of retries?).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
