Balaji, Are you repairing a specific keyspace/table? if the failure is tied to a table, try 'verify' and 'scrub' options on .91...see if you get any errors.
On Thursday, June 29, 2017, 12:12:14 PM PDT, Balaji Venkatesan <venkatesan.bal...@gmail.com> wrote: Thanks. I tried with trace option and there is not much info. Here are the few log lines just before it failed. [2017-06-29 19:01:54,969] /xx.xx.xx.93: Sending REPAIR_MESSAGE message to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to /xx.xx.xx.91[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to /xx.xx.xx.91[2017-06-29 19:02:04,842] Some repair failed[2017-06-29 19:02:04,848] Repair command #1 finished in 1 minute 2 secondserror: Repair job has failed with the error message: [2017-06-29 19:02:04,842] Some repair failed-- StackTrace --java.lang.RuntimeException: Repair job has failed with the error message: [2017-06-29 19:02:04,842] Some repair failed at org.apache.cassandra.tools. RepairRunner.progress( RepairRunner.java:116) at org.apache.cassandra.utils. progress.jmx. JMXNotificationProgressListene r.handleNotification( JMXNotificationProgressListene r.java:77) at com.sun.jmx.remote.internal. ClientNotifForwarder$ NotifFetcher. dispatchNotification( ClientNotifForwarder.java:583) at com.sun.jmx.remote.internal. ClientNotifForwarder$ NotifFetcher.doRun( ClientNotifForwarder.java:533) at com.sun.jmx.remote.internal. ClientNotifForwarder$ NotifFetcher.run( ClientNotifForwarder.java:452) at com.sun.jmx.remote.internal. ClientNotifForwarder$ LinearExecutor$1.run( ClientNotifForwarder.java:108) FYI I am running repair from xx.xx.xx.91 node and its a 5 node cluster xx.xx.xx.91-xx.xx.xx.95 On Wed, Jun 28, 2017 at 5:16 PM, Akhil Mehra <akhilme...@gmail.com> wrote: nodetool repair has a trace option nodetool repair -tr yourkeyspacename see if that provides you with additional information. Regards,Akhil On 28/06/2017, at 2:25 AM, Balaji Venkatesan <venkatesan.bal...@gmail.com> wrote: We use Apache Cassandra 3.10-13 On Jun 26, 2017 8:41 PM, "Michael Shuler" <mich...@pbandjelly.org> wrote: What version of Cassandra? -- Michael On 06/26/2017 09:53 PM, Balaji Venkatesan wrote: > Hi All, > > When I run nodetool repair on a keyspace I constantly get "Some repair > failed" error, there are no sufficient info to debug more. Any help? > > Here is the stacktrace > > ============================== ============================== ========== > [2017-06-27 02:44:34,275] Some repair failed > [2017-06-27 02:44:34,279] Repair command #3 finished in 33 seconds > error: Repair job has failed with the error message: [2017-06-27 > 02:44:34,275] Some repair failed > -- StackTrace -- > java.lang.RuntimeException: Repair job has failed with the error > message: [2017-06-27 02:44:34,275] Some repair failed > at org.apache.cassandra.tools.Rep airRunner.progress(RepairRunne r.java:116) > at > org.apache.cassandra.utils.pro gress.jmx.JMXNotificationProgr essListener. > handleNotification(JMXNotifica tionProgressListener.java:77) > at > com.sun.jmx.remote.internal.Cl ientNotifForwarder$NotifFetche > r.dispatchNotification(ClientN otifForwarder.java:583) > at > com.sun.jmx.remote.internal.Cl ientNotifForwarder$NotifFetche > r.doRun(ClientNotifForwarder. java:533) > at > com.sun.jmx.remote.internal.Cl ientNotifForwarder$NotifFetche > r.run(ClientNotifForwarder. java:452) > at > com.sun.jmx.remote.internal.Cl ientNotifForwarder$LinearExecu > tor$1.run(ClientNotifForwarder .java:108) > ============================== ============================== ========== > > > -- > Thanks, > Balaji Venkatesan. ------------------------------ ------------------------------ --------- To unsubscribe, e-mail: user-unsubscr...@cassandra.apa che.org For additional commands, e-mail: user-h...@cassandra.apache.org -- Thanks, Balaji Venkatesan.
