Thank you so much Kurt - that helped!! So here's the thing, I was logged into the server as my own id and had Cassandra binaries under my home directory with the default configuration. Cassandra was running on the server with a service account and with its own user-id and configuration.
I assumed that like nodetool, this command would also use the Cassandra service JMX or server port. However your suggestion made me realize that that's not the case. So I copied the service account's Cassandra/conf to my local install and things worked. I also looked up the source code and it did not indicate any "connection" via service or jmx port :) So now need to see why the data is not getting purged. Thanks again!! From: kurt greaves <k...@instaclustr.com> Date: Wednesday, January 31, 2018 at 11:27 PM To: User <user@cassandra.apache.org> Subject: Re: TWCS not deleting expired sstables Well, that shouldn't happen. Seems like it's possibly not looking in the correct location for data directories. Try setting CASSANDRA_INCLUDE=<path to cassandra.in.sh<http://cassandra.in.sh>> prior to running the script? e.g: CASSANDRA_INCLUDE=<path_to_cassandra_bin>/cassandra.in.sh<http://cassandra.in.sh> sstableexpiredblockers ae raw_logs_by_user On 30 January 2018 at 15:34, Thakrar, Jayesh <jthak...@conversantmedia.com<mailto:jthak...@conversantmedia.com>> wrote: Thanks Kurt and Kenneth. Now only if they would work as expected. node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>ls -lt | tail -rw-r--r--. 1 vchadoop vchadoop 286889260 Sep 18 14:14 mc-1070-big-Index.db -rw-r--r--. 1 vchadoop vchadoop 12236 Sep 13 20:53 mc-178-big-Statistics.db -rw-r--r--. 1 vchadoop vchadoop 92 Sep 13 20:53 mc-178-big-TOC.txt -rw-r--r--. 1 vchadoop vchadoop 9371211 Sep 13 20:53 mc-178-big-CompressionInfo.db -rw-r--r--. 1 vchadoop vchadoop 10 Sep 13 20:53 mc-178-big-Digest.crc32 -rw-r--r--. 1 vchadoop vchadoop 13609890747<tel:(360)%20989-0747> Sep 13 20:53 mc-178-big-Data.db -rw-r--r--. 1 vchadoop vchadoop 1394968 Sep 13 20:53 mc-178-big-Summary.db -rw-r--r--. 1 vchadoop vchadoop 11172592 Sep 13 20:53 mc-178-big-Filter.db -rw-r--r--. 1 vchadoop vchadoop 190508739 Sep 13 20:53 mc-178-big-Index.db drwxr-xr-x. 2 vchadoop vchadoop 10 Sep 12 21:47 backups node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>sstableexpiredblockers ae raw_logs_by_user Exception in thread "main" java.lang.IllegalArgumentException: Unknown keyspace/table ae.raw_logs_by_user at org.apache.cassandra.tools.SSTableExpiredBlockers.main(SSTableExpiredBlockers.java:66) node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>sstableexpiredblockers system peers No sstables for system.peers node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>ls -l ../../system/peers-37f71aca7dc2383ba70672528af04d4f/ total 308 drwxr-xr-x. 2 vchadoop vchadoop 10 Sep 11 22:59 backups -rw-rw-r--. 1 vchadoop vchadoop 83 Jan 25 02:11 mc-137-big-CompressionInfo.db -rw-rw-r--. 1 vchadoop vchadoop 180369 Jan 25 02:11 mc-137-big-Data.db -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 25 02:11 mc-137-big-Digest.crc32 -rw-rw-r--. 1 vchadoop vchadoop 64 Jan 25 02:11 mc-137-big-Filter.db -rw-rw-r--. 1 vchadoop vchadoop 386 Jan 25 02:11 mc-137-big-Index.db -rw-rw-r--. 1 vchadoop vchadoop 5171 Jan 25 02:11 mc-137-big-Statistics.db -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 25 02:11 mc-137-big-Summary.db -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 25 02:11 mc-137-big-TOC.txt -rw-rw-r--. 1 vchadoop vchadoop 43 Jan 29 21:11 mc-138-big-CompressionInfo.db -rw-rw-r--. 1 vchadoop vchadoop 9723 Jan 29 21:11 mc-138-big-Data.db -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 29 21:11 mc-138-big-Digest.crc32 -rw-rw-r--. 1 vchadoop vchadoop 16 Jan 29 21:11 mc-138-big-Filter.db -rw-rw-r--. 1 vchadoop vchadoop 17 Jan 29 21:11 mc-138-big-Index.db -rw-rw-r--. 1 vchadoop vchadoop 5015 Jan 29 21:11 mc-138-big-Statistics.db -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 29 21:11 mc-138-big-Summary.db -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 29 21:11 mc-138-big-TOC.txt -rw-rw-r--. 1 vchadoop vchadoop 43 Jan 29 21:53 mc-139-big-CompressionInfo.db -rw-rw-r--. 1 vchadoop vchadoop 18908 Jan 29 21:53 mc-139-big-Data.db -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 29 21:53 mc-139-big-Digest.crc32 -rw-rw-r--. 1 vchadoop vchadoop 16 Jan 29 21:53 mc-139-big-Filter.db -rw-rw-r--. 1 vchadoop vchadoop 36 Jan 29 21:53 mc-139-big-Index.db -rw-rw-r--. 1 vchadoop vchadoop 5055 Jan 29 21:53 mc-139-big-Statistics.db -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 29 21:53 mc-139-big-Summary.db -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 29 21:53 mc-139-big-TOC.txt From: Kenneth Brotman <kenbrot...@yahoo.com.INVALID> Date: Tuesday, January 30, 2018 at 7:37 AM To: <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: RE: TWCS not deleting expired sstables Wow! It’s in the DataStax documentation: https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/tools/toolsSStables/toolsSStabExpiredBlockers.html Other nice tools there as well: https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/tools/toolsSStables/toolsSSTableUtilitiesTOC.html Kenneth Brotman From: kurt greaves [mailto:k...@instaclustr.com<mailto:k...@instaclustr.com>] Sent: Monday, January 29, 2018 8:20 PM To: User Subject: Re: TWCS not deleting expired sstables Likely a read repair caused old data to be brought into a newer SSTable. Try running sstableexpiredblockers to find out if there's a newer SSTable blocking that one from being dropped.
