Reading Data from C* Cluster
HI I want to read data from C* Cluster. Schema is CREATE TABLE code( campaign_id_sub_partition text code text, PRIMARY KEY ((campaign_id_sub_partition),code)) ); Where I can have configurable codes in partition key as cassandra supports. campaign_id_sub_partition is appx 10 characters code field is appx 12~20 characters Query : 1) How much data I can put in one partition ? 2) I want to read data from cassandra as SELECT * from code where campaign_id_sub_partition = 'XXX-1'; Is reading whole partition at one time is feasible or shall I use pagination ? -- Raman Gugnani 8588892293
Best compaction strategy
Hi All, I have one table in which i have some data which has TTL of 2days and some data which has TTL of 60 days. What compaction strategy will suits the most. 1. LeveledCompactionStrategy (LCS) 2. SizeTieredCompactionStrategy (STCS) 3. TimeWindowCompactionStrategy (TWCS) -- Raman Gugnani 8588892293
Re: Cassandra LWT Writes inconsistent
Hi Team, Can anyone help on the same. Cassnadra driver says write is not done but eventually write has been done. On Mon, 8 Jul 2019 at 12:31, Upasana Sharma <028upasana...@gmail.com> wrote: > > Hi, > > I am using an LWT Insert transaction similar to: > > INSERT INTO table1 (code, id, subpart) VALUES (:code, :id, :subpart) IF > NOT EXISTS > > With > readConsistency="LOCAL_SERIAL" > writeConsistency="LOCAL_QUORUM" > > Cassandra Driver: 3.6.0 > Cassandra Version: Cassandra 3.11.2 > > > The problem is that I am getting* [applied] false* on the *first write* to > cassandra. > > I have set retry policy as writeTimes = 0, so no retries are attempted. > > My application logs for reference: > > c-7967981443032352 - [INFO ] 2019-06-18T19:46:16.276Z [pool-15-thread-5] > CreateService - SubPartition 104 > c-7967981443032352 - [INFO ] 2019-06-18T19:46:16.805Z [pool-15-thread-5] > *Repository > - Row[false, A, 1, 104]* > c-7967981443032352 - [INFO ] 2019-06-18T19:46:16.805Z [pool-15-thread-5] > CreateService - SubPartition 104 CodeNumber 75191 > *DuplicateCodeGenerated A* > > This is causing my count of writes to tables to differ from required > 10, to 11, writing extra codes. > > Please guide here. > > -- > Regards, > Upasana Sharma > -- Raman Gugnani 8588892293 Principal Engineer *ixigo.com <http://ixigo.com>*
Getting warnings in system.log
Hi I am getting below logs in system.log WARN [StreamReceiveTask:18] 2019-07-22 07:19:11,590 StorageProxy.java:790 - Received base materialized view mutation for key DecoratedKey(6142465457689613197, 35323330332d383630) that does not belong to this node. There is probably a range movement happening (move or decommission),but this node hasn't updated its ring metadata yet. Adding mutation to local batchlog to be replayed later. I did nodetool decomission on one of the node. After that I have started receiving above logs. nodetool status of the node is *UL* -- Raman Gugnani
Cassandra OutOfMemoryError
Hi I am using Apace Cassandra version : [cqlsh 5.0.1 | Cassandra 3.11.2 | CQL spec 3.4.4 | Native protocol v4] I am running a 5 node cluster and recently added one node to the cluster. Cluster is running with G1 GC garbage collector with 16GB -Xmx. Cluster is having one materialised view also; On the newly added node I got OutOfMemory Error. Heap Dump analysis shows below error: BatchlogTasks:1 at java.lang.OutOfMemoryError.()V (OutOfMemoryError.java:48) at java.util.HashMap.resize()[Ljava/util/HashMap$Node; (HashMap.java:704) at java.util.HashMap.putVal(ILjava/lang/Object;Ljava/lang/Object;ZZ)Ljava/lang/Object; (HashMap.java:663) at java.util.HashMap.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (HashMap.java:612) at java.util.HashSet.add(Ljava/lang/Object;)Z (HashSet.java:220) at org.apache.cassandra.batchlog.BatchlogManager.finishAndClearBatches(Ljava/util/ArrayList;Ljava/util/Set;Ljava/util/Set;)V (BatchlogManager.java:281) at org.apache.cassandra.batchlog.BatchlogManager.processBatchlogEntries(Lorg/apache/cassandra/cql3/UntypedResultSet;ILcom/google/common/util/concurrent/RateLimiter;)V (BatchlogManager.java:261) at org.apache.cassandra.batchlog.BatchlogManager.replayFailedBatches()V (BatchlogManager.java:210) at org.apache.cassandra.batchlog.BatchlogManager$$Lambda$269.run()V (Unknown Source) at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run()V (DebuggableScheduledThreadPoolExecutor.java:118) at java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object; (Executors.java:511) at java.util.concurrent.FutureTask.runAndReset()Z (FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)Z (ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V (ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V (ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run()V (ThreadPoolExecutor.java:624) at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(Ljava/lang/Runnable;)V (NamedThreadFactory.java:81) at org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$4.run()V (Unknown Source) at java.lang.Thread.run()V (Thread.java:748) I have found system.bacthes file have huge data on this node. nodetool -u cassandra -pw cassandra tablestats system.batches -H Total number of tables: 65 Keyspace : system Read Count: 3990928 Read Latency: 0.07400208372589032 ms Write Count: 4898771 Write Latency: 0.012194797838069997 ms Pending Flushes: 0 Table: batches SSTable count: 5 Space used (live): 50.89 GiB Space used (total): 50.89 GiB Space used by snapshots (total): 0 bytes Off heap memory used (total): 1.05 GiB SSTable Compression Ratio: 0.38778672943000886 Number of partitions (estimate): 727971046 Memtable cell count: 12 Memtable data size: 918 bytes Memtable off heap memory used: 0 bytes Memtable switch count: 10 Local read count: 0 Local read latency: NaN ms Local write count: 618894 Local write latency: 0.010 ms Pending flushes: 0 Percent repaired: 0.0 Bloom filter false positives: 0 Bloom filter false ratio: 0.0 Bloom filter space used: 906.25 MiB Bloom filter off heap memory used: 906.25 MiB Index summary off heap memory used: 155.86 MiB Compression metadata off heap memory used: 10.6 MiB Compacted partition minimum bytes: 30 Compacted partition maximum bytes: 258 Compacted partition mean bytes: 136 Average live cells per slice (last five minutes): 149.0 Maximum live cells per slice (last five minutes): 149 Average tombstones per slice (last five minutes): 1.0 Maximum tombstones per slice (last five minutes): 1 Dropped Mutations: 0 bytes *Can someone please help, what can be the issue ?* -- Raman Gugnani
best ORM for cassandra
Hi
I am new to cassandra.
I am developing an application with cassandra.
Which is the best ORM for cassandra?
--
Thanks & Regards
Raman Gugnani
*Senior Software Engineer | CaMS*
M: +91 8588892293 | T: 0124-660 | EXT: 14255
ASF Centre A | 2nd Floor | CA-2130 | Udyog Vihar Phase IV |
Gurgaon | Haryana | India
*Disclaimer:* This communication is for the sole use of the addressee and
is confidential and privileged information. If you are not the intended
recipient of this communication, you are prohibited from disclosing it and
are required to delete it forthwith. Please note that the contents of this
communication do not necessarily represent the views of Jasper Infotech
Private Limited ("Company"). E-mail transmission cannot be guaranteed to be
secure or error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The Company,
therefore, does not accept liability for any loss caused due to this
communication. *Jasper Infotech Private Limited, Registered Office: 1st
Floor, Plot 238, Okhla Industrial Estate, New Delhi - 110020 INDIA CIN:
U72300DL2007PTC168097*
Re: Cassandra Data Audit
Hi Alain,
As per the document. Which column of the dse_audit.audit_log will hold the
previous or new data.
On Wed, Feb 24, 2016 at 3:59 PM, Alain RODRIGUEZ wrote:
> Hi Charu,
>
> Are you using DSE or Open source Cassandra ?
>
> I never used it, but DSE brings a feature that seems to be what you are
> looking for -->
> http://docs.datastax.com/en/datastax_enterprise/4.6/datastax_enterprise/sec/secAuditingCassandraTable.html
>
> Never heard about such a thing in the Open source version though.
>
> C*heers,
> -
> Alain Rodriguez
> France
>
> The Last Pickle
> http://www.thelastpickle.com
>
> 2016-02-24 6:36 GMT+01:00 Charulata Sharma (charshar)
> :
>
>> To all Cassandra experts out there,
>>
>> Can you please let me know if there is any inbuilt Cassandra
>> feature that allows audits on Column family data ?
>>
>>
>>
>> When I change any data in a CF, I want to record that change. Probably
>> store the old value as well as the changed one.
>>
>> One way of doing this is to create new CFs , but I wanted to know if
>> there is any standard C* feature that could be used.
>>
>> Any guidance in this and implementation approaches would really help.
>>
>>
>>
>> Thanks,
>>
>> Charu
>>
>
>
--
Thanks & Regards
Raman Gugnani
*Senior Software Engineer | CaMS*
M: +91 8588892293 | T: 0124-660 | EXT: 14255
ASF Centre A | 2nd Floor | CA-2130 | Udyog Vihar Phase IV |
Gurgaon | Haryana | India
*Disclaimer:* This communication is for the sole use of the addressee and
is confidential and privileged information. If you are not the intended
recipient of this communication, you are prohibited from disclosing it and
are required to delete it forthwith. Please note that the contents of this
communication do not necessarily represent the views of Jasper Infotech
Private Limited ("Company"). E-mail transmission cannot be guaranteed to be
secure or error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The Company,
therefore, does not accept liability for any loss caused due to this
communication. *Jasper Infotech Private Limited, Registered Office: 1st
Floor, Plot 238, Okhla Industrial Estate, New Delhi - 110020 INDIA CIN:
U72300DL2007PTC168097*
Not able to insert data through achilles.
Hi
I am to trying to insert data into cassandra using achilles which contains
only partition key and static columns(all other columns and clustering key
are null), but getting error
info.archinnov.achilles.exception.AchillesException: Field 'membername' in
entity of type 'com.xxx.domain.cassandra.entity.TeamMember' should not be
null because it is a clustering column
I am trying below insert through achilles
TeamMember teamMember = new TeamMember();
teamMember.setTeamname("raman");
teamMember.setManager("rahul");
teamMember.setLocation("india");
manager.crud().insert(teamMember).withInsertStrategy(InsertStrategy.NOT_NULL_FIELDS).execute();
But as per the refrence link it is possible to insert static columns only
with partition key.
reference link :
https://blogs.infosupport.com/static-columns-in-cassandra-and-their-benefits/
CREATE TABLE teammember_by_team (
teamname text,
manager text static,
location text static,
membername text,
nationality text,
position text,
PRIMARY KEY ((teamname), membername)
);
INSERT INTO teammember_by_team (teamname, manager, location)
VALUES (‘Red Bull’, ‘Christian Horner’, ‘’);
teamname | membername | location | manager | nationality | position
--++--+--+-+--
Red Bull | null | | Christian Horner | null | null
--
Thanks & Regards
Raman Gugnani
*Senior Software Engineer | CaMS*
M: +91 8588892293 | T: 0124-660 | EXT: 14255
ASF Centre A | 2nd Floor | CA-2130 | Udyog Vihar Phase IV |
Gurgaon | Haryana | India
*Disclaimer:* This communication is for the sole use of the addressee and
is confidential and privileged information. If you are not the intended
recipient of this communication, you are prohibited from disclosing it and
are required to delete it forthwith. Please note that the contents of this
communication do not necessarily represent the views of Jasper Infotech
Private Limited ("Company"). E-mail transmission cannot be guaranteed to be
secure or error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The Company,
therefore, does not accept liability for any loss caused due to this
communication. *Jasper Infotech Private Limited, Registered Office: 1st
Floor, Plot 238, Okhla Industrial Estate, New Delhi - 110020 INDIA CIN:
U72300DL2007PTC168097*
Achilles not picking cassandra ConsistencyLevel
Hi All,
We are using achilles in our code. We are setting Consistency level as
below, but is not being set.Does anyone else have faced this issue.We have
three nodes in our cluster.
PoolingOptions poolingOptions = new PoolingOptions();
poolingOptions.setCoreConnectionsPerHost(HostDistance.REMOTE,
Integer.parseInt(env.getProperty("cassandra.core.connections.host")));
poolingOptions.setMaxConnectionsPerHost(HostDistance.REMOTE,
Integer.parseInt(env.getProperty("cassandra.max.connections.host")));
poolingOptions.setMaxRequestsPerConnection(HostDistance.REMOTE,
Integer.parseInt(env.getProperty("cassandra.max.requests.connections")));
Cluster.Builder
builder=Cluster.builder().addContactPoints(env.getProperty("cassandra.host")).withPoolingOptions(poolingOptions).withPort(
Integer.parseInt(env.getProperty("cassandra.cql.port"))).withClusterName(CLUSTER_NAME);
if(env.getProperty("cassandra.cql.user") != null){
builder.withCredentials(env.getProperty("cassandra.cql.user"),
env.getProperty("cassandra.cql.password"));
}
Cluster cluster = builder.build();
QueryLogger queryLogger =
QueryLogger.builder().withConstantThreshold(300).withMaxQueryStringLength(1000).build();
cluster.register(queryLogger);
String clRead = env.getProperty("cassandra.consistencylevel.read");
String clWrite =
env.getProperty("cassandra.consistencylevel.write");
LOG.info("Starting cassandra cluster for host {} and port {} ",
env.getProperty("cassandra.host"), env.getProperty("cassandra.cql.port"));
LOG.info(" clRead {} and clWrite {}", clRead, clWrite);
final ManagerFactory factory =
ManagerFactoryBuilder.builder(cluster).withDefaultWriteConsistency(
clWrite == null ? ConsistencyLevel.EACH_QUORUM :
ConsistencyLevel.valueOf(clWrite)).withDefaultReadConsistency(
clRead == null ? ConsistencyLevel.ONE :
ConsistencyLevel.valueOf(clRead)).build();
return factory;
--
Thanks & Regards
Raman Gugnani
*Senior Software Engineer | CaMS*
M: +91 8588892293 | T: 0124-660 | EXT: 14255
ASF Centre A | 2nd Floor | CA-2130 | Udyog Vihar Phase IV |
Gurgaon | Haryana | India
*Disclaimer:* This communication is for the sole use of the addressee and
is confidential and privileged information. If you are not the intended
recipient of this communication, you are prohibited from disclosing it and
are required to delete it forthwith. Please note that the contents of this
communication do not necessarily represent the views of Jasper Infotech
Private Limited ("Company"). E-mail transmission cannot be guaranteed to be
secure or error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The Company,
therefore, does not accept liability for any loss caused due to this
communication. *Jasper Infotech Private Limited, Registered Office: 1st
Floor, Plot 238, Okhla Industrial Estate, New Delhi - 110020 INDIA CIN:
U72300DL2007PTC168097*
Re: Data is not syncing up when we add one more Node(DR) to existing 3 node cluster
HI Anil,
Please follow the below link.
https://thelastpickle.com/blog/2019/02/26/data-center-switch.html
Did you ran the command nodetool rebuild old_dc_name to stream the old
historical data to new data center.
Step 9 of the above article.
On Thu, 12 Dec 2019 at 09:39, Anil Kumar Ganipineni <
akganipin...@adaequare.com> wrote:
> Hi All,
>
>
>
> We have 3 node cluster on datacentre DC1 and below is our key space
> declaration. The current data size on the cluster is ~10GB. When we add a
> new node on datacentre DC2, the new node is not syncing up with the data,
> but it is showing UN when I run the *nodetool status*.
>
>
>
> *CREATE* KEYSPACE *production* *WITH* REPLICATION = { 'class' :
> 'org.apache.cassandra.locator.NetworkTopologyStrategy', 'DC1': '3', 'DC2':
> '1' } *AND* DURABLE_WRITES = *true*;
>
>
>
>
>
> Please provide suggestions to make the new node on DC2 to sync up with
> existing cluster. This is required as the DC2 is our DR in a different
> region other than existing cluster.
>
>
>
>
>
> *Regards,*
>
> *Anil Ganipineni*
>
>
>
> *P** Please consider environment before printing this page.*
>
>
>
--
Raman Gugnani
Change of Cassandra TTL
HI all, 1. I have a table with default_time_to_live = 31536000 (1 year) . We want it to reduce the value to 7884000 (3 months). If we alter the table , is there a way to update the existing data? 1. I have a table without TTL we want to add TTL = 7884000 (3 months) on the table. If we alter the table , is there a way to update the existing data? -- Raman Gugnani
Re: Change of Cassandra TTL
Thanks Eric for the update. On Tue, 14 Sept 2021 at 16:50, Erick Ramirez wrote: > You'll need to write an ETL app (most common case is with Spark) to scan > through the existing data and update it with a new TTL. You'll need to make > sure that the ETL job is throttled down so it doesn't overload your > production cluster. Cheers! > >> -- Raman Gugnani
Cassandra Delete Query Doubt
HI Team, I have one table below and want to delete data on this table. DELETE FROM game.tournament USING TIMESTAMP 161692578000 WHERE tournament_id = 1 AND version_id = 1 AND partition_id = 1; Cassandra internally manages the timestamp of each column when some data is updated on the same column. My Query is , *USING TIMESTAMP 161692578000* picks up a timestamp of which column ? CREATE TABLE game.tournament ( tournament_id bigint, version_id bigint, partition_id bigint, user_id bigint, created_at timestamp, rank bigint, score bigint, updated_at timestamp, PRIMARY KEY ((tournament_id, version_id, partition_id), user_id) ) WITH CLUSTERING ORDER BY (user_id ASC) -- Raman Gugnani
Re: Cassandra Delete Query Doubt
Thanks Jeff for the information. On Wed, 10 Nov 2021 at 21:08, Jeff Jirsa wrote: > This type of delete - which doesnt supply a user_id, so it's deleting a > range of rows - creates what is known as a range tombstone. It's not tied > to any given cell, as it covers a range of cells, and supersedes/shadows > them when merged (either in the read path or compaction path). > > > > On Wed, Nov 10, 2021 at 4:27 AM raman gugnani > wrote: > >> HI Team, >> >> >> I have one table below and want to delete data on this table. >> >> >> DELETE FROM game.tournament USING TIMESTAMP 161692578000 WHERE >> tournament_id = 1 AND version_id = 1 AND partition_id = 1; >> >> >> Cassandra internally manages the timestamp of each column when some data >> is updated on the same column. >> >> >> My Query is , *USING TIMESTAMP 161692578000* picks up a timestamp of >> which column ? >> >> >> >> CREATE TABLE game.tournament ( >> >> tournament_id bigint, >> >> version_id bigint, >> >> partition_id bigint, >> >> user_id bigint, >> >> created_at timestamp, >> >> rank bigint, >> >> score bigint, >> >> updated_at timestamp, >> >> PRIMARY KEY ((tournament_id, version_id, partition_id), user_id) >> >> ) WITH CLUSTERING ORDER BY (user_id ASC) >> >> >> >> >> >> >> >> -- >> Raman Gugnani >> > -- Raman Gugnani
