Re: JVM safepoints, mmap, and slow disks

Hi, Page cache is in use even if you disable swap. Swap is anonymous memory, and whatever else the Linux kernel supports paging out. Page cache is data pending flush to disk and data cached from disk. Given how bad the GC pauses are in C* I think it's not the high pole in the tent. Until key thin

Understanding cassandra data directory contents

Hello, I am using Cassandra 3.0.9 and I have encountered an issue where the nodes in my 3 node cluster have vastly different amounts of data even though they should be roughly the same. When I looked through the data directory for my database on two of the nodes, I see a number of directories wi

Re: Re: JVM safepoints, mmap, and slow disks

>Page cache is data pending flush to disk and data cached from disk. Do you mean file cache? Best regards, Vladimir Yudovin, Winguzone - Hosted Cloud Cassandra on Azure and SoftLayer. Launch your cluster in minutes. On Sat, 08 Oct 2016 13:40:19 -0400 Ariel Weisberg

Re: Understanding cassandra data directory contents

Each table has unique id (suffix). If you drop and then recreate table with the same name it gets new id. Try SELECT keyspace_name, table_name, id FROM system_schema.tables ; to determinate actual ID. You can limit request to specific keyspace or table. Best regards, Vladimir Yudovin, Winguzo

Re: JVM safepoints, mmap, and slow disks

Hi, Nope I mean page cache. Linux doesn't call the cache it maintains using free memory a file cache. It uses free (and some of the time not so free!) memory to buffer writes and to cache recently written/read data. http://www.tldp.org/LDP/lki/lki-4.html When Linux decides it needs free memory i

Re: JVM safepoints, mmap, and slow disks

Sorry, I don't catch something. What page (memory) cache can exist if there is no swap file. Where are those page written/read? Best regards, Vladimir Yudovin, Winguzone - Hosted Cloud Cassandra on Azure and SoftLayer. Launch your cluster in minutes. On Sat, 08 Oct 2016 14:09:50 -0400 A

Re: JVM safepoints, mmap, and slow disks

Linux automatically uses free memory as cache. It's not swap. http://www.tldp.org/LDP/lki/lki-4.html On Sat, Oct 8, 2016 at 11:12 AM Vladimir Yudovin wrote: > Sorry, I don't catch something. What page (memory) cache can exist if > there is no swap file. > Where are those page written/read? > >

Re: Understanding cassandra data directory contents

Hi Vladamir, Thanks for the response. I assume then that it is safe to remove the directories that are not current as per the system_schema.tables table. I have dozens of the same table and haven't dropped and added nearly that many times. Do any of the nodetool or other commands clean up these

Ordering by multiple columns?

Is it possible to have multiple clustering keys in cassandra, or some other way to order by multiple columns? For example, say I have a table of songs, and each song has a rating and a date. I want to sort songs by rating first, and then with newer songs on top. So if two songs have 5 rating, an

Re: JVM safepoints, mmap, and slow disks

We don’t use Azul’s Zing, but it does have the nice feature that all threads don’t have to reach safepoints at the same time. That said we make heavy use of Cassandra (with off heap memtables - not directly related but allows us a lot more GC headroom) and SOLR where we switched to mmap because

Re: JVM safepoints, mmap, and slow disks

I haven’t studied the read path that carefully, but there might be a spot at the C* level rather than JVM level where you could effectively do a JNI touch of the mmap region you’re going to need next. > On Oct 8, 2016, at 7:17 PM, Graham Sanderson wrote: > > We don’t use Azul’s Zing, but it do

Do partition keys create skinny or wide rows?

Say I have the following primary key: PRIMARY KEY((organization_id, employee_id)) Will this create 1 row whose primary key is the organization id, but it has a 4 billion column / cell limit? Or will this create 1 row for each employee in the same organization, so if i have 5 employees, they will

Re: Do partition keys create skinny or wide rows?

the last '4 billion rows' should say '4 billion columns / cells' On Sun, Oct 9, 2016 at 6:34 AM, Ali Akhtar wrote: > Say I have the following primary key: > PRIMARY KEY((organization_id, employee_id)) > > Will this create 1 row whose primary key is the organization id, but it > has a 4 billion c

Re: JVM safepoints, mmap, and slow disks

Hi, This is starting to get into dev list territory. Interesting idea to touch every 4K page you are going to read. You could use this to minimize the cost. http://stackoverflow.com/questions/36298111/is-it-possible-to-use-sun-misc-unsafe-to-call-c-functions-without-jni/36309652#36309652 Maybe

Re: JVM safepoints, mmap, and slow disks

I was using the term “touch” loosely to hopefully mean pre-fetch, though I suspect (I think intel has been de-emphasizing) you can still do a sensible prefetch instruction in native code. Even if not you are still better blocking in JNI code - I haven’t looked at the link to see if the correct b

Re: Do partition keys create skinny or wide rows?

Nomenclature is tricky, but PRIMARY KEY((organization_id, employee_id)) will make organization_id, employee_id the partition key which equates roughly to your latter sentence (I’m not sure about the 4 billion limit - that may be the new actual limit, but probably not a good idea). > On Oct 8, 2

Re: Do partition keys create skinny or wide rows?

In the case of PRIMARY KEY((organization_id, employee_id)), could I still do a query like Select ... where organization_id = x, to get all employees in a particular organization? And, this will put all those employees in the same node, right? On Sun, Oct 9, 2016 at 9:17 AM, Graham Sanderson wrot

Re: Ordering by multiple columns?

Yes, look at this example. But you need to add 'fake' partition with the same value for all rows, that will cause them to be stored on the same nodes. Best regards, Vladimir Yudovin, Winguzone - Hosted Cloud Cassandra on Azure and SoftLayer. Launch your cluster in minutes. On Sat, 08 O

Re: Do partition keys create skinny or wide rows?

No the employees would end up in arbitrary partitions, and querying them would be inefficient (impossible? - I am levels back on C* so don’t know if ALLOW FILTERING even works for this). I would be tempted to use organization_id only or organization_Id and maybe a few shard bits (if you are wor

Re: Do partition keys create skinny or wide rows?

>querying them would be inefficient (impossible? Impossible. In the case of multi-column partition key all of them must be restricted in WHERE clause: CREATE TABLE data.table (id1 int, id2 int, primary KEY ((id1,id2))); SELECT * FROM data.table WHERE id1 = 0; InvalidRequest: Error from server: co

Missing table suffix in data directory directories

Hello, In the data directory of my 3.0.9 installation, I have directories with both suffixes and without: periodicReading periodicReadingTemp-76eb7510096811e68a7421c8b9466352 The directories with and without suffixes are being updated and for those with a suffix, the suffix matches the output of

Re: Ordering by multiple columns?

Yes it is possible, read this: http://www.slideshare.net/doanduyhai/datastax-day-2016-cassandra-data-modeling-basics/24 and the following slides On Sun, Oct 9, 2016 at 2:04 AM, Ali Akhtar wrote: > Is it possible to have multiple clustering keys in cassandra, or some > other way to order by mult