There's a few things you can do here that might help.
First off, if you're using the default heap settings, that's a serious
problem. If you've got the head room, my recommendation is to use 16GB
heap with 12 GB new gen and pin your memtable heap space to 2GB. Set your
max tenuring threshold to
Thanks for handling this, Mick!
On Fri, Feb 7, 2020 at 12:02 PM Mick Semb Wever wrote:
>
>
> The Cassandra team is pleased to announce the release of Apache Cassandra
> version 4.0-alpha3.
>
> Apache Cassandra is a fully distributed database. It is the right choice
> when you need scalability an
A while ago, on my first cluster, I decided to do an upgrade by adding
nodes running 1.2 to an existing cluster running version 1.1. This was a
bad decision, and at that point I decided to always play it safe and always
stick to a single version, and never bootstrap in a node running different
ver
Seeds don't bootstrap, don't list new nodes as seeds.
On Thu, Feb 13, 2020 at 5:23 PM Sergio wrote:
> Hi guys!
>
> I don't know how but this is the first time that I see such behavior. I
> wanted to add a new node in the cluster and it looks to be working fine but
> instead to wait for 2-3 hours
I also recommend avoiding them. I've seen too many clusters fall over as a
result of their usage.
On Fri, Feb 28, 2020 at 9:52 AM Max C. wrote:
> The general view of the community is that you should *NOT* use them in
> production, due to multiple serious outstanding issues (see Jira). We used
You can issue a delete using a future timestamp.
http://cassandra.apache.org/doc/latest/cql/dml.html#grammar-token-update-parameter
Look for USING TIMESTAMP.
Jon
On Mon, Mar 2, 2020, 3:28 AM Furkan Cifci wrote:
> Greetings,
> In our C* cluster, one node lost time sync and it went to future(16
It's not going to matter at all.
On Fri, Mar 6, 2020, 2:15 AM Hanauer, Arnulf, Vodacom South Africa
(External) wrote:
> Hi Cassandra folks,
>
>
>
> Is there any difference in performance of general operations if using a
> TEXT based Primary key versus a BIGINT Primary key.
>
>
>
> Our use-case r
I'm on mobile now so I might be mistaken, but I don't think nodetool move
works with multiple tokens
On Fri, May 29, 2020, 1:48 PM Kornel Pal wrote:
> Hi Anthony,
>
> Thank you very much for looking into using the script for initial token
> generation and for providing multiple detailed methods
Generally speaking, don't run mixed versions longer than you have to, and
don't upgrade that way.
Why?
* We don't support it.
* We don't even test it.
* If you run into trouble and ask for help, the first thing people will
tell you is to get all nodes on the same version.
Anyone that's doing so
You could also pull TWCS out of the version of Cassandra you want to
deploy, fix the imports and change the package name. Then you've got the
same version as OSS, just under the name you're using in 2.1. Once you've
moved to 3.11, you can switch to the OSS version.
On Thu, Jul 9, 2020 at 9:09 AM
I worked on a handful of large clusters (> 200 nodes) using vnodes, and
there were some serious issues with both performance and availability. We
had to put in a LOT of work to fix the problems.
I agree with Jeff - it's way better to manage multiple clusters than a
really large one.
On Fri, Jul
I haven't found chunk cache to be particularly useful. It's a fairly small
cache that could only help when you're dealing with a small hot dataset. I
wouldn't bother increasing memory for it.
Key cache can be helpful, but it depends on the workload. I generally
recommend optimizing for your
I can't think of a reason to keep empty directories around, seems like a
reasonable change, but I don't think you're butting up against a thing that
most people would run into, as snapshots are enabled by default (auto_snapshot:
true) and almost nobody changes it.
The use case you described i
Hey everyone,
Over the last several months I've put a lot of work into 2 projects I
started back at The Last Pickle, for stress testing Cassandra and for
building labs in AWS. You may know them as tlp-stress and tlp-cluster.
Since I haven't worked at TLP in almost half a decade, and am the prima
Hey Chris - this looks pretty interesting! It looks like there's a lot of
functionality in here.
* What aspects of Mission Control are dependent on using K8ssandra?
* Can Mission Control work without K8ssandra?
* Is mission control open source?
* I'm not familiar with Vector - does it require an
Hey everyone,
Today starting at 10am PT I'm going to be streaming my session messing with
5.0, looking at UCS. I'm doing this with my easy-cass-lab and
easy-cass-stress tools using a build of C* from last night. I'll also show
some of the cool things you can do with my tools.
I'll be running th
Hey folks,
I'm doing a working session tomorrow at 10am PDT, testing LWTs in C* 5.0.
I'll be running benchmarks and doing some performance analysis. Come hang
out and bring your questions!
Jon
YouTube: https://www.youtube.com/watch?v=IoWh647LRQ0
LinkedIn:
https://www.linkedin.com/events/cassan
Hi,
Unfortunately, the numbers you're posting have no meaning without context.
The speculative retries could be the cause of a problem, or you could
simply be executing enough queries and you have a fairly high variance in
latency which triggers them often. It's unclear how many queries / second
Try changing the chunk length parameter on the compression settings to 4kb,
and reduce read ahead to 16kb if you’re using EBS or 4KB if you’re using
decent local ssd or nvme.
Counters read before write.
—
Jon Haddad
Rustyrazorblade Consulting
rustyrazorblade.com
On Fri, Apr 5, 2024 at 9:27 AM
You shouldn’t decom an entire DC before removing it from replication.
—
Jon Haddad
Rustyrazorblade Consulting
rustyrazorblade.com
On Mon, Apr 8, 2024 at 6:26 AM Michalis Kotsiouros (EXT) via user <
user@cassandra.apache.org> wrote:
> Hello community,
>
> In our deployments, we
Hey all,
Tomorrow at 10:30am PDT I'm taking a look at Trie Memtables tomorrow on my
live stream. I'll do some performance comparisons between it and the
legacy SkipListMemtable implementation and see what I can learn.
https://www.youtube.com/live/Jp5R_-uXORQ?si=NnIoV3jqjHFoD8nF
or if you prefer
Hey everyone,
I'm doing a 4 week YouTube series on the C* storage engine. My first video
was last week where I gave an overview into some of the storage engine
internals [1].
The next 3 weeks are looking at the new Trie indexes coming in 5.0 [2],
running Cassandra on EBS [3], and finally looking
Thanks Aaron!
Just realized I made a mistake, the 4th week's URL is
https://www.youtube.com/watch?v=MAxQ0QygcKk.
Jon
On Tue, Apr 30, 2024 at 4:58 AM Aaron Ploetz wrote:
> Nice! This sounds awesome, Jon.
>
> On Mon, Apr 29, 2024 at 6:25 PM Jon Haddad wrote:
>
>> Hey ev
Unless your cluster is very small, using the method of adding / removing
nodes will eventually result in putting a much larger portion of your
dataset on a very few number of nodes. I *highly* discourage this.
The only correct, safe path is Bowen's suggestion of adding another DC and
decommission
I strongly suggest you don't use materialized views at all. There are edge
cases that in my opinion make them unsuitable for production, both in terms
of cluster stability as well as data integrity.
Jon
On Fri, May 17, 2024 at 8:58 AM Gábor Auth wrote:
> Hi,
>
> I know, I know, the materialize
As an aside, if you're not putting a TTL on your data, it's a good idea to
be proactive and use multiple tables. For example, one per month or year.
This allows you the flexibility to delete your data by dropping old tables.
Storing old data in Cassandra is expensive. Once you get to a certain
p
Erick, that blanket statement about it “never” going in 5.1 is incorrect.
We could absolutely add Java 21 support in 5.1 if we wanted to.
—
Jon Haddad
Rustyrazorblade Consulting
rustyrazorblade.com
On Mon, Jun 17, 2024 at 11:26 AM Erick Ramirez
wrote:
> Cassandra 5.0 will only work with J
Yes, 5.0 already works with 17 and I’ve been running it almost exclusively
in my tests.
—
Jon Haddad
Rustyrazorblade Consulting
rustyrazorblade.com
On Mon, Jun 17, 2024 at 1:08 PM manish khandelwal <
manishkhandelwa...@gmail.com> wrote:
> Thans Erick, Jon and Stefan for the respons
Hey everyone!
If you've already seen the news that 5.0 RC-1 is out, you might be
wondering how you can kick the tires without investing a ton of time into
changing your tooling around. I've got you covered :)
I've just released an update to easy-cass-lab [1], my tooling to spin up
Cassandra envi
Hey everyone!!
With 5.0 right around the corner I'm sure you're wondering if there's a
good reason for you to upgrade, or if you should just ignore it and spend
your time doing other things instead.
ARE YOU KIDDING ME?!?!?! THIS RELEASE IS HUGE.
Come join me as I take an hour to cover some of t
Riding on the coattails of the release...
If folks are interested in trying this release but you don't have the
tooling to spin it up, I've already released an update to easy-cass-lab [1]
to support 4.1.6. It's my tooling that lets you get started with Apache
Cassandra in AWS in under 15 minutes.
Hey Matthias,
Somehow this slipped by me. Here's the video:
https://youtube.com/live/-i-ox-8q0nI
Jon
On Thu, Aug 29, 2024 at 3:31 AM Matthias Pfau via user <
user@cassandra.apache.org> wrote:
> Hey Jon,
> just saw this and wondered if a recording is available?
>
> Thanks!
>
> Matthias
>
>
> Au
Are you using collections?
—
Jon Haddad
Rustyrazorblade Consulting
rustyrazorblade.com
On Tue, Oct 8, 2024 at 10:52 PM Naman kaushik
wrote:
> Hi Community,
>
> We are currently using Cassandra version 4.1.3 and have encountered an
> issue related to tombstone generation. We hav
I've worked with a few hundred teams now, including the major ones that
used single token (Apple, Netflix, Spotify), and pretty much all the rest
used some form of vnodes.
Jeff did a good job of summarizing the tradeoffs and I don't have anything
to add.
I would never, ever, recommend > 4 tokens.
Overwriting non collections does not generate tombstones on compaction.
—
Jon Haddad
Rustyrazorblade Consulting
rustyrazorblade.com
On Wed, Oct 9, 2024 at 9:57 AM James Shaw wrote:
> Hi, Naman:
> How does the client side load large amounts of data ? Most
> likely it ha
I ran into this a few months ago, and in my case I tracked it down to an
issue with ZFS not unlinking commitlogs properly.
https://issues.apache.org/jira/browse/CASSANDRA-19564
On Tue, Nov 5, 2024 at 6:05 AM Dmitry Konstantinov
wrote:
> I am speaking about a thread dump (stack traces for all th
#x27;s a large number of snapshots. These can
> have significant impact on file deletion performance on ZFS. Also worth
> checking the disks, I have seen broken disks that stuck on some operations,
> e.g. when a specific sector is being read, and this will certainly affect
> the filesyste
I think this is the correct explanation. It's very similar to
CASSANDRA-19576, where compaction is unable to finish because we can't
insert into compaction history.
Really good analysis, Jaydeep.
Jon
On Sun, Nov 10, 2024 at 1:51 PM Jaydeep Chovatia
wrote:
> Upon further studying the thread d
Thank you both for the recommendation!
Jon
On Fri, Sep 27, 2024 at 5:12 AM Aaron Ploetz wrote:
> Casting a second vote for Jon Haddad. You can reach out to him on
> LinkedIn: https://www.linkedin.com/in/rustyrazorblade/
>
> Thanks,
>
> Aaron
>
>
> On Fri, Sep 27,
It sounds like enabling the JDK's vector preview api could significantly
improve Vector search. I haven't verified this myself, but it might be
worth trying Java 17 + this flag:
--add-modules jdk.incubator.vector
I'd love to hear how much of a difference this makes.
Jon
On Fri, Nov 8, 2024 at
fficulty of upgrading in general? I'm all for
improving it. It's just not what this thread is about.
Jon
On Wed, Dec 18, 2024 at 10:01 AM Eric Evans
wrote:
>
>
> On Wed, Dec 18, 2024 at 11:43 AM Jon Haddad
> wrote:
>
>> > We (Wikimedia) have had more
ch had very good automation for this sort of thing, I can
>> still see this process taking 3 times as long to complete as a normal
>> upgrade, and this does take up operators time.
>>
>> I can see the advantages of 3 stage process, and all things being equal I
>> would r
thing, I can
>> still see this process taking 3 times as long to complete as a normal
>> upgrade, and this does take up operators time.
>>
>> I can see the advantages of 3 stage process, and all things being equal I
>> would recommend that process as being safe
Just curious, why is a rolling restart difficult? Is it a tooling issue,
stability, just overall fear of messing with things?
You *should* be able to do a rolling restart without it being an issue. I look
at this as a fundamental workflow that every C* operator should have available,
and you
age process, and all things being equal I
> would recommend that process as being safer, however I am getting a lot of
> push back whenever we discuss the upgrade process.
>
> Thanks
>
> Paul
>
> > On 17 Dec 2024, at 19:24, Jon Haddad wrote:
> >
> > Just cu
#x27;s question - clients (i.e. java driver, etc) should be able to
> handle disconnects gracefully and route to other coordinators leaving the
> application-facing symptom being a blip on latency. Are you seeing
> something else more painful, or is it more just not having the built-in
&
I’m not sure if i shared this to the user list… I’m doing a massive series
on C* 5.0 performance and how it relates to node density and cost. First
post is up now.
http://rustyrazorblade.com/post/2025/03-streaming/
The benefit any given feature depends on a lot of factors. Hardware and
workload v
Thanks for bringing this up. I've been meaning to for a while now.
I'm at the early stages of doing some testing of Java 11 G1GC vs 17 w/ G1GC
vs 17 w/ Shenandoah. I plan on sharing my results with the wider
community, and if things look good I'll be proposing we remove the
experimental label.
Just a reminder!
The deadline to submit a proposal closes at 6:59 PM 21 Apr 2025 in Central
Daylight Time (UTC-05:00) timezone.
Jon
On Mon, Mar 10, 2025 at 6:51 AM Paulo Motta wrote:
> Hi,
>
> Please see message below with instructions on submitting talk proposals
> for Community Over Code 202
There's several options for load testing. I'll admit that I'm massively
biased as I wrote one of them.
* easy-cass-stress offers a variety of customizable workloads out of the
box and can stress many features without writing any code or learning new
config. I wrote this with the goal of getting
Can you explain a bit more what you mean by memory spikes?
The defaults we ship use the same settings for min and max JVM heap size,
so you should see all the memory allocated to the JVM at startup. Did you
change anything here? I don't recommend doing so.
If you're referring to files in the pa
I've got 1K nodes in various clusters running on Java 17. I've had great
results with Shenandoah, 30GB heap, and off heap trie memtables.
Pauses are as advertised, between 1-3ms. With smaller heaps or high
throughput you might see the JVM apply pacing - to ensure it can keep up
with allocations.
I agree that managing Cassandra on Kubernetes can be challenging without
prior experience, as understanding all the nuances of Kubernetes takes time.
However, there are ways to address the rescheduling issues, node placement,
and local disk concerns that were mentioned. You can pin pods to specifi
ries about there being
>> additional hidden nuances to implementing the same level of functionality
>> and reliability in ECS and even K8s. We agree that the K8ssandra operator
>> would be the most advantageous and desired aspect of switching to a
>> container-based solu
I don't have the thread handy but I thought we decided to do this already
and it would just be a doc change.
Either way, I'm +1 on removing experimental flag.
Jon
On Tue, Jul 8, 2025 at 1:38 PM Geremy Cohen wrote:
> Hi all, just curious as to when docs will be updated to signify official
> JDK
201 - 255 of 255 matches
Mail list logo
