Re: [EXTERNAL] Re: Connection Pooling in v4.x Java Driver

2019-12-11 Thread Caravaggio, Kevin
Hi Alexandre, Thank you for the explanation. I understand that reasoning very well now. Jon, appreciate the link, and will follow up there for this sort of thing then. Thanks, Kevin From: Alexandre Dutra Reply-To: "user@cassandra.apache.org" Date: Wednesday, December 11, 2019

Connection Pooling in v4.x Java Driver

2019-12-10 Thread Caravaggio, Kevin
n pool configuration. Is anyone aware of the reasoning for this departure from dynamic pool sizing, which I believe was available in v3.x? Thanks, Kevin NOTICE: All information in and attached to the e-mails below may be proprietary, confidential, privileged and oth

Re: Adding a new node with the double of disk space

2017-08-17 Thread Kevin O'Connor
Are you saying if a node had double the hardware capacity in every way it would be a bad idea to up num_tokens? I thought that was the whole idea of that setting though? On Thu, Aug 17, 2017 at 9:52 AM, Carlos Rolo wrote: > No. > > If you would double all the hardware on that node vs the others

unsubscribe

2017-07-17 Thread kevin
unsubscribe

Re: Truncate data from a single node

2017-07-12 Thread Kevin O'Connor
Thanks for the suggestions! Could altering the RF from 2 to 1 cause any issues, or will it basically just be changing the coordinator's write paths and also guiding future repairs/cleans? On Wed, Jul 12, 2017 at 22:29 Jeff Jirsa wrote: > > > On 2017-07-11 20:09 (-0700), &

Truncate data from a single node

2017-07-11 Thread Kevin O'Connor
This might be an interesting question - but is there a way to truncate data from just a single node or two as a test instead of truncating from the entire cluster? We have time series data we don't really care if we're missing gaps in, but it's taking up a huge amount of space and we're looking to

Re: How to avoid flush if the data can fit into memtable

2017-05-31 Thread Kevin O'Connor
Great post Akhil! Thanks for explaining that. On Mon, May 29, 2017 at 5:43 PM, Akhil Mehra wrote: > Hi Preetika, > > After thinking about your scenario I believe your small SSTable size might > be due to data compression. By default, all tables enable SSTable > compression. > > Let go through yo

Fail to add a new node to a exist cluster

2017-05-03 Thread kevin
some other nodes continued to be high, and finally had to give up join. Is it a new node to join the process itself is very slow, or our way of use (too much data per node) improper and cause this problem? Is there any good way to speed up the process of adding new nodes? Thanks, Kevin

iostat -like tool to parse 'nodetool cfstats'

2016-12-20 Thread Kevin Burton
nodetool cfstats has some valuable data but what I would like is a 1 minute delta. Similar to iostat... It's easy to parse this but has anyone done it? I want to see IO throughput and load on C* for each table. -- We’re hiring if you know of any awesome Java Devops or Linux Operations Enginee

Re: STCS Compaction with wide rows & TTL'd data

2016-09-02 Thread Kevin O'Connor
On Fri, Sep 2, 2016 at 9:33 AM, Mark Rose wrote: > Hi Kevin, > > The tombstones will live in an sstable until it gets compacted. Do you > have a lot of pending compactions? If so, increasing the number of > parallel compactors may help. Nope, we are pretty well managed on co

STCS Compaction with wide rows & TTL'd data

2016-09-01 Thread Kevin O'Connor
s pretty surprising. Why would this explicit request to compact an sstable not remove tombstones? Thanks! Kevin

Re: [Marketing Mail] Re: Memory leak and lockup on our 2.2.7 Cassandra cluster.

2016-08-04 Thread Kevin Burton
BTW. we think we tracked this down to using large partitions to implement inverted indexes. C* just doesn't do a reasonable job at all with large partitions so we're going to migrate this use case to using Elasticsearch On Wed, Aug 3, 2016 at 1:54 PM, Ben Slater wrote: > Yep, that was what I w

Re: Mutation of X bytes is too large for the maximum size of Y

2016-08-03 Thread Kevin Burton
(but other drivers should > have a similar exception): > https://github.com/datastax/python-driver/blob/master/cassandra/protocol.py#L288 > > On Wed, Aug 3, 2016 at 1:59 PM Ryan Svihla wrote: > >> Made a Jira about it already >> https://issues.apache.org/jira/plugi

Mutation of X bytes is too large for the maximum size of Y

2016-08-03 Thread Kevin Burton
nt or what CQL is causing the large mutation. Any thoughts on how to mitigate this? Kevin -- We’re hiring if you know of any awesome Java Devops or Linux Operations Engineers! Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ pr

Re: [Marketing Mail] Re: Memory leak and lockup on our 2.2.7 Cassandra cluster.

2016-08-03 Thread Kevin Burton
We usually use 100 per every 5 minutes.. but you're right. We might actually move this use case over to using Elasticsearch in the next couple of weeks. On Wed, Aug 3, 2016 at 11:09 AM, Jonathan Haddad wrote: > Kevin, > > "Our scheme uses large buckets of content where we

Re: Memory leak and lockup on our 2.2.7 Cassandra cluster.

2016-08-03 Thread Kevin Burton
nt, your best >> solution would be to find a way to make your partitions smaller (like >> 1/10th of the size). >> >> Cheers >> Ben >> <https://issues.apache.org/jira/browse/CASSANDRA-11206> >> >> On Wed, 3 Aug 2016 at 12:35 Kevin Burton wrote: &

Re: Memory leak and lockup on our 2.2.7 Cassandra cluster.

2016-08-03 Thread Kevin Burton
to make your partitions smaller (like > 1/10th of the size). > > Cheers > Ben > <https://issues.apache.org/jira/browse/CASSANDRA-11206> > > On Wed, 3 Aug 2016 at 12:35 Kevin Burton wrote: > >> I have a theory as to what I think is happening here. >> >

Re: Memory leak and lockup on our 2.2.7 Cassandra cluster.

2016-08-02 Thread Kevin Burton
index/content_legacy_2016_08_02:1470154500099 (106107128 bytes) On Tue, Aug 2, 2016 at 6:43 PM, Kevin Burton wrote: > We have a 60 node CS cluster running 2.2.7 and about 20GB of RAM allocated > to each C* node. We're aware of the recommended 8GB limit to keep GCs low > but our memory has been cr

Memory leak and lockup on our 2.2.7 Cassandra cluster.

2016-08-02 Thread Kevin Burton
We have a 60 node CS cluster running 2.2.7 and about 20GB of RAM allocated to each C* node. We're aware of the recommended 8GB limit to keep GCs low but our memory has been creeping up (probably) related to this bug. Here's what we're seeing... if we do a low level of writes we think everything g

Re: Are counters faster than CAS or vice versa?

2016-07-20 Thread Kevin Burton
On Wed, Jul 20, 2016 at 11:53 AM, Jeff Jirsa wrote: > Can you tolerate the value being “close, but not perfectly accurate”? If > not, don’t use a counter. > > > yeah.. agreed.. this is a problem which is something I was considering. I guess it depends on whether they are 10x faster.. -- We’r

Are counters faster than CAS or vice versa?

2016-07-20 Thread Kevin Burton
27;re just doing a CAS operation now to read the existing value, then increment it. I think it might have been better to implement this as a counter. Would that be inherently faster or would a CAS be about the same? I can't really test it without deploying it so I figured I would just ask her

Open source equivalents of OpsCenter

2016-07-13 Thread Kevin O'Connor
Now that OpsCenter doesn't work with open source installs, are there any runs at an open source equivalent? I'd be more interested in looking at metrics of a running cluster and doing other tasks like managing repairs/rolling restarts more so than historical data.

Re: Latency overhead on Cassandra cluster deployed on multiple AZs (AWS)

2016-04-12 Thread Kevin O'Connor
Are you in VPC or EC2 Classic? Are you using enhanced networking? On Tue, Apr 12, 2016 at 9:52 AM, Alessandro Pieri wrote: > Hi Jack, > > As mentioned before I've used m3.xlarge instance types together with two > ephemeral disks in raid 0 and, according to Amazon, they have "high" > network perf

Re: Efficiently filtering results directly in CS

2016-04-08 Thread Kevin Burton
Ha.. Yes... C*... I guess I need something like coprocessors in bigtable. On Fri, Apr 8, 2016 at 1:49 AM, vincent gromakowski < vincent.gromakow...@gmail.com> wrote: > c* I suppose > > 2016-04-07 19:30 GMT+02:00 Jonathan Haddad : > >> What is CS? >> >> O

Efficiently filtering results directly in CS

2016-04-07 Thread Kevin Burton
I have a paging model whereby we stream data from CS by fetching 'pages' thereby reading (sequentially) entire datasets. We're using the bucket approach where we write data for 5 minutes, then we can just fetch the bucket for that range. Our app now has TONS of data and we have a piece of middlew

[ANNOUNCE] YCSB 0.7.0 Release

2016-02-26 Thread Kevin Risden
On behalf of the development community, I am pleased to announce the release of YCSB 0.7.0. Highlights: * GemFire binding replaced with Apache Geode (incubating) binding * Apache Solr binding was added * OrientDB binding improvements * HBase Kerberos support and use single connection * Accumulo i

Faster version of 'nodetool status'

2016-02-12 Thread Kevin Burton
Is there a faster way to get the output of 'nodetool status' ? I want us to more aggressively monitor for 'nodetool status' and boxes being DN... I was thinking something like jolokia and REST but I'm not sure if there are variables exported by jolokia for nodetool status. Thoughts? -- We’re

Re: automated CREATE TABLE just nuked my cluster after a 2.0 -> 2.1 upgrade....

2016-01-23 Thread Kevin Burton
fic Jira assigned, and the antipattern doc doesn't appear to > reference this scenario. Maybe a committer can shed some more light. > > -- Jack Krupansky > > On Fri, Jan 22, 2016 at 10:29 PM, Kevin Burton wrote: > >> I sort of agree.. but we are also considering migrating t

Re: automated CREATE TABLE just nuked my cluster after a 2.0 -> 2.1 upgrade....

2016-01-22 Thread Kevin Burton
47 PM, Jonathan Haddad wrote: > Instead of using ZK, why not solve your concurrency problem by removing > it? By that, I mean simply have 1 process that creates all your tables > instead of creating a race condition intentionally? > > On Fri, Jan 22, 2016 at 6:16 PM Kevin Burton wrote: &

automated CREATE TABLE just nuked my cluster after a 2.0 -> 2.1 upgrade....

2016-01-22 Thread Kevin Burton
Not sure if this is a bug or not or kind of a *fuzzy* area. In 2.0 this worked fine. We have a bunch of automated scripts that go through and create tables... one per day. at midnight UTC our entire CQL went offline.. .took down our whole app. ;-/ The resolution was a full CQL shut down and th

Strategy / order for upgradesstables during rolling upgrade.

2016-01-21 Thread Kevin Burton
I think there are two strategies to upgradesstables after a release. We're doing a 2.0 to 2.1 upgrade (been procrastinating here). I think we can go with B below... Would you agree? Strategy A: - foreach server - upgrade to 2.1 - nodetool upgradesstables Strategy B: -

Re: Using cassandra a BLOB store / web cache.

2016-01-20 Thread Kevin Burton
There's also the 'support' issue.. C* is hard enough as it is... maybe you can bring in another system like ES or HDFS but the more you bring in the more your complexity REALLY goes through the roof. Better to keep things simple. I really like the chunking idea for C*... seems like an easy way to

Re: Using cassandra a BLOB store / web cache.

2016-01-19 Thread Kevin Burton
18, 2016 at 6:52 PM, Kevin Burton wrote: > >> Internally we have the need for a blob store for web content. It's >> MOSTLY key, ,value based but we'd like to have lookups by coarse grained >> tags. >> > > I know you know how to operate and scale MySQ

Using cassandra a BLOB store / web cache.

2016-01-18 Thread Kevin Burton
Internally we have the need for a blob store for web content. It's MOSTLY key, ,value based but we'd like to have lookups by coarse grained tags. This needs to store normal web content like HTML , CSS, JPEG, SVG, etc. Highly doubt that anything over 5MB would need to be stored. We also need the

Re: Cassandra is consuming a lot of disk space

2016-01-12 Thread Kevin O'Connor
Have you tried restarting? It's possible there's open file handles to sstables that have been compacted away. You can verify by doing lsof and grepping for DEL or deleted. If it's not that, you can run nodetool cleanup on each node to scan all of the sstables on disk and remove anything that it's

Re: compact/repair shouldn't compete for normal compaction resources.

2015-10-19 Thread Kevin Burton
this would resolve this problem. IF anyone else thinks this is an issue I'll create a JIRA. On Mon, Oct 19, 2015 at 3:38 PM, Robert Coli wrote: > On Mon, Oct 19, 2015 at 9:30 AM, Kevin Burton wrote: > >> I think the point I was trying to make is that on highly loaded boxes, >&g

Re: compact/repair shouldn't compete for normal compaction resources.

2015-10-19 Thread Kevin Burton
ogy, > delivering Apache Cassandra to the world’s most innovative enterprises. > Datastax is built to be agile, always-on, and predictably scalable to any > size. With more than 500 customers in 45 countries, DataStax is the > database technology and transactional backbone of choice for th

Re: Would we have data corruption if we bootstrapped 10 nodes at once?

2015-10-18 Thread Kevin Burton
if done on a single > node, is typically correctable with `nodetool repair`. > > If you do it on many nodes at once, it’s possible that the new nodes > could represent all 3 replicas of the data, but don’t physically have any > of that data, leading to missing records. > > >

compact/repair shouldn't compete for normal compaction resources.

2015-10-18 Thread Kevin Burton
I'm doing a big nodetool repair right now and I'm pretty sure the added overhead is impacting our performance. Shouldn't you be able to throttle repair so that normal compactions can use most of the resources? -- We’re hiring if you know of any awesome Java Devops or Linux Operations Engineers!

Re: Would we have data corruption if we bootstrapped 10 nodes at once?

2015-10-18 Thread Kevin Burton
An shit.. I think we're seeing corruption.. missing records :-/ On Sat, Oct 17, 2015 at 10:45 AM, Kevin Burton wrote: > We just migrated from a 30 node cluster to a 45 node cluster. (so 15 new > nodes) > > By default we have auto_boostrap = false > > so we just push ou

Re: reiserfs - DirectoryNotEmptyException

2015-10-17 Thread Kevin Burton
My advice is to not even consider anything else or make any other changes to your architecture until you get onto a modern and maintained filesystem. VERY VERY VERY few people are deploying anything on ReiserFS so you're going to be the first group encountering any problems. On Thu, Oct 15, 2015

Would we have data corruption if we bootstrapped 10 nodes at once?

2015-10-17 Thread Kevin Burton
auto boostrapped themselves EVEN though auto_bootstrap=false. We don't have any errors. Everything seems functional. I'm just worried about data loss. Thoughts? Kevin -- We’re hiring if you know of any awesome Java Devops or Linux Operations Engineers! Founder/CEO Spinn3r.com Location:

Post portem of a large Cassandra datacenter migration.

2015-10-09 Thread Kevin Burton
ess. However, it was easily 20-30x faster. This probably saved me about 5 hours of sleep! In hindsight, I'm not sure what we would have done differently. Maybe bought more boxes. Maybe upgraded to Cassandra 2.2 and probably java 8 as well. Setting up datacenter migration might have worked out

Re: Does failing to run "nodetool cleanup" end up causing more data to be transferred during bootstrapping?

2015-10-07 Thread Kevin Burton
e technology, > delivering Apache Cassandra to the world’s most innovative enterprises. > Datastax is built to be agile, always-on, and predictably scalable to any > size. With more than 500 customers in 45 countries, DataStax is the > database technology and transactional backbone of cho

Does failing to run "nodetool cleanup" end up causing more data to be transferred during bootstrapping?

2015-10-07 Thread Kevin Burton
Let's say I have 10 nodes, I add 5 more, if I fail to run nodetool cleanup, is excessive data transferred when I add the 6th node? IE do the existing nodes send more data to the 6th node? the documentation is unclear. It sounds like the biggest problem is that the existing data causes things to

Why can't nodetool status include a hostname?

2015-10-07 Thread Kevin Burton
I find it really frustrating that nodetool status doesn't include a hostname Makes it harder to track down problems. I realize it PRIMARILY uses the IP but perhaps cassandra.yml can include an optional 'hostname' parameter that can be set by the user. OR have the box itself include the hostname

Re: Maximum node decommission // bootstrap at once.

2015-10-06 Thread Kevin Burton
TCP tuning, > > On Tue, Oct 6, 2015 at 1:29 PM, Kevin Burton wrote: > >> I'm not sure which is faster/easier. Just joining one box at a time and >> then decommissioning or using replace_address. >> >> this stuff is always something you do rarely and then more comple

Re: Maximum node decommission // bootstrap at once.

2015-10-06 Thread Kevin Burton
rote: > On Tue, Oct 6, 2015 at 12:32 PM, Kevin Burton wrote: > >> How many nodes can we bootstrap at once? How many can we decommission? >> > > short answer : 1 node can join or part at simultaneously > > longer answer : https://issues.apache.org/jira/browse/CASSANDRA-2

Maximum node decommission // bootstrap at once.

2015-10-06 Thread Kevin Burton
How many can we decommission? I remember reading docs for this but hell if I can find it now :-P I know what the answer is theoretically. I just want to make sure we do everything properly. Kevin -- We’re hiring if you know of any awesome Java Devops or Linux Operations Engineers! Founder/

Re: Running Cassandra on Java 8 u60..

2015-09-27 Thread Kevin Burton
k JDK9 will be the one. > > On Sep 25, 2015, at 7:14 PM, Stefano Ortolani wrote: > > I think those were referring to Java7 and G1GC (early versions were buggy). > > Cheers, > Stefano > > > On Fri, Sep 25, 2015 at 5:08 PM, Kevin Burton wrote: > >> Any issu

Using inline JSON is 2-3x faster than using many columns (>20)

2015-09-26 Thread Kevin Burton
I wanted to share this with the community in the hopes that it might help someone with their schema design. I didn't get any red flags early on to limit the number of columns we use. If anything the community pushes for dynamic schema because Cassandra has super nice online ALTER TABLE. However,

Running Cassandra on Java 8 u60..

2015-09-25 Thread Kevin Burton
Any issues with running Cassandra 2.0.16 on Java 8? I remember there is long term advice on not changing the GC but not the underlying version of Java. Thoughts? -- We’re hiring if you know of any awesome Java Devops or Linux Operations Engineers! Founder/CEO Spinn3r.com Location: *San Francis

cassandra-stress on 3.0 with column widths benchmark.

2015-09-13 Thread Kevin Burton
I’m trying to benchmark two scenarios… 10 columns with 150 bytes each vs 150 columns with 10 bytes each. The total row “size” would be 1500 bytes (ignoring overhead). Our app uses 150 columns so I’m trying to see if packing it into a JSON structure using one column would improve performance.

Re: Best strategy for hiring from OSS communities.

2015-09-13 Thread Kevin Burton
should post to general@ … On Fri, Sep 11, 2015 at 5:34 PM, Otis Gospodnetić < otis.gospodne...@gmail.com> wrote: > Hey Kevin - I think there is j...@apache.org > > Otis > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch S

Re: Cassandra 2.2 for time series

2015-09-02 Thread Kevin Burton
Check out kairosd for a time series db on Cassandra. On Aug 31, 2015 7:12 AM, "Peter Lin" wrote: > > I didn't realize they had added max and min as stock functions. > > to get the sample time. you'll probably need to write a custom function. > google for it and you'll find people that have done i

Re: Practical limitations of too many columns/cells ?

2015-08-25 Thread Kevin Burton
ge this, but > it's good to have it on the radar. > > > On Sun, Aug 23, 2015 at 10:31 PM Kevin Burton wrote: > >> Agreed. We’re going to run a benchmark. Just realized we grew to 144 >> columns. Fun. Kind of disappointing that Cassandra is so slow in this &g

Re: Practical limitations of too many columns/cells ?

2015-08-23 Thread Kevin Burton
shows a ton of > different examples, but they’re not scientific, and at this point they’re > old versions (and performance varies version to version). > > - Jeff > > From: on behalf of Kevin Burton > Reply-To: "user@cassandra.apache.org" > Date:

Store JSON as text or UTF-8 encoded blobs?

2015-08-23 Thread Kevin Burton
Hey. I’m considering migrating my DB from using multiple columns to just 2 columns, with the second one being a JSON object. Is there going to be any real difference between TEXT or UTF-8 encoded BLOB? I guess it would probably be easier to get tools like spark to parse the object as JSON if it’

Re: Practical limitations of too many columns/cells ?

2015-08-23 Thread Kevin Burton
ile=nodes Averages from the middle 80% of > values:interval_op_rate : 23489 > > From: on behalf of Kevin Burton > Reply-To: "user@cassandra.apache.org" > Date: Sunday, August 23, 2015 at 1:02 PM > To: "user@cassandra.apache.org" > Subject: Practical limitation

Practical limitations of too many columns/cells ?

2015-08-23 Thread Kevin Burton
Is there any advantage to using say 40 columns per row vs using 2 columns (one for the pk and the other for data) and then shoving the data into a BLOB as a JSON object? To date, we’ve been just adding new columns. I profiled Cassandra and about 50% of the CPU time is spent on CPU doing compactio

Best strategy for hiring from OSS communities.

2015-08-13 Thread Kevin Burton
Mildly off topic but we are looking to hire someone with Cassandra experience.. I don’t necessarily want to spam the list though. We’d like someone from the community who contributes to Open Source, etc. Are there forums for Apache / Cassandra, etc for jobs? I couldn’t fine one. -- Founder/CE

Re: TTLs on tables with *only* primary keys?

2015-08-05 Thread Kevin Burton
RA-9312. > > On Tue, Aug 4, 2015 at 9:22 PM, Kevin Burton wrote: > >> I have a table which just has primary keys. >> >> basically: >> >> create table foo ( >> >> sequence bigint, >> signature text, >> primary key( sequ

TTLs on tables with *only* primary keys?

2015-08-04 Thread Kevin Burton
I have a table which just has primary keys. basically: create table foo ( sequence bigint, signature text, primary key( sequence, signature ) ) I need these to eventually get GCd however it doesn’t seem to work. If I then run: select ttl(sequence) from foo; I get: Cannot use sel

Configuring the java client to retry on write failure.

2015-07-12 Thread Kevin Burton
I can’t seem to find a decent resource to really explain this… Our app seems to fail some write requests, a VERY low percentage. I’d like to retry the write requests that fail due to number of replicas not being correct. http://docs.datastax.com/en/developer/java-driver/2.0/common/drivers/refere

Re: Lots of write timeouts and missing data during decomission/bootstrap

2015-07-01 Thread Kevin Burton
WOW.. nice. you rock!! On Wed, Jul 1, 2015 at 3:18 PM, Robert Coli wrote: > On Wed, Jul 1, 2015 at 2:58 PM, Kevin Burton wrote: > >> Looks like all of this is happening because we’re using CAS operations >> and the driver is going to SERIAL consistency level. >> ... &g

Re: Lots of write timeouts and missing data during decomission/bootstrap

2015-07-01 Thread Kevin Burton
in failures of CAS? This is Cassandra 2.0.9 btw. On Wed, Jul 1, 2015 at 2:22 PM, Kevin Burton wrote: > We get lots of write timeouts when we decommission a node. About 80% of > them are write timeout and just about 20% of them are read timeout. > > We’ve tried to adjust streamthrou

Lots of write timeouts and missing data during decomission/bootstrap

2015-07-01 Thread Kevin Burton
We get lots of write timeouts when we decommission a node. About 80% of them are write timeout and just about 20% of them are read timeout. We’ve tried to adjust streamthroughput (and compaction throughput) for that matter and that doesn’t resolve the issue. We’ve increased write_request_timeout

How the heck do we repair when migrating to 3 replicas on 2.0.x ?

2015-06-11 Thread Kevin Burton
We’re running Cassandra 2.0.9 and just migrated from 2-3 replicas. We changes our consistency level to 2 during this period while we’re running a repair. but we can’t figure out what command to run to repair our data We *think* we have to run “nodetool repair -pr” on each node.. is that right?

Tracking ETA and % complete in nodetool netstats during a decommission ?

2015-05-08 Thread Kevin Burton
I’m trying to track the throughput of nodetool decommission so I can figure out how long until this box is out of service. Basically, I want a % complete, and a ETA on when the job will be done. IS this possible? Without opscenter? -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog:

Re: Timeseries analysis using Cassandra and partition by date period

2015-04-05 Thread Kevin Burton
> Hi, I switched from HBase to Cassandra and try to find problem solution for timeseries analysis on top Cassandra. Depending on what you’re looking for, you might want to check out KairosDB. 0.95 beta2 just shipped yesterday as well so you have good timing. https://github.com/kairosdb/kairosdb

Best way to alert/monitor "nodetool status” down.

2015-03-06 Thread Kevin Burton
What’s the best way to monitor nodetool status being down? IE if a specific server things a node is down (DN). Does this just use JMX? IS there an API we can call? We want to tie it into our zabbix server so we can detect if here is failure. -- Founder/CEO Spinn3r.com Location: *San Francisco

Re: Fastest way to map/parallel read all values in a table?

2015-02-09 Thread Kevin Burton
I had considered using spark for this but: 1. we tried to deploy spark only to find out that it was missing a number of key things we need. 2. our app needs to shut down to release threads and resources. Spark doesn’t have support for this so all the workers would have stale thread leaking aft

Re: High GC activity on node with 4TB on data

2015-02-08 Thread Kevin Burton
Do you have a lot of individual tables? Or lots of small compactions? I think the general consensus is that (at least for Cassandra), 8GB heaps are ideal. If you have lots of small tables it’s a known anti-pattern (I believe) because the Cassandra internals could do a better job on handling the

Fastest way to map/parallel read all values in a table?

2015-02-08 Thread Kevin Burton
What’s the fastest way to map/parallel read all values in a table? Kind of like a mini map only job. I’m doing this to compute stats across our entire corpus. What I did to begin with was use token() and then spit it into the number of splits I needed. So I just took the total key range space w

Disabling the write ahead log with 2 data centers?

2015-01-23 Thread Kevin Burton
The WAL (and walls in general) impose a performance overhead. If one were to just take a machine out of the cluster, permanently, when a machine crashes, you could quickly get all the shards back up to N replicas after a node crashes. So realistically, running with a WAL is somewhat redundant. E

Re: number of replicas per data center?

2015-01-18 Thread Kevin Burton
Ah.. six replicas. At least its super inexpensive that way (sarcasm!) On Sun, Jan 18, 2015 at 8:14 PM, Jonathan Haddad wrote: > Sorry, I left out RF. Yes, I prefer 3 replicas in each datacenter, and > that's pretty common. > > > On Sun Jan 18 2015 at 8:02:12 PM Kevin Bur

Re: number of replicas per data center?

2015-01-18 Thread Kevin Burton
> > On Sun Jan 18 2015 at 7:52:10 PM Kevin Burton wrote: > >> How do people normally setup multiple data center replication in terms of >> number of *local* replicas? >> >> So say you have two data centers, do you have 2 local replicas, for a >> total of 4

number of replicas per data center?

2015-01-18 Thread Kevin Burton
How do people normally setup multiple data center replication in terms of number of *local* replicas? So say you have two data centers, do you have 2 local replicas, for a total of 4 replicas? Or do you have 2 in one datacenter, and 1 in another? If you only have one in a local datacenter then w

Re: "Not enough replica available” when consistency is ONE?

2015-01-18 Thread Kevin Burton
are > quorum-based ... > > This kicks in whenever you do CAS operations (eg, IF NOT EXISTS). > Otherwise a cluster which became network partitioned would end up being > able to have two separate CAS statements which both succeeded, but which > disagreed with each other. > > On

"Not enough replica available” when consistency is ONE?

2015-01-18 Thread Kevin Burton
I’m really confused here. I”m calling: acquireInsert.setConsistencyLevel( ConsistencyLevel.ONE ); but I”m still getting the exception: com.datastax.driver.core.exceptions.UnavailableException: Not enough replica available for query at consistency SERIAL (2 required but only 1 alive) Do

Re: is primary key( foo, bar) the same as primary key ( foo ) with a ‘set' of bars?

2015-01-01 Thread Kevin Burton
of data > 2) collections and maps are loaded entirely by Cassandra for each query, > whereas with clustering columns you can select a slice of columns > > > > On Thu, Jan 1, 2015 at 7:46 PM, Kevin Burton wrote: > >> I think the two tables are the same. Correct? >>

is primary key( foo, bar) the same as primary key ( foo ) with a ‘set' of bars?

2015-01-01 Thread Kevin Burton
I think the two tables are the same. Correct? create table foo ( source text, target text, primary key( source, target ) ) vs create table foo ( source text, target set, primary key( source ) ) … meaning that the first one, under the covers is represented the same as

Re: limit vs sample for indexing a small amount of data quickly?

2014-12-31 Thread Kevin Burton
, Dec 31, 2014 at 7:09 PM, Ganelin, Ilya wrote: > You want to use take() or takeOrdered. > > > > Sent with Good (www.good.com) > > > > -Original Message- > *From: *Kevin Burton [bur...@spinn3r.com] > *Sent: *Wednesday, December 31, 2014 10:02 P

bootstrapping manually when auto_bootstrap=false ?

2014-12-17 Thread Kevin Burton
I’m trying to figure out the best way to bootstrap our nodes. I *think* I want our nodes to be manually bootstrapped. This way an admin has to explicitly bring up the node in the cluster and I don’t have to worry about a script accidentally provisioning new nodes. The problem is HOW do you do it

Re: nodetool breaks on firewall ?

2014-12-13 Thread Kevin Burton
Fri, Dec 12, 2014 at 2:34 PM, Kevin Burton wrote: >> >> Oh. and if I specify —host it still doesn’t work. Very weird. >> >> On Fri, Dec 12, 2014 at 12:33 PM, Kevin Burton >> wrote: >> >>> OK..I’m stracing it and it’s definitely trying to connect to 173…

Re: nodetool breaks on firewall ?

2014-12-12 Thread Kevin Burton
be localhost anymore > > > https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java > > lines 87 and 88 > private static final String DEFAULT_HOST = "127.0.0.1"; > private static final int DEFAULT_PORT = 7199; >

Re: nodetool breaks on firewall ?

2014-12-12 Thread Kevin Burton
Oh. and if I specify —host it still doesn’t work. Very weird. On Fri, Dec 12, 2014 at 12:33 PM, Kevin Burton wrote: > OK..I’m stracing it and it’s definitely trying to connect to 173… here’s > the log line below. (anonymized). > > the question is why.. is cassandra configur

Re: nodetool breaks on firewall ?

2014-12-12 Thread Kevin Burton
gt; nodetool status -h 10.1.1.100 > > > > On Thu, Dec 11, 2014 at 6:38 PM, Kevin Burton wrote: >> >> I have a firewall I need to bring up to keep our boxes off the Internet >> (obviously). >> >> The problem is that once I do nodetool doesn’t work anym

nodetool breaks on firewall ?

2014-12-11 Thread Kevin Burton
I have a firewall I need to bring up to keep our boxes off the Internet (obviously). The problem is that once I do nodetool doesn’t work anymore. There’s a bunch of advice on this on the Internet: http://stackoverflow.com/questions/17430872/cassandra-1-2-nodetool-getting-failed-to-connect-when-t

does safe cassandra shutdown require disable binary?

2014-11-30 Thread Kevin Burton
I’m trying to figure out a safe way to do a rolling restart. http://devblog.michalski.im/2012/11/25/safe-cassandra-shutdown-and-restart/ It has the following command which make sense: root@cssa01:~# nodetool -h cssa01.michalski.im disablegossiproot@cssa01:~# nodetool -h cssa01.michalski.im disab

Re: RAM vs SSD for real world performance?

2014-11-25 Thread Kevin Burton
I imagine I’d generally be happy if we were CPU bound :-) … as long as the number of transactions per second is generally reasonable. On Tue, Nov 25, 2014 at 7:35 PM, Robert Coli wrote: > On Tue, Nov 25, 2014 at 5:31 PM, Kevin Burton wrote: > >> Curious what other people have

RAM vs SSD for real world performance?

2014-11-25 Thread Kevin Burton
The new SSDs that we have (as well as Fusion IO) in theory can saturate the gigabit ethernet port. The 4k random read and write IOs they’re doing now can easily add up quick and they’re faster than gigabit and even two gigabit. However, not all of that 4k is actually used. I suspect that on aver

Unsubscribe

2014-11-25 Thread Kevin Daly
Only BlueCat IPAM delivers true network intelligence, a smarter way to manage your network and devices. Read more at www.bluecatnetworks.com/networkintelligence . This e-mail and any attachments are for the sole use of the intended recipien

What causes NoHostAvailableException, WriteTimeoutException, and UnavailableException?

2014-11-24 Thread Kevin Burton
I’m trying to track down some exceptions in our production cluster. I bumped up our write load and now I’m getting a non-trivial number of these exceptions. Somewhere on the order of 100 per hour. All machines have a somewhat high CPU load because they’re doing other tasks. I’m worried that per

Re: IF NOT EXISTS on UPDATE statements?

2014-11-18 Thread Kevin Burton
. At least that’s the way I see it. Kevin -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts> <http://spinn3r.com>

Re: IF NOT EXISTS on UPDATE statements?

2014-11-17 Thread Kevin Burton
ed] > --- > True > > cqlsh:test> SELECT * FROM simple ; > > id | val > +- > 1 | new val > > (1 rows) > > On Tue, Nov 18, 2014 at 12:12 AM, Kevin Burton wrote: > >> >> "you can still do IF on UPDAT

Re: IF NOT EXISTS on UPDATE statements?

2014-11-17 Thread Kevin Burton
> "you can still do IF on UPDATE though… but it’s not possible to do IF > mycolumn IS NULL" --> If mycolumn = null should work > > Alas.. it doesn’t :-/ -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile

IF NOT EXISTS on UPDATE statements?

2014-11-17 Thread Kevin Burton
There’s still a lot of weirdness in CQL. For example, you can do an INSERT with an UPDATE .. .which I’m generally fine with. Kind of make sense. However, with INSERT you can do IF NOT EXISTS. … but you can’t do the same thing on UPDATE. So I foolishly wrote all my code assuming that INSERT/UPD

conditional batches across two tables?

2014-11-16 Thread Kevin Burton
I’m trying to have some code acquire a lock by first at performing a table mutation, and then if it wins, performing a second table insert. I don’t think this is possible with batches though. I don’t think I can say “update this table, and if you are able to set the value, and the value doesn’t a

Re: Reading the write time of each value in a set?

2014-11-16 Thread Kevin Burton
Nov 15 2014 at 12:51:55 AM DuyHai Doan > wrote: > >> Why don't you use map to store write time as value and data as key? >> Le 15 nov. 2014 00:24, "Kevin Burton" a écrit : >> >> I’m trying to build a histograph in CQL for various records. I’d like to

  1   2   3   4   >