Re: Cannot TRUNCATE

2013-11-25 Thread Robert Wille
stem nuke it and try again :) Was there more than one node at any time ? Does nodetool status show only one node ? Cheers - Aaron Morton New Zealand @aaronmorton Co-Founder & Principal Consultant Apache Cassandra Consulting http://www.thelastpickle.com On 21/11/2013,

WRITETIME() sometimes returns 0

2013-11-26 Thread Robert Wille
I've got a Cassandra 2.0.2 server with a single node. I've written a test harness that populates the database, looks at the write times for each column, runs the test cases, and then checks to see what columns have been updated. However, when I query WRITETIME() (prior to running the test cases), o

Re: Inefficiency with large set of small documents?

2013-11-28 Thread Robert Wille
I recently created a test database with about 400 million small records. The disk space consumed was about 30 GB, or 75 bytes per record. From: onlinespending Reply-To: Date: Monday, November 25, 2013 at 2:18 PM To: Subject: Inefficiency with large set of small documents? I¹m trying to de

Recommended amount of free disk space for compaction

2013-11-28 Thread Robert Wille
I¹m trying to estimate our disk space requirements and I¹m wondering about disk space required for compaction. My application mostly inserts new data and performs updates to existing data very infrequently, so there will be very few bytes removed by compaction. It seems that if a major compaction

Re: Exactly one wide row per node for a given CF?

2013-12-10 Thread Robert Wille
I have a question about this statement: When rows get above a few 10¹s of MB things can slow down, when they get above 50 MB they can be a pain, when they get above 100MB it¹s a warning sign. And when they get above 1GB, well you you don¹t want to know what happens then. I tested a data model t

Re: What is the fastest way to get data into Cassandra 2 from a Java application?

2013-12-11 Thread Robert Wille
I use hand-rolled batches a lot. You can get a *lot* of performance improvement. Just make sure to sanitize your strings. I¹ve been wondering, what¹s the limit, practical or hard, on the length of a query? Robert On 12/11/13, 3:37 AM, "David Tinker" wrote: >Yes thats what I found. > >This is f

Re: What is the fastest way to get data into Cassandra 2 from a Java application?

2013-12-11 Thread Robert Wille
Network latency is the reason why the batched query is fastest. One trip to Cassandra versus 1000. If you execute the inserts in parallel, then that eliminates the latency issue. From: Sylvain Lebresne Reply-To: Date: Wednesday, December 11, 2013 at 5:40 AM To: "user@cassandra.apache.org" S

Re: What is the fastest way to get data into Cassandra 2 from a Java application?

2013-12-11 Thread Robert Wille
: Date: Wednesday, December 11, 2013 at 6:52 AM To: "user@cassandra.apache.org" Subject: Re: What is the fastest way to get data into Cassandra 2 from a Java application? On Wed, Dec 11, 2013 at 1:52 PM, Robert Wille wrote: > Network latency is the reason why the batched que

Conference or training recommendations

2013-12-15 Thread Robert Wille
I¹d like to attend a conference or some form of training to become more proficient and knowledgable about Cassandra. Any suggestions?

Gotchas when creating a lot of tombstones

2014-01-01 Thread Robert Wille
I¹ve seen a number of people on this forum that have run into a variety of issues when creating a lot of tombstones. This makes me concerned about an application that I¹m writing. It works great on my test server, but I¹m concerned about what will happen when I move my production server to it. I¹m

Keyspaces on different volumes

2014-01-07 Thread Robert Wille
I¹d like to have my keyspaces on different volumes, so that some can be on SSD and others on spinning disk. Is such a thing possible or advisable?

Re: Keyspaces on different volumes

2014-01-07 Thread Robert Wille
: Date: Tuesday, January 7, 2014 at 6:07 AM To: Subject: Re: Keyspaces on different volumes That is a fine option and can make perfect sense if you have keyspaces with very different runtime characteristics. -Tupshin On Jan 7, 2014 7:30 AM, "Robert Wille" wrote: > I¹d li

Re: Read/Write consistency issue

2014-01-10 Thread Robert Wille
Cassandra is a last-write wins kind of a deal. The last write is determined by the timestamp. There are two problems with this: 1. If your clocks are not synchronized, you¹re totally screwed. Note that the 2nd and 3rd to last operations occurred just 2 milliseconds apart. A clock skew of 2 millisec

Re: Read/Write consistency issue

2014-01-10 Thread Robert Wille
: Date: Friday, January 10, 2014 at 3:28 PM To: "user@cassandra.apache.org" Subject: RE: Read/Write consistency issue That, or roll your own locking. Means multiple updates, but it works reliably. tc From: Robert Wille [mailto:rwi...@fold3.com] Sent: Friday, January 10, 2014

Re: Read/Write consistency issue

2014-01-10 Thread Robert Wille
Interested in knowing more on why read-before-write is an anti-pattern. In the next month or so, I intend to use Cassandra as a doc store. One very common operation will be to read the document, make a change, and write it back. These would be interactive users modifying their own documents, so rap

Re: Read/Write consistency issue

2014-01-10 Thread Robert Wille
#x27;s generally a Cassandra anti-pattern to do >>> read-before-write in any case, not just because of this issue. I'd agree >>> with Robert's suggestion earlier in this thread of writing each update >>> independently and aggregating on read. >>> >>> S

Re: Gotchas when creating a lot of tombstones

2014-01-10 Thread Robert Wille
not be reclaimed > with gc grace=0 because it has not compacted yet. For this you might want to > look at tombstone_threshold > > > On Wed, Jan 8, 2014 at 10:31 AM, Tyler Hobbs wrote: >> >> On Wed, Jan 1, 2014 at 7:53 AM, Robert Wille wrote: >>> >>&g

Re: java.lang.StackOverflowError with big IN list

2014-01-10 Thread Robert Wille
I had a problem in my code that produced a big IN list (several tens of thousands). I got a timeout error, not a stack overflow. 2.0.4 with java driver 2.0 rc3. From: Dave Brosius Reply-To: Date: Friday, January 10, 2014 at 5:53 PM To: Subject: Re: java.lang.StackOverflowError with big IN

Re: Upserting the same values multiple times

2014-01-21 Thread Robert Wille
No tombstones, just many copies of the same data until compaction occurs. From: Sanjeeth Kumar Reply-To: Date: Tuesday, January 21, 2014 at 8:37 PM To: Subject: Upserting the same values multiple times Hi, I have a table A, one of the fields of which is a text column called body. This

Fatal error with limit > 10000

2014-01-22 Thread Robert Wille
I have a table with a bunch of records that have 10,000 keys per partition key (not sure if that¹s the right terminology). Here¹s the schema: CREATE TABLE bdn_index_pub ( tshard VARCHAR, pord INT, ord INT, hpath VARCHAR, page BIGINT, PRIMARY KEY (tshard, pord) ) WITH gc_grace_seconds = 0;

Re: Any Limits on number of items in a collection column type

2014-01-22 Thread Robert Wille
Yes, I¹ve experienced this as well. It looks like you¹re getting the number of items inserted mod 64K. From: Manoj Khangaonkar Reply-To: Date: Wednesday, January 22, 2014 at 7:17 PM To: Subject: Any Limits on number of items in a collection column type Hi, On C* 2.0.0. 3 Node cluster. I

Re: Any Limits on number of items in a collection column type

2014-01-22 Thread Robert Wille
I didn¹t read your question properly. Collections are limited to 64K items, not 64K bytes per item. From: Manoj Khangaonkar Reply-To: Date: Wednesday, January 22, 2014 at 7:17 PM To: Subject: Any Limits on number of items in a collection column type Hi, On C* 2.0.0. 3 Node cluster. I ha

Heavy update dataset and compaction

2014-01-28 Thread Robert Wille
I have a dataset which is heavy on updates. The updates are actually performed by inserting new records and deleting the old ones the following day. Some records might be updated (replaced) a thousand times before they are finished. As I watch SSTables get created and compacted on my staging serve

Re: Heavy update dataset and compaction

2014-01-28 Thread Robert Wille
> > Perhaps a log structured database with immutable data files is not best suited > for this use case? Perhaps not, but I have other data structures I¹m moving to Cassandra as well. This is just the first. Cassandra has actually worked quite well for this first step, in spite of it not being an

GC taking a long time

2014-01-29 Thread Robert Wille
I read through the recent thread "Cassandra mad GC", which seemed very similar to my situation, but didn¹t really help. Here is what I get from my logs when I grep for GCInspector. Note that this is the middle of the night on a dev server, so there should have been almost no load. INFO [Schedule

Re: GC taking a long time

2014-01-29 Thread Robert Wille
d 1.7.0_17-b02) Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode) Thanks Robert From: Robert Wille Reply-To: Date: Wednesday, January 29, 2014 at 4:06 AM To: "user@cassandra.apache.org" Subject: GC taking a long time I read through the recent thread "Cassandra m

Lots of deletions results in death by GC

2014-02-01 Thread Robert Wille
A few days ago I posted about an issue I¹m having where GC takes a long time (20-30 seconds), and it happens repeatedly and basically no work gets done. I¹ve done further investigation, and I now believe that I know the cause. If I do a lot of deletes, it creates memory pressure until the memtables

Re: Lots of deletions results in death by GC

2014-02-04 Thread Robert Wille
d" increasing? If so, play around with turning up memtable_flush_writers and memtable_flush_queue_size and see if that helps. On Sat, Feb 1, 2014 at 9:03 AM, Robert Wille wrote: > A few days ago I posted about an issue I¹m having where GC takes a long time > (20-30 seconds), and it happ

Re: Lots of deletions results in death by GC

2014-02-05 Thread Robert Wille
exclusively deletes for this table? On 5 February 2014 00:10, Robert Wille wrote: > I ran my test again, and Flush Writer¹s ³All time blocked² increased to 2 and > then shortly thereafter GC went into its death spiral. I doubled > memtable_flush_writers (to 2) and memtable_flush_queue_s

Re: Lots of deletions results in death by GC

2014-02-05 Thread Robert Wille
lease window, so you'll either need to grab the development branch once it's committed or wait until 2.0.6 On 5 February 2014 15:09, Robert Wille wrote: > Yes. It¹s kind of an unusual workload. An insertion phase followed by a > deletion phase, generally not overlapping. > > Fr

Re: [OT]: Can I have a non-delivering subscription?

2014-02-22 Thread Robert Wille
Yeah, it¹s called a rule. Set one up to delete everything from user@cassandra.apache.org. On 2/22/14, 10:32 AM, "Paul "LeoNerd" Evans" wrote: >A question about the mailing list itself, rather than Cassandra. > >I've re-subscribed simply because I have to be subscribed in order to >send to the li

Re: Flushing after dropping a column family

2014-02-26 Thread Robert Wille
I use truncate between my test cases. Never had a problem with one test case inheriting the data from the previous one. I¹m using a single node, so that may be why. On 2/26/14, 9:27 AM, "Ben Hood" <0x6e6...@gmail.com> wrote: >On Wed, Feb 26, 2014 at 3:58 PM, DuyHai Doan wrote: >> Try truncate fo

Re: Securing Cassandra database

2014-04-05 Thread Robert Wille
Password protection doesn¹t protect against an engineer accidentally running test cases using the live config file instead of the test config file. To protect against that, our RDBMS system will only accept connections from certain IP addresses. Is there an equivalent thing in Cassandra, or should

Exception inserting UUID's into Cassandra using JDBC driver

2013-05-02 Thread Robert Wille
Just downloaded the latest Cassandra and JDBC driver. When I try to insert UUID's into a column family, I get an exception. Here's how I created the column family: CREATE TABLE browse.tree (tree_id UUID PRIMARY KEY, activation_time TIMESTAMP, tree_lock_id INT, sql_publication_id INT); Here's som

Iterating through large numbers of rows with JDBC

2013-05-11 Thread Robert Wille
I'm using the JDBC driver to access Cassandra. I'm wondering if its possible to iterate through a large number of records (e.g. to perform maintenance on a large column family). I tried calling Connection.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY), but it times out, so

Re: Iterating through large numbers of rows with JDBC

2013-05-12 Thread Robert Wille
/12/13 3:41 AM, "Oleg Dulin" wrote: >On 2013-05-11 14:42:32 +, Robert Wille said: > >> I'm using the JDBC driver to access Cassandra. I'm wondering if its >> possible to iterate through a large number of records (e.g. to perform >> maintenanc

Limit on the size of a list

2013-05-12 Thread Robert Wille
I designed a data model for my data that uses a list of UUID's in a column. When I designed my data model, my expectation was that most of the lists would have fewer than a hundred elements, with a few having several thousand. I discovered in my data a list that has nearly 400,000 items in it. When

Re: Unsubscribe?

2013-06-10 Thread Robert Wille
I unsubscribed a while ago and then resubscribed. It took about four unsubscribe attempts before it actually worked. From: "Fatih P." Reply-To: Date: Mon, 10 Jun 2013 17:46:30 +0300 To: Subject: Re: Unsubscribe? i tried the same and receiving mails. On Mon, Jun 10, 2013 at 5:34 PM, Luke

Re: very inefficient operation with tombstones

2013-07-02 Thread Robert Wille
I've seen the same thing From: Sylvain Lebresne Reply-To: Date: Tue, 2 Jul 2013 08:32:06 +0200 To: "user@cassandra.apache.org" Subject: Re: very inefficient operation with tombstones This is https://issues.apache.org/jira/browse/CASSANDRA-5677. -- Sylvain On Tue, Jul 2, 2013 at 6:04 AM

<    1   2