Re: cassandra spark-connector-sqlcontext too many tasks

2018-03-17 Thread Ben Slater
KBs > for javaFunctions scenario > > Sent using Zoho Mail <https://www.zoho.com/mail/> > > > > -- *Ben Slater* *Chief Product Officer <https://www.instaclustr.com/>* <https://www.facebook.com/instaclustr> <https://twitter.com/instaclus

Re: Cassandra client tuning

2018-03-18 Thread Ben Slater
use createSStable mechanism. Is there any tuning which i'm missing > in client side, cause the server side is already tuned with datastax > recomendations. > > Sent using Zoho Mail <https://www.zoho.com/mail/> > > > -- *Ben Slater* *Chief Product Officer <https:

Re: Cassandra client tuning

2018-03-18 Thread Ben Slater
oughput but much less > CPU usage at client side. > > Sent using Zoho Mail <https://www.zoho.com/mail/> > > > On Sun, 18 Mar 2018 12:46:02 +0330 *Ben Slater > >* wrote > > You will probably find grouping writes into small batches improves overall >

Re: Cassandra client tuning

2018-03-18 Thread Ben Slater
Sent using Zoho Mail <https://www.zoho.com/mail/> > > > ---- On Sun, 18 Mar 2018 13:14:54 +0330 *Ben Slater > >* wrote > > When you say batch was worth than async in terms of throughput are you > comparing throughput with the same number of threads or something? I w

Re: Does Cassandra supports ACID txn

2018-04-19 Thread Ben Slater
meout you won’t have a way to >>> know if it worked or not. >>> >>> There is no way to rollback a statement/batch, the only way is to send >>> an update to modify the partition to its previous state. >>> >>> >>> >>> Regards, >>> >

Re: read repair with consistency one

2018-04-21 Thread Ben Slater
s it mean that in my case read repair does not work at >> all? Is there any way to enable read repair across DCs and stay will >> consistency ONE for reads? >> >> >> #1 https://www.datastax.com/dev/blog/common-mistakes-and-misconceptions >> #2 >> https://docs.datastax.co

Re: Does Cassandra supports ACID txn

2018-04-25 Thread Ben Slater
>> assumption is these operations would need to be atomic( and may be >> supporting other ACID properties) . How these kind of operations/usecases >> being handled in Cassandra/NoSQL world? >> >> Appreciate your response. >> >> Thanks, >> Rajesh >

Re: Basic Copy vs Snapshot for backup

2018-05-10 Thread Ben Slater
pany has taken reasonable precautions to > ensure no viruses are present in this email. The company cannot accept > responsibility for any loss or damage arising from the use of this email or > attachment." > -- *Ben Slater* *Chief Product Officer <https://www.i

Re: Interesting Results - Cassandra Benchmarks over Time Series Data for IoT Use Case I

2018-05-17 Thread Ben Slater
possibly improve this and share widely. > [1] Cassandra Benchmarks over Time Series Data for IoT Use Case > <https://sites.google.com/an10.io/timeseries-results> >https://sites.google.com/an10.io/timeseries-results > > > -- > Regards, > Arbab Khalil > Soft

Re: Certified Cassandra for Enterprise use

2018-05-29 Thread Ben Slater
Is there any third party who provides security patches/releases for Apache > cassandra > > For Enterprise use is there any third party who provides certified Apache > cassandra packages ?? > > Thanks > Pranay > -- *Ben Slater* *Chief Product Officer <https://

Re: Timeout for only one keyspace in cluster

2018-07-21 Thread Ben Slater
ngth_in_kb': '64', 'class': >> 'org.apache.cassandra.io.compress.LZ4Compressor'} >> >> AND crc_check_chance = 1.0 >> >> AND dclocal_read_repair_chance = 0.1 >> >> AND default_time_to_live = 0 >> >

Re: Re: [EXTERNAL] full text search on some text columns

2018-07-31 Thread Ben Slater
> Date : Wed, 01 Aug 2018 08:16:06 +0430 > Subject : Re: [EXTERNAL] full text search on some text columns > Forwarded message > > Maybe this plugin could do the job: > https://github.com/Stratio/cassandra-lucene-index > > On Tue, 31 Jul 2018 at 2

Re: Secure data

2018-08-01 Thread Ben Slater
e the keystore in kms >>> or vault. If any of you already configured please direct me to >>> documentation for it. >>> >> > > -- > Jon Haddad > http://www.rustyrazorblade.com > twitter: rustyrazorblade > -- *Ben Slater* *Chief Product Officer <h

Re: cold vs hot data

2018-09-13 Thread Ben Slater
o PDF e-mail accounts will be archived and may > be scanned by us and/or by external service providers to detect and prevent > threats to our systems, investigate illegal or inappropriate behavior, > and/or eliminate unsolicited promotional e-mails (“spam”). If you have any > concerns

Re: [ANNOUNCE] StratIO's Lucene plugin fork

2018-10-30 Thread Ben Slater
nyone out there > has established knowledge about the plugin we'd be grateful for any > assistance! > > You can find our fork here: > https://github.com/instaclustr/cassandra-lucene-index > At the moment, the only difference is that there is a 3.11.3 branch which > just h

Re: Multiple cluster for a single application

2018-11-07 Thread Ben Slater
t equal so easy calculations here) for >>> example inserts to table X would go to cluster Y >>> 2. shard data at loader level by some business logic grouping of data, >>> for example all rows with some column starting with X would go to cluster Y >>> >>> I would appreciate sh

Re: [EXTERNAL] Is Apache Cassandra supports Data at rest

2018-11-14 Thread Ben Slater
a destructive nature, which may be contained in this attachment > and shall not be liable for direct, indirect, consequential or special > damages in connection with this e-mail message or its attachment. > -- *Ben Slater* *Chief Product Officer <https://www.instaclustr.com/>*

Re: Cassandra single unreachable node causing total cluster outage

2018-11-27 Thread Ben Slater
In what way does the cluster become unstable (ie more specifically what are the symptoms)? My first thought would be the loss of the node causing the other nodes to become overloaded but that doesn’t seem to fit with your point 2. Cheers Ben --- *Ben Slater* *Chief Product Officer* <ht

Re: Authenticate cassandra-stress with cqlshrc

2019-01-08 Thread Ben Slater
(in which case I don’t think there is one but stress has a hell of a lot of options so I could be wrong). Cheers Ben --- *Ben Slater* *Chief Product Officer* <https://www.facebook.com/instaclustr> <https://twitter.com/instaclustr> <https://www.linkedin.com/company/instaclustr>

Re: Authenticate cassandra-stress with cqlshrc

2019-01-08 Thread Ben Slater
Yep, cassandra-stress doesn’t attempt to use the cqlshrc file. Seems to me it could be convenient so might make a nice contribution to the project. Cheers Ben --- *Ben Slater* *Chief Product Officer* <https://www.facebook.com/instaclustr> <https://twitter.com/instaclust

Re: How to read the Index.db file

2019-02-07 Thread Ben Slater
They don’t do exactly what you want but depending on why you are trying to get this info you might find our sstable-tools useful: https://github.com/instaclustr/cassandra-sstable-tools --- *Ben Slater* *Chief Product Officer* <https://www.facebook.com/instaclustr> <https://tw

Re: ***UNCHECKED*** Query regarding cassandra column write time set by client Timestamp Generator

2019-04-09 Thread Ben Slater
Not in the logs but I think you should be able to use the WRITETIME function to view via CQL (see https://cassandra.apache.org/doc/latest/cql/dml.html#select) Cheers Ben --- *Ben Slater* *Chief Product Officer* <https://www.facebook.com/instaclustr> <https://twitter.com/instaclust

Re: ***UNCHECKED*** Query regarding cassandra column write time set by client Timestamp Generator

2019-04-09 Thread Ben Slater
Maybe stabledump can help you? https://cassandra.apache.org/doc/4.0/tools/sstable/sstabledump.html --- *Ben Slater* *Chief Product Officer* <https://www.facebook.com/instaclustr> <https://twitter.com/instaclustr> <https://www.linkedin.com/company/instaclustr> Read our lat

Re: A cluster (RF=3) not recovering after two nodes are stopped

2019-04-23 Thread Ben Slater
Is it possible that stress is overloading node 1 so it’s not recovering state properly when node 2 comes up? Have you tried running with a lower load (say 2 or 3 threads)? Cheers Ben --- *Ben Slater* *Chief Product Officer* <https://www.facebook.com/instaclustr> <https://tw

Re: A cluster (RF=3) not recovering after two nodes are stopped

2019-04-25 Thread Ben Slater
Ben --- *Ben Slater**Chief Product Officer* <https://www.instaclustr.com/platform/> <https://www.facebook.com/instaclustr> <https://twitter.com/instaclustr> <https://www.linkedin.com/company/instaclustr> Read our latest technical blog posts here <https://www.in

Re: different query result after a rerun of the same query

2019-04-29 Thread Ben Slater
inserted (relative to your attempt to query it)? Cheers Ben --- *Ben Slater**Chief Product Officer* <https://www.instaclustr.com/platform/> <https://www.facebook.com/instaclustr> <https://twitter.com/instaclustr> <https://www.linkedin.com/company/instaclustr> Read our lat

Re: different query result after a rerun of the same query

2019-04-29 Thread Ben Slater
, changes in topology without following proper procedure, etc. Cheers Ben --- *Ben Slater**Chief Product Officer* <https://www.instaclustr.com/platform/> <https://www.facebook.com/instaclustr> <https://twitter.com/instaclustr> <https://www.linkedin.com/company/instaclu

Re: different query result after a rerun of the same query

2019-04-30 Thread Ben Slater
monitor the dropped mutations metrics. Cheers Ben --- *Ben Slater**Chief Product Officer* <https://www.instaclustr.com/platform/> <https://www.facebook.com/instaclustr> <https://twitter.com/instaclustr> <https://www.linkedin.com/company/instaclustr> Read our latest t

Re: CL=LQ, RF=3: Can a Write be Lost If Two Nodes ACK'ing it Die

2019-05-02 Thread Ben Slater
In scenario 2, if the row has been written to node 3 it will be replaced on the other nodes via rebuild or repair. --- *Ben Slater**Chief Product Officer* <https://www.instaclustr.com/platform/> <https://www.facebook.com/instaclustr> <https://twitter.com/inst

Re: CL=LQ, RF=3: Can a Write be Lost If Two Nodes ACK'ing it Die

2019-05-02 Thread Ben Slater
many not have made its way to the third node by the time the first two replicas are lost. If there is a replica on the third node it can be recovered to the other two nodes by either rebuild (actually replace) or repair. Cheers Ben --- *Ben Slater**Chief Product Officer* <ht

Re: CL=LQ, RF=3: Can a Write be Lost If Two Nodes ACK'ing it Die

2019-05-03 Thread Ben Slater
the two nodes dieing (although it’s very likely there will be three replicas from the start if the cluster is in a healthy state at the time of the write). Cheers Ben --- *Ben Slater**Chief Product Officer* <https://www.instaclustr.com/platform/> <https://www.facebook.com/instaclustr&

Re: high write latency on a single table

2019-07-22 Thread Ben Slater
Is the size of the data in your “state” column variable? The higher write latencies at the 95%+ could line up with large volumes of data for particular rows in that column (the one column not in both tables)? Cheers Ben --- *Ben Slater**Chief Product Officer* <https://www.instaclustr.

Re: Cassandra-stress testing

2019-08-20 Thread Ben Slater
://www.instaclustr.com/certified-apache-cassandra/ Cheers Ben --- *Ben Slater**Chief Product Officer* <https://www.instaclustr.com/platform/> <https://www.facebook.com/instaclustr> <https://twitter.com/instaclustr> <https://www.linkedin.com/company/instaclustr> Read our latest technica

Re: Cassandra-stress testing

2019-08-21 Thread Ben Slater
any useful response from the community. Cheers Ben --- *Ben Slater**Chief Product Officer* <https://www.instaclustr.com/platform/> <https://www.facebook.com/instaclustr> <https://twitter.com/instaclustr> <https://www.linkedin.com/company/instaclustr> Read our latest t

Re: Warning message even for batches targeting single partition

2016-05-08 Thread Ben Slater
]. > You should use a logged batch for atomicity, or asynchronous writes for > performance. > > If I have identified the partition and writing bunch of rows into that > using unlogged batch, isnt that an ideal case? > > Regards, > Bhuvan > -- Ben Slater Chief Product Officer, Instaclustr +61 437 929 798

Re: Accessing Cassandra data from Spark Shell

2016-05-09 Thread Ben Slater
; > Thanks, > LCassa > -- Ben Slater Chief Product Officer, Instaclustr +61 437 929 798

Re: Repair schedules for new clusters

2016-05-17 Thread Ben Slater
s this sound reasonable? Are there any pitfalls I should be aware of? > > Thanks, > Ashic. > -- Ben Slater Chief Product Officer, Instaclustr +61 437 929 798

Re: restore cassandra snapshots on a smaller cluster

2016-05-17 Thread Ben Slater
at 10:35 Luigi Tagliamonte wrote: > Hi everyone, > i'm wondering if it is possible to restore all the snapshots of a cluster > (10 nodes) in a smaller cluster (3 nodes)? If yes how to do it? > > -- > Luigi > --- > “The only way to get smarter is by playing a smarter

Re: Accessing Cassandra data from Spark Shell

2016-05-17 Thread Ben Slater
gt;> >> >> >> From this point onward, you have complete access to the DataFrame API. >> You can even register it as a temporary table, if you would prefer to use >> SQL/HiveQL. >> >> >> >> Mohammed >> >> Author: Big Data Analytics

Re: Recovery for deleted SSTables files for one column family.

2016-05-19 Thread Ben Slater
we do not have any backup of it. > > Thank you. > > Regards, > Mike Yeap > -- Ben Slater Chief Product Officer, Instaclustr +61 437 929 798

Re: Unable to use native library in C* trigger

2016-06-07 Thread Ben Slater
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_25] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) > [main/:na] > at > org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:106) [main/:na] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25] > > I am just running one, local node. > > Thanks, > Brian > -- Ben Slater Chief Product Officer, Instaclustr +61 437 929 798

Re: Empty snapshot created

2016-06-09 Thread Ben Slater
> 'max_threshold': '32', 'min_threshold': '4'} > AND compression = {'chunk_length_in_kb': '64', 'class': > 'org.apache.cassandra.io.compress.LZ4Compressor'} > AND crc_check_chance = 1.0 > AND dclocal_read_repair_chance = 0.1 > AND default_time_to_live = 0 > AND gc_grace_seconds = 864000 > AND max_index_interval = 2048 > AND memtable_flush_period_in_ms = 0 > AND min_index_interval = 128 > AND read_repair_chance = 0.0 > AND speculative_retry = '99PERCENTILE'; > > -- Ben Slater Chief Product Officer, Instaclustr +61 437 929 798

Re: java.lang.OutOfMemoryError: Java heap space

2016-06-10 Thread Ben Slater
d them but nothing seems appropriate. > > Debug log part 1: http://pastebin.com/b8ZSYtqV > Debug log part 2: http://pastebin.com/1Bbb7Vf8 > > Thanks for any suggestions, > Tobin > > -- Ben Slater Chief Product Officer, Instaclustr +61 437 929 798

Re: java.lang.OutOfMemoryError: Java heap space

2016-06-11 Thread Ben Slater
g on VMs with 2 cores and 4gig RAM. Neither they > nor the hypervisor are showing much load. > > Thanks for your help, > Tobin > > > On 10 Jun 2016, at 22:18, Ben Slater wrote: > > > > The short-term fix is probably to try increasing heap space (in > cassandra-env.

Re: java.lang.OutOfMemoryError: Java heap space

2016-06-11 Thread Ben Slater
I should add - there is probably an option (c) of fiddling with a bunch of tuning parameters to try to nurse things through with your current config but I’m not sure that’s useful unless you really need to make the current set up work for some reason. On Sun, 12 Jun 2016 at 15:23 Ben Slater

Re: Spark Memory Error - Not enough space to cache broadcast

2016-06-14 Thread Ben Slater
across 0 tasks(s)) = 2.6 GB. Storage limit = 2.6 GB. >> 16/06/13 21:26:02 WARN MemoryStore: Persisting block broadcast_69652 to disk >> instead. >> 16/06/13 21:26:02 INFO BlockManager: Found block rdd_100761_1 locally >> 16/06/13 21:26:02 INFO Executor: Finished task

Re: Data lost in Cassandra 3.5 single instance via Erlang driver

2016-06-14 Thread Ben Slater
>> 0x00d0: 4964 000a 7465 7374 746f Idtestto >> 0x00e0: 7069 6332 0002 0004 pic2 >> 0x00f0: 03e8 0000 0064 .d >> 17:16:24.834414 IP localhost.9042 > localhost.49128: Flags [P.], seq >> 14:27, ack 396, win 579, options [nop,nop,TS val 72982872 ecr 72982868], >> length 13 >> 0x: 4500 0041 f907 4000 4006 43ad 7f00 0001 E..A..@.@.C. >> 0x0010: 7f00 0001 2352 bfe8 bfe6 87a4 83eb 4bcb #RK. >> 0x0020: 8018 0243 fe35 0101 080a 0459 a158 ...C.5...Y.X >> 0x0030: 0459 a154 *8400 0800 0400 * .Y.T >> 0x0040: *01* . >> 17:16:24.834452 IP localhost.49128 > localhost.9042: Flags [.], ack 27, >> win 530, options [nop,nop,TS val 72982872 ecr 72982872], length 0 >> 0x: 4500 0034 370d 4000 4006 05b5 7f00 0001 E..47.@.@... >> 0x0010: 7f00 0001 bfe8 2352 83eb 4bcb bfe6 87b1 ..#R..K. >> 0x0020: 8010 0212 fe28 0101 080a 0459 a158 .(...Y.X >> 0x0030: 0459 a158.Y.X >> > > > Check the TCP flow, the data send and ack succeed, but not sure why the > data doesn't insert in Server side successful. Any tool or method can I > debug the Cassandra Server, or am I missing something? > > Thanks, > Linbo > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: how to force cassandra-stress to actually generate enough data

2016-06-15 Thread Ben Slater
tion of any offer to buy any financial product, or an official > statement of ICAP or its affiliates. Non-Transactable Pricing Terms and > Conditions apply to any non-transactable pricing provided. All terms and > conditions referenced herein available at www.icapterms.com. Please >

Re: Multi-DC Cluster w/ non-replicated Keyspace

2016-06-16 Thread Ben Slater
plicated > to the other? > > Whenever we add a new keyspace it seems to get replicated even if we don't > specify the other DC in the keyspace's NetworkTopologyStrategy. > > -J > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: Multi-DC Cluster w/ non-replicated Keyspace

2016-06-16 Thread Ben Slater
J > > On Thu, Jun 16, 2016 at 7:55 PM, Ben Slater > wrote: > >> Do you mean the data is getting replicated or just the schema? >> >> On Fri, 17 Jun 2016 at 11:48 Jason J. W. Williams < >> jasonjwwilli...@gmail.com> wrote: >> >>> Hi Guys, >

Re: Is my cluster normal?

2016-07-07 Thread Ben Slater
t;>>> fit into the row cache. >>>>> >>>>> >>>>> *...* >>>>> >>>>> >>>>> >>>>> *Daemeon C.M. ReiydelleUSA (+1) 415.501.0198 >>>>> <%28%2B1%29%20415.501.0198>Londo

Re: Re : Purging tombstones from a particular row in SSTable

2016-07-27 Thread Ben Slater
gt; >>> Also, we changed the tombstone_compaction_interval : 1800 >>> and tombstone_threshold : 0.1, but it did not help. >>> >>> >>> >>> thanks >>> >>> >>> >>> >>> >>> >>> >>>

Re: Memory leak and lockup on our 2.2.7 Cassandra cluster.

2016-08-03 Thread Ben Slater
Dcom.sun.management.jmxremote.ssl=false >> -Dcom.sun.management.jmxremote.authenticate=false >> -Djava.library.path=/usr/share/cassandra/lib/sigar-bin >> -XX:+UnlockCommercialFeatures >> -XX:+FlightRecorder >> -Dlogback.configurationFile=logback.xml >> -Dcassandra.logdir=/var/log/cassandra >> -Dcassandra.storagedir= >> -Dcassandra-pidfile=/var/run/cassandra/cassandra.pid >> >> >> -- >> >> We’re hiring if you know of any awesome Java Devops or Linux Operations >> Engineers! >> >> Founder/CEO Spinn3r.com >> Location: *San Francisco, CA* >> blog: http://burtonator.wordpress.com >> … or check out my Google+ profile >> <https://plus.google.com/102718274791889610666/posts> >> >> > > > -- > > We’re hiring if you know of any awesome Java Devops or Linux Operations > Engineers! > > Founder/CEO Spinn3r.com > Location: *San Francisco, CA* > blog: http://burtonator.wordpress.com > … or check out my Google+ profile > <https://plus.google.com/102718274791889610666/posts> > > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: [Marketing Mail] Re: Memory leak and lockup on our 2.2.7 Cassandra cluster.

2016-08-03 Thread Ben Slater
y the safest upgrade is 2.1 to 3.0.x > > Best, > > Romain > > > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: Read Repairs and CL

2016-08-28 Thread Ben Slater
for the read during the read repair > is based off the CL of the query. However I don't think that makes sense at > other CLs. Anyway, I just want to clarify what CL the read for the read > repair occurs at for cases where the overall query CL is not ALL. > > Thanks, > Kurt. >

Re: Bootstrapping multiple C* nodes in AWS

2016-08-29 Thread Ben Slater
xperiences here. > > Thanks > > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: Read Repairs and CL

2016-08-30 Thread Ben Slater
ce and speculative_retry > options. THe main gotcha is global read repair (via read_repair_chance) > which will trigger cross-dc repairs at CL ALL in the case of a digest > mismatch, even if the requested CL is DC-local. > > > On Sun, Aug 28, 2016 at 11:55 AM, Ben Slater > w

Re: ServerError: An unexpected error occurred server side; in cassandra java driver

2016-09-01 Thread Ben Slater
t; io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:419) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:391) > at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112) > at java.lang.Thread.run(Thread.java:745) > > Thanks > Siddharth > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: Streaming Process: How can we speed it up?

2016-09-15 Thread Ben Slater
end >> more than one file at a time is probably the most obvious area for >> improvement, and being able to better deal with the CPU / garbage generated >> on the receiving side is just behind that. You'll likely be able to stream >> 10-15 MB/s per sending server or cpu core, whichever is less (in a vnode >> setup, you'll be cpu bound - in a single-token setup, you'll be stream >> bound). >> >> >> > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: Export/Importing keyspace from a different sized cluster

2016-09-19 Thread Ben Slater
cluster to run > sstableloader, what methods can I use to load a single keyspace from one > cluster to another cluster of different size? > > > > Appreciate the help… > > > > Thanks, > > Justin > > > -- Ben Slater Chief Product Officer Instacl

Re: Question about replica and replication factor

2016-09-19 Thread Ben Slater
e each node has half data? But in the > calculator it show 1: You are really reading from 1 node every time. > >Any suggestions? Thanks! > > Jun > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: Question about replica and replication factor

2016-09-19 Thread Ben Slater
ecyrd.com/cassandracalculator/, > every node will store 50% of the data. > > When I want to read all data from the cluster, how many nodes should I > read from, 2 or 1? Is it 2, because each node has half data? But in the > calculator it show 1: You are really reading from 1 node every

Re: Question about replica and replication factor

2016-09-19 Thread Ben Slater
ending the full data we need, we need to chose > the best possible replica to ask, since if all we get back is checksums we > have nothing useful to return to the user. The dynamic snitch handles this > task by monitoring the performance of reads from the various replicas and > choosin

Re: Exceptions whenever compaction happens

2016-09-26 Thread Ben Slater
t > org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1797) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2466) > ~[apache-cassandra-3.0.9.jar:3.0.9] &g

Re: crash with OOM

2016-09-27 Thread Ben Slater
MemoryError: Java heap space > ERROR [SharedPool-Worker-157] 2016-09-26 20:51:14,124 Message.java:538 - > Unexpected exception during request; channel = [id: 0xce43a388, / > 13.13.13.80:55536 :> /13.13.13.149:9042] > > My server has total 256G memory so I set the MAX_HEAP

Re: ask for help about exmples of Data Types the document shows

2016-09-27 Thread Ben Slater
'landline' : { country_code: 1, number: '...' } } > } > 'work' : { > street: '1600 Pennsylvania Ave NW', > city: 'Washington', > zip: '20500', > phones: { 'fax' : { country_code: 1, number: '...' } } > } > }) > > error: > > SyntaxException: message="line 10:14 mismatched input 'work' expecting '}' (...: '...' } } > } ['wor]k' :...)"> > > Is the any suggestion about the problem 2? > > Best wishes for everyone, thank you for your watching ! > > -- > zha...@easemob.com > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: Re: ask for help about exmples of Data Types the document shows

2016-09-27 Thread Ben Slater
My best guess it that you need to remove the quotes from around the zip values (ie change if to zip: 20500 rather than zip: ‘20500’ ) as zip is defined as an int. Cheers Ben On Wed, 28 Sep 2016 at 14:38 zha...@easemob.com wrote: > Hi, Ben Slater, thank you very much for your replay! >

Re: failure node rejoin

2016-10-16 Thread Ben Slater
> I'm not sure this causes data lost. All data can be read properly just > after this rejoin. > But some rows are lost when I kill&restart C* for destructive tests after > this rejoin. > > Thanks. > > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: failure node rejoin

2016-10-17 Thread Ben Slater
ile, what you need > to do is to is remove the node first before doing "rejoin". > > Cheers, > > Yabin > > On Mon, Oct 17, 2016 at 1:48 AM, Ben Slater > wrote: > > To cassandra, the node where you deleted the files looks like a brand new > machine. It doesn

Re: failure node rejoin

2016-10-17 Thread Ben Slater
wiped, it's going to start again without a > bootstrap, and make a single node cluster or join an existing cluster if > the seed list is valid > > > > -- > Jeff Jirsa > > > On Oct 17, 2016, at 8:51 PM, Ben Slater > wrote: > > OK, sorry - I think understand

Re: failure node rejoin

2016-10-20 Thread Ben Slater
n C* node is restarted? (Currently, I > just restarted C* process) > > Regards, > > > On Tue, Oct 18, 2016 at 2:18 PM, Ben Slater > wrote: > > OK, that’s a bit more unexpected (to me at least) but I think the solution > of running a rebuild or repair still applies. &

Re: failure node rejoin

2016-10-20 Thread Ben Slater
> 2) What replication factor and replication strategy is used by the test > keyspace? What consistency level is used by your operations? > > - create keyspace testkeyspace WITH REPLICATION = > {'class':'SimpleStrategy','replication_factor':3}; >

Re: failure node rejoin

2016-10-20 Thread Ben Slater
? Cheers Ben On Fri, 21 Oct 2016 at 17:00 Yuji Ito wrote: > > Are you certain your tests don’t generate any overlapping inserts (by > PK)? > > Yes. The operation 2) also checks the number of rows just after all > insertions. > > > On Fri, Oct 21, 2016 at 2:51 PM, Ben S

Re: What is the maximum value of Cassandra Counter Column?

2016-10-23 Thread Ben Slater
http://cassandra.apache.org/doc/latest/cql/types.html?highlight=counter#counters On Sun, 23 Oct 2016 at 19:15 Kant Kodali wrote: > where does it say counter is implemented as long? > > On Sun, Oct 23, 2016 at 1:13 AM, Ali Akhtar wrote: > > Probably: > https://docs.oracle.com/javase/8/docs/api/j

Re: Hadoop vs Cassandra

2016-10-23 Thread Ben Slater
It’s reasonably common to use Cassandra to cover both online and analytics requirements, particularly using it in conjunction with Spark. You can use Cassandra’s multi-DC functionality to have online and analytics DCs for a reasonable degree of workload separation without having to build ETL (or so

Re: failure node rejoin

2016-10-23 Thread Ben Slater
ll report the result to you next Monday. > > Thanks > > > On Fri, Oct 21, 2016 at 3:05 PM, Ben Slater > wrote: > > Just to confirm, are you saying: > a) after operation 2, you select all and get 1000 rows > b) after operation 3 (which only does updates and read) you sel

Re: Lightweight transaction inside a batch : request rejected

2016-10-24 Thread Ben Slater
Move item_id_by_key into a collection field in item table? (Would probably be a “better” C* data model anyway.) On Tue, 25 Oct 2016 at 16:08 Mickael Delanoë wrote: > Ok, I understand, thanks. > So now i would like to know if there is some best practices to do what i > want. > I.e inserting entri

Re: Lightweight transaction inside a batch : request rejected

2016-10-24 Thread Ben Slater
://issues.apache.org/jira/browse/CASSANDRA-7396 Cheers Ben On Tue, 25 Oct 2016 at 16:25 Mickael Delanoë wrote: > I can't do this, otherwhise i won't be able to query the item_id using a > key with a query like : > Select * from item_id_by_key where user_id=... and key= > > Le

Re: Commercial Support Providers?

2016-11-03 Thread Ben Slater
I can confirm that we do offer support contracts for OSS Apache Cassandra at Instaclustr (in addition to our managed service) - either drop me an email direct (signature below) or contact sa...@instaclustr.com and would be happy to discuss details. Cheers Ben On Fri, 4 Nov 2016 at 14:02 Max C wr

Re: Is it a memory issue?

2016-11-06 Thread Ben Slater
This sounds to me like your writes go ahead of compactions trying to keep up which can eventually cause issues. Keep an eye on nodetool compactionstats if the number of compactions continually climbs then you are writing faster than Cassandra can actually process. If this is happening then you need

Re: Is it a memory issue?

2016-11-06 Thread Ben Slater
Compaction cargts > eventdata.eventdata_event_time_idx 32098562606 42616107664 bytes > 75.32% > Active compaction remaining time : 0h11m12s > > > *From:* Ben Slater > *Date:* 2016-11-07 11:41 > *To:* user > *Subject:* Re: Is it a memory issue? >

Re: large number of pending compactions, sstables steadily increasing

2016-11-07 Thread Ben Slater
What I’ve seen happen a number of times is you get in a negative feedback loop: not enough capacity to keep up with compactions (often triggered by repair or compaction hitting a large partition) -> more sstables -> more expensive reads -> even less capacity to keep up with compactions -> repeat T

Re: failure node rejoin

2016-11-08 Thread Ben Slater
. > If the flush failed on more than one node, the test would retry step 2.) > However, if so, the problem would happen without deleting Cassandra data. > > Regards, > yuji > > > On Mon, Oct 24, 2016 at 8:37 AM, Ben Slater > wrote: > > Definitely sounds to me like

Re: failure node rejoin

2016-11-10 Thread Ben Slater
at of 3.0?) > > Do you know other bug fixes related to commitlog? > > Regards > yuji > > On Wed, Nov 9, 2016 at 11:34 AM, Ben Slater > wrote: > > There have been a few commit log bugs around in the last couple of months > so perhaps you’ve hit something that was fixed recent

Re: cassandra python driver routing requests to one node?

2016-11-13 Thread Ben Slater
What load balancing policies are you using in your client code ( https://datastax.github.io/python-driver/api/cassandra/policies.html)? Cheers Ben On Mon, 14 Nov 2016 at 16:22 Andrew Bialecki wrote: > We have an odd situation where all of a sudden of our cluster started > seeing a disproportion

Re: failure node rejoin

2016-11-23 Thread Ben Slater
be more careful to add failure nodes to the existing cluster? > > Thanks, > yuji > > > On Fri, Nov 11, 2016 at 1:00 PM, Ben Slater > wrote: > > From a quick look I couldn’t find any defects other than the ones you’ve > found that seem potentially relevant to your issue (if any

Re: generate different sizes of request from single client

2016-11-24 Thread Ben Slater
If targetting two different tables for the different sizes works then I’ve submitted a patch for cassandra-stress that allows you to do that: https://issues.apache.org/jira/browse/CASSANDRA-8780 It would be nice to see someone else test it if you have the appetite to build it and try it out. Chee

Re: Does recovery continue after truncating a table?

2016-11-25 Thread Ben Slater
Nice detective work! Seems to me that it’s a best an undocumented limitation and potentially could be viewed as a bug - maybe log another JIRA? One node - there is a nodetool truncatehints command that could be used to clear out the hints ( http://cassandra.apache.org/doc/latest/tools/nodetool/tru

Re: Does recovery continue after truncating a table?

2016-11-26 Thread Ben Slater
finitely a bug to be solved asap. > > >Ben > What do you mean "an undocumented limitation" ? > > Thanks, > Hiro > > On Sat, Nov 26, 2016 at 3:13 PM, Ben Slater > wrote: > > Nice detective work! Seems to me that it’s a best an undocumented > limitatio

Re: Single cluster node restore

2016-11-29 Thread Ben Slater
You can have situations where rebuilding a node via streaming is painful and slow (generally because there is something bad about the data model like misused secondary indexes or massive partitions). Also, overstreaming can mean you need more disk space to bootstrap a node than you’ll require once

Re: All nodes hosting replicas down

2016-12-18 Thread Ben Slater
And I’m not aware of any case where it’s a good idea to use SimpleStrategy in Prod (be interested to hear if anyone else knows of one). NetworkTopologyStrategy behaves the same as SimpleStrategy in the basic case of one DC and Rack but gives you a good path to migrate to more sophisticated topologi

Re: Cassandra cluster performance

2016-12-21 Thread Ben Slater
You would expect some drop when moving to single multiple nodes but on the face of it that feels extreme to me (although I’ve never personally tested the difference). Some questions that might help provide an answer: - what consistency level are you using for the test? - what is your keyspace defin

Re: Cassandra cluster performance

2016-12-21 Thread Ben Slater
quot;*; > > > > - yes, the client is on separate VM > > - In our project we use Cassandra API version 3.0.2 but the database > (cluster) is version 3.9 > > - for 2node cluster: > > first VM: 25 GB RAM, 16 CPUs > > second VM: 16 GB RAM, 16 C

Re: Query

2016-12-29 Thread Ben Slater
I wasn’t familiar with Gizzard either so I thought I’d take a look. The first things on their github readme is: *NB: This project is currently not recommended as a base for new consumers.* (And no commits since 2013) So, Cassandra definitely looks like a better choice as your datastore for a new p

Re: [RELEASE] Apache Cassandra 3.10 released

2017-02-03 Thread Ben Slater
; [1]: (CHANGES.txt) https://goo.gl/J0VghF > [2]: (NEWS.txt) https://goo.gl/00KNVW > [3]: https://issues.apache.org/jira/browse/CASSANDRA > > > Great job all on this release. > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: Current data density limits with Open Source Cassandra

2017-02-08 Thread Ben Slater
format, is that 3-5TB still a valid limit? > > Does anyone have experience on running Cassandra with 3-5TB compressed > data ? > > Cheers, > Hannu -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: DELETE/SELECT with multi-column PK and IN

2017-02-09 Thread Ben Slater
%207161%203048806> · Fax +49 7161 304880-1 > <+49%207161%203048801> > AG Ulm · HRB 731058 · Managing Director: Jens Kammerer > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

Re: DELETE/SELECT with multi-column PK and IN

2017-02-09 Thread Ben Slater
ROM ks.cf WHERE (partitionkey1, partitionkey2) IN ((1, 2), (1, > 3), (2, 3), (3, 4)); > > If want to delete or select a bunch of records identified by their > multi-partitionkey tuples. > > 2017-02-09 10:18 GMT+01:00 Ben Slater : > > Are you looking this to be equivalent to (PK1=

Re: DELETE/SELECT with multi-column PK and IN

2017-02-09 Thread Ben Slater
e roughly equivalent? And probably > no more complex to generate in the client? > > On Thu, 9 Feb 2017 at 20:22 Benjamin Roth wrote: > > Maybe that makes it clear: > > DELETE FROM ks.cf WHERE (partitionkey1, partitionkey2) IN ((1, 2), (1, > 3), (2, 3), (3, 4)); > > If want

Re: Pluggable throttling of read and write queries

2017-02-20 Thread Ben Slater
ted experience I would > find this a terrible idea from the perspective of maintenance (remember the > old saying about basket and eggs?) > > What potential benefits do you see? > > Regards, > -- > Alex > > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798

  1   2   >