Vivek, By default, Cassandra uses ΒΌ of the system memory, so in your case, it will be around 8GB, which is fine.
If you have more Cassandra related question, it is better to post it on the Cassandra mailing list. Also feel free to email me directly. Mohammed Author: Big Data Analytics with Spark<http://www.amazon.com/Big-Data-Analytics-Spark-Practitioners/dp/1484209656/> From: Ted Yu [mailto:yuzhih...@gmail.com] Sent: Friday, January 22, 2016 6:37 PM To: vivek.meghanat...@wipro.com Cc: user Subject: Re: Spark Cassandra clusters I am not Cassandra developer :-) Can you use http://search-hadoop.com/ or ask on Cassandra mailing list. Cheers On Fri, Jan 22, 2016 at 6:35 PM, <vivek.meghanat...@wipro.com<mailto:vivek.meghanat...@wipro.com>> wrote: Thanks Ted, also what is the suggested memory setting for Cassandra process? Regards Vivek On Sat, Jan 23, 2016 at 7:57 am, Ted Yu <yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>> wrote: From your description, putting Cassandra daemon on Spark cluster should be feasible. One aspect to be measured is how much locality can be achieved in this setup - Cassandra is distributed NoSQL store. Cheers On Fri, Jan 22, 2016 at 6:13 PM, <vivek.meghanat...@wipro.com<mailto:vivek.meghanat...@wipro.com>> wrote: + spark standalone cluster On Sat, Jan 23, 2016 at 7:33 am, Vivek Meghanathan (WT01 - NEP) <vivek.meghanat...@wipro.com<mailto:vivek.meghanat...@wipro.com>> wrote: We have the setup on Google cloud platform. Each node has 8 CPU + 30GB memory. 10 nodes for spark another 9nodes for Cassandra. We are using spark 1.3.0 and Datastax bundle 4.5.9(which has 2.0.x Cassandra). Spark master and worker daemon uses Xmx & Xms 4G. We have not changed the default setting of Cassandra, should we be increasing the JVM memory? we have 9 streaming jobs the core usage varies from 2-6 and memory usage from 1 - 4 gb. We have budget to use higher CPU or higher memory systems hence was planning to have them together on more efficient nodes. Regards Vivek On Sat, Jan 23, 2016 at 7:13 am, Ted Yu <yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>> wrote: Can you give us a bit more information ? How much memory does each node have ? What's the current heap allocation for Cassandra process and executor ? Spark / Cassandra release you are using Thanks On Fri, Jan 22, 2016 at 5:37 PM, <vivek.meghanat...@wipro.com<mailto:vivek.meghanat...@wipro.com>> wrote: Hi All, What is the right spark Cassandra cluster setup - having Cassandra cluster and spark cluster in different nodes or they should be on same nodes. We are having them in different nodes and performance test shows very bad result for the spark streaming jobs. Please let us know. Regards Vivek The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com<http://www.wipro.com> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com<http://www.wipro.com> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com<http://www.wipro.com>
