Re: Backup strategy

Snapshot would flush your memtable to disk and you could stream your sstables out. Incremental backups would be the differences that have occurred since your last snapshot as far as I'm aware. Since it's reasonably unfeasible to constantly stream out full snapshots (depending on the density of yo

Re: Backup strategy

Periodic snapshots + incremental backups I think are pretty good in terms of restoring to point in time. But you must manage cleaning up your snapshots + incremental backups on your own. I believe that tablesnap ( https://github.com/JeremyGrosser/tablesnap) is a pretty decent approach in terms of

Re: Backup strategy

On Thu, Jun 16, 2016 at 7:30 PM, Bhuvan Rawal wrote: > 2. Snapshotting : Hardlinks of sstables will get created. This is a very > fast process and latest data is captured into sstables after flushing > memtables, snapshots will be created in snapshots directory. But snapshot > does not provide you

Re: Backup strategy

Also if we talk about backup strategy for Cassandra Data then essentially there are couple of strategies that are adopted: 1. Incremental Backups. The old sstables will remain inside a backup directory and can be shipped to a storage location like AWS Glacier, etc. 2. Snapshotting : Hardlinks of s

Re: Backup strategy

What kind of data are we talking here? Is it time series data with infrequent updates and only inserts or frequently updated data. How frequently is old data read. I ask this because your Node size planning and Compaction Strategy will essentially depend on these. I have known people go upto 3-5 T

Re: Backup strategy

Bhuvan, Thanks for the info but actually I'm not looking for migration strategy. just want to backup strategy and retention policy best practices Thanks, Vasu > On Jun 16, 2016, at 6:51 PM, Bhuvan Rawal wrote: > > Hi Vasu, > > Planet Cassandra has a documentation page for basic info about m

Re: Backup strategy

Hi Vasu, Planet Cassandra has a documentation page for basic info about migrating to cassandra from MySQL. What to expect and what not to. It can be found here . I had a look at this slide

Re: Backup strategy

Thanks for sharing tablesnap. It's just what I have been looking for. On Thu, Nov 7, 2013 at 5:10 PM, Robert Coli wrote: > On Thu, Nov 7, 2013 at 6:28 AM, Sridhar Chellappa > wrote: > >> Yes. I am taking a Snapshot and then offloading the full data into S3. >> How will Table Snap help? >> >

Re: Backup strategy

On Thu, Nov 7, 2013 at 6:28 AM, Sridhar Chellappa wrote: > Yes. I am taking a Snapshot and then offloading the full data into S3. > How will Table Snap help? > As I detailed in my previous mail : 1) incremental style backup, instead of snapshot + full 2) tracks meta information about backup set

Re: Backup strategy

Yes. I am taking a Snapshot and then offloading the full data into S3. How will Table Snap help? On Wed, Nov 6, 2013 at 6:57 AM, Robert Coli wrote: > On Tue, Nov 5, 2013 at 4:36 PM, Sridhar Chellappa > wrote: > >> >>1. *If not, what is the right backup strategy ?* >> >> You didn't specif

Re: Backup strategy

On Tue, Nov 5, 2013 at 4:36 PM, Sridhar Chellappa wrote: > >1. *If not, what is the right backup strategy ?* > > You didn't specify, but it sounds like you are doing a snapshot and then a full offhost backup of the sstables? Perhaps instead of point in time full backups, you could continuous

Re: Backup strategy

I don't understand how the creation of a snapshot causes any load whatsoever. By definition, a snapshot is a hard link of an existing SSTable. The SSTable is not being physically copied so there is no disk I/O, it's just a reference to an inode. -- Ray //o-o\\ On Tue, Nov 5, 2013 at 8:09 PM, A

Re: Backup strategy

Why not just have a small DC/ring of nodes which you just do your snapshots/backups from? I wouldn't take nodes offline from the ring just to back them up. The other option is to add sufficient nodes to handle your existing request I/O + backups. Sounds like you might be already I/O constrained.

Re: backup strategy

Assuming you are using the SimpleStrategy or the NetworkTopologyStrategy and one rack per DC. If you backed up every 2nd node you would get one copy *IF* all nodes were consistent on disk. That can be a reasonably large if that you need to monitor. It's easier to back up all the nodes it will a

Re: Backup Strategy

On 11/9/10 5:15 AM, Wayne wrote: We are trying to use snapshots etc. to back up the data but it is slow (hours) and slows down the entire node. The snapshot process (as I understand it, and with the caveat that this is the code path without JNA available) first flushes all memtables (this can

Re: Backup Strategy

Thanks for the details. I think we were slowly starting to realize a similar pattern, but you definitely helped fill in the gaps: home brew rsync with lzop in the middle. We have raid1 system/commit log drives we are copying to once a day, and off cluster...maybe once a week. Thanks On Tue, Nov

Re: Backup Strategy

On Tue, Nov 9, 2010 at 8:15 AM, Wayne wrote: > I got some very good advice on manual compaction so I thought I would throw > out another question on raid/backup strategies for production clusters. > > We are debating going with raid 0 vs. raid 10 on our nodes for data storage. > Currently all stor