Re: datamodelling

2019-02-05 Thread Jonathan Haddad
We (The Last Pickle) wrote a blog post on scaling time series: http://thelastpickle.com/blog/2017/08/02/time-series-data-modeling-massive-scale.html Rather than an agent_type, you can use a application determined bucket, so that agents with more data use more buckets. That'll keep your partition

Re: datamodelling

2019-02-05 Thread Bobbie Haynes
even if i try to create a agent_type it will be same issue again because agent_id and agent_type have same values... On Tue, Feb 5, 2019 at 11:36 AM Bobbie Haynes wrote: > unfortunately i do not have different of agents(agent_type) .. i only have > agent_id which is also a UUID type. > > On Tue,

Re: datamodelling

2019-02-05 Thread Bobbie Haynes
unfortunately i do not have different of agents(agent_type) .. i only have agent_id which is also a UUID type. On Tue, Feb 5, 2019 at 11:34 AM Nitan Kainth wrote: > You could consider a sudo column like agent_type and make it a compound > partition key. It will limit break your partition into sm

Re: datamodelling

2019-02-05 Thread Nitan Kainth
You could consider a sudo column like agent_type and make it a compound partition key. It will limit break your partition into smaller ones but you will have to query with agent_id and agent_type in that case. On Tue, Feb 5, 2019 at 12:59 PM Bobbie Haynes wrote: > Hi Everyone, >

Re: DataModelling to query date range

2016-03-24 Thread Vidur Malik
Hi Chris, I had something slightly different in mind. You would treat it as time series data, and have one record for each of the days the route was valid. In your case: start | end| valid New York Washington 2016-01-01 New York Washington 2016-01-02 New York

RE: DataModelling to query date range

2016-03-24 Thread Peer, Oded
init.co.uk] Sent: Thursday, March 24, 2016 9:40 AM To: user@cassandra.apache.org Subject: Re: DataModelling to query date range Ah- that looks interesting! I'm actaully still on cassandra 2.x but I was planning on updgrading anyway. Once I do so I'll check this one out. Chris On T

Re: DataModelling to query date range

2016-03-24 Thread Chris Martin
Ah- that looks interesting! I'm actaully still on cassandra 2.x but I was planning on updgrading anyway. Once I do so I'll check this one out. Chris On Thu, Mar 24, 2016 at 2:57 AM, Henry M wrote: > I haven't tried the new SASI indexer but it may help: > https://github.com/apache/cassandra/

Re: DataModelling to query date range

2016-03-24 Thread Chris Martin
Hi Vidur, I had a go at your solution but the problem is that it doesn't match routes which are valid all throughtout the range queried. For example if I have route that is valid for all of Jan 2016. I will have a table that looks something like this: start | end| vali

Re: DataModelling to query date range

2016-03-23 Thread Henry M
I haven't tried the new SASI indexer but it may help: https://github.com/apache/cassandra/blob/trunk/doc/SASI.md On Wed, Mar 23, 2016 at 2:08 PM, Chris Martin wrote: > Hi all, > > I have a table that represents a train timetable and looks a bit like this: > > CREATE TABLE routes ( > start text,

Re: DataModelling to query date range

2016-03-23 Thread Vidur Malik
Flip the problem over. Instead of storing validTo and validFrom, simply store a valid field and partition by (start, end). This may sound wasteful, but disk is cheap: CREATE TABLE routes ( start text, end text, valid timestamp, PRIMARY KEY ((start, end), valid) ); Now, you can execute something l