Re: Maximum number of columns in a table

I listened to a talk about the new Cassandra 3 file (sstable) format. One takeaway was that the new format supports sparse data better. That is, is you have 2000 columns, but only setting some of the columns, the disk usage will be much less. Cheers, Jens On Thu, Sep 15, 2016 at 10:24 PM Dorian H

Re: Maximum number of columns in a table

@DuyHai I know they don't support. I need key+value mapping, not just "values" or just "keys". I'll use the lucene index. On Thu, Sep 15, 2016 at 10:23 PM, DuyHai Doan wrote: > I'd advise anyone against using the old native secondary index ... You'll > get poor performance (that's the main re

Re: Maximum number of columns in a table

I do agree on that. > On 15 Sep 2016, at 16:23, DuyHai Doan wrote: > > I'd advise anyone against using the old native secondary index ... You'll get > poor performance (that's the main reason why some people developed SASI). > > On Thu, Sep 15, 2016 at 10:20 PM, Hannu Kröger

Re: Maximum number of columns in a table

I'd advise anyone against using the old native secondary index ... You'll get poor performance (that's the main reason why some people developed SASI). On Thu, Sep 15, 2016 at 10:20 PM, Hannu Kröger wrote: > Hi, > > The ‘old-fashioned’ secondary indexes do support index of collection > values: >

Re: Maximum number of columns in a table

Hi, The ‘old-fashioned’ secondary indexes do support index of collection values: https://docs.datastax.com/en/cql/3.1/cql/ddl/ddlIndexColl.html Br, Hannu > On 15 Sep 2016, at 15:59, DuyHai Doan wrote: > > "But the problem is I c

Re: Maximum number of columns in a table

"But the problem is I can't use secondary indexing "where int25=5", while with normal columns I can." You have many objectives that contradict themselves in term of impl. Right now you're unlucky, SASI does not support indexing collections yet (it may come in future, when ? ¯\_(ツ)_/¯ ) If you'r

Re: Maximum number of columns in a table

Yes that makes more sense. But the problem is I can't use secondary indexing "where int25=5", while with normal columns I can. On Thu, Sep 15, 2016 at 8:23 PM, sfesc...@gmail.com wrote: > I agree a single blob would also work (I do that in some cases). The > reason for the map is if you need mor

Re: Maximum number of columns in a table

I agree a single blob would also work (I do that in some cases). The reason for the map is if you need more flexible updating. I think your solution of a map/data type works well. On Thu, Sep 15, 2016 at 11:10 AM DuyHai Doan wrote: > "But I need rows together to work with them (indexing etc)" >

Re: Maximum number of columns in a table

"But I need rows together to work with them (indexing etc)" What do you mean rows together ? You mean that you want to fetch a single row instead of 1 row per property right ? In this case, the map might be the solution: CREATE TABLE generic_with_maps( object_id uuid boolean_map map tex

Re: Maximum number of columns in a table

@DuyHai Yes, that's another case, the "entity" model used in rdbms. But I need rows together to work with them (indexing etc). @sfespace The map is needed when you have a dynamic schema. I don't have a dynamic schema (may have, and will use the map if I do). I just have thousands of schemas. One u

Re: Maximum number of columns in a table

"Another possible alternative is to use a single map column" --> how do you manage the different types then ? Because maps in Cassandra are strongly typed Unless you set the type of map value to blob, in this case you might as well store all the object as a single blob column On Thu, Sep 15, 201

Re: Maximum number of columns in a table

Another possible alternative is to use a single map column. On Thu, Sep 15, 2016 at 7:19 AM Dorian Hoxha wrote: > Since I will only have 1 table with that many columns, and the other > tables will be "normal" tables with max 30 columns, and the memory of 2K > columns won't be that big, I'm gonna

Re: Maximum number of columns in a table

"The data model is too dynamic" --> then create a table to cope with dynamic data types. Example CREATE TABLE dynamic_data( object_id uuid, property_name text, property_type text, bool_value boolean, long_value bigint, decimal_value double, text_value text,

Re: Maximum number of columns in a table

Since I will only have 1 table with that many columns, and the other tables will be "normal" tables with max 30 columns, and the memory of 2K columns won't be that big, I'm gonna guess I'll be fine. The data model is too dynamic, the alternative would be to create a table for each user which will

Re: Maximum number of columns in a table

There is no real limit in term of number of columns in a table, I would say that the impact of having a lot of columns is the amount of meta data C* needs to keep in memory for encoding/decoding each row. Now, if you have a table with 1000+ columns, the problem is probably your data model... On T

Maximum number of columns in a table

Is there alot of overhead with having a big number of columns in a table ? Not unbounded, but say, would 2000 be a problem(I think that's the maximum I'll need) ? Thank You