Tommaso, looking at your description of the architecture the idea came up. You can perform sharding on cassandra client and write to different cassandra clusters to keep the number of column families reasonable.
With best regards, Ilya On Thu, Jul 3, 2014 at 10:55 PM, tommaso barbugli <tbarbu...@gmail.com> wrote: > thank you for the replies; I am rethinking the schema design, one possible > solution is to "implode" one dimension and get N times less CFs. > With this approach I would come up with (cql) tables with up to 100 > columns; would that be a problem? > > Thank You, > Tommaso > > > 2014-07-02 23:43 GMT+02:00 Jack Krupansky <j...@basetechnology.com>: > > The official answer, engraved in stone tablets, and carried down from >> the mountain: “Although having more than dozens or hundreds of tables >> defined is almost certainly a Bad Idea (just as it is a design smell in a >> relational database), it's relatively straightforward to allow disabling >> the SlabAllocator.” Emphasis on “almost certainly a Bad Idea.” >> >> See: >> https://issues.apache.org/jira/browse/CASSANDRA-5935 >> “Allow disabling slab allocation” >> >> IOW, this is considered an anti-pattern, but... >> >> -- Jack Krupansky >> >> *From:* tommaso barbugli <tbarbu...@gmail.com> >> *Sent:* Wednesday, July 2, 2014 2:16 PM >> *To:* user@cassandra.apache.org >> *Subject:* Re: keyspace with hundreds of columnfamilies >> >> Hi, >> thank you for you replies on this; regarding the arena memory is this a >> fixed memory allocation or is some sort of in memory caching? I ask because >> I think that a substantial portion of the column families created will not >> be queried that frequently (and some will become inactive and stay like >> that really long time) >> >> Thank you, >> Tommaso >> >> >> 2014-07-02 18:35 GMT+02:00 Romain HARDOUIN <romain.hardo...@urssaf.fr>: >> >>> Arena allocation is an improvement feature, not a limitation. >>> It was introduced in Cassandra 1.0 in order to lower memory >>> fragmentation (and therefore promotion failure). >>> AFAIK It's not intended to be tweaked so it might not be a good idea to >>> change it. >>> >>> Best, >>> Romain >>> >>> tommaso barbugli <tbarbu...@gmail.com> a écrit sur 02/07/2014 17:40:18 : >>> >>> > De : tommaso barbugli <tbarbu...@gmail.com> >>> > A : user@cassandra.apache.org, >>> > Date : 02/07/2014 17:40 >>> > Objet : Re: keyspace with hundreds of columnfamilies >>> > >>> > 1MB per column family sounds pretty bad to me; is this something I >>> > can tweak/workaround somehow? >>> > >>> > Thanks >>> > Tommaso >>> > >>> >>> > 2014-07-02 17:21 GMT+02:00 Romain HARDOUIN <romain.hardo...@urssaf.fr >>> >: >>> > The trap is that each CF will consume 1 MB of memory due to arena >>> allocation. >>> > This might seem harmless but if you plan thousands of CF it means >>> > thousands of mega bytes... >>> > Up to 1,000 CF I think it could be doable, but not 10,000. >>> > >>> > Best, >>> > >>> > Romain >>> > >>> > >>> > tommaso barbugli <tbarbu...@gmail.com> a écrit sur 02/07/2014 >>> 10:13:41 : >>> > >>> > > De : tommaso barbugli <tbarbu...@gmail.com> >>> > > A : user@cassandra.apache.org, >>> > > Date : 02/07/2014 10:14 >>> > > Objet : keyspace with hundreds of columnfamilies >>> > > >>> > > Hi, >>> > > Are there any known issues, shortcomings about organising data in >>> > > hundreds of column families? >>> > > At this present I am running with 300 column families but I expect >>> > > that to get to a couple of thousands. >>> > > Is this something discouraged / unsupported (I am using Cassandra >>> 2.0). >>> > > >>> > > Thanks >>> > > Tommaso >>> >> >> > >
