On Sun, Sep 29, 2019 at 9:42 AM DuyHai Doan <doanduy...@gmail.com> wrote:
> Thanks Jeff for sharing the ideas. I have some question though: > > - CQLSSTableWriter and explicitly break between windows --> Even if > you break between windows, If we have data worth of 1 years it would > requires us to use CQLSSTableWriter during 1 year (365 days) because > the write time taken into account when flushing to SSTable is the > current clock timestamp : > > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java#L252-L259 > Well, it's not obvious from that block of code alone if one cannot override the write time. But if that's the case, maybe you want to subclass it and extend for that possibility. What we're looking for is a way to load 1 year of data and forcing > write timestamp to the past so that the initial loading operation is > seen by TWCS as if we have loaded the data normally day by day during > 1 year > > - Use the normal write path for a single window at a time, explicitly > calling flush between windows. --> I don't understand how calling > flush will trigger windowing in TWCS, as far as I know, it is based on > write time. And by the way, can we load data using normal CQL and just > forcing the write time to be in the past so that TWCS will trigger > compaction properly ? > Exactly. If you go for this option, you should specify the write time explicitly by adding `USING TIMESTAMP :xxx` to your CQL statement. -- Alex
