One point I would like to add to this; I was already looking into how to extend this but what I saw in SSTableReader was that it is very much "file system oriented". There was not any possibility to actually hook something like that there. I think what importing does is that it will use SSTableReader / Writer stuff so I think that the modification of these classes to accommodate this idea would be necessary.
On Fri, 22 Oct 2021 at 11:02, Stefan Miklosovic <stefan.mikloso...@instaclustr.com> wrote: > > Hi Jacek, > > Thanks for taking the lead on this. > > There was importing of SSTables introduced in 4.0 via > StorageService#importNewSSTables. The "problem" with this is that > SSTables need to be physically located at disk so Cassandra can read > them. If a backup is taken and SSTables are uploaded to, for example, > S3 bucket, then upon restore, all these SSTables need to be downloaded > first and then imported. What about downloading them / importing them > directly from S3? Or any custom source for that matter? Importing of > SSTables is a very nice feature in 4.0, we do not need to copy / hard > link / refresh, it is all handled internally. > > I am not sure if your work is related to this idea but I would > appreciate it if this is pluggable as well for the sake of simplicity > and effectiveness as we would not have to download all sstables before > importing them. > > If it is not related, feel free to skip that completely and I guess I > would have to try to push that forward myself. > > Regards > > > On Fri, 22 Oct 2021 at 10:24, Jacek Lewandowski > <lewandowski.ja...@gmail.com> wrote: > > > > I'd like to start a discussion about SSTable format API proposal (CEP-17) > > > > Jira: https://issues.apache.org/jira/browse/CASSANDRA-17056 > > CEP: > > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-17%3A+SSTable+format+API > > > > Thanks, > > Jacek > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
