Re: [DISCUSS] CEP-23: Enhancement for Sparse Data Serialization

2022-09-08 Thread Benedict
I was referring to Column*s.*Serializer, which has serializeSubset methods. > On 8 Sep 2022, at 07:07, Claude Warren via dev > wrote: > > I have looked through the code mentioned. What I found in the > ColumnSerializer was the use of VInt encoding. Are you proposing switching > directly to

Re: [DISCUSS] CEP-23: Enhancement for Sparse Data Serialization

2022-09-07 Thread Claude Warren via dev
I have looked through the code mentioned.  What I found in the ColumnSerializer was the use of VInt encoding.  Are you proposing switching directly to VInt encoding for sizes rather than one of the other encodings?  Using a -2 as the first length to signal that the new encoding is in use so tha

Re: [DISCUSS] CEP-23: Enhancement for Sparse Data Serialization

2022-09-06 Thread Benedict
So, looking more closely at your proposal I realise what you are trying to do. The thing that threw me was your mention of lists and other collections. This will likely not work as there is no index that is possible to define on a list (or other collection) within a single sstable - a list is de

Re: [DISCUSS] CEP-23: Enhancement for Sparse Data Serialization

2022-09-06 Thread Josh McKenzie
> if that is standard for this project I will move the information there. It is. I'd go to a CEP if you have something you think might be controversial (due to design, size, whatever) and you want to get early consensus on before going to deep on implementation. I'm in favor of JIRA + DISCUSS (+

Re: [DISCUSS] CEP-23: Enhancement for Sparse Data Serialization

2022-09-05 Thread Claude Warren via dev
I am just learning the ropes here so perhaps it is not CEP worthy.  That being said, It felt like there was a lot of information to put into and track in a ticket, particularly when I expected discussion about how to best encode, changes to the algorithms etc.  It feels like it would be difficu

Re: [DISCUSS] CEP-23: Enhancement for Sparse Data Serialization

2022-09-05 Thread Abe Ratnofsky
Looking at this link: https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-23%3A++Enhancement+for+Sparse+Data+Serialization Do you have any plans to include benchmarks in your test plan? It would be useful to include disk usage / read performance / write performance comparisons with the ne

Re: [DISCUSS] CEP-23: Enhancement for Sparse Data Serialization

2022-09-05 Thread Josh McKenzie
Could you post a link to that? I don't see it on the wiki: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652201 On Mon, Sep 5, 2022, at 4:57 AM, Claude Warren via dev wrote: > I have just posted a CEP covering an Enhancement for Sparse Data > Serialzation. This is in respo

[DISCUSS] CEP-23: Enhancement for Sparse Data Serialization

2022-09-05 Thread Claude Warren via dev
I have just posted a CEP  covering an Enhancement for Sparse Data Serialzation.  This is in response to CASSANDRA-8959 I look forward to responses.