Hi all, I sincerely want to thank everyone who chimed in on this thread and the slack conversation.
I want to make sure that I cover the background on CEP-1 for posterity's sake. Cassandra Sidecar was proposed as a component / sub-project prior to the existence of CEP. The discussion around this proposal exposed the need for a formal process to adopt such large changes. We first achieved consensus on the creation of an official Cassandra Sidecar sub-project[1]. Subsequently we discussed the creation of a process and governance structure so future sub-projects can have a paved path[2]. This came almost a year _after_ the Sidecar was adopted. This process was named CEP (Fun fact: We actually started off by calling it CIP[3]). C* Sidecar (Management Process) was therefore named 'CEP-1'. I agree with the approach Joey laid out to move forward here. CEP-1's scope is broad and we're working towards it and it might take some time but in the meanwhile we have a lot of functionality that is usable and adds value so we can release it incrementally. The major functionality here being CEP-28 (Reading and Writing Cassandra Data with Spark Bulk Analytics) and CEP-44 (Kafka integration for Cassandra CDC using Sidecar). We're actively working to merge the latter in and address any gaps. Thanks, Dinesh [1] https://lists.apache.org/thread/xyg8n5hkt7xrfqv48k91tx1jwp0pvcpw [2] https://lists.apache.org/thread/3hkghj3h44fvn3rq2b7b3gdwzxbdzmzv [3] https://lists.apache.org/thread/ljr4j20qgtbz6w27hxsr3p0mkv58jb73 On Sat, Nov 16, 2024 at 6:56 PM Joseph Lynch <joe.e.ly...@gmail.com> wrote: > Just bringing this back to the list. Patrick, Francisco, Jon and I > discussed this a bit over in slack [1] and I think we got to a rough > consensus that: > > 1. We're going to leave CEP-1 archived, if someone calls out functionality > in the current sidecar code they think is problematic we'll open a > discuss thread or if needed a CEP for that particular issue. > 2. Francisco et. al. will focus on closing the identified issues (the > jiras previously mentioned), and work towards a standard release vote - a > summary of what is included in the release will likely be helpful to that > vote. > 3. Francisco, myself and Patrick will collaborate on a blog post about > what is being released when we are closer to that release so the community > can learn more about it. > > If folks are concerned with this course of action please speak up! > > Cheers! > -Joey > > [1] https://the-asf.slack.com/archives/CK23JSY2K/p1731775603199999 > > On Sat, Nov 16, 2024 at 10:20 AM Joseph Lynch <joe.e.ly...@gmail.com> > wrote: > >> Oh I didn't agree we needed a new CEP, I thought we were agreeing on >> focusing on releasing the sidecar as is. CEP-1 was already voted on, we >> built consensus on the controversial part (having functionality outside the >> main process) and developers already started the project many years ago and >> teams are already testing and using the pre-release version in production. >> Releasing it should be like releasing any other C* project at this point (a >> release vote is called). This isn't a donation, it has been in the project >> from the beginning. >> >> I agreed that _if_ there were new features requiring a CEP, similar to >> any other new feature big enough for a CEP we should have one - but fixing >> bugs, adding documentation and closing (already existing) JIRAs with >> minimal blockers so we can get to a release vote doesn't seem to qualify to >> me. Which of those JIRAs do you think justifies a CEP Patrick? >> >> When we are closer to that vote thread, if PMC members feel they can't >> vote on that release without a new summary, let's clearly state that >> expectation, but I personally don't think we need a new CEP to vote on >> releasing the code that already exists in a project that was already >> approved. >> >> -Joey >> >> On Sat, Nov 16, 2024 at 10:03 AM Josh McKenzie <jmcken...@apache.org> >> wrote: >> >>> The process we all agreed to is being avoided, so I hope we can get back >>> on track here. >>> >>> Words matter. I don't think anyone is avoiding anything here, I think >>> CEP-1 was a first draft, way too large, and it's been chaotic and confusing >>> to transition from that to where we are now with a realistic scope of >>> things being considered for a cut of the sidecar release. >>> >>> So to clarify what we all agreed on (which I don't remember the details >>> of already; a month in 2024 is roughly 10 dog-years or so /cry) so I'm >>> going to "guess": >>> >>> >>> 1. We draft a new CEP talking about the scope for the actual first >>> sidecar release >>> 2. We discuss it on the dev list >>> 3. We vote >>> >>> That about right? I'd only advocate for doing this for this release >>> because it's superseding the old CEP-1, and don't expect any process like >>> this for subsequent releases. Just my .02; receptive to other opinions and >>> perspectives as always. >>> >>> On Fri, Nov 15, 2024, at 6:33 PM, Patrick McFadin wrote: >>> >>> It's less the release than the community agreeing to what is being >>> released. If this is an official Cassandra project, the PMC has >>> oversight and there needs to be a consensus vote on what we are >>> including in the project. Even pre-existing code that was donated as a >>> sub-project has gone through the CEP process: >>> >>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+DataStax+Drivers+Donation >>> >>> On October 15, the discussion was about creating a new CEP-1, so the >>> previous version was archived. The process we all agreed to is being >>> avoided, so I hope we can get back on track here. >>> >>> Patrick >>> >>> On Fri, Nov 15, 2024 at 12:08 PM Francisco Guerrero <fran...@apache.org> >>> wrote: >>> > >>> > Hi Patrick, >>> > >>> > Correct me if I'm wrong, but I don't think there's any precedent >>> > on creating a CEP for a release. My understanding is that releases >>> > are driven by release vote. >>> > >>> > Best, >>> > - Francisco >>> > >>> > On 2024/11/15 20:04:10 Patrick McFadin wrote: >>> > > Just a reminder of the CEP. That's something that needs to be >>> > > completed and voted on before any release. >>> > > >>> > > Patrick >>> > > >>> > > On Fri, Nov 15, 2024 at 11:46 AM Francisco Guerrero < >>> fran...@apache.org> wrote: >>> > > > >>> > > > Hi community, >>> > > > >>> > > > Saranya and I wanted to give an update on where things stand >>> > > > with the Sidecar release. We've been working towards completing >>> > > > the remaining JIRAs[1][2][3][4][5][6] in preparation for the >>> Sidecar >>> > > > release. >>> > > > >>> > > > We've already completed the authentication piece [7] and it has >>> been >>> > > > merged as of last week (Special thanks to Yifan Cai for all the >>> feedback >>> > > > on the feature). >>> > > > >>> > > > We expect to have everything ready mid-December, so we can >>> > > > propose cutting a release of Sidecar. >>> > > > >>> > > > Looking forward to any feedback from the community. >>> > > > >>> > > > Best, >>> > > > - Saranya Krishnakumar & Francisco Guerrero >>> > > > >>> > > > [1] https://issues.apache.org/jira/browse/CASSANDRASC-120 >>> > > > [2] https://issues.apache.org/jira/browse/CASSANDRASC-121 >>> > > > [3] https://issues.apache.org/jira/browse/CASSANDRASC-122 >>> > > > [4] https://issues.apache.org/jira/browse/CASSANDRASC-160 >>> > > > [5] https://issues.apache.org/jira/browse/CASSANDRASC-161 >>> > > > [6] https://issues.apache.org/jira/browse/CASSANDRASC-159 >>> > > > [7] https://issues.apache.org/jira/browse/CASSANDRASC-156 >>> > > > >>> > > > On 2024/10/15 18:47:39 Patrick McFadin wrote: >>> > > > > I have moved the original CEP-1 to the Archived section of >>> Confluence: >>> > > > > https://cwiki.apache.org/confluence/x/gImzBQ >>> > > > > >>> > > > > You can go ahead a create v2 of CEP-1 using the CEP template. >>> > > > > >>> > > > > Patrick >>> > > > > >>> > > > > On Tue, Oct 15, 2024 at 10:12 AM Josh McKenzie < >>> jmcken...@apache.org> wrote: >>> > > > > >>> > > > > > Should make sure we snapshot it and/or history retention is >>> sufficient for >>> > > > > > us to preserve the effort that went into the original. To >>> Joey's point, >>> > > > > > there's a lot of hard work that went into it as it stands it'd >>> be a shame >>> > > > > > to lose. >>> > > > > > >>> > > > > > On Tue, Oct 15, 2024, at 1:07 PM, Dinesh Joshi wrote: >>> > > > > > >>> > > > > > My preference would be to simply update CEP-1 instead of >>> starting a new >>> > > > > > one. It achieves the same end goal and we can create new CEPs >>> for the scope >>> > > > > > that is deferred. >>> > > > > > >>> > > > > > Dinesh >>> > > > > > >>> > > > > > On Mon, Oct 14, 2024 at 4:51 PM Patrick McFadin < >>> pmcfa...@gmail.com> >>> > > > > > wrote: >>> > > > > > >>> > > > > > I think that all sounds good. Let's get that new CEP started >>> then. I think >>> > > > > > there are opinions flying around based on last week's >>> discussions that need >>> > > > > > to coalesce. >>> > > > > > >>> > > > > > Patrick >>> > > > > > >>> > > > > > On Mon, Oct 14, 2024 at 3:19 PM Francisco Guerrero < >>> fran...@apache.org> >>> > > > > > wrote: >>> > > > > > >>> > > > > > From my point of view CEP-1 is overly broad in terms it wants >>> to achieve. >>> > > > > > My understanding is that CEPs have to have a well-defined >>> scope. >>> > > > > > >>> > > > > > I agree with Joey that we should close CEP-1 with the features >>> I have >>> > > > > > proposed earlier in this thread. And have any future Sidecar >>> work captured >>> > > > > > in >>> > > > > > subsequent CEPs. >>> > > > > > >>> > > > > > Best, >>> > > > > > - Francisco >>> > > > > > >>> > > > > > On 2024/10/14 21:58:55 Joseph Lynch wrote: >>> > > > > > > Hi everyone! >>> > > > > > > >>> > > > > > > I am curious what Dinesh's perspective is but I think the >>> specific >>> > > > > > > enumerated scope in CEP-1 isn't super critical to be honest. >>> That CEP >>> > > > > > > successfully (imo) built consensus that the community wants >>> a separate >>> > > > > > > management process, and that sidecar both exists today and >>> has useful >>> > > > > > > functionality (which is great!). I agree with Jon and others >>> in these >>> > > > > > > threads that the functionality isn't super accessible until >>> some of the >>> > > > > > > tickets Francisco mentioned are worked on and a release is >>> made. I also >>> > > > > > > agree with Francisco that getting to a release, even if it >>> is an alpha, >>> > > > > > > should be the goal - let's get to a release. >>> > > > > > > >>> > > > > > > I am personally fine closing CEP-1 and saying "This CEP has >>> been >>> > > > > > superseded >>> > > > > > > by subsequent CEPs - future work in the sidecar requiring >>> community >>> > > > > > > consensus will have separate CEPs as needed". That doesn't >>> mean that the >>> > > > > > > scope in CEP-1 isn't useful, and I hope some of the gaps are >>> added in the >>> > > > > > > future, but I think we can release without them and the >>> focus should be >>> > > > > > on >>> > > > > > > the gaps for release not the gaps in functionality with >>> CEP-1. >>> > > > > > > >>> > > > > > > Also I should mention I am extremely excited to see all the >>> great >>> > > > > > features >>> > > > > > > that have landed and that there is a place outside the main >>> DB for those >>> > > > > > > kinds of innovations to be tried out that doesn't conflict >>> with the main >>> > > > > > > process. In my mind, having that place was the purpose of >>> CEP-1, which is >>> > > > > > > done - the specific enumerated features are less important >>> in my opinion. >>> > > > > > > >>> > > > > > > -Joey >>> > > > > > > >>> > > > > > > On Mon, Oct 14, 2024 at 4:09 PM Patrick McFadin < >>> pmcfa...@gmail.com> >>> > > > > > wrote: >>> > > > > > > >>> > > > > > > > What are we going to do with CEP-1? Cut and rescope in a >>> new CEP or >>> > > > > > > > rewrite CEP-1? >>> > > > > > > > >>> > > > > > > > On Mon, Oct 14, 2024 at 11:18 AM Francisco Guerrero < >>> > > > > > fran...@apache.org> >>> > > > > > > > wrote: >>> > > > > > > > >>> > > > > > > >> Hi folks, >>> > > > > > > >> >>> > > > > > > >> It was great meeting some of you in person at Community >>> over Code >>> > > > > > where >>> > > > > > > >> we had a chance to discuss the Cassandra Sidecar project. >>> A lot of >>> > > > > > > >> you expressed interest in having a release of Sidecar to >>> start using >>> > > > > > the >>> > > > > > > >> existing features in the project: >>> > > > > > > >> >>> > > > > > > >> - C* adapters for versions 4.0, 4.1, 5.0 and trunk >>> > > > > > > >> - Cassandra Analytics support >>> > > > > > > >> - Restoring SSTables from S3-compatible object storage >>> > > > > > > >> - Mutual TLS authentication >>> > > > > > > >> - Role base access control >>> > > > > > > >> - Cluster health checks >>> > > > > > > >> - Observability >>> > > > > > > >> - Sidecar Client >>> > > > > > > >> >>> > > > > > > >> Some of the call-outs in this thread of things we need >>> for a release: >>> > > > > > > >> >>> > > > > > > >> - Documentation >>> > > > > > > >> - Bug fixes [1][2][3] >>> > > > > > > >> >>> > > > > > > >> These are other things mentioned in the thread, where we >>> would like >>> > > > > > help >>> > > > > > > >> from the community: >>> > > > > > > >> >>> > > > > > > >> - vNode support [4] >>> > > > > > > >> - Backup support [5] >>> > > > > > > >> - Bulk APIs [6] >>> > > > > > > >> >>> > > > > > > >> Once we have documentation and the bug fixes mentioned >>> above, I will >>> > > > > > > >> start a thread vote for a release of Cassandra Sidecar >>> 0.1 alpha. >>> > > > > > > >> >>> > > > > > > >> We want the community to benefit from the features >>> already present in >>> > > > > > > >> Sidecar. The more community engagement we have, the more >>> feature-rich >>> > > > > > > >> will become. >>> > > > > > > >> >>> > > > > > > >> Additionally, we can leverage easy-cass-stress to have >>> Cassandra >>> > > > > > Sidecar >>> > > > > > > >> included in your Cassandra clusters, so we can easily >>> start trying it >>> > > > > > out. >>> > > > > > > >> >>> > > > > > > >> Please let me know your thoughts on this. >>> > > > > > > >> >>> > > > > > > >> Best, >>> > > > > > > >> - Francisco >>> > > > > > > >> >>> > > > > > > >> [1] https://issues.apache.org/jira/browse/CASSANDRASC-120 >>> > > > > > > >> [2] https://issues.apache.org/jira/browse/CASSANDRASC-121 >>> > > > > > > >> [3] https://issues.apache.org/jira/browse/CASSANDRASC-122 >>> > > > > > > >> [4] https://issues.apache.org/jira/browse/CASSANDRASC-149 >>> > > > > > > >> [5] https://issues.apache.org/jira/browse/CASSANDRASC-148 >>> > > > > > > >> [6] https://issues.apache.org/jira/browse/CASSANDRASC-3 >>> > > > > > > >> >>> > > > > > > >> On 2024/10/03 12:05:14 Josh McKenzie wrote: >>> > > > > > > >> > I'm tentatively in favor of the idea of us cutting >>> releases for all >>> > > > > > our >>> > > > > > > >> ecosystem dependencies as a blocker to cutting a major >>> with Cassandra >>> > > > > > > >> proper. I say tentatively since we've had trouble getting >>> majors >>> > > > > > across the >>> > > > > > > >> line for awhile so adding more dependencies to that feels >>> risky, >>> > > > > > however I >>> > > > > > > >> think the improvement to user experience justifies it. >>> > > > > > > >> > >>> > > > > > > >> > Also - I suspect the effort required to get subprojects >>> across the >>> > > > > > line >>> > > > > > > >> will be quite a bit less than the mainline DB. >>> > > > > > > >> > >>> > > > > > > >> > If this is something there's agreement / consensus on, >>> I'd be happy >>> > > > > > to >>> > > > > > > >> take on the role of driving that coordination in the >>> future. >>> > > > > > > >> > >>> > > > > > > >> > On Wed, Oct 2, 2024, at 3:16 PM, Jon Haddad wrote: >>> > > > > > > >> > > > Mostly Analytics, which should not be a blocker for >>> Sidecar. >>> > > > > > > >> > > >>> > > > > > > >> > > Yes, agreed. I'm just trying to understand the >>> context of the >>> > > > > > vnode >>> > > > > > > >> statement and how we're framing 1.0 sidecar. >>> > > > > > > >> > > >>> > > > > > > >> > > > We definitely need to support 5.0 for the Analytics >>> release, but >>> > > > > > > >> that's orthogonal to Sidecar. >>> > > > > > > >> > > >>> > > > > > > >> > > It is, unless we're endorsing the analytics library >>> as a primary >>> > > > > > > >> reason to use sidecar. Then it becomes a dependency >>> people rely on, >>> > > > > > and I >>> > > > > > > >> don't want it blocking people from upgrading. >>> > > > > > > >> > > >>> > > > > > > >> > > > Yeah, definitely. I agree that we need to support >>> the latest >>> > > > > > > >> released version of Cassandra in ecosystem projects. >>> However, without >>> > > > > > > >> official releases there won't be adoption and without >>> adoption there >>> > > > > > won't >>> > > > > > > >> be feedback from the community on what features / >>> improvements are >>> > > > > > needed. >>> > > > > > > >> > > >>> > > > > > > >> > > I don't expect that our first release will be feature >>> complete, >>> > > > > > but >>> > > > > > > >> it should be at least compelling. I'm still not aware of >>> what >>> > > > > > > >> functionality exists that would meet that requirement. >>> > > > > > > >> > > >>> > > > > > > >> > > Think about this from the perspective of reading a >>> blog post. >>> > > > > > We're >>> > > > > > > >> excited to announce sidecar 1.0! Here's what you can do: >>> > > > > > > >> > > >>> > > > > > > >> > > 1. Backup / restore? >>> > > > > > > >> > > 2. ? >>> > > > > > > >> > > 3. ? >>> > > > > > > >> > > >>> > > > > > > >> > > All I'm asking for are 3 reasons why people should >>> care. If one >>> > > > > > of >>> > > > > > > >> them is backups, do we provide more flexible backup >>> targets than S3, >>> > > > > > and if >>> > > > > > > >> we provide incremental / continuous backup options? Is >>> there a >>> > > > > > scheduler >>> > > > > > > >> or do people need to roll their own? Is it coordinated, >>> or is the >>> > > > > > intent >>> > > > > > > >> that people handle it on their own? >>> > > > > > > >> > > >>> > > > > > > >> > > I work with a lot of end users - this is the type of >>> thing that >>> > > > > > > >> affects people and can either help or harm the project >>> image. >>> > > > > > > >> > > >>> > > > > > > >> > > Jon >>> > > > > > > >> > > >>> > > > > > > >> > > >>> > > > > > > >> > > On Wed, Oct 2, 2024 at 2:40 PM Francisco Guerrero < >>> > > > > > fran...@apache.org> >>> > > > > > > >> wrote: >>> > > > > > > >> > >> > By vnode support, do you mean the analytics >>> library? Or do >>> > > > > > other >>> > > > > > > >> features >>> > > > > > > >> > >> > in sidecar not work with vnodes? >>> > > > > > > >> > >> >>> > > > > > > >> > >> Mostly Analytics, which should not be a blocker for >>> Sidecar. >>> > > > > > > >> However, I do >>> > > > > > > >> > >> feel there should be more testing around vnodes in >>> Sidecar. >>> > > > > > > >> > >> >>> > > > > > > >> > >> > If we're talking about analytics, that gets a >>> little >>> > > > > > complicated. >>> > > > > > > >> Are we >>> > > > > > > >> > >> > also then talking about 1.0'ing analytics? If so, >>> I think we >>> > > > > > need >>> > > > > > > >> support >>> > > > > > > >> > >> > for 5.0 and BTI there. >>> > > > > > > >> > >> >>> > > > > > > >> > >> We need to release Analytics at some point as well, >>> we should >>> > > > > > have a >>> > > > > > > >> similar >>> > > > > > > >> > >> discuss thread regarding Analytics. We definitely >>> need to support >>> > > > > > > >> 5.0 for the >>> > > > > > > >> > >> Analytics release, but that's orthogonal to Sidecar. >>> > > > > > > >> > >> >>> > > > > > > >> > >> > Increasing the size of the ecosystem is >>> challenging... >>> > > > > > > >> > >> >>> > > > > > > >> > >> Yeah, definitely. I agree that we need to support >>> the latest >>> > > > > > > >> released version >>> > > > > > > >> > >> of Cassandra in ecosystem projects. However, without >>> official >>> > > > > > > >> releases >>> > > > > > > >> > >> there won't be adoption and without adoption there >>> won't be >>> > > > > > feedback >>> > > > > > > >> from >>> > > > > > > >> > >> the community on what features / improvements are >>> needed. >>> > > > > > > >> > >> >>> > > > > > > >> > >> On 2024/10/02 18:05:42 Jon Haddad wrote: >>> > > > > > > >> > >> > By vnode support, do you mean the analytics >>> library? Or do >>> > > > > > other >>> > > > > > > >> features >>> > > > > > > >> > >> > in sidecar not work with vnodes? >>> > > > > > > >> > >> > >>> > > > > > > >> > >> > If we're talking about analytics, that gets a >>> little >>> > > > > > complicated. >>> > > > > > > >> Are we >>> > > > > > > >> > >> > also then talking about 1.0'ing analytics? If so, >>> I think we >>> > > > > > need >>> > > > > > > >> support >>> > > > > > > >> > >> > for 5.0 and BTI there. >>> > > > > > > >> > >> > >>> > > > > > > >> > >> > In my opinion, if something core to the project >>> has official >>> > > > > > > >> releases, such >>> > > > > > > >> > >> > as drivers or operational tooling that people >>> depend on, it >>> > > > > > should >>> > > > > > > >> also >>> > > > > > > >> > >> > support the latest version of Cassandra, on >>> release. It >>> > > > > > doesn't >>> > > > > > > >> look good >>> > > > > > > >> > >> > if we release C* without usable drivers or tooling >>> to operate >>> > > > > > it. >>> > > > > > > >> It is >>> > > > > > > >> > >> > massively deflating to announce we just released >>> 5.0 >>> > > > > > (exciting!) >>> > > > > > > >> but you >>> > > > > > > >> > >> > can't use it for 6 months because you're using >>> analytics lib >>> > > > > > and >>> > > > > > > >> the people >>> > > > > > > >> > >> > that contribute to it has better things to do with >>> their >>> > > > > > time. I >>> > > > > > > >> think >>> > > > > > > >> > >> > part of the obligation of maintaining these >>> projects needs to >>> > > > > > be >>> > > > > > > >> keeping up >>> > > > > > > >> > >> > with latest releases. If that can't be done, we >>> should >>> > > > > > continue >>> > > > > > > >> to treat >>> > > > > > > >> > >> > it as a use-as-your-own-risk thing without >>> official releases. >>> > > > > > > >> > >> > >>> > > > > > > >> > >> > Increasing the size of the ecosystem is >>> challenging... >>> > > > > > > >> > >> > >>> > > > > > > >> > >> > Jon >>> > > > > > > >> > >> > >>> > > > > > > >> > >> > >>> > > > > > > >> > >> > >>> > > > > > > >> > >> > >>> > > > > > > >> > >> > On Wed, Oct 2, 2024 at 1:21 PM Francisco Guerrero < >>> > > > > > > >> fran...@apache.org> >>> > > > > > > >> > >> > wrote: >>> > > > > > > >> > >> > >>> > > > > > > >> > >> > > Hi folks, >>> > > > > > > >> > >> > > >>> > > > > > > >> > >> > > Thanks for all the input. I'm trying to gather >>> all the >>> > > > > > comments, >>> > > > > > > >> and from >>> > > > > > > >> > >> > > what I >>> > > > > > > >> > >> > > can gather it seems that most of the opinions >>> are converging >>> > > > > > > >> towards >>> > > > > > > >> > >> > > scoping >>> > > > > > > >> > >> > > a Sidecar release. These are the items that were >>> called out >>> > > > > > that >>> > > > > > > >> we will >>> > > > > > > >> > >> > > need >>> > > > > > > >> > >> > > for a release: >>> > > > > > > >> > >> > > - Documentation >>> > > > > > > >> > >> > > - Authorization / Authentication >>> > > > > > > >> > >> > > - vnode support >>> > > > > > > >> > >> > > >>> > > > > > > >> > >> > > There are some smaller bug fixes that need to be >>> included >>> > > > > > that >>> > > > > > > >> we can label >>> > > > > > > >> > >> > > as part of the 1.0 release.[1][2][3] >>> > > > > > > >> > >> > > >>> > > > > > > >> > >> > > If there are any other tasks we need to >>> completed, I >>> > > > > > encourage >>> > > > > > > >> the >>> > > > > > > >> > >> > > community >>> > > > > > > >> > >> > > to create JIRAs that can be in the release >>> milestone for the >>> > > > > > > >> Sidecar. >>> > > > > > > >> > >> > > >>> > > > > > > >> > >> > > I agree with Stefan that OpenAPI is desirable. >>> OpenAPI is >>> > > > > > > >> something I've >>> > > > > > > >> > >> > > been >>> > > > > > > >> > >> > > looking into as well. I'm also glad to see >>> Abhijeet can help >>> > > > > > > >> with the >>> > > > > > > >> > >> > > documentation effort, I can also help on that >>> front. >>> > > > > > > >> > >> > > >>> > > > > > > >> > >> > > Hopefully, I've captured your comments as truly >>> as possible. >>> > > > > > > >> > >> > > >>> > > > > > > >> > >> > > Thanks again for all the feedback. >>> > > > > > > >> > >> > > >>> > > > > > > >> > >> > > Best, >>> > > > > > > >> > >> > > - Francisco >>> > > > > > > >> > >> > > >>> > > > > > > >> > >> > > [1] >>> https://issues.apache.org/jira/browse/CASSANDRASC-120 >>> > > > > > > >> > >> > > [2] >>> https://issues.apache.org/jira/browse/CASSANDRASC-121 >>> > > > > > > >> > >> > > [3] >>> https://issues.apache.org/jira/browse/CASSANDRASC-122 >>> > > > > > > >> > >> > > >>> > > > > > > >> > >> > > On 2024/10/02 15:29:20 Jon Haddad wrote: >>> > > > > > > >> > >> > > > When I developed some of the original sidecar >>> code, it was >>> > > > > > > >> based on REST >>> > > > > > > >> > >> > > > Easy, which would have allowed us to generate >>> the spec >>> > > > > > > >> automatically. I >>> > > > > > > >> > >> > > > did this in a similar project. >>> > > > > > > >> > >> > > > >>> > > > > > > >> > >> > > > That was removed here: >>> > > > > > > >> > >> > > > >>> https://issues.apache.org/jira/browse/CASSANDRASC-57 >>> > > > > > > >> > >> > > > >>> > > > > > > >> > >> > > > But unfortunately it looks like you can't >>> generate the spec >>> > > > > > > >> from the >>> > > > > > > >> > >> > > code. >>> > > > > > > >> > >> > > > >>> > > > > > > >> > >> > > > Disappointing, that functionality was really >>> useful. I >>> > > > > > wish >>> > > > > > > >> someone had >>> > > > > > > >> > >> > > > asked me before gutting it. >>> > > > > > > >> > >> > > > >>> > > > > > > >> > >> > > > Jon >>> > > > > > > >> > >> > > > >>> > > > > > > >> > >> > > > >>> > > > > > > >> > >> > > > >>> > > > > > > >> > >> > > > On Wed, Oct 2, 2024 at 11:16 AM Štefan >>> Miklošovič < >>> > > > > > > >> > >> > > smikloso...@apache.org> >>> > > > > > > >> > >> > > > wrote: >>> > > > > > > >> > >> > > > >>> > > > > > > >> > >> > > > > Something like this: >>> > > > > > > >> > >> > > > > >>> > > > > > > >> > >> > > > > >>> > > > > > https://instaclustr.github.io/instaclustr-icarus-go-client/ >>> > > > > > > >> > >> > > > > >>> > > > > > > >> > >> > > > > On Wed, Oct 2, 2024 at 4:54 PM Štefan >>> Miklošovič < >>> > > > > > > >> > >> > > smikloso...@apache.org> >>> > > > > > > >> > >> > > > > wrote: >>> > > > > > > >> > >> > > > > >>> > > > > > > >> > >> > > > >> While documenting endpoints please use >>> something like >>> > > > > > > >> OpenAPI >>> > > > > > > >> > >> > > > >> specification. The sidecar should expose >>> this document >>> > > > > > > >> itself so when >>> > > > > > > >> > >> > > I go >>> > > > > > > >> > >> > > > >> to this and that URL, I see all endpoints, >>> I put the >>> > > > > > > >> payloads / >>> > > > > > > >> > >> > > parameters >>> > > > > > > >> > >> > > > >> for them and I can just directly call that, >>> no messing >>> > > > > > with >>> > > > > > > >> curl / >>> > > > > > > >> > >> > > wget or >>> > > > > > > >> > >> > > > >> programmatically or whatever like that. The >>> barrier to >>> > > > > > > >> exercise the >>> > > > > > > >> > >> > > basic >>> > > > > > > >> > >> > > > >> functionality has to virtually not be there. >>> > > > > > > >> > >> > > > >> >>> > > > > > > >> > >> > > > >> On Wed, Oct 2, 2024 at 4:13 PM Abhijeet >>> Dubey < >>> > > > > > > >> > >> > > dubey.abhijee...@gmail.com> >>> > > > > > > >> > >> > > > >> wrote: >>> > > > > > > >> > >> > > > >> >>> > > > > > > >> > >> > > > >>> Hi folks, >>> > > > > > > >> > >> > > > >>> >>> > > > > > > >> > >> > > > >>> I have been using Sidecar recently and >>> have found some >>> > > > > > of >>> > > > > > > >> its >>> > > > > > > >> > >> > > > >>> functionalities to be quite useful. Hari >>> and I are also >>> > > > > > > >> working on >>> > > > > > > >> > >> > > CEP-40 >>> > > > > > > >> > >> > > > >>> which aims to introduce live migration >>> features in >>> > > > > > Sidecar >>> > > > > > > >> in the >>> > > > > > > >> > >> > > > >>> near future. >>> > > > > > > >> > >> > > > >>> >>> > > > > > > >> > >> > > > >>> However, as others have mentioned, I agree >>> that it >>> > > > > > > >> currently lacks >>> > > > > > > >> > >> > > > >>> proper documentation. >>> > > > > > > >> > >> > > > >>> >>> > > > > > > >> > >> > > > >>> Since this is an official Apache project, >>> I believe >>> > > > > > that >>> > > > > > > >> creating >>> > > > > > > >> > >> > > > >>> comprehensive documentation would be >>> beneficial. This >>> > > > > > > >> documentation >>> > > > > > > >> > >> > > should >>> > > > > > > >> > >> > > > >>> include an overview, architecture, a list >>> and >>> > > > > > description >>> > > > > > > >> of various >>> > > > > > > >> > >> > > > >>> endpoints, and some examples or tutorials >>> on how to use >>> > > > > > > >> Sidecar's >>> > > > > > > >> > >> > > features. >>> > > > > > > >> > >> > > > >>> >>> > > > > > > >> > >> > > > >>> This documentation would help people get >>> started with >>> > > > > > > >> Sidecar and >>> > > > > > > >> > >> > > lower >>> > > > > > > >> > >> > > > >>> the entry barrier for many. We can update >>> the >>> > > > > > documentation >>> > > > > > > >> > >> > > incrementally >>> > > > > > > >> > >> > > > >>> as needed, along with future enhancements >>> and new >>> > > > > > > >> features. However, >>> > > > > > > >> > >> > > > >>> creating some form of formal documentation >>> would be >>> > > > > > very >>> > > > > > > >> helpful. >>> > > > > > > >> > >> > > > >>> >>> > > > > > > >> > >> > > > >>> To this end I'm willing and highly >>> interested in >>> > > > > > writing >>> > > > > > > >> some form of >>> > > > > > > >> > >> > > > >>> formal documentation for the Sidecar >>> project. Please >>> > > > > > let >>> > > > > > > >> me know your >>> > > > > > > >> > >> > > > >>> thoughts/opinions on this proposal. >>> > > > > > > >> > >> > > > >>> >>> > > > > > > >> > >> > > > >>> >>> > > > > > > >> > >> > > > >>> >>> > > > > > > >> > >> > > > >>> On Wed, Oct 2, 2024 at 6:46 PM Štefan >>> Miklošovič < >>> > > > > > > >> > >> > > smikloso...@apache.org> >>> > > > > > > >> > >> > > > >>> wrote: >>> > > > > > > >> > >> > > > >>> >>> > > > > > > >> > >> > > > >>>> Totally agree with Jon here basically on >>> all fronts. >>> > > > > > > >> Apache >>> > > > > > > >> > >> > > Cassandra >>> > > > > > > >> > >> > > > >>>> Sidecar was always a hard nut to crack >>> for me, that is >>> > > > > > > >> probably why >>> > > > > > > >> > >> > > I have >>> > > > > > > >> > >> > > > >>>> not been involved with that a lot even >>> that is a great >>> > > > > > > >> tool to have >>> > > > > > > >> > >> > > and be >>> > > > > > > >> > >> > > > >>>> invested in as I was writing my own >>> sidecar and I >>> > > > > > found a >>> > > > > > > >> lot of >>> > > > > > > >> > >> > > > >>>> similarities and problems Apache's >>> sidecar tries to >>> > > > > > fix. >>> > > > > > > >> There was >>> > > > > > > >> > >> > > some >>> > > > > > > >> > >> > > > >>>> invisible barrier I have never managed to >>> jump over. I >>> > > > > > > >> was looking >>> > > > > > > >> > >> > > around >>> > > > > > > >> > >> > > > >>>> and I am very sorry if I just have not >>> found it yet >>> > > > > > but >>> > > > > > > >> there is >>> > > > > > > >> > >> > > not a list >>> > > > > > > >> > >> > > > >>>> of endpoints a sidecar has, is there? In >>> readme and >>> > > > > > dev >>> > > > > > > >> docs there >>> > > > > > > >> > >> > > is just >>> > > > > > > >> > >> > > > >>>> nothing. Taking it at a face value I just >>> don't know >>> > > > > > what >>> > > > > > > >> Sidecar is >>> > > > > > > >> > >> > > > >>>> capable of and how to use it. I see in >>> the commit >>> > > > > > history >>> > > > > > > >> there is >>> > > > > > > >> > >> > > a bunch >>> > > > > > > >> > >> > > > >>>> of commits mentioning S3 but it is a >>> total blackbox >>> > > > > > for >>> > > > > > > >> me as a >>> > > > > > > >> > >> > > potential >>> > > > > > > >> > >> > > > >>>> user. >>> > > > > > > >> > >> > > > >>>> >>> > > > > > > >> > >> > > > >>>> On Wed, Oct 2, 2024 at 2:52 PM Jon Haddad >>> < >>> > > > > > > >> j...@rustyrazorblade.com> >>> > > > > > > >> > >> > > > >>>> wrote: >>> > > > > > > >> > >> > > > >>>> >>> > > > > > > >> > >> > > > >>>>> I don't think we should release sidecar >>> 1.0 without >>> > > > > > any >>> > > > > > > >> docs. >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> I took a look through the closed JIRAs >>> to see what's >>> > > > > > > >> there. Here's >>> > > > > > > >> > >> > > > >>>>> what I found, please correct me if >>> there's more: >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> - Lots of stuff related to analytics. >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> I would be pretty excited for this, but >>> the analytics >>> > > > > > > >> library only >>> > > > > > > >> > >> > > > >>>>> works with single token clusters. Most >>> folks don't >>> > > > > > run >>> > > > > > > >> Cassandra >>> > > > > > > >> > >> > > this >>> > > > > > > >> > >> > > > >>>>> way. I realize there's some element of >>> everyone >>> > > > > > needs >>> > > > > > > >> to scratch >>> > > > > > > >> > >> > > their own >>> > > > > > > >> > >> > > > >>>>> itch, but I don't think we can really >>> call this a >>> > > > > > useful >>> > > > > > > >> feature >>> > > > > > > >> > >> > > if the >>> > > > > > > >> > >> > > > >>>>> overwhelming majority of folks can't use >>> it. I've >>> > > > > > > >> worked with a >>> > > > > > > >> > >> > > couple >>> > > > > > > >> > >> > > > >>>>> hundred teams over the years and can >>> only think of 1 >>> > > > > > org >>> > > > > > > >> outside >>> > > > > > > >> > >> > > of Apple >>> > > > > > > >> > >> > > > >>>>> and Netflix that used 1 token, and It >>> was a cluster >>> > > > > > that >>> > > > > > > >> predated >>> > > > > > > >> > >> > > v-nodes. >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> The analytics repo says it's compatible >>> with >>> > > > > > Cassandra >>> > > > > > > >> 4, but not >>> > > > > > > >> > >> > > 5. >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> - Backup & Restore from S3 >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> Is this compatible with other cloud >>> providers or >>> > > > > > object >>> > > > > > > >> stores? It >>> > > > > > > >> > >> > > > >>>>> specifically lists S3 in JIRA. I >>> haven't looked at >>> > > > > > the >>> > > > > > > >> source >>> > > > > > > >> > >> > > yet. Am I >>> > > > > > > >> > >> > > > >>>>> correct in reading it supports backing >>> up snapshots, >>> > > > > > no >>> > > > > > > >> continuous >>> > > > > > > >> > >> > > > >>>>> backups? Seems like we should have at >>> least feature >>> > > > > > > >> parity with >>> > > > > > > >> > >> > > Medusa if >>> > > > > > > >> > >> > > > >>>>> we're going to release something here. >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> All the other closed JIRAs look related >>> to these two >>> > > > > > > >> items. So the >>> > > > > > > >> > >> > > > >>>>> question is, are we releasing 1.0 as an >>> limited S3 >>> > > > > > > >> backup and >>> > > > > > > >> > >> > > restore >>> > > > > > > >> > >> > > > >>>>> tool? One that prevents you from >>> upgrading to >>> > > > > > Cassandra >>> > > > > > > >> 5 if you >>> > > > > > > >> > >> > > happen to >>> > > > > > > >> > >> > > > >>>>> use single token clusters? >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> Who is the target audience? >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> Jon >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>> On Wed, Oct 2, 2024 at 2:41 AM Dinesh >>> Joshi < >>> > > > > > > >> djo...@apache.org> >>> > > > > > > >> > >> > > wrote: >>> > > > > > > >> > >> > > > >>>>> >>> > > > > > > >> > >> > > > >>>>>> Currently the Sidecar has a lot of >>> functionality >>> > > > > > that is >>> > > > > > > >> > >> > > immediately >>> > > > > > > >> > >> > > > >>>>>> usable by the community. Apart from >>> minor fixes, the >>> > > > > > > >> AuthN/Z >>> > > > > > > >> > >> > > story would be >>> > > > > > > >> > >> > > > >>>>>> wrapped up soon. Post this, I would >>> propose moving >>> > > > > > > >> forward with >>> > > > > > > >> > >> > > cutting a >>> > > > > > > >> > >> > > > >>>>>> release with the existing feature set >>> so we can get >>> > > > > > > >> this in the >>> > > > > > > >> > >> > > hands of >>> > > > > > > >> > >> > > > >>>>>> our community. >>> > > > > > > >> > >> > > > >>>>>> >>> > > > > > > >> > >> > > > >>>>>> On Tue, Oct 1, 2024 at 8:27 PM guo >>> Maxwell < >>> > > > > > > >> cclive1...@gmail.com> >>> > > > > > > >> > >> > > > >>>>>> wrote: >>> > > > > > > >> > >> > > > >>>>>> >>> > > > > > > >> > >> > > > >>>>>>> Have the same question ： what ‘s the >>> plan ？ >>> > > > > > > >> > >> > > > >>>>>>> >>> > > > > > > >> > >> > > > >>>>>>> Jeff Jirsa <jji...@gmail.com >>> >于2024年10月2日 >>> > > > > > 周三上午10:43写道： >>> > > > > > > >> > >> > > > >>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> On Oct 1, 2024, at 7:26 PM, Josh >>> McKenzie < >>> > > > > > > >> jmcken...@apache.org >>> > > > > > > >> > >> > > > >>> > > > > > > >> > >> > > > >>>>>>>> wrote: >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> However it is used by a number of >>> other features >>> > > > > > as a >>> > > > > > > >> dependency >>> > > > > > > >> > >> > > > >>>>>>>> such as analytics, backup/restore, >>> repair, >>> > > > > > metrics, >>> > > > > > > >> and CDC >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> It seems like a natural pressure >>> relief valve for >>> > > > > > > >> moving >>> > > > > > > >> > >> > > operations >>> > > > > > > >> > >> > > > >>>>>>>> out of a core C* node that are well >>> served out of >>> > > > > > > >> process. >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> Yea, but the point of the foundation >>> is to RELEASE >>> > > > > > > >> software for >>> > > > > > > >> > >> > > the >>> > > > > > > >> > >> > > > >>>>>>>> public good, and the link asserting >>> consensus was >>> > > > > > > >> dec2018, so >>> > > > > > > >> > >> > > its’ 5.5 >>> > > > > > > >> > >> > > > >>>>>>>> years and no releases. >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> What’s the plan here? >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>>>>>>> >>> > > > > > > >> > >> > > > >>> >>> > > > > > > >> > >> > > > >>> -- >>> > > > > > > >> > >> > > > >>> *Abhijeet* >>> > > > > > > >> > >> > > > >>> >>> > > > > > > >> > >> > > > >> >>> > > > > > > >> > >> > > > >>> > > > > > > >> > >> > > >>> > > > > > > >> > >> > >>> > > > > > > >> > >>> > > > > > > >> >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> > > > > >>> > > >>> >>> >>>
