Re: In need of reviewers

[dinesh.jo...@yahoo.com.INVALID]

I can help out.
Dinesh 

On Friday, May 11, 2018, 4:37:07 PM PDT, kurt greaves 
 wrote:  
 
 Oh I know, there are just tickets relevant to people I work with.

On Fri., 11 May 2018, 22:10 Josh McKenzie,  wrote:

> May be easier to just provide a link to the JQL, since there's quite a few
> more tickets than just what you've mentioned above:
>
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20cassandra%20AND%20resolution%20%3D%20unresolved%20AND%20status%20in%20(%22Patch%20Available%22%2C%20%22Awaiting%20Feedback%22)%20AND%20reviewer%20is%20EMPTY%20ORDER%20BY%20updated%20DESC
>
> or JQL:
> project = cassandra AND resolution = unresolved AND status in ("Patch
> Available", "Awaiting Feedback") AND reviewer is EMPTY ORDER BY updated
> DESC
>
> On Fri, May 11, 2018 at 6:43 AM, kurt greaves 
> wrote:
>
> > We've got a bunch of tickets that are either in need of review or just a
> > bit of feedback. Would be very grateful for any help here :).
> >
> > Bugs:
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14365&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=dzgugNQ_-Xp_IhoZ_
> > yXOhK90rU63tOa5VWtbNCJXB1I&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14204&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=
> > pITz0kdAo5jrNc3TDzeXdBJ14yDl4HHi2aowk1yA4NI&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14162&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=tiN83_
> > Q68L9l6coMUD2KdBIptE4F-xfi--cWEzadVLM&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14126&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=
> > LBThldSjUG6z912bxdp2202kNA-ma-TbReAfPuaKOho&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14365&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=dzgugNQ_-Xp_IhoZ_
> > yXOhK90rU63tOa5VWtbNCJXB1I&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14099&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=RYPlsfyk_
> > m9AUKPjTqS99FUIrHUR15T9gYFNkAvHmT4&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14073&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=YJs46FqZ1P_M9xiLcl-
> > ynUduvhhfzCWoElcDXMioThY&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14063&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=
> > vjJGUzOXIG0UXbVZWOb42nx6kBHhHhJG-p6eyw2JpLs&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14056&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=XItQA2Xa1jdbjLCZhYRhdm-
> > HgfoKgJwspC99fdQv_iM&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14054&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=FvTGFmMgSIp4wS-
> > Knt7lzAJONsk2DthnBLPPcgyMEfg&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14013&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=
> > ngN2nSGlKr5iNVL4NVGzH84CJvQABXjBqmvtqjVeiy0&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D13841&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=
> > 2NH6xOyQU1mIPOOcWLDkPm3tiuzj59vJbPORWIjnjrs&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D13698&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=
> > AlEWZmIpKOTRjOOSLQVllahk4dgHPpDEmHHJUXNjLS4&e=
> >
> > Improvements:
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CA

Re: Which approach should we use for exposing metrics through Virtual tables?

[dinesh.jo...@yahoo.com.INVALID]

+1 on doing this on a case-by-case basis. The threadpool_metrics looks 
reasonable. It's best not to shoehorn all metrics into a single table with all 
possible columns.
Dinesh 

On Friday, June 22, 2018, 8:11:33 AM PDT, Chris Lohfink 
 wrote:  
 
 I think this can really be case by case. In tpstats (I have patch for that by 
the way in CASSANDRA-14523 
) is pretty intuitive in 
way you listed. Table metrics is another beast and we will likely have many 
tables for them, ie a table for viewing latencies, caches, on disk 
statistics... they can be discussed in their respective tickets.

Having a general table for viewing all metrics is I think an additional thing 
(ie like the table_stats below), not as a general use browsing thing but to 
provide alternative to JMX. The custom tables that expose things in a nice 
(attempted at least) intuitive manner wont have _all_ the metrics and its very 
likely that people will want them for reporting. Unfortunately the metrics are 
currently not something you can readily expose in a single table as there are 
type/scope on some while others have type/keyspace/scope, type/keyspace, and 
others type/path/scope so there will likely need to be some breakup here with 
things like "table_metrics", "client_metrics", "streaming_metrics" etc.

I agree with benedict that we should attempt to not expose the internal 
implementation details in the metrics for when there are changes again, there 
are always changes. However it is kinda necessary at some level for this 
"generalized" metrics. This is something the "custom" tables that expose data 
in the nodetool way don't have as much issues with, and what I personally have 
been working on first.

Chris

> On Jun 22, 2018, at 5:14 AM, Benjamin Lerer  
> wrote:
> 
> Hi,
> 
> I would like to start working on exposing the metrics through virtual
> tables in CASSANDRA-14537
> 
> 
> We had some long discussion already in CASSANDRA-7622 about which schema to
> use to expose the metrics, unfortunately in the end I was not truly
> convinced by any solution (including my own).
> 
> I would like to expose the possible solutions and there limitations and
> advantages to find out which is the solution that people prefer or to see
> if somebody can come up with another solution.
> 
> In CASSANDRA-7622, Chris Lohfink proposed to expose the table metric using
> the following schema:
> 
> VIRTUAL TABLE table_stats (
>    keyspace_name TEXT,
>    table_name TEXT,
>    metric TEXT,
>    value DOUBLE,
>    fifteen_min_rate DOUBLE,
>    five_min_rate DOUBLE,
>    mean_rate DOUBLE,
>    one_min_rate DOUBLE,
>    p75th DOUBLE,
>    p95th DOUBLE,
>    p99th DOUBLE,
>    p999th DOUBLE,
>    min BIGINT,
>    max BIGINT,
>    mean DOUBLE,
>    std_dev DOUBLE,
>    median DOUBLE,
>    count BIGINT,
>    PRIMARY KEY( keyspace_name,  table_name , metric));
> 
> This approach has some advantages:
> 
>  - It is easy to use for all the metric categories that we have (http://
>  cassandra.apache.org/doc/latest/operating/metrics.html)
>  - The number of column is relatively small and fit in the cqlsh console.
> 
> 
> The main disadvantage that I see with that approach is that it might not
> always be super readable. Gauge or a Counter metric will have data for only
> one column and will return NULL for all the others. If you know precisely
> which metric is what and you only target that type of metric you can build
> your query in such a way that the output is nicely formatted.
> Unfortunately, I do not expect every user to know which metric is what.
> The output format can also be problematic for monitoring tools as they
> might have to use some extra logic to determine how to process each metric.
> 
> My preferred approach was to use metrics has columns. For example for the
> threadpool metrics it will have given the following schema:
> 
> VIRTUAL TABLE threadpool_metrics (
>    pool_name TEXT,
>    active INT,
>    pending INT,
>    completed BIGINT,
>    blocked BIGINT,
>    total_blocked BIGINT,
>    max_pool_size INT,
>    PRIMARY KEY( pool_name )
> )
> 
> That approach provide an output similar to the one of the nodetool
> tpstats which will be, in my opinion, more readable that the previous
> approach.
> 
> Unfortunately, it also has several serious drawbacks:
> 
> 
>  - It does work for small set of metrics but do not work well for the
>  table or keyspace metrics where we have more than 63 metrics. If you
>  split the histograms, meters and timers into multiple columns you easily
>  reach more than a hundred columns. As Chris pointed out in CASSANDRA-7622
>  it makes the all thing unusable.
>  - It also does not work properly for set of metrics like the commit log
>  metrics because you can not get a natural primary key and will have to
>  somehow create a fake one.
> 
> 
> Nodetool solved the table and keyspace metric problems by splitting 

Re: Testing 4.0 Post-Freeze

[dinesh.jo...@yahoo.com.INVALID]

I think the call to action for the community here is to focus efforts on 
testing and bug fixes than spending time on reviewing features. That said, 
tlp-stress looks interesting.
Dinesh 

On Tuesday, July 3, 2018, 1:03:54 PM PDT, Jonathan Haddad 
 wrote:  
 
 I agree with Josh. I don’t see how changing the convention around trunk
will improve the process, seems like it’ll only introduce a handful of
rollback commits when people forget.

Other than that, it all makes sense to me.

I’ve been working on a workload centric stress tool on and off for a little
bit in an effort to create something that will help with wider adoption in
stress testing. It differs from the stress we ship by including fully
functional stress workloads as well as a validation process. The idea being
to be flexible enough to test both performance and correctness in LWT and
MVs as well as other arbitrary workloads.

https://github.com/thelastpickle/tlp-stress

Jon


On Tue, Jul 3, 2018 at 12:28 PM Josh McKenzie  wrote:

> Why not just branch a 4.0-rel and bugfix there and merge up while still
> accepting new features or improvements on trunk?
>
> I don't think the potential extra engagement in testing will balance out
> the atrophy and discouraging contributions / community engagement we'd get
> by deferring all improvements and new features in an open-ended way.
>
> On Tue, Jul 3, 2018 at 1:33 PM sankalp kohli 
> wrote:
>
> > Hi cassandra-dev@,
> >
> > With the goal of making Cassandra's 4.0 the most stable major release to
> > date, we would like all committers of the project to consider joining us
> in
> > dedicating their time and attention to testing, running, and fixing
> issues
> > in 4.0 between the September freeze and the 4.0 beta release. This would
> > result in a freeze of new feature development on trunk or branches during
> > this period, instead focusing on writing, improving, and running tests or
> > fixing and reviewing bugs or performance regressions found in 4.0 or
> > earlier.
> >
> > How would this work?
> >
> > We propose that between the September freeze date and beta, a new branch
> > would not be created and trunk would only have bug fixes and performance
> > improvements committed to it. At the same time we do not want to
> discourage
> > community contributions. Not all contributors can be expected to be aware
> > of such a decision or may be new to the project. In cases where new
> > features are contributed during this time, the contributor can be
> informed
> > of the current status of the release process, be encouraged to contribute
> > to testing or bug fixing, and have their feature reviewed after the beta
> is
> > reached.
> >
> >
> > What happens when beta is reached?
> >
> > Ideally, contributors who have made significant contributions to the
> > release will stick around to continue testing between beta and final
> > release. Any additional folks who continue this focus would also be
> greatly
> > appreciated.
> >
> > What about before the freeze?
> >
> > Testing new features is of course important. This isn't meant to
> discourage
> > development – only to enable us to focus on testing and hardening 4.0 to
> > deliver Cassandra's most stable major release. We would like to see
> > adoption of 4.0 happen much more quickly than its predecessor.
> >
> > Thanks for considering this proposal,
> > Sankalp Kohli
>
-- 
Jon Haddad
http://www.rustyrazorblade.com
twitter: rustyrazorblade  

Re: JIRAs in Review

[dinesh.jo...@yahoo.com.INVALID]

Kurt was looking at some help with this ticket - 
https://issues.apache.org/jira/browse/CASSANDRA-14525
Dinesh 

On Tuesday, July 17, 2018, 12:35:25 PM PDT, sankalp kohli 
 wrote:  
 
 Hi,
    We are 7 weeks away from 4.0 freeze and there are ~150 JIRAs waiting
for review. It is hard to know which ones should be prioritized as some of
them could be not valid(fixes 2.0 bug), some of them will have the assignee
who no longer is active, etc.

If anyone is *not* getting traction on the JIRA to get it reviewed, please
use this thread to send your JIRA number and optionally why it is
important.

Thanks,
Sankalp
  

Re: reroll the builds?

[dinesh.jo...@yahoo.com.INVALID]

I can help out with the triage / rerunning dtests if needed.
Dinesh 

On Monday, July 23, 2018, 10:22:18 AM PDT, Jason Brown 
 wrote:  
 
 I spoke with some people over here, and I'm going to spend a day doing a
quick triage of the failing dtests. There are some fixes for data loss bugs
that are critical to get out in these builds, so I'll ensure the current
failures are within an acceptable level of flakey-ness in order to unblock
those fixes.

Will have an update shortly ...

-Jason

On Mon, Jul 23, 2018 at 9:18 AM, Jason Brown  wrote:

> Hi all,
>
> First, thanks Joey for running the tests. Your pass/fail counts are
> basically what in line with what I've seen for the last several months. (I
> don't have an aggregated list anywhere, just observations from recent runs).
>
> Second, it's beyond me why there's such inertia to actually cutting a
> release. We're getting up to almost *six months* since the last release.
> Are there any grand objections at this point?
>
> Thanks,
>
> -Jason
>
>
> On Tue, Jul 17, 2018 at 4:01 PM, Joseph Lynch 
> wrote:
>
>> We ran the tests against 3.0, 2.2 and 3.11 using circleci and there are
>> various failing dtests but all three have green unit tests.
>>
>> 3.11.3 tentative (31d5d87, test branch
>> > cassandra_3.11_temp_testing>,
>> unit tests 
>> pass, 5
>>  and 6
>> > tests/containers/8>
>> dtest failures)
>> 3.0.17 tentative (d52c7b8, test branch
>> ,
>> unit
>> tests  pass, 14
>>  and 15
>>  dtest failures)
>> 2.2.13 tentative (3482370, test branch
>> > dra/tree/2.2-testing>,
>> unit tests 
>> pass, 9
>>  and 10
>> > tests/containers/8>
>> dtest failures)
>>
>> It looks like many (~6) of the failures in 3.0.x are related to
>> snapshot_test.TestArchiveCommitlog. I'm not sure if this is abnormal.
>>
>> I don't see a good historical record to know if these are just flakes, but
>> if we only want to go on green builds perhaps we can either disable the
>> flakey tests or fix them up? If someone feels strongly we should fix
>> particular tests up please link a jira and I can take a whack at some of
>> them.
>>
>> -Joey
>>
>> On Tue, Jul 17, 2018 at 9:35 AM Michael Shuler 
>> wrote:
>>
>> > On 07/16/2018 11:27 PM, Jason Brown wrote:
>> > > Hey all,
>> > >
>> > > The recent builds were -1'd, but it appears the issues have been
>> resolved
>> > > (2.2.13 with CASSANDRA-14423, and 3.0.17 / 3.11.3 reverting
>> > > CASSANDRA-14252). Can we go ahead and reroll now?
>> >
>> > Could someone run through the tests on 2.2, 3.0, 3.11 branches and link
>> > them?  Thanks!
>> >
>> > Michael
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> > For additional commands, e-mail: dev-h...@cassandra.apache.org
>> >
>> >
>>
>
>
  

Re: reroll the builds?

[dinesh.jo...@yahoo.com.INVALID]

Hi Jason,
I agree - we should release with the dataloss bug fix. I went over the gist - 
apart from the Python errors and test teardown failures, there seem to be a few 
failures that look legitimate. Any chance you can run the dtests on the 
previous release SHAs and compare the dtest failures? If they're the same / 
similar, we know at least we're at parity with the previous release :)
Dinesh 

On Tuesday, July 24, 2018, 8:18:50 AM PDT, Jason Brown 
 wrote:  
 
 TL;DR We are in a better place than we were for the 3.0.16 and 3.11.2
releases. The current fails are not fatal, although they warrant
investigation. My opinion is that due the critical data loss bugs that are
fixed by CASSANDRA-14513 and CASSANDRA-14515, we should cut the builds now.

I've run the HEAD of the 3.0 and 3.11 branches vs the 3.0.16 and 3.11.2
release shas, and there are far less failing dtests now. In comparison:

- 3.11
-- HEAD - 5-6 failing tests
-- 3.11.2 - 18-20 failures

- 3.0
-- HEAD - 14-16 failures
-- 3.0.16 - 22-25 failures

The raw dump of my work can be found here:
https://gist.github.com/jasobrown/e7ecf6d0bf875d1f4a08ee06ac7eaba0. I've
applied no effort to clean it up, but it's available (includes links to the
circleci runs). I haven't completed an exhautive analysis of the failures
to see how far they go back as things become tricky (or, at least, very
time intensive to research) with the pytest/python-3 update with
CASSANDRA-14134. Thus some of the failures might be in the dtests
themselves (I suspect a couple of the failures are), but most are proabaly
legit failures.

As this thread is about cutting the releases, I'll save any significiant
analysis for a followup thread. I will say that the current failures are a
subset of the previous release's failures, and those failures are not data
loss bugs.

Overall, I feel far more comfortable getting the data loss fixes out
without any further delay than waiting for a few minor fixes. I will triage
the dtest failures over the coming days. There are some open tickets, and
I'll try to corral those with any new ones.

Thanks,

-Jason


On Mon, Jul 23, 2018 at 10:26 AM, dinesh.jo...@yahoo.com.INVALID <
dinesh.jo...@yahoo.com.invalid> wrote:

> I can help out with the triage / rerunning dtests if needed.
> Dinesh
>
>    On Monday, July 23, 2018, 10:22:18 AM PDT, Jason Brown <
> jasedbr...@gmail.com> wrote:
>
>  I spoke with some people over here, and I'm going to spend a day doing a
> quick triage of the failing dtests. There are some fixes for data loss bugs
> that are critical to get out in these builds, so I'll ensure the current
> failures are within an acceptable level of flakey-ness in order to unblock
> those fixes.
>
> Will have an update shortly ...
>
> -Jason
>
> On Mon, Jul 23, 2018 at 9:18 AM, Jason Brown  wrote:
>
> > Hi all,
> >
> > First, thanks Joey for running the tests. Your pass/fail counts are
> > basically what in line with what I've seen for the last several months.
> (I
> > don't have an aggregated list anywhere, just observations from recent
> runs).
> >
> > Second, it's beyond me why there's such inertia to actually cutting a
> > release. We're getting up to almost *six months* since the last release.
> > Are there any grand objections at this point?
> >
> > Thanks,
> >
> > -Jason
> >
> >
> > On Tue, Jul 17, 2018 at 4:01 PM, Joseph Lynch 
> > wrote:
> >
> >> We ran the tests against 3.0, 2.2 and 3.11 using circleci and there are
> >> various failing dtests but all three have green unit tests.
> >>
> >> 3.11.3 tentative (31d5d87, test branch
> >> <https://circleci.com/gh/vinaykumarchella/cassandra/tree/
> >> cassandra_3.11_temp_testing>,
> >> unit tests <https://circleci.com/gh/vinaykumarchella/cassandra/258>
> >> pass, 5
> >> <https://circleci.com/gh/vinaykumarchella/cassandra/256> and 6
> >> <https://circleci.com/gh/vinaykumarchella/cassandra/256#
> >> tests/containers/8>
> >> dtest failures)
> >> 3.0.17 tentative (d52c7b8, test branch
> >> <https://circleci.com/gh/jolynch/workflows/cassandra/tree/3.0-testing>,
> >> unit
> >> tests <https://circleci.com/gh/jolynch/cassandra/110> pass, 14
> >> <https://circleci.com/gh/jolynch/cassandra/112> and 15
> >> <https://circleci.com/gh/jolynch/cassandra/111> dtest failures)
> >> 2.2.13 tentative (3482370, test branch
> >> <https://circleci.com/gh/sumanth-pasupuleti/workflows/cassan
> >> dra/tree/2.2-testing>,
> >> unit tests <https://circleci.com/gh/sumanth-pasupuleti/cassandra/20>
> >> pass,

Re: reroll the builds?

[dinesh.jo...@yahoo.com.INVALID]

If that's the case, I'm +1 on rerolling the builds.
Dinesh 

On Tuesday, July 24, 2018, 9:18:14 AM PDT, Jason Brown 
 wrote:  
 
 I did run the dtests against the last release shas (3.0.16 and 3.11.2).
Notes are all the way at the bottom of the gist about those runs. Circleci
URLs: https://circleci.com/workflow-run/5a1df5a1-f0c1-4ab4-a7db-e5551e7a5d38
/ https://circleci.com/workflow-run/a4369ab0-ae11-497a-8e10-de3995d10f25.

Current HEAD of 3.0 & 3.11 branches have significantly lower failing
dtests, and the failing tests on HEAD are a subset of those from the last
release.


On Tue, Jul 24, 2018 at 9:03 AM, dinesh.jo...@yahoo.com.INVALID <
dinesh.jo...@yahoo.com.invalid> wrote:

> Hi Jason,
> I agree - we should release with the dataloss bug fix. I went over the
> gist - apart from the Python errors and test teardown failures, there seem
> to be a few failures that look legitimate. Any chance you can run the
> dtests on the previous release SHAs and compare the dtest failures? If
> they're the same / similar, we know at least we're at parity with the
> previous release :)
> Dinesh
>
>    On Tuesday, July 24, 2018, 8:18:50 AM PDT, Jason Brown <
> jasedbr...@gmail.com> wrote:
>
>  TL;DR We are in a better place than we were for the 3.0.16 and 3.11.2
> releases. The current fails are not fatal, although they warrant
> investigation. My opinion is that due the critical data loss bugs that are
> fixed by CASSANDRA-14513 and CASSANDRA-14515, we should cut the builds now.
>
> I've run the HEAD of the 3.0 and 3.11 branches vs the 3.0.16 and 3.11.2
> release shas, and there are far less failing dtests now. In comparison:
>
> - 3.11
> -- HEAD - 5-6 failing tests
> -- 3.11.2 - 18-20 failures
>
> - 3.0
> -- HEAD - 14-16 failures
> -- 3.0.16 - 22-25 failures
>
> The raw dump of my work can be found here:
> https://gist.github.com/jasobrown/e7ecf6d0bf875d1f4a08ee06ac7eaba0. I've
> applied no effort to clean it up, but it's available (includes links to the
> circleci runs). I haven't completed an exhautive analysis of the failures
> to see how far they go back as things become tricky (or, at least, very
> time intensive to research) with the pytest/python-3 update with
> CASSANDRA-14134. Thus some of the failures might be in the dtests
> themselves (I suspect a couple of the failures are), but most are proabaly
> legit failures.
>
> As this thread is about cutting the releases, I'll save any significiant
> analysis for a followup thread. I will say that the current failures are a
> subset of the previous release's failures, and those failures are not data
> loss bugs.
>
> Overall, I feel far more comfortable getting the data loss fixes out
> without any further delay than waiting for a few minor fixes. I will triage
> the dtest failures over the coming days. There are some open tickets, and
> I'll try to corral those with any new ones.
>
> Thanks,
>
> -Jason
>
>
> On Mon, Jul 23, 2018 at 10:26 AM, dinesh.jo...@yahoo.com.INVALID <
> dinesh.jo...@yahoo.com.invalid> wrote:
>
> > I can help out with the triage / rerunning dtests if needed.
> > Dinesh
> >
> >    On Monday, July 23, 2018, 10:22:18 AM PDT, Jason Brown <
> > jasedbr...@gmail.com> wrote:
> >
> >  I spoke with some people over here, and I'm going to spend a day doing a
> > quick triage of the failing dtests. There are some fixes for data loss
> bugs
> > that are critical to get out in these builds, so I'll ensure the current
> > failures are within an acceptable level of flakey-ness in order to
> unblock
> > those fixes.
> >
> > Will have an update shortly ...
> >
> > -Jason
> >
> > On Mon, Jul 23, 2018 at 9:18 AM, Jason Brown 
> wrote:
> >
> > > Hi all,
> > >
> > > First, thanks Joey for running the tests. Your pass/fail counts are
> > > basically what in line with what I've seen for the last several months.
> > (I
> > > don't have an aggregated list anywhere, just observations from recent
> > runs).
> > >
> > > Second, it's beyond me why there's such inertia to actually cutting a
> > > release. We're getting up to almost *six months* since the last
> release.
> > > Are there any grand objections at this point?
> > >
> > > Thanks,
> > >
> > > -Jason
> > >
> > >
> > > On Tue, Jul 17, 2018 at 4:01 PM, Joseph Lynch 
> > > wrote:
> > >
> > >> We ran the tests against 3.0, 2.2 and 3.11 using circleci and there
> are
> > >> various failing dtests but all three have

Re: [VOTE] Release Apache Cassandra 3.11.3 (Take 2)

[dinesh.jo...@yahoo.com.INVALID]

+1
Dinesh 

On Wednesday, July 25, 2018, 12:46:20 AM PDT, Michael Shuler 
 wrote:  
 
 I propose the following artifacts for release as 3.11.3.

sha1: 31d5d870f9f5b56391db46ba6cdf9e0882d8a5c0
Git:
http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.11.3-tentative
Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1164/org/apache/cassandra/apache-cassandra/3.11.3/
Staging repository:
https://repository.apache.org/content/repositories/orgapachecassandra-1164/

The Debian and RPM packages are available here:
http://people.apache.org/~mshuler

The vote will be open for 72 hours (longer if needed).

[1]: CHANGES.txt:
http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/3.11.3-tentative
[2]: NEWS.txt:
http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/3.11.3-tentative

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: [VOTE] Release Apache Cassandra 3.0.17 (Take 2)

[dinesh.jo...@yahoo.com.INVALID]

+1

Dinesh 

On Wednesday, July 25, 2018, 12:47:34 AM PDT, Michael Shuler 
 wrote:  
 
 I propose the following artifacts for release as 3.0.17.

sha1: d52c7b8c595cc0d06fc3607bf16e3f595f016bb6
Git:
http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.0.17-tentative
Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1165/org/apache/cassandra/apache-cassandra/3.0.17/
Staging repository:
https://repository.apache.org/content/repositories/orgapachecassandra-1165/

The Debian and RPM packages are available here:
http://people.apache.org/~mshuler

The vote will be open for 72 hours (longer if needed).

[1]: CHANGES.txt:
http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/3.0.17-tentative
[2]: NEWS.txt:
http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/3.0.17-tentative

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: [VOTE] Release Apache Cassandra 2.2.13

[dinesh.jo...@yahoo.com.INVALID]

+1

Dinesh 

On Wednesday, July 25, 2018, 12:49:09 AM PDT, Michael Shuler 
 wrote:  
 
 I propose the following artifacts for release as 2.2.13.

sha1: 3482370df5672c9337a16a8a52baba53b70a4fe8
Git:
http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/2.2.13-tentative
Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1167/org/apache/cassandra/apache-cassandra/2.2.13/
Staging repository:
https://repository.apache.org/content/repositories/orgapachecassandra-1167/

The Debian and RPM packages are available here:
http://people.apache.org/~mshuler

The vote will be open for 72 hours (longer if needed).

[1]: CHANGES.txt:
http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/2.2.13-tentative
[2]: NEWS.txt:
http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/2.2.13-tentative

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: GitHub PR ticket spam

[dinesh.jo...@yahoo.com.INVALID]

It is useful to have a historical record. However, it could definitely be 
better (huge diffs are pointless).
Thanks,
Dinesh 

On Monday, July 30, 2018, 1:27:26 AM PDT, Stefan Podkowinski 
 wrote:  
 
 Looks like we had some active PRs recently to discuss code changes in
detail on GitHub, which I think is something we agreed is perfectly
fine, in addition to the usual Jira ticket.

What bugs me a bit is that for some reasons any comments on the PR would
be posted to the Jira ticket as well. I'm not sure what would be the
exact reason for this, I guess it's because the PR is linked in the
ticket? I find this a bit annoying while subscribed to commits@,
especially since we created pr@ for these kind of messages. Also I don't
really see any value in mirroring all github comments to the ticket.
#14556 is a good example how you could end up with tons of unformatted
code in the ticket that will also mess up search in jira. Does anyone
think this is really useful, or can we stop linking the PR in the future
(at least for highly active PRs)?


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: GitHub PR ticket spam

[dinesh.jo...@yahoo.com.INVALID]

+1 for preserving it as worklog.
Dinesh 
P.S.: Apologies for the github spam :-)
On Monday, August 6, 2018, 3:09:28 PM PDT, Mick Semb Wever 
 wrote:  
 
 
> 
> Great idea. +1 to moving it to the work log.
> 


https://issues.apache.org/jira/browse/INFRA-16879 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: GitHub PR ticket spam

[dinesh.jo...@yahoo.com.INVALID]

Nice! Thanks for getting this done.
Dinesh 

On Wednesday, August 8, 2018, 2:21:22 PM PDT, Mick Semb Wever 
 wrote:  
 
 > > 
> > Great idea. +1 to moving it to the work log.
> > 
> 
> https://issues.apache.org/jira/browse/INFRA-16879 


This is working now.

See eg https://issues.apache.org/jira/browse/CASSANDRA-12926 
 (which I hijacked for testing and the comments in the PR have since been 
deleted)

cheers,
mick

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: upgrade guava on trunk before 9/1?

[dinesh.jo...@yahoo.com.INVALID]

Jason,
Given that we're so close to the 9/1 date, I would err on the side of caution 
especially given the low value prop. If someone does run into Guava 
compatibility issues (and someone somewhere will), we can revisit this question 
then.
Dinesh 

On Wednesday, August 15, 2018, 11:42:31 PM PDT, Dinesh A. Joshi 
 wrote:  
 
 Jason,
Given that we're so close to the 9/1 date, I would err on the side of caution 
especially given the low value prop. If someone does run into Guava 
compatibility issues (and someone somewhere will), we can revisit this question 
then.
Dinesh 

On Wednesday, August 15, 2018, 8:22:28 AM PDT, Salih Gedik  
wrote:  
 
 Hi,

Change logs are on Github releases page. It seems like only hash flooding 
protection which is added to ImmutableMap is relevant to Cassandra code. I 
haven’t checked whether we use deprecated APIs. But there isn’t much on table 
from what I see.

Salih
On 15 Aug 2018 17:55 +0300, Ariel Weisberg , wrote:
> Hi,
>
> They don't even do release notes after 23. Also no API diffs. I mean I'm fine 
> with it, but it's mostly just changing to another arbitrary version that 
> won't match what is in apps.
>
> Ariel
>
> On Wed, Aug 15, 2018, at 10:48 AM, Jason Brown wrote:
> > Hey Ariel,
> >
> > Tbqh, not that much. I was mostly thinking from the "I have conflicts on
> > guava versions in my app because I pull in cassandra and XYZ libraries, and
> > the transitive dependencies on guava use different versions" POV. Further,
> > we'll be on this version of guava for 4.0 for at least two years from now.
> >
> > As I asked, "does anybody feeling strongly?". Personally, I'm sorta +0 to
> > +0.5, but I was just throwing this out there in case someone does really
> > think it best we upgrade (and wants to make a contribution).
> >
> > -Jason
> >
> >
> >
> >
> > On Wed, Aug 15, 2018 at 7:25 AM, Ariel Weisberg  wrote:
> >
> > > Hi,
> > >
> > > What do we get from Guava in exchange for upgrading?
> > >
> > > Ariel
> > >
> > > On Wed, Aug 15, 2018, at 10:19 AM, Jason Brown wrote:
> > > > Hey all,
> > > >
> > > > Does anyone feel strongly about upgrading guava on trunk before the 9/1
> > > > feature freeze for 4.0? We are currently at 23.3 (thanks to
> > > > CASSANDRA-13997), and the current is 26.0.
> > > >
> > > > I took a quick look, and there's about 17 compilation errors. They fall
> > > > into two categories, both of which appear not too difficult to resolve 
> > > > (I
> > > > didn't look too closely, tbh).
> > > >
> > > > If anyone wants to tackle this LHF I can rustle up some review time.
> > > >
> > > > Thanks,
> > > >
> > > > -Jason
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>

Re: Proposing an Apache Cassandra Management process

[dinesh.jo...@yahoo.com.INVALID]

Both approaches have merits. However the general consensus seems to be that we 
should put this in a separate repo and I agree.
Dinesh 

On Friday, August 17, 2018, 10:46:04 AM PDT, Rahul Singh 
 wrote:  
 
 I understand the issues of managing different versions of two correlated 
components — but it is possible to create unit tests with core components of 
both. It takes more effort but it is possible.

That being said, in my experience using Reaper and in the DataStax distribution 
, using OpsCenter , I prefer a separate project that is loosely tied to the 
system and not connected at the hips. Whenever there is an update to Reaper or 
OpsCenter, I can always pull it down and test it before rolling it out - and 
this is much more frequently than if I were rolling out updates to a C* cluster.


Rahul
On Aug 17, 2018, 9:41 AM -0700, Jonathan Haddad , wrote:
> Speaking from experience (Reaper), I can say that developing a sidecar
> admin / repair tool out of tree that's compatible with multiple versions
> really isn't that big of a problem. We've been doing it for a while now.
>
> https://github.com/thelastpickle/cassandra-reaper/blob/master/.travis.yml
>
> On Fri, Aug 17, 2018 at 9:39 AM Joseph Lynch  wrote:
>
> > While I would love to use a different build system (e.g. gradle) for the
> > sidecar, I agree with Dinesh that a separate repo would make sidecar
> > development much harder to verify, especially on the testing and
> > compatibility front. As Jeremiah mentioned we can always choose later to
> > release the sidecar artifact separately or more frequently than the main
> > server regardless of repo choice and as per Roopa's point having a separate
> > release artifact (jar or deb/rpm) is probably a good idea until the default
> > Cassandra packages don't automatically stop and start Cassandra on install.
> >
> > While we were developing the repair scheduler in a separate repo we had a
> > number of annoying issues that only started surfacing once we started
> > merging it directly into the trunk tree:
> > 1. It is hard to compile/test against unreleased versions of Cassandra
> > (e.g. the JMX interfaces changed a lot with 4.x, and it was pretty tricky
> > to compile against that as the main project doesn't release nightly
> > snapshots or anything like that, so we had to build local trunk jars and
> > then manually dep them).
> > 2. Integration testing and cross version compatibility is extremely hard.
> > The sidecar is going to be involved in multi node coordination (e.g.
> > monitoring, metrics, maintenance) and will be tightly coupled to JMX
> > interface choices in the server and trying to make sure that it all works
> > with multiple versions of Cassandra is much harder if it's a separate repo
> > that has to have a mirroring release cycle to Cassandra. It seems much
> > easier to have it in tree and just be like "the in tree sidecar is tested
> > against that version of Cassandra". Every time we cut a Cassandra server
> > branch the sidecar branches with it.
> >
> > We experience these pains all the time with Priam being in a separate repo,
> > where every time we support a new Cassandra version a bunch of JMX
> > endpoints break and we have to refactor the code to either call JMX methods
> > by string or cut a new Priam branch. A separate artifact is pretty
> > important, but a separate repo just allows drifts. Furthermore from the
> > Priam experience I also don't think it's realistic to say one version of a
> > sidecar artifact can actually support multiple versions.
> >
> > -Joey
> >
> > On Fri, Aug 17, 2018 at 12:00 PM Jeremiah D Jordan 
> > wrote:
> >
> > > Not sure why the two things being in the same repo means they need the
> > > same release process. You can always do interim releases of the
> > management
> > > artifact between server releases, or even have completely decoupled
> > > releases.
> > >
> > > -Jeremiah
> > >
> > > > On Aug 17, 2018, at 10:52 AM, Blake Eggleston 
> > > wrote:
> > > >
> > > > I'd be more in favor of making it a separate project, basically for all
> > > the reasons listed below. I'm assuming we'd want a management process to
> > > work across different versions, which will be more awkward if it's in
> > tree.
> > > Even if that's not the case, keeping it in a different repo at this point
> > > will make iteration easier than if it were in tree. I'd imagine (or at
> > > least hope) that validating the management process for release would be
>

Re: Apache Cassandra Blog is now live

[dinesh.jo...@yahoo.com.INVALID]

Hi Jeff,
Thanks for the patch. I assigned the ticket to you. I'll be happy to review the 
ticket later today.
Dinesh 

On Friday, August 17, 2018, 1:24:33 PM PDT, Jeff Beck  
wrote:  
 
 I submitted a patch for the feed to that ticket it also seemed like we
needed bundler to ensure we always use the same gems.

On Thu, Aug 9, 2018 at 2:44 AM Jacques-Henri Berthemet <
jacques-henri.berthe...@genesys.com> wrote:

> Hi Scott,
>
> Here is the ticket: https://issues.apache.org/jira/browse/CASSANDRA-14631
>
> Regards,
> --
> Jacques-Henri Berthemet
>
> -Original Message-
> From: Scott Andreas 
> Sent: Wednesday, August 08, 2018 6:53 PM
> To: dev@cassandra.apache.org
> Subject: Re: Apache Cassandra Blog is now live
>
> Please feel free to file a ticket (label: Documentation and Website).
>
> It looks like Jekyll, the static site generator used to build the website,
> has a plugin that generates Atom feeds if someone would like to work on
> adding one: https://github.com/jekyll/jekyll-feed
>
> 
> From: Jacques-Henri Berthemet 
> Sent: Wednesday, August 8, 2018 1:32:23 AM
> To: dev@cassandra.apache.org
> Subject: RE: Apache Cassandra Blog is now live
>
> It is possible to setup RSS on the blog? Should I create a Jira about that?
>
> --
> Jacques-Henri Berthemet
>
> -Original Message-
> From: Mick Semb Wever 
> Sent: Wednesday, August 08, 2018 8:17 AM
> To: dev@cassandra.apache.org
> Subject: Re: Apache Cassandra Blog is now live
>
>
> > We are also interested in contributing to the blog, what's the process?
>
>
> Dikang,
> it's part of the website,
> https://svn.apache.org/repos/asf/cassandra/site/src/README
>
> regards,
> Mick
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
  

Re: Proposing an Apache Cassandra Management process

[dinesh.jo...@yahoo.com.INVALID]

On Mon, Aug 20, 2018 at 4:23 AM Mick Semb Wever  wrote:
>
> We are looking to contribute Reaper to the Cassandra project.
>
> Looking at the patch it's very similar in its base design already, but
> Reaper does has a lot more to offer. We have all been working hard to move
> it to also being a side-car so it can be contributed. This raises a number
> of relevant questions to this thread: would we then accept both works in
> the Cassandra project, and what burden would it put on the current PMC to
> maintain both works.
I think the comparison is not fair. The patch that has landed is new and is the 
beginning of a sidecar within Cassandra. It would be unfair to compare its 
features with Reaper which has been around for some time now. 
I proposed a management process (not exactly a sidecar) for Cassandra about 4 
months ago. Had you guys indicated interest in contributing Reaper, we would 
not be discussing two separate implementations. Don't get me wrong, I'm happy 
that we're talking about this right now.
> This seems at odds when we're already struggling to keep up with the
> incoming patches/contributions, and there could be other git repos in the
> project we will need to support in the future too. 
I think this is a great problem to have for a project. This is a sign that the 
pool of contributors is greater than reviewers / committers. I personally have 
been volunteering my time reviewing tickets, fixing flaky tests and generally 
helping out. I definitely think we need more people actively contributing.
> The Reaper project has worked hard in building both its user and
> contributor base. And I would have thought these, including having the
> contributor base overlap with the C* PMC, were prerequisites before moving
> a larger body of work into the project (separate git repo or not). I guess
You're talking about donating a body of code i.e. Reaper which is different 
from building a new feature from scratch.
> There's been little effort in evaluating these two bodies of work, one
> which is largely unknown to us, and my concern is how we would fairly
> support both going into the future?
I don't think we should have two separate implementations as part of the 
project. It would be best if we could have a single sidecar that could have 
features from Reaper as well as the proposed patch.
Thanks,
Dinesh  

Re: Side Car New Repo vs not

[dinesh.jo...@yahoo.com.INVALID]

An option is to create a mono repo with Cassandra and SideCar as modules that 
could be built independently. This would keep source for both artifacts in the 
same repo and have their own release cadences. That said, I don't have any 
strong opinions at this point. We can try going with a separate repo and 
reevaluate it if it doesn't work out.
Dinesh 

On Monday, August 20, 2018, 9:21:33 PM PDT, Blake Eggleston 
 wrote:  
 
 If the sidecar is going to be on a different release cadence, or support 
interacting with mixed mode clusters, then it should definitely be in a 
separate repo. I don’t even know how branching and merging would work in a repo 
that supports 2 separate release targets and/or mixed mode compatibility, but 
I’m pretty sure it would be a mess.

As a cluster management tool, mixed mode is probably going to be a goal at some 
point. As a new project, it will benefit from not being tied to the C* release 
cycle (which would probably delay any sidecar release until whenever 4.1 is 
cut).


On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.ly...@gmail.com) wrote:

I think that the pros of incubating the sidecar in tree as a tool first  
outweigh the alternatives at this point of time. Rough tradeoffs that I see:  

Unique pros of in tree sidecar:  
* Faster iteration speed in general. For example when we need to add a new  
JMX endpoint that the sidecar needs, or change something from JMX to a  
virtual table (e.g. for repair, or monitoring) we can do all changes  
including tests as one commit within the main repository and don't have to  
commit to main repo, sidecar repo, and dtest repo (juggling version  
compatibility along the way).  
* We can in the future more easily move serious background functionality  
like compaction or repair itself (not repair scheduling, actual repairing)  
into the sidecar with a single atomic commit, we don't have to do two phase  
commits where we add some IPC mechanism to allow us to support it in both,  
then turn it on in the sidecar, then turn it off in the server, etc...  
* I think that the verification is much easier (sounds like Jonathan  
disagreed on the other thread, I could certainly be wrong), and we don't  
have to worry about testing matrices to assure that the sidecar works with  
various versions as the version of the sidecar that is released with that  
version of Cassandra is the only one we have to certify works. If people  
want to pull in new versions or maintain backports they can do that at  
their discretion/testing.  
* We can iterate and prove value before committing to a choice. Since it  
will be a separate artifact from the start we can always move the artifact  
to a separate repo later (but moving the other way is harder).  
* Users will get the sidecar "for free" when they install the daemon, they  
don't need to take affirmative action to e.g. be able to restart their  
cluster, run repair, or back their data up; it just comes out of the box  
for free.  

Unique pros of a separate repository sidecar:  
* We can use a more modern build system like gradle instead of ant  
* Merging changes is less "scary" I guess (I feel like if you're not  
touching the daemon this is already true but I could see this being less  
worrisome for some).  
* Releasing a separate artifact is somewhat easier from a separate repo  
(especially if we have gradle which makes e.g. building debs and rpms  
trivial).  
* We could backport to previous versions without getting into arguments  
about bug fixes vs features.  
* Committers could be different from the main repo, which ... may be a  
useful thing  

Non unique pros of a sidecar (could be achieved in the main repo or in a  
separate repo):  
* A separate build artifact .jar/.deb/.rpm that can be installed  
separately. It's slightly easier with a separate repo but certainly not out  
of reach within a single repo (indeed the current patch already creates a  
separate jar, and we could create a separate .deb reasonably easily).  
Personally I think having a separate .deb/.rpm is premature at this point  
(for companies that really want it they can build their own packages using  
the .jars), but I think it really is a distracting issue from where the  
patch should go as we can always choose to remove experimental .jar files  
that the main daemon doesn't touch.  
* A separate process lifecycle. No matter where the sidecar goes, we get  
the benefit of restarting it being less dangerous for availability than  
restarting the main daemon.  

That all being said, these are strong opinions weakly held and I would  
rather get something actually committed so that we can prove value one way  
or the other and am therefore, of course, happy to put sidecar patches  
wherever someone can review and commit it.  

-Joey  

On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli   
wrote:  

> Hi,  
> I am starting a new thread to get consensus on where the side car  
> should be contributed.  
>  
> Pleas

Re: Supporting multiple JDKs

[dinesh.jo...@yahoo.com.INVALID]

I think we should have the ability to build & run unit tests and dtests against 
a specified JDK. If we support multiple JDKs, at a minimum we should compile 
code against those JDKs. It would be ideal if we could run unit tests and 
dtests but given the availability of CircleCI resources that could be time 
consuming.
Dinesh 

On Wednesday, August 22, 2018, 10:12:35 AM PDT, Sumanth Pasupuleti 
 wrote:  
 
 Hi,

The goal of this email is to arrive at a solution (probably resulting in a
few follow-ups) to ensure support for multiple JDKs for different versions
of Cassandra.
This comes out of Jason's comment in
https://issues.apache.org/jira/browse/CASSANDRA-14563.

Below is the proposal. Please provide your thoughts.

C* 2.2
Goal:
Need to support Java7 and Java8.
Current State:
CircleCI workflow to build, run UTs and DTests against Java8 (run by
default)
Proposed State:
* Add CircleCI workflow to build against Java7 (
https://issues.apache.org/jira/browse/CASSANDRA-14625), and optionally run
UTs and DTests against Java7
* Add a tool like AnimalSniffer into build process (proposed by Mick in
https://issues.apache.org/jira/browse/CASSANDRA-14563), so that build would
fail as part of the development process in case of JDK incompatibility.


C* 3.0 and 3.11
Goal:
Support Java8.
Current state:
CircleCI workflow to build, run UTs and DTests against Java8 (run by
default)
Proposed State:
* No change. Same as current.


C* 4.0
Goal:
Support Java8 and Java11.
Current state:
CircleCI workflow to build, run UTs and DTests against Java8 (run by
default)
Proposed State:
* Add CircleCI workflow to build against Java11, and optionally run UTs and
DTests against Java11

Looking forward to any feedback.

Thanks,
Sumanth
  

Re: Reaper as cassandra-admin

[dinesh.jo...@yahoo.com.INVALID]

On Tuesday, August 28, 2018, 2:52:03 PM PDT, Blake Eggleston 
 wrote:
> I’m sure reaper will bring tech debt with it, but I doubt it's a hopeless 
> mess. 
FTR nobody has called Reaper a "hopeless mess".
> It would bring a relatively mature project as well as a community of users> 
> and developers that the other options won’t. It’s probably a lot less work to 
> > rework whatever shortcomings reaper has, add new-hotness repair

You can bring in parts of a relatively mature project that minimize refactoring 
& changes that need to be made once imported. You can also bring in best parts 
of multiples projects without importing entire codebases.
Dinesh 


On August 28, 2018 at 1:40:59 PM, Roopa Tangirala 
(rtangir...@netflix.com.invalid) wrote:
I share Dinesh's concern too regarding tech debt with existing codebase.  
Its good we have multiple solutions for repairs which have been always  
painful in Cassandra. It would be great to see the community take the best  
pieces from the available solutions and roll it into the fresh side car  
which will help ease Cassandra's maintenance for lot of folks.  

My main concern with starting with an existing codebase is that it comes  
with tech debt. This is not specific to Reaper but to any codebase that is  
imported as a whole. This means future developers and patches have to work  
within the confines of the decisions that were already made. Practically  
speaking once a codebase is established there is inertia in making  
architectural changes and we're left dealing with technical debt.  



*Regards,*  

*Roopa Tangirala*  

Engineering Manager CDE  

*(408) 438-3156 - mobile*  






On Mon, Aug 27, 2018 at 10:49 PM Dinesh Joshi  
 wrote:  

> > On Aug 27, 2018, at 5:36 PM, Jonathan Haddad  wrote:  
> > We're hoping to get some feedback on our side if that's something people  
> > are interested in. We've gone back and forth privately on our own  
> > preferences, hopes, dreams, etc, but I feel like a public discussion  
> would  
> > be healthy at this point. Does anyone share the view of using Reaper as  
> a  
> > starting point? What concerns to people have?  
>  
>  
> I have briefly looked at the Reaper codebase but I am yet to analyze it  
> better to have a real, meaningful opinion.  
>  
> My main concern with starting with an existing codebase is that it comes  
> with tech debt. This is not specific to Reaper but to any codebase that is  
> imported as a whole. This means future developers and patches have to work  
> within the confines of the decisions that were already made. Practically  
> speaking once a codebase is established there is inertia in making  
> architectural changes and we're left dealing with technical debt.  
>  
> As it stands I am not against the idea of using Reaper's features and I  
> would very much like using mature code that has been tested. I would  
> however like to propose piece-mealing it into the codebase. This will give  
> the community a chance to review what is going in and possibly change some  
> of the design decisions upfront. This will also avoid a situation where we  
> have to make many breaking changes in the initial versions due to  
> refactoring.  
>  
> I would also like it if we could compare and contrast the functionality  
> with Priam or any other interesting sidecars that folks may want to call  
> out. In fact it would be great if we could bring in the best functionality  
> from multiple implementations.  
>  
> Dinesh  
> -  
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org  
> For additional commands, e-mail: dev-h...@cassandra.apache.org  
>  
>    

Re: [Discuss] Accept GoCQL driver donation

[dinesh.jo...@yahoo.com.INVALID]

I like the idea of having an officially supported Go Driver under ASF. It would 
mean easier contributions.
I don't think we should necessarily limit it to a reference implementation. The 
industry has a strong interest in building server side as well as client 
software in Go.
Dinesh
On Friday, August 31, 2018, 7:14:03 AM PDT, Nate McCall 
 wrote:  
 
 Hi folks,
So I was recently talking with, Chris Bannister  the gocql [0]
maintainer, and he expressed an interest in donating the driver to the
ASF.

We could accept this along the same lines as how we took in the dtest
donation - going through the incubator IP clearance process [1], but
in this case it's much simpler as an individual (Chris) owns the
copyright.

I think the end goal here is to have a reference protocol
implementation controlled by the project at the least, potentially
replace cqlsh with a GoLang statically compiled binary eventually (?).

What are other folks' thoughts about this? (we are discussing, not voting).

[0] https://github.com/gocql/gocql
[1] https://incubator.apache.org/guides/ip_clearance.html

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: Request for post-freeze merge exception

[dinesh.jo...@yahoo.com.INVALID]

+1
Dinesh 

On Tuesday, September 4, 2018, 12:51:49 PM PDT, Ariel Weisberg 
 wrote:  
 
 +1 Transient Replication had some rebase pain as well, but we were able to get 
through it at the last minute. The traffic on the last few days was pretty 
heavy with several substantial commits.

On Tue, Sep 4, 2018, at 2:19 PM, Jeff Jirsa wrote:
> Seems like a reasonable thing to merge to me. Nothing else has been
> committed, it was approved pre-freeze, seems like the rush to merge was
> bound to have some number of rebase casualties.
> 
> On Tue, Sep 4, 2018 at 11:15 AM Sam Tunnicliffe  wrote:
> 
> > Hey all,
> >
> > On 2018-31-08 CASSANDRA-14145 had been +1'd by two reviewers and CI was
> > green, and so it was marked Ready To Commit. This was before the 4.0
> > feature freeze but before it landed, CASSANDRA-14408, which touched a few
> > common areas of the code, was merged. I didn't have chance to finish the
> > rebase over the weekend but in the end it turned out that most of the
> > conflicts were in test code and were straightforward to resolve. I'd like
> > to commit this now; the rebase is done (& has been re-reviewed), and the CI
> > is still green so I suspect most of the community would probably be ok with
> > that. We did vote for a freeze though and I don't want to subvert or
> > undermine that decision, so I wanted to check and give a chance for anyone
> > to raise objections before I did.
> >
> > I'll wait 24 hours, and if nobody objects before then I'll merge to trunk.
> >
> > Thanks,
> > Sam
> >

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: Proposing an Apache Cassandra Management process

[dinesh.jo...@yahoo.com.INVALID]

I have a few clarifications -
The scope of the management process is not to simply run repair scheduling. 
Repair scheduling is one of the many features we could implement or adopt from 
existing sources. So could we please split the Management Process discussion 
and the repair scheduling?
After re-reading the management process proposal, I see we missed to 
communicate a basic idea in the document. We wanted to take a pluggable 
approach to various activities that the management process could perform. This 
could accommodate different implementations of common activities such as 
repair. The management process would provide the basic framework and it would 
have default implementations for some of the basic activities. This would allow 
for speedier iteration cycles and keep things extensible.
Turning to some questions that Jon and others have raised, when I +1, my 
intention is to fully contribute and stay with this community. That said, 
things feel rushed for some but for me it feels like analysis paralysis. We're 
looking for actionable feedback and to discuss the management process _not_ 
repair scheduling solutions.
Thanks,
Dinesh



On Sep 12, 2018, at 6:24 PM, sankalp kohli  wrote:
Here is a list of open discussion points from the voting thread. I think
some are already answered but I will still gather these questions here.

>From several people:
1. Vote is rushed and we need more time for discussion.

>From Sylvain
2. About the voting process...I think that was addressed by Jeff Jirsa and
deserves a separate thread as it is not directly related to this thread.
3. Does the project need a side car.

>From Jonathan Haddad
4. Are people doing +1 willing to contribute

>From Jonathan Ellis
5. List of feature set, maturity, maintainer availability from Reaper or
any other project being donated.

Mick Semb Wever
6. We should not vote on these things and instead build consensus.

Open Questions from this thread
7. What technical debts we are talking about in Reaper. Can someone give
concrete examples.
8. What is the timeline of donating Reaper to Apache Cassandra.

On Wed, Sep 12, 2018 at 3:49 PM sankalp kohli 
wrote:


(Using this thread and not the vote thread intentionally)
For folks talking about vote being rushed. I would use the email from
Joseph to show this is not rushed. There was no email on this thread for 4
months until I pinged.


Dec 2016: Vinay worked with Jon and Alex to try to collaborate on Reaper to
come up with design goals for a repair scheduler that could work at Netflix
scale.

~Feb 2017: Netflix believes that the fundamental design gaps prevented us
from using Reaper as it relies heavily on remote JMX connections and
central coordination.

Sep. 2017: Vinay gives a lightning talk at NGCC about a highly available
and distributed repair scheduling sidecar/tool. He is encouraged by
multiple committers to build repair scheduling into the daemon itself and
not as a sidecar so the database is truly eventually consistent.

~Jun. 2017 - Feb. 2018: Based on internal need and the positive feedback at
NGCC, Vinay and myself prototype the distributed repair scheduler within
Priam and roll it out at Netflix scale.

Mar. 2018: I open a Jira (CASSANDRA-14346) along with a detailed 20 page
design document for adding repair scheduling to the daemon itself and open
the design up for feedback from the community. We get feedback from Alex,
Blake, Nate, Stefan, and Mick. As far as I know there were zero proposals
to contribute Reaper at this point. We hear the consensus that the
community would prefer repair scheduling in a separate distributed sidecar
rather than in the daemon itself and we re-work the design to match this
consensus, re-aligning with our original proposal at NGCC.

Apr 2018: Blake brings the discussion of repair scheduling to the dev list
(

https://lists.apache.org/thread.html/760fbef677f27aa5c2ab4c375c7efeb81304fea428deff986ba1c2eb@%3Cdev.cassandra.apache.org%3E
).
Many community members give positive feedback that we should solve it as
part of Cassandra and there is still no mention of contributing Reaper at
this point. The last message is my attempted summary giving context on how
we want to take the best of all the sidecars (OpsCenter, Priam, Reaper) and
ship them with Cassandra.

Apr. 2018: Dinesh opens CASSANDRA-14395 along with a public design document
for gathering feedback on a general management sidecar. Sankalp and Dinesh
encourage Vinay and myself to kickstart that sidecar using the repair
scheduler patch

Apr 2018: Dinesh reaches out to the dev list (

https://lists.apache.org/thread.html/a098341efd8f344494bcd2761dba5125e971b59b1dd54f282ffda253@%3Cdev.cassandra.apache.org%3E
)
about the general management process to gain further feedback. All feedback
remains positive as it is a potential place for multiple community members
to contribute their various sidecar functionality.

May-Jul 2017: Vinay and I work on creating a basic sidecar for running the
repair scheduler bas

Re: Proposing an Apache Cassandra Management process

[dinesh.jo...@yahoo.com.INVALID]

I will update the document to add that point. The document did not mean to 
serve as a design or architectural document but rather something that would 
spark a discussion on the idea.
Dinesh 

On Thursday, September 13, 2018, 10:59:34 AM PDT, Jonathan Haddad 
 wrote:  
 
 Most of the discussion and work was done off the mailing list - there's a
big risk involved when folks disappear for months at a time and resurface
with big pile of code plus an agenda that you failed to loop everyone in
on. In addition, by your own words the design document didn't accurately
describe what was being built.  I don't write this to try to argue about
it, I just want to put some perspective for those of us that weren't part
of this discussion on a weekly basis over the last several months.  Going
forward let's keep things on the ML so we can avoid confusion and
frustration for all parties.

With that said - I think Blake made a really good point here and it's
helped me understand the scope of what's being built better.  Looking at it
from a different perspective it doesn't seem like there's as much overlap
as I had initially thought.  There's the machinery that runs certain tasks
(what Joey has been working on) and the user facing side of exposing that
information in management tool.

I do appreciate (and like) the idea of not trying to boil the ocean, and
working on things incrementally.  Putting a thin layer on top of Cassandra
that can perform cluster wide tasks does give us an opportunity to move in
the direction of a general purpose user-facing admin tool without
committing to trying to write the full stack all at once (or even make
decisions on it now).  We do need a sensible way of doing rolling restarts
/ scrubs / scheduling and Reaper wasn't built for that, and even though we
can add it I'm not sure if it's the best mechanism for the long term.

So if your goal is to add maturity to the project by making cluster wide
tasks easier by providing a framework to build on top of, I'm in favor of
that and I don't see it as antithetical to what I had in mind with Reaper.
Rather, the two are more complementary than I had originally realized.

Jon




On Thu, Sep 13, 2018 at 10:39 AM dinesh.jo...@yahoo.com.INVALID
 wrote:

> I have a few clarifications -
> The scope of the management process is not to simply run repair
> scheduling. Repair scheduling is one of the many features we could
> implement or adopt from existing sources. So could we please split the
> Management Process discussion and the repair scheduling?
> After re-reading the management process proposal, I see we missed to
> communicate a basic idea in the document. We wanted to take a pluggable
> approach to various activities that the management process could perform.
> This could accommodate different implementations of common activities such
> as repair. The management process would provide the basic framework and it
> would have default implementations for some of the basic activities. This
> would allow for speedier iteration cycles and keep things extensible.
> Turning to some questions that Jon and others have raised, when I +1, my
> intention is to fully contribute and stay with this community. That said,
> things feel rushed for some but for me it feels like analysis paralysis.
> We're looking for actionable feedback and to discuss the management process
> _not_ repair scheduling solutions.
> Thanks,
> Dinesh
>
>
>
> On Sep 12, 2018, at 6:24 PM, sankalp kohli  wrote:
> Here is a list of open discussion points from the voting thread. I think
> some are already answered but I will still gather these questions here.
>
> From several people:
> 1. Vote is rushed and we need more time for discussion.
>
> From Sylvain
> 2. About the voting process...I think that was addressed by Jeff Jirsa and
> deserves a separate thread as it is not directly related to this thread.
> 3. Does the project need a side car.
>
> From Jonathan Haddad
> 4. Are people doing +1 willing to contribute
>
> From Jonathan Ellis
> 5. List of feature set, maturity, maintainer availability from Reaper or
> any other project being donated.
>
> Mick Semb Wever
> 6. We should not vote on these things and instead build consensus.
>
> Open Questions from this thread
> 7. What technical debts we are talking about in Reaper. Can someone give
> concrete examples.
> 8. What is the timeline of donating Reaper to Apache Cassandra.
>
> On Wed, Sep 12, 2018 at 3:49 PM sankalp kohli 
> wrote:
>
>
> (Using this thread and not the vote thread intentionally)
> For folks talking about vote being rushed. I would use the email from
> Joseph to show this is not rushed. There was no email on this thread for 4
> months until I pinged.
>
>
> Dec 2016: Vin

Re: Proposing an Apache Cassandra Management process

[dinesh.jo...@yahoo.com.INVALID]

I have created a sub-task - CASSANDRA-14783. Could we get some feedback before 
we begin implementing anything?

Dinesh 

On Thursday, September 20, 2018, 11:22:33 PM PDT, Dinesh Joshi 
 wrote:  
 
 I have updated the doc with a short paragraph providing the clarification. 
Sankalp's suggestion is already part of the doc. If there aren't further 
objections could we move this discussion over to the jira (CASSANDRA-14395)?

Dinesh

> On Sep 18, 2018, at 10:31 AM, sankalp kohli  wrote:
> 
> How about we start with a few basic features in side car. How about starting 
> with this
> 1. Bulk nodetool commands: User can curl any sidecar and be able to run a 
> nodetool command in bulk across the cluster. 
> :/bulk/nodetool/tablestats?arg0=keyspace_name.table_name&arg1=  required>
> 
> And later 
> 2: Health checks. 
> 
> On Thu, Sep 13, 2018 at 11:34 AM dinesh.jo...@yahoo.com.INVALID 
>  wrote:
> I will update the document to add that point. The document did not mean to 
> serve as a design or architectural document but rather something that would 
> spark a discussion on the idea.
> Dinesh 
> 
>    On Thursday, September 13, 2018, 10:59:34 AM PDT, Jonathan Haddad 
>mailto:j...@jonhaddad.com>> wrote:  
> 
>  Most of the discussion and work was done off the mailing list - there's a
> big risk involved when folks disappear for months at a time and resurface
> with big pile of code plus an agenda that you failed to loop everyone in
> on. In addition, by your own words the design document didn't accurately
> describe what was being built.  I don't write this to try to argue about
> it, I just want to put some perspective for those of us that weren't part
> of this discussion on a weekly basis over the last several months.  Going
> forward let's keep things on the ML so we can avoid confusion and
> frustration for all parties.
> 
> With that said - I think Blake made a really good point here and it's
> helped me understand the scope of what's being built better.  Looking at it
> from a different perspective it doesn't seem like there's as much overlap
> as I had initially thought.  There's the machinery that runs certain tasks
> (what Joey has been working on) and the user facing side of exposing that
> information in management tool.
> 
> I do appreciate (and like) the idea of not trying to boil the ocean, and
> working on things incrementally.  Putting a thin layer on top of Cassandra
> that can perform cluster wide tasks does give us an opportunity to move in
> the direction of a general purpose user-facing admin tool without
> committing to trying to write the full stack all at once (or even make
> decisions on it now).  We do need a sensible way of doing rolling restarts
> / scrubs / scheduling and Reaper wasn't built for that, and even though we
> can add it I'm not sure if it's the best mechanism for the long term.
> 
> So if your goal is to add maturity to the project by making cluster wide
> tasks easier by providing a framework to build on top of, I'm in favor of
> that and I don't see it as antithetical to what I had in mind with Reaper.
> Rather, the two are more complementary than I had originally realized.
> 
> Jon
> 
> 
> 
> 
> On Thu, Sep 13, 2018 at 10:39 AM dinesh.jo...@yahoo.com.INVALID
> mailto:dinesh.jo...@yahoo.com>.invalid> wrote:
> 
> > I have a few clarifications -
> > The scope of the management process is not to simply run repair
> > scheduling. Repair scheduling is one of the many features we could
> > implement or adopt from existing sources. So could we please split the
> > Management Process discussion and the repair scheduling?
> > After re-reading the management process proposal, I see we missed to
> > communicate a basic idea in the document. We wanted to take a pluggable
> > approach to various activities that the management process could perform.
> > This could accommodate different implementations of common activities such
> > as repair. The management process would provide the basic framework and it
> > would have default implementations for some of the basic activities. This
> > would allow for speedier iteration cycles and keep things extensible.
> > Turning to some questions that Jon and others have raised, when I +1, my
> > intention is to fully contribute and stay with this community. That said,
> > things feel rushed for some but for me it feels like analysis paralysis.
> > We're looking for actionable feedback and to discuss the management process
> > _not_ repair scheduling solutions.
> > Thanks,
> > Dinesh
> >
> >
> >
> > On Sep 12, 2018, at 6:24

Re: [DISCUSS] changing default token behavior for 4.0

[dinesh.jo...@yahoo.com.INVALID]

Logistics aside, I think it is a good idea to default 1 token (or a low 
number). Let the user understand what it means to go beyond 1 and tune things 
based on their needs.
Dinesh 

On Friday, September 21, 2018, 5:06:14 PM PDT, Jonathan Haddad 
 wrote:  
 
 One thing that's really, really bothered me for a while is how we default
to 256 tokens still.  There's no experienced operator that leaves it as is
at this point, meaning the only people using 256 are the poor folks that
just got started using C*.  I've worked with over a hundred clusters in the
last couple years, and I think I only worked with one that had lowered it
to something else.

I think it's time we changed the default to 4 (or 8, up for debate).

To improve the behavior, we need to change a couple other things.  The
allocate_tokens_for_keyspace setting is... odd.  It requires you have a
keyspace already created, which doesn't help on new clusters.  What I'd
like to do is add a new setting, allocate_tokens_for_rf, and set it to 3 by
default.

To handle clusters that are already using 256 tokens, we could prevent the
new node from joining unless a -D flag is set to explicitly allow
imbalanced tokens.

We've agreed to a trunk freeze, but I feel like this is important enough
(and pretty trivial) to do now.  I'd also personally characterize this as a
bug fix since 256 is horribly broken when the cluster gets to any
reasonable size, but maybe I'm alone there.

I honestly can't think of a use case where random tokens is a good choice
anymore, so I'd be fine / ecstatic with removing it completely and
requiring either allocate_tokens_for_keyspace (for existing clusters)
or allocate_tokens_for_rf
to be set.

Thoughts?  Objections?
-- 
Jon Haddad
http://www.rustyrazorblade.com
twitter: rustyrazorblade
  

Re: Proposing an Apache Cassandra Management process

[dinesh.jo...@yahoo.com.INVALID]

*bump*
Mick - could you enumerate the list of tools you mentioned earlier?

Dinesh 

On Sunday, September 23, 2018, 6:22:48 PM PDT, Dinesh Joshi 
 wrote:  
 
 Hi Mick,

Thanks for the feedback. Please find my clarifications inline.

> There's also been suggestions to take a step away from any focus on code and 
> go back to documentation and brainstorming. I appreciate the work already 
> done in the gdoc, but there's a lot left to the imagination (or just not 
> properly summarised in one place).

As I mentioned in my previous emails, the gdoc was just a starting point for a 
discussion / brainstorming.

> So could we take a step back from even designing and go to brainstorming. 
> Examples questions i can think of are running through the different scenarios 
> we're thinking of improving/solving, surveying the different C* tooling out 
> there and what they solve, need or could benefit from, ("feature set, 
> maturity, maintainer availability, etc"). I can think of a number of tools 
> already that provide the ability for cluster-wide commands. Such a page 
> listing all the tools and what they offer would be of benefit to the 
> community.

Great point, I would appreciate enumerating the list of tools you know. This is 
the kind of feedback we would've liked on the gdoc. We can look at all the 
features they provide. However, I am curious other than looking at the feature 
set, what is the motivation for surveying these tools?

> But I also thought we agreed to take a piecemeal approach between solutions, 
> rather than jumping in and developing the one and only defined sub-task from 
> scratch.

Yes I think we have a consensus on the piecemeal approach (if something doesn't 
agree, please speak up!). 

> And I can't say I'm a huge fan of google doc. Any chance we could be 
> collaborating in the Cassandra wiki instead?
> https://cwiki.apache.org/confluence/display/CASSANDRA

Sure, I don't think I have a particular preference for gdoc. It's easier to 
collaborate. I am not sure if everyone has write access to the apache wiki. It 
also requires a separate account.

I also haven't seen a process to propose & discuss larger changes to Cassandra. 
The Cassandra contribution[1] guide possibly needs to be updated. Some 
communities have a process which facilitate things. See Kafka Improvement 
Process[2], Spark Improvement Process[3].

Thanks,

Dinesh

[1] http://cassandra.apache.org/doc/latest/development/index.html
[2] 
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
[3] http://spark.apache.org/improvement-proposals.html


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org
  

Re: Proposing an Apache Cassandra Management process

[dinesh.jo...@yahoo.com.INVALID]

Thanks for starting this, Mick. I will flesh it out.
Dinesh 

On Sunday, October 21, 2018, 1:52:10 AM PDT, Mick Semb Wever 
 wrote:  
 
 
> But I'll try to put together a strawman proposal for the doc(s) over the 
> weekend. 


I've thrown something quickly together here:
 - https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652201
 - 
https://cwiki.apache.org/confluence/display/CASSANDRA/CIP-1%3A+Proposing+an+Apache+Cassandra+Management+process

The former is a blatant rip-off from the Kafka and Spark design proposal pages 
that Dinesh previously mentioned. I'd hoped to do more of an analysis of the 
existing C* habits and precedence on design proposals (implicit in jira 
tickets), but in lei of that this is a strawman to start the discussion.

The latter still needs to be fleshed out. Dinesh, can you do this? I can add a 
subpage/section that describes the alternative/consuming third-party tools out 
there.

regards,
Mick

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: Request for reviewer: CASSANDRA-14829

[dinesh.jo...@yahoo.com.INVALID]

Hi Georg,
I took a look at your patch and left some comments on the ticket.
Thanks,
Dinesh 

On Friday, November 16, 2018, 12:04:39 PM PST, Jeff Jirsa 
 wrote:  
 
 The assignment is just so you get “credit” for the patch - asking for a 
reviewer is good but not strictly necessary. 

(Some of the committers will try to review it when we can, usually waiting for 
someone who’s comfortable with that code to come along)

-- 
Jeff Jirsa


> On Nov 16, 2018, at 11:33 AM, Georg Dietrich  wrote:
> 
> Hi here,
> 
> I've posted https://issues.apache.org/jira/browse/CASSANDRA-14829 together 
> with a pull request, now I've been assigned the task... I assume that means I 
> should go look for a reviewer?
> 
> Regards
> Georg
> 
> --
> 
> Georg Dietrich
> Senior System Developer
> imbus TestBench
> Tel. +49 9131 7518-944
> E-Mail: georg.dietr...@imbus.de
> 
> Tel. +49 9131 7518-0, Fax +49 9131 7518-50
> i...@imbus.de www.imbus.de
> 
> imbus AG, Kleinseebacher Str. 9,  91096 Möhrendorf, DEUTSCHLAND
> Vorsitzender des Aufsichtsrates: Wolfgang Wieser
> Vorstand: Tilo Linz, Bernd Nossem, Thomas Roßner
> Sitz der Gesellschaft: Möhrendorf; Registergericht: Fürth/Bay, HRB 8365
> 
> Post/Besuchsadresse: imbus AG, Hauptstraße 8a, 91096 Möhrendorf, Deutschland
> =
> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org
  

Re: Proposing an Apache Cassandra Management process

[dinesh.jo...@yahoo.com.INVALID]

Thanks, Mick & Stefan for your inputs. What should we do as next steps to move 
this proposal forward?
Dinesh 

On Sunday, November 18, 2018, 11:57:54 AM PST, Stefan Podkowinski 
 wrote:  
 
 My goal for a side car would be to enable more people to contribute to
the project, by making it more accessible for anyone who’s not familiar
with the Cassandra code base, or not familiar with Java development in
general. Although most of the functionality described in the proposal
sounds useful to have, I’d already be happy to have a solid REST API for
the existing nodetool and JMX functionality. If an official side car,
installed separately on each node, would provide that, I’m sure we’d see
lots of new tools created by the community (web UIs, cli tools, ..)
based on that. This would also be a good foundation for other existing
tool to converge upon, e.g. by calling the REST APIs for repair
scheduling and progress tracking instead of JMX, or by continually
integrating and sharing useful helper calls. This would also give
Cassandra devs more leeway to replace some of the existing tooling
related code in Cassandra, e.g. by migrating to virtual tables, while at
the same time keep providing a stable API through the side car.

What I’d also like to point out here is that implementing such a project
as an *official* side car, also implies to me having the same standards
when it comes to release quality. I’d also really prefer having feature
sets matching between Cassandra and the side car, e.g. authentication
and SSL should also be supported in the side car from the beginning,
ideally without any additional configuration.


On 06.11.18 10:40, Dinesh Joshi wrote:
> Hi all,
>
> Joey, Vinay & I have fleshed out the Management process proposal as the very 
> first CIP document (with Jason’s inputs). It is available on the cwiki - 
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652224
>
> Please comment on it and provide us any input that you may have. We want to 
> ideally time-box the period to 2 weeks so we avoid waiting indefinitely.
>
> Thanks,
>
> Dinesh
>
>> On Oct 22, 2018, at 7:30 AM, "dinesh.jo...@yahoo.com.INVALID" 
>>  wrote:
>>
>> Thanks for starting this, Mick. I will flesh it out.
>> Dinesh 
>>
>>    On Sunday, October 21, 2018, 1:52:10 AM PDT, Mick Semb Wever 
>> wrote:  
>>
>>
>>> But I'll try to put together a strawman proposal for the doc(s) over the 
>>> weekend. 
>>
>> I've thrown something quickly together here:
>> - https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652201
>> - 
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CIP-1%3A+Proposing+an+Apache+Cassandra+Management+process
>>
>> The former is a blatant rip-off from the Kafka and Spark design proposal 
>> pages that Dinesh previously mentioned. I'd hoped to do more of an analysis 
>> of the existing C* habits and precedence on design proposals (implicit in 
>> jira tickets), but in lei of that this is a strawman to start the discussion.
>>
>> The latter still needs to be fleshed out. Dinesh, can you do this? I can add 
>> a subpage/section that describes the alternative/consuming third-party tools 
>> out there.
>>
>> regards,
>> Mick
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: I/O threads busy error

[dinesh.jo...@yahoo.com.INVALID]

You're probably hitting this - 
https://github.com/datastax/cpp-driver/blob/2.0/src/session.cpp#L740
>From my reading it feels you may want to throttle your queries or play around 
>with the driver settings. Essentially it seems the number of queries you're 
>issuing is greater than what the cluster can handle and therefore they're 
>queuing up in the driver.
Dinesh 

On Monday, November 19, 2018, 10:24:10 PM PST, vishal1.sha...@ril.com 
 wrote:  
 
 Dear all,
I've got a 2 node Cassandra cluster(3.11.2). I'm using Datastax's c++ 
driver(ver 2.7) in my application to fetch/add data from the cluster(RHEL 6.9). 
Sometimes I'm getting the error "All connections on all I/O threads are busy" 
in my application.  I'm unable to figure out why I am getting this error 
infrequently and not everytime. 

Has anyone faced such errors before? Please let me know any possible reasons 
behind the issue.

Thanks and regards,
Vishal Sharma

P.S. I don't really know whether this is a Datastax driver's issue or 
Cassandra's issue that's why I'm posting this query here.
"Confidentiality Warning: This message and any attachments are intended only 
for the use of the intended recipient(s). 
are confidential and may be privileged. If you are not the intended recipient. 
you are hereby notified that any 
review. re-transmission. conversion to hard copy. copying. circulation or other 
use of this message and any attachments is 
strictly prohibited. If you are not the intended recipient. please notify the 
sender immediately by return email. 
and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure 
no viruses are present in this email. 
The company cannot accept responsibility for any loss or damage arising from 
the use of this email or attachment."
B�CB��[��X��ܚX�KK[XZ[�]�][��X��ܚX�P�\��[��K�\X�K�ܙ�B��܈Y][ۘ[��[X[��K[XZ[�]�Z[�\��[��K�\X�K�ܙ�B�B
  

Re: Request to review feature-freeze proposed tickets

[dinesh.jo...@yahoo.com.INVALID]

Kurt, I don't believe this should be subject of "heated debate". If those 
tickets were sitting in patch available state prior to the freeze they *should* 
get in.
Vinay, I can help review the tickets.
Dinesh 

On Tuesday, November 20, 2018, 2:59:18 PM PST, kurt greaves 
 wrote:  
 
 Thanks Vinay. While I suspect this will be subject to heated debate, I'm
also for this. The time to review for this project is incredibly
demotivating, and it stems from a lack of contributors that are interested
in the general health of the project. I think this can be quite easily
remedied by making more committers/PMC, however there is a catch-22 that to
achieve this our existing set of committers needs to be dedicated to
reviewing contributions from non-committers.

If we can get dedicated reviewers for the listed tickets I'll take on some
of the work to get the tickets up to scratch.

On Wed, 21 Nov 2018 at 02:12, Ariel Weisberg  wrote:

> Hi,
>
> I would like to get as many of these as is feasible in. Before the feature
> freeze started 1 out of 17 JIRAs that were patch available were reviewed
> and committed.
>
> If you didn’t have access reviewers and committers, as the one out of the
> 17 did, it has been essentially impossible to get your problems with
> Cassandra fixed in 4.0.
>
> This is basically the same as saying that despite the fact Cassandra is
> open source it does you no good because it will be years before the issues
> impacting you get fixed even if you contribute the fixes yourself.
>
> Pulling up the ladder after getting “your own” fixes in is a sure fire way
> to fracture the community into a collection of private forks containing the
> fixes people can’t live without, and pushing people to look at alternatives.
>
> Private forks are a serious threat to the project. The people on them are
> at risk of getting left behind and Cassandra stagnates for them and becomes
> uncompetitive. Those with the resources to maintain a seriously diverged
> fork are also the ones better positioned to be active contributors.
>
> Regards,
> Ariel
>
> > On Nov 18, 2018, at 9:18 PM, Vinay Chella 
> wrote:
> >
> > Hi,
> >
> > We still have 15 Patch Available/ open tickets which were requested for
> > reviews before the Sep 1, 2018 freeze. I am starting this email thread to
> > resurface and request a review of community tickets as most of these
> > tickets address vital correctness, performance, and usability bugs that
> > help avoid critical production issues. I tried to provide context on why
> we
> > feel these tickets are important to get into 4.0. If you would like to
> > discuss the technical details of a particular ticket, let's try to do
> that
> > in JIRA.
> >
> > CASSANDRA-14525: Cluster enters an inconsistent state after bootstrap
> > failures. (Correctness bug, Production impact, Ready to Commit)
> >
> > CASSANDRA-14459: DES sends requests to the wrong nodes routinely. (SLA
> > breaking latencies, Production impact, Review in progress)
> >
> > CASSANDRA-14303 and CASSANDRA-14557: Currently production 3.0+ clusters
> > cannot be rebuilt after node failure due to 3.0’s introduction of the
> > system_auth keyspace with rf of 1. These tickets both fix the regression
> > introduced in 3.0 by letting operators configure rf=3 and prevent future
> > outages (Usability bug, Production impact, Patch Available).
> >
> > CASSANDRA-14096: Cassandra 3.11.1 Repair Causes Out of Memory. We believe
> > this may also impact 3.0 (Title says it all, Production impact, Patch
> > Available)
> >
> > CASSANDRA-10023: It is impossible to accurately determine local
> read/write
> > calls on C*. This patch allows users to detect when they are choosing
> > incorrect coordinators. (Usability bug (troubleshoot), Review in
> progress)
> >
> > CASSANDRA-10789: There is no way to safely stop bad clients bringing down
> > C* nodes. This patch would give operators a very important tool to use
> > during production incidents to mitigate impact. (Usability bug,
> Production
> > Impact (recovery), Patch Available)
> >
> > CASSANDRA-13010: No visibility into which disk is being compacted to.
> > (Usability bug, Production Impact (troubleshoot), Review in progress)
> >
> > CASSANDRA-12783 - Break up large MV mutations to prevent OOMs (Title says
> > it all, Production Impact, Patch InProgress/ Awaiting Feedback)
> >
> > CASSANDRA-14319 - nodetool rebuild from DC lets you pass invalid
> > datacenters (Usability bug, Production impact, Patch available)
> >
> > CASSANDRA-13841 - Smarter nodetool rebuild. Kind of a bug but would be
> nice
> > to get it in 4.0. (Production Impact (recovery), Patch Available)
> >
> > CASSANDRA-9452: Cleanup of old configuration, confusing to new C*
> > operators. (Cleanup, Patch Available)
> >
> > CASSANDRA-14309: Hint window persistence across the record. This way
> hints
> > that are accumulated over a period of time when nodes are creating are
> less
> > likely to take down the entire cluster. (Potential Production I

Re: RES: RES: Implicit Casts for Arithmetic Operators

[dinesh.jo...@yahoo.com.INVALID]

To clarify send an empty email (no subject or body) to 
dev-unsubscr...@cassandra.apache.org
You will then get a confirmation email with a link. Click that.
Dinesh 

On Tuesday, November 20, 2018, 6:20:34 PM GMT+2, Michael Shuler 
 wrote:  
 
 On 11/20/18 10:15 AM, Versátil wrote:
> 
> I already requested as you said and it did not help. And I NEVER asked to 
> enter into this discussion. Please request to withdraw my email 

 | | | | | | | | | |
\/\/\/\/\/\/\/\/\/\/

> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: Revisit the proposal to use github PR

[dinesh.jo...@yahoo.com.INVALID]

I've been already using github PRs for some time now. Once you specify the 
ticket number, the comments and discussion are persisted in Apache Jira as work 
log so it can be audited if desired. However, committers usually squash and 
commit the changes once the PR is approved. We don't use the merge feature in 
github. I don't believe github we can merge the commit into multiple branches 
through the UI. We would need to merge it into one branch and then manually 
merge that commit into other branches. The big upside of using github PRs is 
that it makes collaborating a lot easier. Downside is that it makes it very 
difficult to follow along the progress in Apache Jira. The messages that github 
posts back include huge diffs and are aweful.
Dinesh 

On Thursday, December 13, 2018, 1:10:12 AM GMT+5:30, Benedict Elliott Smith 
 wrote:  
 
 Perhaps somebody could summarise the tradeoffs?  I’m a little concerned about 
how it would work for our multi-branch workflow.  Would we open multiple PRs?

Could we easily link with external CircleCI?

It occurs to me, in JIRA proposal mode, that an extra required field for a 
permalink to GitHub for the patch would save a lot of time I spend hunting for 
a branch in the comments.




> On 12 Dec 2018, at 19:20, jay.zhu...@yahoo.com.INVALID wrote:
> 
> It was discussed 1 year's ago: 
> https://www.mail-archive.com/dev@cassandra.apache.org/msg11810.html
> As all Apache projects are moving to gitbox: 
> https://reference.apache.org/committer/github, should we revisit that and 
> change our review/commit process to use github PR?A good example is 
> Spark:"Changes to Spark source code are proposed, reviewed and committed via 
> Github pull requests" (https://spark.apache.org/contributing.html).
> /jay


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org
  

Re: CASSANDRA-14925 DecimalSerializer.toString() can OOM

[dinesh.jo...@yahoo.com.INVALID]

I think it makes sticking to trunk as this change will affect log messages and 
may break tooling that depends on certain patterns.
Dinesh 

On Friday, December 14, 2018, 4:09:51 PM GMT+5:30, Jasonstack Zhao Yang 
 wrote:  
 
 Hi,

Would like to get some feedback for CASSANDRA-14925.

In order to avoid potential OOM attack, we propose to change
DecimalSerializer.toString() from `BigDecimal.toPlainString()` to
`BigDecimal.toString()` on Trunk.

This change should not cause any compatibility issues..

Thanks
Zhao Yang
  

Re: about ticket "CASSANDRA-14847"

[dinesh.jo...@yahoo.com.INVALID]

Hi Fumiya, I'm looking at it this week.
Dinesh 

On Sunday, February 3, 2019, 9:46:35 PM PST, Fumiya Yamashita 
 wrote:  
 
 Hello.

I have created a ticket for the improvement of nodetool.
(https://issues.apache.org/jira/browse/CASSANDRA-14847)

I’d like this patch to be merged to Cassandra 4.0,
So which version should I choose as Fix version?
{4.0 | 4.0.x | 4.x}

Thank you.

---
Fumiya Yamashita(fyama...@yahoo-corp.jp)
Yahoo Japan Corporation

  

cqlsh tests and Python 3

[dinesh.jo...@yahoo.com.INVALID]

Hey all,
We've gotten the cqlsh tests running in the Cassandra repo (these are distinct 
from the cqlsh tests in dtests repo). They're in Python 2.7 and using the 
nosetests. We'd like to make them consistent with the rest of the tests which 
means moving them to Python 3 & Pytest framework. However this would involve 
migrating cqlsh to Python 3. Does anybody have any concerns if we move cqlsh to 
Python 3? Please note that Python 2 is EOL'd and will be unsupported in about 
10 months.
So here are the options -
1. Leave cqlsh in Python 2.7 & nosetests. Just make sure they're running as 
part of the build process.2. Move cqlsh to Python 3 & pytests.3. Leave cqlsh in 
Python 2.7 but move to Pytests. This option doesn't really add much value 
though.
Thanks,
Dinesh

Re: cqlsh tests and Python 3

[dinesh.jo...@yahoo.com.INVALID]

I saw that thread and the tickets. They haven't had any activity recently. 
Given that it is already Feb 2019 and Python 2.7 is getting close to EOL'd, I 
think it's worth moving forward with deprecating Python 2.7 support and adding 
3.0 support prior to 4.0 release. I am not sure what the timeline is looking 
like for 4.0 right now but it would be silly if we release 4.0 so close to 
Python 2.7 EOL.
Dinesh 

On Tuesday, February 12, 2019, 7:00:45 AM PST, Stefan Podkowinski 
 wrote:  
 
 Previous discussion can be found here:

https://lists.apache.org/thread.html/cbc50f5ac085ac759b52eb7e87277a3b82e2773c6d507c4b525d@%3Cdev.cassandra.apache.org%3E


On 11.02.19 19:58, Ariel Weisberg wrote:
> Hi,
>
> Do you mean Python 2/3 compatibility?
>
> This has been discussed earlier and I think that being compatible with both 
> is an easier sell.
>
> Ariel
>
>> On Feb 11, 2019, at 1:24 PM, dinesh.jo...@yahoo.com.INVALID 
>>  wrote:
>>
>> Hey all,
>> We've gotten the cqlsh tests running in the Cassandra repo (these are 
>> distinct from the cqlsh tests in dtests repo). They're in Python 2.7 and 
>> using the nosetests. We'd like to make them consistent with the rest of the 
>> tests which means moving them to Python 3 & Pytest framework. However this 
>> would involve migrating cqlsh to Python 3. Does anybody have any concerns if 
>> we move cqlsh to Python 3? Please note that Python 2 is EOL'd and will be 
>> unsupported in about 10 months.
>> So here are the options -
>> 1. Leave cqlsh in Python 2.7 & nosetests. Just make sure they're running as 
>> part of the build process.2. Move cqlsh to Python 3 & pytests.3. Leave cqlsh 
>> in Python 2.7 but move to Pytests. This option doesn't really add much value 
>> though.
>> Thanks,
>> Dinesh
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: cqlsh tests and Python 3

[dinesh.jo...@yahoo.com.INVALID]

Hi Patrick,
I can help with validating 14298. We recently merged in changes that got the 
nosetests for cqlshlib working on trunk. The test failures are mostly due to 
4.0 changes. We can either fix them and then migrate to Python 3 or we can 
directly fix the test failures once we move to Python 3.
Dinesh 

On Tuesday, February 12, 2019, 1:17:08 PM PST, Patrick Bannister 
 wrote:  
 
 
I did a lot of work to make cqlsh compatible with Python 3 (and also Python 
2.7) under CASSANDRA-10190. CASSANDRA-10190 has been blocked by 
CASSANDRA-14298, which got about two thirds of the cqlsh dtests to work.
If somebody could commit to reviewing CASSANDRA-14298, I'd be willing to pick 
it back up and get it working with the current head of develop. That way we 
would be in a better place to address CASSANDRA-10190 and the Python 3 issue.

Patrick Bannister

On Tue, Feb 12, 2019 at 2:46 PM dinesh.jo...@yahoo.com.INVALID 
 wrote:

I saw that thread and the tickets. They haven't had any activity recently. 
Given that it is already Feb 2019 and Python 2.7 is getting close to EOL'd, I 
think it's worth moving forward with deprecating Python 2.7 support and adding 
3.0 support prior to 4.0 release. I am not sure what the timeline is looking 
like for 4.0 right now but it would be silly if we release 4.0 so close to 
Python 2.7 EOL.
Dinesh 

    On Tuesday, February 12, 2019, 7:00:45 AM PST, Stefan Podkowinski 
 wrote:  

 Previous discussion can be found here:

https://lists.apache.org/thread.html/cbc50f5ac085ac759b52eb7e87277a3b82e2773c6d507c4b525d@%3Cdev.cassandra.apache.org%3E


On 11.02.19 19:58, Ariel Weisberg wrote:
> Hi,
>
> Do you mean Python 2/3 compatibility?
>
> This has been discussed earlier and I think that being compatible with both 
> is an easier sell.
>
> Ariel
>
>> On Feb 11, 2019, at 1:24 PM, dinesh.jo...@yahoo.com.INVALID 
>>  wrote:
>>
>> Hey all,
>> We've gotten the cqlsh tests running in the Cassandra repo (these are 
>> distinct from the cqlsh tests in dtests repo). They're in Python 2.7 and 
>> using the nosetests. We'd like to make them consistent with the rest of the 
>> tests which means moving them to Python 3 & Pytest framework. However this 
>> would involve migrating cqlsh to Python 3. Does anybody have any concerns if 
>> we move cqlsh to Python 3? Please note that Python 2 is EOL'd and will be 
>> unsupported in about 10 months.
>> So here are the options -
>> 1. Leave cqlsh in Python 2.7 & nosetests. Just make sure they're running as 
>> part of the build process.2. Move cqlsh to Python 3 & pytests.3. Leave cqlsh 
>> in Python 2.7 but move to Pytests. This option doesn't really add much value 
>> though.
>> Thanks,
>> Dinesh
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  
  

Re: CASSANDRA-14482

[dinesh.jo...@yahoo.com.INVALID]

Thanks Ariel & Jonathan.
Michael, I have addressed the comments in the issue so we're using the latest 
jar now. I've posted the updated patch in the latest comment of the ticket. In 
addition there is a JMH performance benchmark in the C* repo 
(CompressorPerformance) that tests various Compressors. Depending on the 
`compression_level` setting we can see a boost in compression speed. However, 
the major win with Zstd is not only the speed but the superior compression 
ratio. While lz4 can achieve about 2.101, Zstd can achieve a ratio of 2.877 
which is a huge difference.
The comparisons are here - https://github.com/facebook/zstd
Dinesh 

On Friday, February 15, 2019, 9:56:49 AM PST, Michael Shuler 
 wrote:  
 
 +0.5

I skimmed the jira and github diff and a few things came to mind:
- There are multiple comments about using an older jar than the latest
version.
- I did not see any performance test results to form an opinion on any
gains/caveats as a user. This was the first thing I looked for.
- I did not see any conf/cassandra.yaml comment in the diff for the
valid class_name configuration option to use.

Seems interesting, and there are comments about 4.0 being in a freeze
and all, but OK on it being non-default.

-- 
Michael

On 2/15/19 11:43 AM, Ariel Weisberg wrote:
> Hi,
> 
> I am +1 since it's an additional compressor and not the default.
> 
> Ariel
> 
> On Fri, Feb 15, 2019, at 11:41 AM, Dinesh Joshi wrote:
>> Hey folks,
>>
>> Just wanted to get a pulse on whether we can proceed with ZStd support. 
>> The consensus on the ticket was that it’s a very valuable addition 
>> without any risk of destabilizing 4.0. It’s ready to go if there aren’t 
>> any objections.
>>
>> Dinesh
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: CASSANDRA-14482

[dinesh.jo...@yahoo.com.INVALID]

Thanks all for your input. The consensus is to go forward with this ticket.
Dinesh 

On Friday, February 15, 2019, 12:54:20 PM PST, Sumanth Pasupuleti 
 wrote:  
 
 +1

On Fri, Feb 15, 2019 at 12:14 PM Dikang Gu  wrote:

> +1
>
> On Fri, Feb 15, 2019 at 10:27 AM Vinay Chella 
> wrote:
>
> > We have been using Zstd compressor across different products/services
> here
> > and have seen significant improvements, getting this in 4.0 would be a
> big
> > win.
> >
> > +1
> >
> > Thanks,
> > Vinay Chella
> >
> >
> > On Fri, Feb 15, 2019 at 10:19 AM Jeff Jirsa  wrote:
> >
> > > +1
> > >
> > > --
> > > Jeff Jirsa
> > >
> > >
> > > > On Feb 15, 2019, at 9:35 AM, Jonathan Ellis 
> wrote:
> > > >
> > > > IMO "add a new compression class that has demonstrable benefits to
> > Sushma
> > > > and Joseph" is sufficiently noninvasive that we should allow it into
> > 4.0.
> > > >
> > > > On Fri, Feb 15, 2019 at 10:48 AM Dinesh Joshi
> > > >  wrote:
> > > >
> > > >> Hey folks,
> > > >>
> > > >> Just wanted to get a pulse on whether we can proceed with ZStd
> > support.
> > > >> The consensus on the ticket was that it’s a very valuable addition
> > > without
> > > >> any risk of destabilizing 4.0. It’s ready to go if there aren’t any
> > > >> objections.
> > > >>
> > > >> Dinesh
> > > >>
> > > >>
> -
> > > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> > > >>
> > > >>
> > > >
> > > > --
> > > > Jonathan Ellis
> > > > co-founder, http://www.datastax.com
> > > > @spyced
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> > >
> >
>
>
> --
> Dikang
>  

Apache Cassandra meetup @ Instagram HQ

[dinesh.jo...@yahoo.com.INVALID]

Hi all,

Apologies for the cross-post. In case you're in the SF Bay Area, Instagram is 
hosting a meetup. Interesting talks on Cassandra Traffic management, Cassandra 
on Kubernetes. See details in the attached link -
https://www.eventbrite.com/e/cassandra-traffic-management-at-instagram-cassandra-and-k8s-with-instaclustr-tickets-54986803008

Thanks,
Dinesh

Re: Apache Cassandra meetup @ Instagram HQ

[dinesh.jo...@yahoo.com.INVALID]

Just heard back. Unfortunately they cannot live stream but they'll try to 
record it and post it online.
Dinesh 

On Tuesday, February 19, 2019, 3:55:15 PM PST, Sundaramoorthy, Natarajan 
 wrote:  
 
 Dinesh Thank you.



-Original Message-
From: andre wilkinson [mailto:antonio...@me.com.INVALID] 
Sent: Tuesday, February 19, 2019 5:14 PM
To: dev@cassandra.apache.org
Subject: Re: Apache Cassandra meetup @ Instagram HQ

Thank you for reaching out 

Sent from my iPhone

> On Feb 19, 2019, at 8:45 AM, Dinesh Joshi  
> wrote:
> 
> I’ve emailed the organizers. They didn’t intend to live stream it but will 
> get back to us if they can.
> 
> Cheers,
> 
> Dinesh
> 
>> On Feb 19, 2019, at 1:11 AM, andre wilkinson  
>> wrote:
>> 
>> Yes is there anyway to attend remotely?
>> 
>> Sent from my iPhone
>> 
>>> On Feb 18, 2019, at 6:40 PM, Shaurya Gupta  wrote:
>>> 
>>> Hi,
>>> 
>>> This looks very interesting to me. Can I attend this remotely?
>>> 
>>> Thanks
>>> Shaurya
>>> 
>>> 
>>> On Tue, Feb 19, 2019 at 5:37 AM dinesh.jo...@yahoo.com.INVALID
>>>  wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> Apologies for the cross-post. In case you're in the SF Bay Area, Instagram
>>>> is hosting a meetup. Interesting talks on Cassandra Traffic management,
>>>> Cassandra on Kubernetes. See details in the attached link -
>>>> 
>>>> 
>>>> https://www.eventbrite.com/e/cassandra-traffic-management-at-instagram-cassandra-and-k8s-with-instaclustr-tickets-54986803008
>>>> 
>>>> Thanks,
>>>> 
>>>> Dinesh
>>>> 
>>> 
>>> 
>>> -- 
>>> Shaurya Gupta
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>> 
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.
B�CB��[��X��ܚX�KK[XZ[�]�][��X��ܚX�P�\��[��K�\X�K�ܙ�B��܈Y][ۘ[��[X[��K[XZ[�]�Z[�\��[��K�\X�K�ܙ�B�B
  

Re: How Apache Cassandra handles flaky tests

[dinesh.jo...@yahoo.com.INVALID]

+1 to everything Jeff said. As someone who has worked on flaky tests not just 
in Cassandra's context, I know it can be hard to deal with them. 
However, it's best to root cause them. I have found some flaky tests were 
genuine issues that needed fixing in Cassandra. Sometimes the flakiness is due 
to underpowered VMs running low on resources or in one case tests failed due to 
the kernel settings different between systems. Explore tuning the VM settings 
used for the test execution. I usually don't prefer adding retries but in some 
cases retries can be helpful. Rewriting the tests to reduce dependencies on 
external systems or using mocks is another useful method in reducing the 
flakiness. Try breaking up tests if they're too big. Finally deleting tests can 
also be a solution but use it sparingly. 
I am believe in the broken windows theory so it is critical that you spend time 
fixing them else everyone ignores them and attributes all failures to 
"flakiness" leading to real issues sneaking in.
Dinesh

On Tuesday, February 26, 2019, 12:06:10 PM PST, Jeff Jirsa 
 wrote:  
 
 


> On Feb 26, 2019, at 8:26 AM, Stanislav Kozlovski 
>  wrote:
> 
> Hey there Cassandra community,
> 
> I work on a fellow open-source project - Apache Kafka - and there we have 
> been fighting flaky tests a lot. We run Java 8 and Java 11 builds on every 
> Pull Request and due to test flakiness, almost all of them turn out red with 
> 1 or 2 tests (completely unrelated to the change in the PR) failing. This has 
> resulted in committers ignoring them and merging the changes either way, or 
> in the worst case - rerunning the hour-long build until it becomes green.

I hope most committers wont commit unless the flakey rest is definitely not in 
the subsystem they touched. But yes, one of the motivations for speeding up 
tests (parallelized on a containerized hosted CI platform) was to cut down the 
time for (re-)running
 
> This test flakiness has also slowed down our releases significantly.
> 
> In general, I was just curious to understand if this is a problem that 
> Cassandra faces as well.

Yes


> Does your project have a lot of intermittently failing tests,

Sometimes more than others. There were a few big pushes to get green, though it 
naturally regresses a bit over time 

> do you have any active process of addressing such tests (during the initial 
> review, after realizing it is flaky, etc). Any pointers will be greatly 
> appreciated!

I don’t think we’ve solved this convincingly. Different large (corporate) 
contributors have done long one time passes, and that helped a ton, but I don’t 
think there are any silver bullets yet.
-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org