Re: restrictions on IN operator
Sorry, it seems that I did not provide enough details. IN restrictions are supported on any clustering key as long as ALL the previous clustering keys are restricted by an equality restrictions ( = or IN ). The only way to have a restriction on a clustering column if a previous one has been restricted by a slice restriction (<=, <, > or >=) is by using filtering (since 3.6). Unfortunatly filtering does not support IN restrictions for the moment. On Tue, Sep 13, 2016 at 8:32 PM, Samba wrote: > I am still getting the following error when trying to run a query with > non-equal conditions in where clause > > > Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: > Clustering column "author" cannot be restricted (preceding column "timing" > is restricted by a non-EQ relation) > > > Here is the version details of cassandra: > > show version; > [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 | Native protocol v4] > > when IN conditions can be run on any (clustering) column, then i suppose > non-equal conditions should also be supported. is my expectation wrong? > > > On Tue, Sep 6, 2016 at 10:08 PM, Benjamin Lerer < > benjamin.le...@datastax.com > > wrote: > > > Since 2.2, IN restrictions are supported on any partition key or > clustering > > colum in SELECT statement. For UPDATE and DELETE statement they are > > supported since 3.0. > > > > Benjamin > > > > On Tue, Sep 6, 2016 at 11:19 AM, Samba wrote: > > > > > Hi, > > > > > > I understand, from the documentation, that IN operator is permitted > only > > on > > > the last column in partition and/or on the last column in the > clustering > > > key. > > > > > > I can understand that IN on partition key column being indeterministic > > but > > > i wonder why is IN permitted only on one (last) clustering column. > aren't > > > all the records differing only in clustering columns stay on the same > > node? > > > is it something impossible or is scheduled for future? > > > > > > > > > alternatively, why not distribute the query to all the nodes matching > the > > > IN condition, in parallel, and join the result sets or return as > futures? > > > perhaps this is what map-reduce does -- but why not a distributed > > database > > > execute its queries (functions & aggregates too) on the matching nodes > in > > > its cluster? > > > > > > could you please try explain the rationale behind why it has been done > > so, > > > and if there are any plans to enhancing this behaviour in the near > > future? > > > > > > Thanks & Regards, > > > Samba > > > > > >
Proposal - 3.5.1
Unfortunately CASSANDRA-11618 was fixed in 3.6 but was not back ported to 3.5 as well, and it makes Cassandra effectively unusable if someone is using any of the 4 types affected in any of their schema. I have cherry picked & merged the patch back to here and will put it in a JIRA as well tonight, I just wanted to get the ball rolling asap on this. https://github.com/rustyrazorblade/cassandra/tree/fix_commitlog_exception Jon
Re: Proposal - 3.5.1
What's preventing the use of the 3.6 or 3.7 releases where this bug is already fixed? This is also fixed in the 3.0.6/7/8 releases. Michael On 09/14/2016 08:30 PM, Jonathan Haddad wrote: > Unfortunately CASSANDRA-11618 was fixed in 3.6 but was not back ported to > 3.5 as well, and it makes Cassandra effectively unusable if someone is > using any of the 4 types affected in any of their schema. > > I have cherry picked & merged the patch back to here and will put it in a > JIRA as well tonight, I just wanted to get the ball rolling asap on this. > > https://github.com/rustyrazorblade/cassandra/tree/fix_commitlog_exception > > Jon >
Re: Proposal - 3.5.1
Common sense is what prevents someone from upgrading to yet another completely unknown version with new features which have probably broken even more stuff that nobody is aware of. The folks I'm helping right deployed 3.5 when they got started because cassandra.apache.org suggests it's acceptable for production. It turns out using 4 of the built in datatypes of the database result in the server being unable to restart without clearing out the commit logs and running a repair. That screams critical to me. You shouldn't even be able to install 3.5 without the patch I've supplied - that bug is a ticking time bomb for anyone that installs it. On Wed, Sep 14, 2016 at 8:12 PM Michael Shuler wrote: > What's preventing the use of the 3.6 or 3.7 releases where this bug is > already fixed? This is also fixed in the 3.0.6/7/8 releases. > > Michael > > On 09/14/2016 08:30 PM, Jonathan Haddad wrote: > > Unfortunately CASSANDRA-11618 was fixed in 3.6 but was not back ported to > > 3.5 as well, and it makes Cassandra effectively unusable if someone is > > using any of the 4 types affected in any of their schema. > > > > I have cherry picked & merged the patch back to here and will put it in a > > JIRA as well tonight, I just wanted to get the ball rolling asap on this. > > > > > https://github.com/rustyrazorblade/cassandra/tree/fix_commitlog_exception > > > > Jon > > > >
Re: Proposal - 3.5.1
We did 3.1.1 and 3.2.1, so there’s SOME precedent for emergency fixes, but we certainly didn’t/won’t go back and cut new releases from every branch for every critical bug in future releases, so I think we need to draw the line somewhere. If it’s fixed in 3.7 and 3.0.x (x >= 6), it seems like you’ve got options (either stay on the tick and go up to 3.7, or bail down to 3.0.x) Perhaps, though, this highlights the fact that tick/tock may not be the best option long term. We’ve tried it for a year, perhaps we should instead discuss whether or not it should continue, or if there’s another process that gives us a better way to get useful patches into versions people are willing to run in production. On 9/14/16, 8:55 PM, "Jonathan Haddad" wrote: >Common sense is what prevents someone from upgrading to yet another >completely unknown version with new features which have probably broken >even more stuff that nobody is aware of. The folks I'm helping right >deployed 3.5 when they got started because >https://urldefense.proofpoint.com/v2/url?u=http-3A__cassandra.apache.org&d=DQIBaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=MZ9nLcNNhQZkuXyH0NBbP1kSEE2M-SYgyVqZ88IJcXY&s=pLP3udocOcAG6k_sAb9p8tcAhtOhpFm6JB7owGhPQEs&e= > suggests >it's acceptable for production. It turns out using 4 of the built in >datatypes of the database result in the server being unable to restart >without clearing out the commit logs and running a repair. That screams >critical to me. You shouldn't even be able to install 3.5 without the >patch I've supplied - that bug is a ticking time bomb for anyone that >installs it. > >On Wed, Sep 14, 2016 at 8:12 PM Michael Shuler >wrote: > >> What's preventing the use of the 3.6 or 3.7 releases where this bug is >> already fixed? This is also fixed in the 3.0.6/7/8 releases. >> >> Michael >> >> On 09/14/2016 08:30 PM, Jonathan Haddad wrote: >> > Unfortunately CASSANDRA-11618 was fixed in 3.6 but was not back ported to >> > 3.5 as well, and it makes Cassandra effectively unusable if someone is >> > using any of the 4 types affected in any of their schema. >> > >> > I have cherry picked & merged the patch back to here and will put it in a >> > JIRA as well tonight, I just wanted to get the ball rolling asap on this. >> > >> > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rustyrazorblade_cassandra_tree_fix-5Fcommitlog-5Fexception&d=DQIBaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=MZ9nLcNNhQZkuXyH0NBbP1kSEE2M-SYgyVqZ88IJcXY&s=ktY5tkT-nO1jtyc0EicbgZHXJYl03DvzuxqzyyOgzII&e= >> >> > >> > Jon >> > >> >> smime.p7s Description: S/MIME cryptographic signature
Re: Proposal - 3.5.1
In this particular case, I'd say adding a bug fix release for every version
that's affected would be the right thing. The issue is so easily
reproducible and will likely result in massive data loss for anyone on 3.X
WHERE X < 6 and uses the "date" type.
This is how easy it is to reproduce:
1. Start Cassandra 3.5
2. create KEYSPACE test WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 1};
3. use test;
4. create table fail (id int primary key, d date);
5. delete d from fail where id = 1;
6. Stop Cassandra
7. Start Cassandra
You will get this, and startup will fail:
ERROR 05:32:09 Exiting due to error while processing commit log during
initialization.
org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException:
Unexpected error deserializing mutation; saved to
/var/folders/0l/g2p6cnyd5kx_1wkl83nd3y4rgn/T/mutation6313332720566971713dat.
This may be caused by replaying a mutation against a table with the same
name but incompatible schema. Exception follows:
org.apache.cassandra.serializers.MarshalException: Expected 4 byte long for
date (0)
I mean.. come on. It's an easy fix. It cleanly merges against 3.5 (and
probably the other releases) and requires very little investment from
anyone.
On Wed, Sep 14, 2016 at 9:40 PM Jeff Jirsa
wrote:
> We did 3.1.1 and 3.2.1, so there’s SOME precedent for emergency fixes, but
> we certainly didn’t/won’t go back and cut new releases from every branch
> for every critical bug in future releases, so I think we need to draw the
> line somewhere. If it’s fixed in 3.7 and 3.0.x (x >= 6), it seems like
> you’ve got options (either stay on the tick and go up to 3.7, or bail down
> to 3.0.x)
>
> Perhaps, though, this highlights the fact that tick/tock may not be the
> best option long term. We’ve tried it for a year, perhaps we should instead
> discuss whether or not it should continue, or if there’s another process
> that gives us a better way to get useful patches into versions people are
> willing to run in production.
>
>
>
> On 9/14/16, 8:55 PM, "Jonathan Haddad" wrote:
>
> >Common sense is what prevents someone from upgrading to yet another
> >completely unknown version with new features which have probably broken
> >even more stuff that nobody is aware of. The folks I'm helping right
> >deployed 3.5 when they got started because
> https://urldefense.proofpoint.com/v2/url?u=http-3A__cassandra.apache.org&d=DQIBaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=MZ9nLcNNhQZkuXyH0NBbP1kSEE2M-SYgyVqZ88IJcXY&s=pLP3udocOcAG6k_sAb9p8tcAhtOhpFm6JB7owGhPQEs&e=
> suggests
> >it's acceptable for production. It turns out using 4 of the built in
> >datatypes of the database result in the server being unable to restart
> >without clearing out the commit logs and running a repair. That screams
> >critical to me. You shouldn't even be able to install 3.5 without the
> >patch I've supplied - that bug is a ticking time bomb for anyone that
> >installs it.
> >
> >On Wed, Sep 14, 2016 at 8:12 PM Michael Shuler
> >wrote:
> >
> >> What's preventing the use of the 3.6 or 3.7 releases where this bug is
> >> already fixed? This is also fixed in the 3.0.6/7/8 releases.
> >>
> >> Michael
> >>
> >> On 09/14/2016 08:30 PM, Jonathan Haddad wrote:
> >> > Unfortunately CASSANDRA-11618 was fixed in 3.6 but was not back
> ported to
> >> > 3.5 as well, and it makes Cassandra effectively unusable if someone is
> >> > using any of the 4 types affected in any of their schema.
> >> >
> >> > I have cherry picked & merged the patch back to here and will put it
> in a
> >> > JIRA as well tonight, I just wanted to get the ball rolling asap on
> this.
> >> >
> >> >
> >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rustyrazorblade_cassandra_tree_fix-5Fcommitlog-5Fexception&d=DQIBaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=MZ9nLcNNhQZkuXyH0NBbP1kSEE2M-SYgyVqZ88IJcXY&s=ktY5tkT-nO1jtyc0EicbgZHXJYl03DvzuxqzyyOgzII&e=
> >> >
> >> > Jon
> >> >
> >>
> >>
>
Failing tests 2016-09-14
cassandra-3.9 === testall: 8 failures org.apache.cassandra.cql3.ViewFilteringTest .testPartitionKeyAndClusteringKeyFilteringRestrictions org.apache.cassandra.cql3.ViewFilteringTest .testMVCreationSelectRestrictions org.apache.cassandra.cql3.ViewTest.testCompoundPartitionKey org.apache.cassandra.cql3.validation.entities.UFTest.testEmptyString org.apache.cassandra.cql3.validation.operations.AggregationTest .testFunctionsWithCompactStorage org.apache.cassandra.cql3.validation.operations.SelectTest .testAllowFiltering These six test failures are due to environmental timeouts. org.apache.cassandra.db.compaction .TimeWindowCompactionStrategyTest .testDropExpiredSSTables-compression New flaky failure. CASSANDRA-12645 opened. org.apache.cassandra.service.RemoveTest.testBadHostId CASSANDRA-12487. Flaky failure in a test utility setup method. === dtest: 1 failure user_types_test.TestUserTypes.test_type_as_part_of_pkey Should have been fixed as part of CASSANDRA-11031. Incorrect version gating still - I'll follow up and get this fixed tomorrow. === novnode: 4 failures user_types_test.TestUserTypes.test_type_as_part_of_pkey Same as above. cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest .test_bulk_round_trip_with_single_core New failure - looks like a schema agreement problem. A JIRA hasn't been created yet. cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest .test_reading_max_insert_errors New failure - looks like Netty detected a leak. A JIRA hasn't been created yet. batch_test.TestBatch.logged_batch_doesnt_throw_uae_test CASSANDRA-12383. Flaky failure. === upgrade: 1 failure upgrade_tests.cql_tests .TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_3_x .bug_5732_test CASSANDRA-12457. Patch available that needs a reviewer. Since there's a few open opportunities based on 3.9 failures, I'm only covering 3.9 on today's email.
Re: Failing tests 2016-09-14
> CASSANDRA-11031 Yes, sorry for delay with #11031 dtests. I've ran updated dtests yesterday and they were clean to merge. I just wanted to make sure someone else takes a quick glance. By now they're merged, so hopefully today it's going to be better. As regards environmental timeouts, it looks like certain methods are more prone to this (in particular, view filtering test does quite a lot). I realise they don't hang, they just execute slower on CI machine than we anticipate. But what should we do with it generally? Increasing timeouts won't really help, so what comes to mind is: * Splitting tests * Modularising to make sure unnecessary components don't get started * Running "slow-prone" tests sequentially to make sure they get enough processor time * Taking a deeper look, might be there's a performance issue hiding behind * Thread-dumping in case there is some sort of deadlock that's hard to reproduce on "faster" machine (however improbable that might sound) Since it looks like generally tests are in much better shape, it might be a good point to start thinking about those timing out ones. On Thu, Sep 15, 2016 at 7:51 AM Joel Knighton wrote: > cassandra-3.9 > === > testall: 8 failures > org.apache.cassandra.cql3.ViewFilteringTest > .testPartitionKeyAndClusteringKeyFilteringRestrictions > > org.apache.cassandra.cql3.ViewFilteringTest > .testMVCreationSelectRestrictions > > org.apache.cassandra.cql3.ViewTest.testCompoundPartitionKey > > org.apache.cassandra.cql3.validation.entities.UFTest.testEmptyString > > org.apache.cassandra.cql3.validation.operations.AggregationTest > .testFunctionsWithCompactStorage > > org.apache.cassandra.cql3.validation.operations.SelectTest > .testAllowFiltering > These six test failures are due to environmental timeouts. > > org.apache.cassandra.db.compaction > .TimeWindowCompactionStrategyTest > .testDropExpiredSSTables-compression > New flaky failure. CASSANDRA-12645 opened. > > org.apache.cassandra.service.RemoveTest.testBadHostId > CASSANDRA-12487. Flaky failure in a test utility setup method. > > === > dtest: 1 failure > user_types_test.TestUserTypes.test_type_as_part_of_pkey > Should have been fixed as part of CASSANDRA-11031. Incorrect > version gating still - I'll follow up and get this fixed tomorrow. > > === > novnode: 4 failures > user_types_test.TestUserTypes.test_type_as_part_of_pkey > Same as above. > > cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest > .test_bulk_round_trip_with_single_core > New failure - looks like a schema agreement problem. A JIRA > hasn't been created yet. > > cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest > .test_reading_max_insert_errors > New failure - looks like Netty detected a leak. A JIRA hasn't been > created yet. > > batch_test.TestBatch.logged_batch_doesnt_throw_uae_test > CASSANDRA-12383. Flaky failure. > > === > upgrade: 1 failure > upgrade_tests.cql_tests > .TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_3_x > .bug_5732_test > CASSANDRA-12457. Patch available that needs a reviewer. > > > Since there's a few open opportunities based on 3.9 failures, I'm only > covering 3.9 on today's email. > -- Alex Petrov
