date:20151111

Re: Cassandra compaction stuck? Should I disable?

2015-11-11 Thread PenguinWhispererThe .

Does compactionstats shows only stats for completed compactions (100%)? It
might be that the compaction is running constantly, over and over again.
In that case I need to know what I might be able to do to stop this
constant compaction so I can start a nodetool repair.

Note that there is a lot of traffic on this columnfamily so I'm not sure if
temporary disabling compaction is an option. The repair will probably take
long as well.

Sebastian and Rob: do you might have any more ideas about the things I put
in this thread? Any help is appreciated!

2015-11-10 20:03 GMT+01:00 PenguinWhispererThe . <
th3penguinwhispe...@gmail.com>:

> Hi Sebastian,
>
> Thanks for your response.
>
> No swap is used. No offense, I just don't see a reason why having swap
> would be the issue here. I put swapiness on 1. I also have jna installed.
> That should prevent java being swapped out as wel AFAIK.
>
>
> 2015-11-10 19:50 GMT+01:00 Sebastian Estevez <
> sebastian.este...@datastax.com>:
>
>> Turn off Swap.
>>
>>
>> http://docs.datastax.com/en/cassandra/2.1/cassandra/install/installRecommendSettings.html?scroll=reference_ds_sxl_gf3_2k__disable-swap
>>
>>
>> All the best,
>>
>>
>> [image: datastax_logo.png] 
>>
>> Sebastián Estévez
>>
>> Solutions Architect | 954 905 8615 | sebastian.este...@datastax.com
>>
>> [image: linkedin.png]  [image:
>> facebook.png]  [image: twitter.png]
>>  [image: g+.png]
>> 
>> 
>> 
>>
>>
>> 
>>
>> DataStax is the fastest, most scalable distributed database technology,
>> delivering Apache Cassandra to the world’s most innovative enterprises.
>> Datastax is built to be agile, always-on, and predictably scalable to any
>> size. With more than 500 customers in 45 countries, DataStax is the
>> database technology and transactional backbone of choice for the worlds
>> most innovative companies such as Netflix, Adobe, Intuit, and eBay.
>>
>> On Tue, Nov 10, 2015 at 1:48 PM, PenguinWhispererThe . <
>> th3penguinwhispe...@gmail.com> wrote:
>>
>>> I also have the following memory usage:
>>> [root@US-BILLINGDSX4 cassandra]# free -m
>>>  total   used   free sharedbuffers cached
>>> Mem: 12024   9455   2569  0110   2163
>>> -/+ buffers/cache:   7180   4844
>>> Swap: 2047  0   2047
>>>
>>> Still a lot free and a lot of free buffers/cache.
>>>
>>> 2015-11-10 19:45 GMT+01:00 PenguinWhispererThe . <
>>> th3penguinwhispe...@gmail.com>:
>>>
 Still stuck with this. However I enabled GC logging. This shows the
 following:

 [root@myhost cassandra]# tail -f gc-1447180680.log
 2015-11-10T18:41:45.516+: 225.428: [GC
 2721842K->2066508K(6209536K), 0.0199040 secs]
 2015-11-10T18:41:45.977+: 225.889: [GC
 2721868K->2066511K(6209536K), 0.0221910 secs]
 2015-11-10T18:41:46.437+: 226.349: [GC
 2721871K->2066524K(6209536K), 0.0222140 secs]
 2015-11-10T18:41:46.897+: 226.809: [GC
 2721884K->2066539K(6209536K), 0.0224140 secs]
 2015-11-10T18:41:47.359+: 227.271: [GC
 2721899K->2066538K(6209536K), 0.0302520 secs]
 2015-11-10T18:41:47.821+: 227.733: [GC
 2721898K->2066557K(6209536K), 0.0280530 secs]
 2015-11-10T18:41:48.293+: 228.205: [GC
 2721917K->2066571K(6209536K), 0.0218000 secs]
 2015-11-10T18:41:48.790+: 228.702: [GC
 2721931K->2066780K(6209536K), 0.0292470 secs]
 2015-11-10T18:41:49.290+: 229.202: [GC
 2722140K->2066843K(6209536K), 0.0288740 secs]
 2015-11-10T18:41:49.756+: 229.668: [GC
 2722203K->2066818K(6209536K), 0.0283380 secs]
 2015-11-10T18:41:50.249+: 230.161: [GC
 2722178K->2067158K(6209536K), 0.0218690 secs]
 2015-11-10T18:41:50.713+: 230.625: [GC
 2722518K->2067236K(6209536K), 0.0278810 secs]

 This is a VM with 12GB of RAM. Highered the HEAP_SIZE to 6GB and
 HEAP_NEWSIZE to 800MB.

 Still the same result.

 This looks very similar to following issue:

 http://mail-archives.apache.org/mod_mbox/cassandra-user/201411.mbox/%3CCAJ=3xgRLsvpnZe0uXEYjG94rKhfXeU+jBR=q3a-_c3rsdd5...@mail.gmail.com%3E

 Is the only possibility to upgrade memory? I mean, I can't believe it's
 just loading all it's data in memory. That would require to keep scaling up
 the node to keep it work?


 2015-11-10 9:36 GMT+01:00 PenguinWhispererThe . <
 th3penguinwhispe...@gmail.com>:

> Correction...
> I was grepping on Segmentation on the strace and it happens a lot.
>
> Do I need to run a scrub?
>
> 2015-11-10 9:30 GMT+01:00 PenguinWhispererThe . <
> th3penguinwhispe...@gmail.com>:
>
>> Hi Rob,
>>
>>

答复: org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:218 throws java.lang.AssertionError

2015-11-11 Thread 李建奇

Hello,

 Reading more codes. I  can reproduce  the exception. Over long
slice parameter can cause the exception.

   Code :

 char[] chs = new char[7];   //over 64k
 Arrays.fill(chs,'b');
 String end = String.valueOf(chs);
 Composite c2 =  nameType.make(new Object[] {2, end ,11});
 ColumnSlice slice = new ColumnSlice(c1,c2);
 ColumnSlice.Serializer ser = new ColumnSlice.Serializer(nameType);
 DataOutputPlus out = new DataOutputBuffer();
 SliceQueryFilter slf = new SliceQueryFilter(c1,c2,false,1000);
 nameType.sliceQueryFilterSerializer().serialize(slf,out,
MessagingService.current_version);

 

  Question:

   I wonder if have some validators before send the ReadCommand to data
nodes . 

  

Thanks

 

 

发件人: 李建奇 [mailto:lijia...@jd.com] 
发送时间: 2015年11月10日 10:51
收件人: user@cassandra.apache.org
主题:
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.jav
a:218 throws java.lang.AssertionError

 

Hi, All,

 

  We have a 12 nodes cluster with 2.1.9 version for near one month. Last
week it have an exception . Cluster’s write and read latency will go up to
4 seconds from 0.4ms average after exception.

  I suspect OutboundTcpConnection is broken .I try to disablegossip then
enablegossip to rescue OutboundTcpConnction ,but fail .

  I have to restart every node to restore cluster to normal . Cluster’s
load is low. The client use datastax java driver 2.1.7.1 . CF with RF 3. 

  

  Question:

 Which situation to trigger this AssertionError?   I read
OutboundTcpConnection.java source code,  line 228 comments
“writeConnected() is reasonably robust”.

  

Thanks

 

 

Attachment :

 

ERROR [MessagingService-Outgoing-/172.20.114.13] 2015-11-08 10:36:28,763
CassandraDaemon.java:223 - Exception in thread
Thread[MessagingService-Outgoing-/172.20.114.13,5,main]

java.lang.AssertionError: 78251

at
org.apache.cassandra.utils.ByteBufferUtil.writeWithShortLength(ByteBufferUti
l.java:290) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.composites.AbstractCType$Serializer.serialize(Abstra
ctCType.java:392) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.composites.AbstractCType$Serializer.serialize(Abstra
ctCType.java:381) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.filter.ColumnSlice$Serializer.serialize(ColumnSlice.
java:271) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.filter.ColumnSlice$Serializer.serialize(ColumnSlice.
java:259) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.filter.SliceQueryFilter$Serializer.serialize(SliceQu
eryFilter.java:503) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.filter.SliceQueryFilter$Serializer.serialize(SliceQu
eryFilter.java:490) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.SliceFromReadCommandSerializer.serialize(SliceFromRe
adCommand.java:168) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.ReadCommandSerializer.serialize(ReadCommand.java:143
) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.ReadCommandSerializer.serialize(ReadCommand.java:132
) ~[apache-cassandra-2.1.9.jar:2.1.9]

at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:121)
~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.net.OutboundTcpConnection.writeInternal(OutboundTcpConn
ection.java:330) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpCon
nection.java:282) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.jav
a:218) ~[apache-cassandra-2.1.9.jar:2.1.9]

ERROR [MessagingService-Outgoing-/172.20.114.19] 2015-11-08 10:36:28,763
CassandraDaemon.java:223 - Exception in thread
Thread[MessagingService-Outgoing-/172.20.114.19,5,main]

java.lang.AssertionError: 78251

at
org.apache.cassandra.utils.ByteBufferUtil.writeWithShortLength(ByteBufferUti
l.java:290) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.composites.AbstractCType$Serializer.serialize(Abstra
ctCType.java:392) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.composites.AbstractCType$Serializer.serialize(Abstra
ctCType.java:381) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.filter.ColumnSlice$Serializer.serialize(ColumnSlice.
java:271) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.filter.ColumnSlice$Serializer.serialize(ColumnSlice.
java:259) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.filter.SliceQueryFilter$Serializer.serialize(SliceQu
eryFilter.java:503) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.filter.SliceQueryFilter$Serializer.serialize(SliceQu
eryFilter.java:490) ~[apache-cassandra-2.1.9.jar:2.1.9]

at
org.apache.cassandra.db.SliceFromReadCommandSerializer.serialize(SliceFromRe
adCommand.java:168) ~[apache-cassa

Re: Cassandra compaction stuck? Should I disable?

2015-11-11 Thread Sebastian Estevez

Use 'nodetool compactionhistory'

all the best,

Sebastián
On Nov 11, 2015 3:23 AM, "PenguinWhispererThe ." <
th3penguinwhispe...@gmail.com> wrote:

> Does compactionstats shows only stats for completed compactions (100%)? It
> might be that the compaction is running constantly, over and over again.
> In that case I need to know what I might be able to do to stop this
> constant compaction so I can start a nodetool repair.
>
> Note that there is a lot of traffic on this columnfamily so I'm not sure
> if temporary disabling compaction is an option. The repair will probably
> take long as well.
>
> Sebastian and Rob: do you might have any more ideas about the things I put
> in this thread? Any help is appreciated!
>
> 2015-11-10 20:03 GMT+01:00 PenguinWhispererThe . <
> th3penguinwhispe...@gmail.com>:
>
>> Hi Sebastian,
>>
>> Thanks for your response.
>>
>> No swap is used. No offense, I just don't see a reason why having swap
>> would be the issue here. I put swapiness on 1. I also have jna installed.
>> That should prevent java being swapped out as wel AFAIK.
>>
>>
>> 2015-11-10 19:50 GMT+01:00 Sebastian Estevez <
>> sebastian.este...@datastax.com>:
>>
>>> Turn off Swap.
>>>
>>>
>>> http://docs.datastax.com/en/cassandra/2.1/cassandra/install/installRecommendSettings.html?scroll=reference_ds_sxl_gf3_2k__disable-swap
>>>
>>>
>>> All the best,
>>>
>>>
>>> [image: datastax_logo.png] 
>>>
>>> Sebastián Estévez
>>>
>>> Solutions Architect | 954 905 8615 | sebastian.este...@datastax.com
>>>
>>> [image: linkedin.png]  [image:
>>> facebook.png]  [image: twitter.png]
>>>  [image: g+.png]
>>> 
>>> 
>>> 
>>>
>>>
>>> 
>>>
>>> DataStax is the fastest, most scalable distributed database technology,
>>> delivering Apache Cassandra to the world’s most innovative enterprises.
>>> Datastax is built to be agile, always-on, and predictably scalable to any
>>> size. With more than 500 customers in 45 countries, DataStax is the
>>> database technology and transactional backbone of choice for the worlds
>>> most innovative companies such as Netflix, Adobe, Intuit, and eBay.
>>>
>>> On Tue, Nov 10, 2015 at 1:48 PM, PenguinWhispererThe . <
>>> th3penguinwhispe...@gmail.com> wrote:
>>>
 I also have the following memory usage:
 [root@US-BILLINGDSX4 cassandra]# free -m
  total   used   free sharedbuffers
 cached
 Mem: 12024   9455   2569  0110
 2163
 -/+ buffers/cache:   7180   4844
 Swap: 2047  0   2047

 Still a lot free and a lot of free buffers/cache.

 2015-11-10 19:45 GMT+01:00 PenguinWhispererThe . <
 th3penguinwhispe...@gmail.com>:

> Still stuck with this. However I enabled GC logging. This shows the
> following:
>
> [root@myhost cassandra]# tail -f gc-1447180680.log
> 2015-11-10T18:41:45.516+: 225.428: [GC
> 2721842K->2066508K(6209536K), 0.0199040 secs]
> 2015-11-10T18:41:45.977+: 225.889: [GC
> 2721868K->2066511K(6209536K), 0.0221910 secs]
> 2015-11-10T18:41:46.437+: 226.349: [GC
> 2721871K->2066524K(6209536K), 0.0222140 secs]
> 2015-11-10T18:41:46.897+: 226.809: [GC
> 2721884K->2066539K(6209536K), 0.0224140 secs]
> 2015-11-10T18:41:47.359+: 227.271: [GC
> 2721899K->2066538K(6209536K), 0.0302520 secs]
> 2015-11-10T18:41:47.821+: 227.733: [GC
> 2721898K->2066557K(6209536K), 0.0280530 secs]
> 2015-11-10T18:41:48.293+: 228.205: [GC
> 2721917K->2066571K(6209536K), 0.0218000 secs]
> 2015-11-10T18:41:48.790+: 228.702: [GC
> 2721931K->2066780K(6209536K), 0.0292470 secs]
> 2015-11-10T18:41:49.290+: 229.202: [GC
> 2722140K->2066843K(6209536K), 0.0288740 secs]
> 2015-11-10T18:41:49.756+: 229.668: [GC
> 2722203K->2066818K(6209536K), 0.0283380 secs]
> 2015-11-10T18:41:50.249+: 230.161: [GC
> 2722178K->2067158K(6209536K), 0.0218690 secs]
> 2015-11-10T18:41:50.713+: 230.625: [GC
> 2722518K->2067236K(6209536K), 0.0278810 secs]
>
> This is a VM with 12GB of RAM. Highered the HEAP_SIZE to 6GB and
> HEAP_NEWSIZE to 800MB.
>
> Still the same result.
>
> This looks very similar to following issue:
>
> http://mail-archives.apache.org/mod_mbox/cassandra-user/201411.mbox/%3CCAJ=3xgRLsvpnZe0uXEYjG94rKhfXeU+jBR=q3a-_c3rsdd5...@mail.gmail.com%3E
>
> Is the only possibility to upgrade memory? I mean, I can't believe
> it's just loading all it's data in memory. That would require to keep
> scaling up the node to keep it work?
>
>
> 2015-11-10 9:36 GMT+01:00 PenguinWhispererThe . <
> th3penguinwhispe...

Re: scylladb

2015-11-11 Thread Carlos Alonso

Any update about this?

@Carlos Rolo, did you tried it? Thoughts?

Carlos Alonso | Software Engineer | @calonso 

On 5 November 2015 at 14:07, Carlos Rolo  wrote:

> Something to do on a expected rainy weekend. Thanks for the information.
>
> Regards,
>
> Carlos Juzarte Rolo
> Cassandra Consultant
>
> Pythian - Love your data
>
> rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
> *
> Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
> www.pythian.com
>
> On Thu, Nov 5, 2015 at 12:07 PM, Dani Traphagen <
> dani.trapha...@datastax.com> wrote:
>
>> As of two days ago, they say they've got it @cjrolo.
>>
>> https://github.com/scylladb/scylla/wiki/RELEASE-Scylla-0.11-Beta
>>
>>
>> On Thursday, November 5, 2015, Carlos Rolo  wrote:
>>
>>> I will not try until multi-DC is implemented. More than an month has
>>> passed since I looked for it, so it could possibly be in place, if so I may
>>> take some time to test it.
>>>
>>> Regards,
>>>
>>> Carlos Juzarte Rolo
>>> Cassandra Consultant
>>>
>>> Pythian - Love your data
>>>
>>> rolo@pythian | Twitter: @cjrolo | Linkedin: 
>>> *linkedin.com/in/carlosjuzarterolo
>>> *
>>> Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
>>> www.pythian.com
>>>
>>> On Thu, Nov 5, 2015 at 9:37 AM, Jon Haddad 
>>> wrote:
>>>
 Nope, no one I know.  Let me know if you try it I'd love to hear your
 feedback.

 > On Nov 5, 2015, at 9:22 AM, tommaso barbugli 
 wrote:
 >
 > Hi guys,
 >
 > did anyone already try Scylladb (yet another fastest NoSQL database
 in town) and has some thoughts/hands-on experience to share?
 >
 > Cheers,
 > Tommaso


>>>
>>> --
>>>
>>>
>>>
>>>
>>
>> --
>> Sent from mobile -- apologizes for brevity or errors.
>>
>
>
> --
>
>
>
>

Re: scylladb

2015-11-11 Thread Carlos Rolo

Not yet, but not far from doing it. No rain here yet! :)

On a more serious tone, should be done before end of the Month.

-- 


--

Re: scylladb

2015-11-11 Thread Dani Traphagen

Killer, @cjrolo. Will you update via this thread?

On Wed, Nov 11, 2015 at 7:57 AM, Carlos Rolo  wrote:

> Not yet, but not far from doing it. No rain here yet! :)
>
> On a more serious tone, should be done before end of the Month.
>
> --
>
>
>
>


-- 
[image: datastax_logo.png] 

DANI TRAPHAGEN

Technical Enablement Lead | dani.trapha...@datastax.com

[image: twitter.png]  [image: linkedin.png]

Re: scylladb

2015-11-11 Thread Carlos Rolo

Sure! I have a lot of blog post on backlog to blog asap about this,
otherwise I would only share results mid 2016 :P

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Wed, Nov 11, 2015 at 4:46 PM, Dani Traphagen  wrote:

> Killer, @cjrolo. Will you update via this thread?
>
> On Wed, Nov 11, 2015 at 7:57 AM, Carlos Rolo  wrote:
>
>> Not yet, but not far from doing it. No rain here yet! :)
>>
>> On a more serious tone, should be done before end of the Month.
>>
>> --
>>
>>
>>
>>
>
>
> --
> [image: datastax_logo.png] 
>
> DANI TRAPHAGEN
>
> Technical Enablement Lead | dani.trapha...@datastax.com
>
> [image: twitter.png]  [image:
> linkedin.png] 
> 
>
>
>

-- 


--

Re: UnknownColumnFamily exception / schema inconsistencies

2015-11-11 Thread Maciek Sakrejda

On Tue, Nov 10, 2015 at 3:20 PM, Sebastian Estevez <
sebastian.este...@datastax.com> wrote:

> #1 The cause of this problem is a CREATE TABLE statement collision. Do not 
> generate tables
> dynamically from multiple clients, even with IF NOT EXISTS. First thing you 
> need to do is
> fix your code so that this does not happen. Just create your tables manually 
> from cqlsh allowing
> time for the schema to settle.
>
> #2 Here's the fix:
>
> 1) Change your code to not automatically re-create tables (even with IF NOT 
> EXISTS).
>
> 2) Run a rolling restart to ensure schema matches across nodes. Run nodetool 
> describecluster
>
> around your cluster. Check that there is only one schema version.
>
> Thanks, that seems to have resolved the schema version inconsistency
(though I'm still getting the original error).

> ON EACH NODE:
>
> 3) Check your filesystem and see if you have two directories for the table in
>
> question in the data directory.
>
> Stupid question, but how do I find the problem table? The error message
complains about a keyspace (by uuid); I haven't seen errors relating to a
specific table. I've poked around in the data directory, but I'm not sure
what I'm looking for.

Re: UnknownColumnFamily exception / schema inconsistencies

2015-11-11 Thread Sebastian Estevez

>
> Stupid question, but how do I find the problem table? The error message
> complains about a keyspace (by uuid); I haven't seen errors relating to a
> specific table. I've poked around in the data directory, but I'm not sure
> what I'm looking for.


Is the message complaining about a *keyspace* or abou*t a table (cfid)*?
You'r original was complaining about a table:

at=IncomingTcpConnection.run UnknownColumnFamilyException reading from
> socket; closing org.apache.cassandra.db.UnknownColumnFamilyException:
> Couldn't find *cfId=3ecce750-84d3-11e5-bdd9-**dd7717dcdbd5*


All the best,


[image: datastax_logo.png] 

Sebastián Estévez

Solutions Architect | 954 905 8615 | sebastian.este...@datastax.com

[image: linkedin.png]  [image:
facebook.png]  [image: twitter.png]
 [image: g+.png]







DataStax is the fastest, most scalable distributed database technology,
delivering Apache Cassandra to the world’s most innovative enterprises.
Datastax is built to be agile, always-on, and predictably scalable to any
size. With more than 500 customers in 45 countries, DataStax is the
database technology and transactional backbone of choice for the worlds
most innovative companies such as Netflix, Adobe, Intuit, and eBay.

On Wed, Nov 11, 2015 at 12:26 PM, Maciek Sakrejda  wrote:

> On Tue, Nov 10, 2015 at 3:20 PM, Sebastian Estevez <
> sebastian.este...@datastax.com> wrote:
>
>> #1 The cause of this problem is a CREATE TABLE statement collision. Do not 
>> generate tables
>> dynamically from multiple clients, even with IF NOT EXISTS. First thing you 
>> need to do is
>> fix your code so that this does not happen. Just create your tables manually 
>> from cqlsh allowing
>> time for the schema to settle.
>>
>> #2 Here's the fix:
>>
>> 1) Change your code to not automatically re-create tables (even with IF NOT 
>> EXISTS).
>>
>> 2) Run a rolling restart to ensure schema matches across nodes. Run nodetool 
>> describecluster
>>
>> around your cluster. Check that there is only one schema version.
>>
>> Thanks, that seems to have resolved the schema version inconsistency
> (though I'm still getting the original error).
>
>> ON EACH NODE:
>>
>> 3) Check your filesystem and see if you have two directories for the table in
>>
>> question in the data directory.
>>
>> Stupid question, but how do I find the problem table? The error message
> complains about a keyspace (by uuid); I haven't seen errors relating to a
> specific table. I've poked around in the data directory, but I'm not sure
> what I'm looking for.
>
>

Re: Multi-column slice restrictions not respected by the returned result

2015-11-11 Thread Tyler Hobbs

This is a known problem with multi-column slices and mixed ASC/DESC
clustering orders.  See https://issues.apache.org/jira/browse/CASSANDRA-7281
for details.

On Tue, Nov 10, 2015 at 11:02 PM, Yuri Shkuro  wrote:

> According to this blog:
> http://www.datastax.com/dev/blog/a-deep-look-to-the-cql-where-clause
>
> I should be able to do multi-column restrictions on clustering columns, as
> in the blog example: WHERE (server, time) >= (‘196.8.0.0’, 12:00) AND
> (server, time) <= (‘196.8.255.255’, 14:00)
>
> However, I am getting data returned from such query that does not match
> the restrictions. Tried on Cassandra 2.17 and 2.2.3. Here's an example:
>
> CREATE TABLE IF NOT EXISTS dur (
> s  text,
> nd bigint,
> ts bigint,
> tidbigint,
> PRIMARY KEY (s, nd, ts)
> ) WITH CLUSTERING ORDER BY (nd ASC, ts DESC);
>
> insert INTO dur (s, nd, ts, tid) values ('x', 1, 10, 99);
> insert INTO dur (s, nd, ts, tid) values ('x', 2, 11, 98) ;
> insert INTO dur (s, nd, ts, tid) values ('x', 3, 10, 97) ;
> insert INTO dur (s, nd, ts, tid) values ('x', 1, 11, 96) ;
> insert INTO dur (s, nd, ts, tid) values ('x', 1, 12, 95) ;
> insert INTO dur (s, nd, ts, tid) values ('x', 2, 10, 94) ;
> insert INTO dur (s, nd, ts, tid) values ('x', 2, 12, 93) ;
> insert INTO dur (s, nd, ts, tid) values ('x', 3, 11, 92) ;
> insert INTO dur (s, nd, ts, tid) values ('x', 3, 12, 91) ;
>
> select * from dur where s='x' and (nd,ts) > (2, 11);
>
>  s | nd | ts | tid
> ---+++-
>  x |  2 | 10 |  94
>  x |  3 | 12 |  91
>  x |  3 | 11 |  92
>  x |  3 | 10 |  97
> (4 rows)
>
> The first row in the result does not satisfy the restriction (nd,ts) >
> (2, 11). Am I doing something incorrectly?
>
> Thanks,
> --Yuri
>



-- 
Tyler Hobbs
DataStax

Re: Multi-column slice restrictions not respected by the returned result

2015-11-11 Thread Yuri Shkuro

Thanks, Tyler.

I also realized that I misunderstood multi-column restriction. Evidently,
(a, b) > (x, y) does not imply component-wise restriction (a>x && b>y) in
CQL, it only implies full tuple comparison. That explains why my condition
(a, b) > (2, 10) was matching row (2, 11).

On Wed, Nov 11, 2015 at 2:31 PM, Tyler Hobbs  wrote:

> This is a known problem with multi-column slices and mixed ASC/DESC
> clustering orders.  See
> https://issues.apache.org/jira/browse/CASSANDRA-7281 for details.
>
> On Tue, Nov 10, 2015 at 11:02 PM, Yuri Shkuro  wrote:
>
>> According to this blog:
>> http://www.datastax.com/dev/blog/a-deep-look-to-the-cql-where-clause
>>
>> I should be able to do multi-column restrictions on clustering columns,
>> as in the blog example: WHERE (server, time) >= (‘196.8.0.0’, 12:00) AND
>> (server, time) <= (‘196.8.255.255’, 14:00)
>>
>> However, I am getting data returned from such query that does not match
>> the restrictions. Tried on Cassandra 2.17 and 2.2.3. Here's an example:
>>
>> CREATE TABLE IF NOT EXISTS dur (
>> s  text,
>> nd bigint,
>> ts bigint,
>> tidbigint,
>> PRIMARY KEY (s, nd, ts)
>> ) WITH CLUSTERING ORDER BY (nd ASC, ts DESC);
>>
>> insert INTO dur (s, nd, ts, tid) values ('x', 1, 10, 99);
>> insert INTO dur (s, nd, ts, tid) values ('x', 2, 11, 98) ;
>> insert INTO dur (s, nd, ts, tid) values ('x', 3, 10, 97) ;
>> insert INTO dur (s, nd, ts, tid) values ('x', 1, 11, 96) ;
>> insert INTO dur (s, nd, ts, tid) values ('x', 1, 12, 95) ;
>> insert INTO dur (s, nd, ts, tid) values ('x', 2, 10, 94) ;
>> insert INTO dur (s, nd, ts, tid) values ('x', 2, 12, 93) ;
>> insert INTO dur (s, nd, ts, tid) values ('x', 3, 11, 92) ;
>> insert INTO dur (s, nd, ts, tid) values ('x', 3, 12, 91) ;
>>
>> select * from dur where s='x' and (nd,ts) > (2, 11);
>>
>>  s | nd | ts | tid
>> ---+++-
>>  x |  2 | 10 |  94
>>  x |  3 | 12 |  91
>>  x |  3 | 11 |  92
>>  x |  3 | 10 |  97
>> (4 rows)
>>
>> The first row in the result does not satisfy the restriction (nd,ts) >
>> (2, 11). Am I doing something incorrectly?
>>
>> Thanks,
>> --Yuri
>>
>
>
>
> --
> Tyler Hobbs
> DataStax 
>

Re: Multi-column slice restrictions not respected by the returned result

2015-11-11 Thread Tyler Hobbs

Correct, it's a full tuple comparison.

On Wed, Nov 11, 2015 at 1:43 PM, Yuri Shkuro  wrote:

> Thanks, Tyler.
>
> I also realized that I misunderstood multi-column restriction. Evidently,
> (a, b) > (x, y) does not imply component-wise restriction (a>x && b>y) in
> CQL, it only implies full tuple comparison. That explains why my condition
> (a, b) > (2, 10) was matching row (2, 11).
>
> On Wed, Nov 11, 2015 at 2:31 PM, Tyler Hobbs  wrote:
>
>> This is a known problem with multi-column slices and mixed ASC/DESC
>> clustering orders.  See
>> https://issues.apache.org/jira/browse/CASSANDRA-7281 for details.
>>
>> On Tue, Nov 10, 2015 at 11:02 PM, Yuri Shkuro  wrote:
>>
>>> According to this blog:
>>> http://www.datastax.com/dev/blog/a-deep-look-to-the-cql-where-clause
>>>
>>> I should be able to do multi-column restrictions on clustering columns,
>>> as in the blog example: WHERE (server, time) >= (‘196.8.0.0’, 12:00) AND
>>> (server, time) <= (‘196.8.255.255’, 14:00)
>>>
>>> However, I am getting data returned from such query that does not match
>>> the restrictions. Tried on Cassandra 2.17 and 2.2.3. Here's an example:
>>>
>>> CREATE TABLE IF NOT EXISTS dur (
>>> s  text,
>>> nd bigint,
>>> ts bigint,
>>> tidbigint,
>>> PRIMARY KEY (s, nd, ts)
>>> ) WITH CLUSTERING ORDER BY (nd ASC, ts DESC);
>>>
>>> insert INTO dur (s, nd, ts, tid) values ('x', 1, 10, 99);
>>> insert INTO dur (s, nd, ts, tid) values ('x', 2, 11, 98) ;
>>> insert INTO dur (s, nd, ts, tid) values ('x', 3, 10, 97) ;
>>> insert INTO dur (s, nd, ts, tid) values ('x', 1, 11, 96) ;
>>> insert INTO dur (s, nd, ts, tid) values ('x', 1, 12, 95) ;
>>> insert INTO dur (s, nd, ts, tid) values ('x', 2, 10, 94) ;
>>> insert INTO dur (s, nd, ts, tid) values ('x', 2, 12, 93) ;
>>> insert INTO dur (s, nd, ts, tid) values ('x', 3, 11, 92) ;
>>> insert INTO dur (s, nd, ts, tid) values ('x', 3, 12, 91) ;
>>>
>>> select * from dur where s='x' and (nd,ts) > (2, 11);
>>>
>>>  s | nd | ts | tid
>>> ---+++-
>>>  x |  2 | 10 |  94
>>>  x |  3 | 12 |  91
>>>  x |  3 | 11 |  92
>>>  x |  3 | 10 |  97
>>> (4 rows)
>>>
>>> The first row in the result does not satisfy the restriction (nd,ts) >
>>> (2, 11). Am I doing something incorrectly?
>>>
>>> Thanks,
>>> --Yuri
>>>
>>
>>
>>
>> --
>> Tyler Hobbs
>> DataStax 
>>
>
>


-- 
Tyler Hobbs
DataStax

Re: UnknownColumnFamily exception / schema inconsistencies

2015-11-11 Thread Maciek Sakrejda

On Wed, Nov 11, 2015 at 9:55 AM, Sebastian Estevez <
sebastian.este...@datastax.com> wrote:

> Stupid question, but how do I find the problem table? The error message
>> complains about a keyspace (by uuid); I haven't seen errors relating to a
>> specific table. I've poked around in the data directory, but I'm not sure
>> what I'm looking for.
>
>
> Is the message complaining about a *keyspace* or abou*t a table (cfid)*?
> You'r original was complaining about a table:
>

> at=IncomingTcpConnection.run UnknownColumnFamilyException reading from
>> socket; closing org.apache.cassandra.db.UnknownColumnFamilyException:
>> Couldn't find *cfId=3ecce750-84d3-11e5-bdd9-**dd7717dcdbd5*
>
>
Sorry, you're absolutely right--it's the table from this error message. I
confused myself. But now I was able to find it:

cursors-3ecce75084d311e5bdd9dd7717dcdbd5
cursors-3ed23e8084d311e583b30fc0205655f5

The second uuid is the one that shows up via the schema_columnfamilies
query, but on two of the nodes, the directory with the *other* uuid exists.
Can I just rename the directory on these two nodes? Or how should I proceed?

Timeout with static column

2015-11-11 Thread Brice Figureau

Hi,

Following my previous Read query timing out, I'm now running in another timeout 
issue, on cassandra 2.1.11.

Still with the same schema from the Akka Persistence Cassandra journal:
CREATE TABLE akka.messages (
persistence_id text,
partition_nr bigint,
sequence_nr bigint,
message blob,
used boolean static,
PRIMARY KEY ((persistence_id, partition_nr), sequence_nr)
) WITH CLUSTERING ORDER BY (sequence_nr ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression = {'sstable_compression': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 216000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';


The following query:
SELECT used from akka.messages WHERE
  persistence_id = 'player-SW11f03e20b8802000' AND
  partition_nr = 0;

times out, or when the timeout is increased (or using a faster cassandra 
cluster), it reports the following trace:

 activity   
 | timestamp  | source | 
source_elapsed
-+++

  Execute CQL3 query | 2015-11-11 19:38:34.424000 | 192.168.169.10 |
  0
  READ message received from /192.168.169.10 
[MessagingService-Incoming-/192.168.169.10] | 2015-11-11 19:38:31.621000 | 
192.168.169.20 | 30
  Executing single-partition query on messages 
[SharedPool-Worker-1] | 2015-11-11 19:38:31.623000 | 192.168.169.20 |   
 221
  Acquiring sstable references 
[SharedPool-Worker-1] | 2015-11-11 19:38:31.624000 | 192.168.169.20 |   
 237
   Merging memtable tombstones 
[SharedPool-Worker-1] | 2015-11-11 19:38:31.625000 | 192.168.169.20 |   
 270
  Key cache hit for sstable 15 
[SharedPool-Worker-1] | 2015-11-11 19:38:31.626000 | 192.168.169.20 |   
 438
   Seeking to partition beginning in data file 
[SharedPool-Worker-1] | 2015-11-11 19:38:31.627000 | 192.168.169.20 |   
 445
 Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones 
[SharedPool-Worker-1] | 2015-11-11 19:38:31.628000 | 192.168.169.20 |   
 876
Merging data from memtables and 1 sstables 
[SharedPool-Worker-1] | 2015-11-11 19:38:31.628000 | 192.168.169.20 |   
 884
  Parsing  
[SharedPool-Worker-1] | 2015-11-11 19:38:34.424000 | 192.168.169.10 |   
  83
   Preparing statement 
[SharedPool-Worker-1] | 2015-11-11 19:38:34.424000 | 192.168.169.10 |   
 273
 reading data from /192.168.169.20 
[SharedPool-Worker-1] | 2015-11-11 19:38:34.425000 | 192.168.169.10 |   
 766
 Sending READ message to /192.168.169.20 
[MessagingService-Outgoing-/192.168.169.20] | 2015-11-11 19:38:34.425000 | 
192.168.169.10 |920
   Read 101 live and 0 tombstone cells 
[SharedPool-Worker-1] | 2015-11-11 19:38:37.837000 | 192.168.169.20 | 
215791
 Enqueuing response to /192.168.169.10 
[SharedPool-Worker-1] | 2015-11-11 19:38:37.85 | 192.168.169.20 | 
228498
 Sending REQUEST_RESPONSE message to /192.168.169.10 
[MessagingService-Outgoing-/192.168.169.10] | 2015-11-11 19:38:37.851000 | 
192.168.169.20 | 228572
  REQUEST_RESPONSE message received from /192.168.169.20 
[MessagingService-Incoming-/192.168.169.20] | 2015-11-11 19:38:40.754000 | 
192.168.169.10 | 330080
  Processing response from /192.168.169.20 
[SharedPool-Worker-2] | 2015-11-11 19:38:40.754000 | 192.168.169.10 | 
330177

Request complete | 2015-11-11 19:38:40.813963 | 192.168.169.10 | 
389963

This specific key has about 1900 records of around 50/100 bytes each which 
makes it quite large (compared to others), and the `used` static column is True.

I know this is a C* anti-pattern, but regular

Repair Hangs while requesting Merkle Trees

2015-11-11 Thread Anuj Wadehra

Hi,
We have 2 DCs at remote locations with 10GBps connectivity.We are able to 
complete repair (-par -pr) on 5 nodes. On only one node in DC2, we are unable 
to complete repair as it always hangs. Node sends Merkle Tree requests, but one 
or more nodes in DC1 (remote) never show that they sent the merkle tree reply 
to requesting node.
Repair hangs infinitely. 

After increasing request_timeout_in_ms on affected node, we were able to 
successfully run repair on one of the two occassions.

Any comments, why this is happening on just one node? In 
OutboundTcpConnection.java,  when isTimeOut method always returns false for 
non-droppable verb such as Merkle Tree Request(verb=REPAIR_MESSAGE),why 
increasing request timeout solved problem on one occasion ?

Thanks
Anuj Wadehra

Re: Repair Hangs while requesting Merkle Trees

2015-11-11 Thread Anuj Wadehra

Hi,
we are using 2.0.14. We have 2 DCs at remote locations with 10GBps 
connectivity.We are able to complete repair (-par -pr) on 5 nodes. On only one 
node in DC2, we are unable to complete repair as it always hangs. Node sends 
Merkle Tree requests, but one or more nodes in DC1 (remote) never show that 
they sent the merkle tree reply to requesting node.
Repair hangs infinitely. 

After increasing request_timeout_in_ms on affected node, we were able to 
successfully run repair on one of the two occassions.

Any comments, why this is happening on just one node? In 
OutboundTcpConnection.java,  when isTimeOut method always returns false for 
non-droppable verb such as Merkle Tree Request(verb=REPAIR_MESSAGE),why 
increasing request timeout solved problem on one occasion ?

Thanks
Anuj Wadehra 


 On Thursday, 12 November 2015 2:35 AM, Anuj Wadehra 
 wrote:
   

 Hi,
We have 2 DCs at remote locations with 10GBps connectivity.We are able to 
complete repair (-par -pr) on 5 nodes. On only one node in DC2, we are unable 
to complete repair as it always hangs. Node sends Merkle Tree requests, but one 
or more nodes in DC1 (remote) never show that they sent the merkle tree reply 
to requesting node.
Repair hangs infinitely. 

After increasing request_timeout_in_ms on affected node, we were able to 
successfully run repair on one of the two occassions.

Any comments, why this is happening on just one node? In 
OutboundTcpConnection.java,  when isTimeOut method always returns false for 
non-droppable verb such as Merkle Tree Request(verb=REPAIR_MESSAGE),why 
increasing request timeout solved problem on one occasion ?

Thanks
Anuj Wadehra

Re: Repair Hangs while requesting Merkle Trees

2015-11-11 Thread daemeon reiydelle

Have you checked the network statistics on that machine? (netstats -tas)
while attempting to repair ... if netstats show ANY issues you have a
problem. If you can put the command in a loop running every 60 seconds for
maybe 15 minutes and post back?

Out of curiousity, how many remote DC nodes are getting successfully
repaired?



*...*






*“Life should not be a journey to the grave with the intention of arriving
safely in apretty and well preserved body, but rather to skid in broadside
in a cloud of smoke,thoroughly used up, totally worn out, and loudly
proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
(+1) 415.501.0198London (+44) (0) 20 8144 9872*

On Wed, Nov 11, 2015 at 1:06 PM, Anuj Wadehra 
wrote:

> Hi,
>
> we are using 2.0.14. We have 2 DCs at remote locations with 10GBps
> connectivity.We are able to complete repair (-par -pr) on 5 nodes. On only
> one node in DC2, we are unable to complete repair as it always hangs. Node
> sends Merkle Tree requests, but one or more nodes in DC1 (remote) never
> show that they sent the merkle tree reply to requesting node.
> Repair hangs infinitely.
>
> After increasing request_timeout_in_ms on affected node, we were able to
> successfully run repair on one of the two occassions.
>
> Any comments, why this is happening on just one node? In
> OutboundTcpConnection.java,  when isTimeOut method always returns false for
> non-droppable verb such as Merkle Tree Request(verb=REPAIR_MESSAGE),why
> increasing request timeout solved problem on one occasion ?
>
>
> Thanks
> Anuj Wadehra
>
>
>
> On Thursday, 12 November 2015 2:35 AM, Anuj Wadehra <
> anujw_2...@yahoo.co.in> wrote:
>
>
> Hi,
>
> We have 2 DCs at remote locations with 10GBps connectivity.We are able to
> complete repair (-par -pr) on 5 nodes. On only one node in DC2, we are
> unable to complete repair as it always hangs. Node sends Merkle Tree
> requests, but one or more nodes in DC1 (remote) never show that they sent
> the merkle tree reply to requesting node.
> Repair hangs infinitely.
>
> After increasing request_timeout_in_ms on affected node, we were able to
> successfully run repair on one of the two occassions.
>
> Any comments, why this is happening on just one node? In
> OutboundTcpConnection.java,  when isTimeOut method always returns false for
> non-droppable verb such as Merkle Tree Request(verb=REPAIR_MESSAGE),why
> increasing request timeout solved problem on one occasion ?
>
>
> Thanks
> Anuj Wadehra
>
>
>

Re: Cassandra compaction stuck? Should I disable?

答复: org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:218 throws java.lang.AssertionError

Re: Cassandra compaction stuck? Should I disable?

Re: scylladb

Re: scylladb

Re: scylladb

Re: scylladb

Re: UnknownColumnFamily exception / schema inconsistencies

Re: UnknownColumnFamily exception / schema inconsistencies

Re: Multi-column slice restrictions not respected by the returned result

Re: Multi-column slice restrictions not respected by the returned result

Re: Multi-column slice restrictions not respected by the returned result

Re: UnknownColumnFamily exception / schema inconsistencies

Timeout with static column

Repair Hangs while requesting Merkle Trees

Re: Repair Hangs while requesting Merkle Trees

Re: Repair Hangs while requesting Merkle Trees

17 matches

Site Navigation

Mail list logo

Footer information