Collecting slow queries

[Jan]

HI Folks; 
is there a way for 'Collecting slow queries'  in the Apache Cassandra. ?I am 
aware of the DSE product offering such an option, but need the solution on 
Apache Cassandra. 
ThanksJan

Re: Collecting slow queries

[Jan]

Hello Yoshi-san; 
is this fix rolled into  Cassandra 3.7.0 ?  I do not see it in the 
cassandra.yaml file.
Is there anything special that needs to be downloaded for this feature to show 
up that I am missing in  Cassandra 3.7.0.
Thank you for your prompt response earlier, Jan
 

On Monday, December 5, 2016 3:42 PM, Yoshi Kimoto  
wrote:
 

 This? : https://issues.apache.org/jira/browse/CASSANDRA-12403

2016-12-06 6:36 GMT+09:00 Jeff Jirsa :

> Should we reopen 6226? Tracing 0.1% doesn’t help find the outliers that
> are slow but don’t time out (slow query log could help find large
> partitions for users with infrequent but painful large partitions, far
> easier than dumping sstables to json to identify them).
>
>
> On 12/5/16, 1:28 PM, "sankalp kohli"  wrote:
>
> >This is duped by a JIRA which is fixed in 3.2
> >
> > https://issues.apache.org/jira/browse/CASSANDRA-6226
> >
> >On Mon, Dec 5, 2016 at 12:15 PM, Jan  wrote:
> >
> >> HI Folks;
> >> is there a way for 'Collecting slow queries'  in the Apache Cassandra.
> ?I
> >> am aware of the DSE product offering such an option, but need the
> solution
> >> on Apache Cassandra.
> >> ThanksJan
>

   

OpsCenter with client to node encryption

[Jan Kesten]

Hi all,

I am trying to setup internode and client encryption on cassandra. I set 
up a small ca, generated the certificates, distributed them and 
configured the nodes to use them.


Internode encryption worked straight forward, cqlsh after I added "--ssl".

But I am not able to setup OpsCenter (running 5.1.1). Two issues:

1. I added the ca file path, for me /etc/opscenter/cassandra_ca.pem, as 
asked. I cant save the cluster until I add a keystore even if I did not 
set a mark for client verification - also I cant find any documentation 
which keystore is meant here. Since OpsCenter is python these are 
obviously not the jks keystores from cassandra.


I guess that it is meant in that way, the individual nodes present thier 
certificate to opscenter which would verify it against the ca-store.


2. Trying to connect gives me an error in opscenterd.log:

2015-05-26 10:34:27+ []  INFO: Using SSL when checking thrift 
connection: /etc/opscenter/cassandra_ca.pem, client_pem=None, 
client_key=None,

validate=True
2015-05-26 10:34:27+ []  INFO: Starting factory 
0x7fa490ff97a0>

2015-05-26 10:34:27+ [] Unhandled Error
Traceback (most recent call last):
  File 
"/usr/share/opscenter/lib/py-debian/2.7/amd64/twisted/python/log.py", 
line 84, in callWithLogger

return callWithContext({"system": lp}, func, *args, **kw)
  File 
"/usr/share/opscenter/lib/py-debian/2.7/amd64/twisted/python/log.py", 
line 69, in callWithContext

return context.call({ILogContext: newCtx}, func, *args, **kw)
  File 
"/usr/share/opscenter/lib/py-debian/2.7/amd64/twisted/python/context.py", line 
59, in callWithContext
return self.currentContext().callWithContext(ctx, func, 
*args, **kw)
  File 
"/usr/share/opscenter/lib/py-debian/2.7/amd64/twisted/python/context.py", line 
37, in callWithContext

return func(*args,**kw)
---  ---
  File 
"/usr/share/opscenter/lib/py-debian/2.7/amd64/twisted/internet/epollreactor.py", 
line 220, in _doReadOrWrite

why = selectable.doWrite()
  File 
"/usr/share/opscenter/lib/py-debian/2.7/amd64/twisted/internet/tcp.py", 
line 664, in doConnect

self._connectDone()
  File 
"/usr/share/opscenter/lib/py-debian/2.7/amd64/twisted/internet/ssl.py", 
line 160, in _connectDone

self.startTLS(self.ctxFactory)
  File 
"/usr/share/opscenter/lib/py-debian/2.7/amd64/twisted/internet/tcp.py", 
line 561, in startTLS

if Connection.startTLS(self, ctx, client):
  File 
"/usr/share/opscenter/lib/py-debian/2.7/amd64/twisted/internet/tcp.py", 
line 402, in startTLS

self.socket = SSL.Connection(ctx.getContext(), self.socket)
  File 
"/usr/lib/python2.7/dist-packages/opscenterd/SslUtils.py", line 54, in 
getContext


  File "/usr/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 
303, in load_verify_locations

raise TypeError("cafile must be None or a byte string")
exceptions.TypeError: cafile must be None or a byte string

2015-05-26 10:34:27+ []  INFO: instance at 0x7fa490ff9a70> will retry in 2 seconds

2015-05-26 10:34:27+ []  INFO: Unhandled error in Deferred:
2015-05-26 10:34:27+ [] Unhandled Error
Traceback (most recent call last):
Failure: twisted.internet.error.ConnectError: An error occurred 
while connecting: [Failure instance: Traceback (failure with no frames): 
: cafile must be None or a byte string

].

Any hints about this?

Thanks in advance,
Jan

Should we align JMX Authentication with internal Cassandra authentication?

[Jan Karlsson]

Hello fellow devs,

Currently JMX authentication is handled by using a password file. I was 
thinking we could make this a little more sophisticated.
What I propose is, we plug JMX so that it uses the internal authenticator 
Cassandra has to authenticate the user. This will allow us to authenticate 
towards the Cassandra database. (No pesky password file)
We can make this pluggable so that users could add their own authenticator and 
authorizer.

I have created a JIRA ticket on this as I have been working on a solution which 
would let us do precisely this. 
https://issues.apache.org/jira/browse/CASSANDRA-10091

What are your thoughts on this?

Best Regards
Jan Karlsson

RE: Should we align JMX Authentication with internal Cassandra authentication?

[Jan Karlsson]

If by Permission class you are referring to the enum class which holds the 
different permissions. Then that’s one possible solution. The other would be 
expanding the existing permission class. We then use the internal authorizer in 
Cassandra to authorize against.
We would need a way to create permissions via CQL and create a new type of 
resource which represents a JMX resource.

-Original Message-
From: Jonathan Ellis [mailto:jbel...@gmail.com] 
Sent: August 17, 2015 6:00 PM
To: dev
Subject: Re: Should we align JMX Authentication with internal Cassandra 
authentication?

So we would create a new permission class for JMX?

On Mon, Aug 17, 2015 at 8:34 AM, Jan Karlsson 
wrote:

> Hello fellow devs,
>
> Currently JMX authentication is handled by using a password file. I 
> was thinking we could make this a little more sophisticated.
> What I propose is, we plug JMX so that it uses the internal 
> authenticator Cassandra has to authenticate the user. This will allow 
> us to authenticate towards the Cassandra database. (No pesky password 
> file) We can make this pluggable so that users could add their own 
> authenticator and authorizer.
>
> I have created a JIRA ticket on this as I have been working on a 
> solution which would let us do precisely this.
> https://issues.apache.org/jira/browse/CASSANDRA-10091
>
> What are your thoughts on this?
>
> Best Regards
> Jan Karlsson
>



--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder, http://www.datastax.com
@spyced

Nodetool Cleanup Problem

[Jan Ali]

Hi All, 

I use cassandra 3.4.When running 'nodetool cleanup' command , see this error?
error: Expecting URI in variable: [cassandra.config]. Found[cassandra.yaml]. 
Please prefix the file with [file:///] for local files and [file:///] 
for remote files. If you are executing this from an external tool, it needs to 
set Config.setClientMode(true) to avoid loading configuration.
-- StackTrace --
org.apache.cassandra.exceptions.ConfigurationException: Expecting URI in 
variable: [cassandra.config]. Found[cassandra.yaml]. Please prefix the file 
with [file:///] for local files and [file:///] for remote files. If you 
are executing this from an external tool, it needs to set 
Config.setClientMode(true) to avoid loading configuration.
    at 
org.apache.cassandra.config.YamlConfigurationLoader.getStorageConfigURL(YamlConfigurationLoader.java:78)
    at 
org.apache.cassandra.config.YamlConfigurationLoader.(YamlConfigurationLoader.java:92)
    at 
org.apache.cassandra.config.DatabaseDescriptor.loadConfig(DatabaseDescriptor.java:134)
    at 
org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:121)
    at 
org.apache.cassandra.config.CFMetaData$Builder.(CFMetaData.java:1160)
    at 
org.apache.cassandra.config.CFMetaData$Builder.create(CFMetaData.java:1175)
    at 
org.apache.cassandra.config.CFMetaData$Builder.create(CFMetaData.java:1170)
    at 
org.apache.cassandra.cql3.statements.CreateTableStatement.metadataBuilder(CreateTableStatement.java:118)
    at org.apache.cassandra.config.CFMetaData.compile(CFMetaData.java:413)
    at 
org.apache.cassandra.schema.SchemaKeyspace.compile(SchemaKeyspace.java:238)
    at 
org.apache.cassandra.schema.SchemaKeyspace.(SchemaKeyspace.java:88)
    at org.apache.cassandra.config.Schema.(Schema.java:96)
    at org.apache.cassandra.config.Schema.(Schema.java:50)
    at org.apache.cassandra.tools.nodetool.Cleanup.execute(Cleanup.java:45)
    at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:248)
    at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:162)

Can anyone help me?
Best regards,
Jan Ali

Unit tests in IDE

[Jan Kotek]

Hi,

I would like to run and debug unit tests directly in Intellij Idea.
Current Ant runner does not work for me.

I studied the problem a bit. Tests use java-agent that complicates it a
bit. Also I would like to make minimal changes. So far my approach is:

- Run special test case under regular Ant

- It saves JVM parameters generated by Ant (properties, JVM args, java
agents, classpath) into text file

- Invoke Ant target to convert this text file into Run Configuration
for IDE

- Click Run in Idea... :-)

I would be happy to polish it a bit more. Would you merge such patch?

Regards,
Jan Kotek

Re: Unit tests in IDE

[Jan Kotek]

It does, I missed that, thanks

On Thu, 2016-06-09 at 11:53 -0500, Josh McKenzie wrote:
> Does ant generate-idea-files
> <https://wiki.apache.org/cassandra/RunningCassandraInIDEA> cover
> this?
> 
> On Thu, Jun 9, 2016 at 11:26 AM, Jan Kotek  wrote:
> 
> > 
> > Hi,
> > 
> > I would like to run and debug unit tests directly in Intellij Idea.
> > Current Ant runner does not work for me.
> > 
> > I studied the problem a bit. Tests use java-agent that complicates
> > it a
> > bit. Also I would like to make minimal changes. So far my approach
> > is:
> > 
> > - Run special test case under regular Ant
> > 
> > - It saves JVM parameters generated by Ant (properties, JVM args,
> > java
> > agents, classpath) into text file
> > 
> > - Invoke Ant target to convert this text file into Run
> > Configuration
> > for IDE
> > 
> > - Click Run in Idea... :-)
> > 
> > I would be happy to polish it a bit more. Would you merge such
> > patch?
> > 
> > Regards,
> > Jan Kotek
> > 

Re: Unit tests in IDE

[Jan Kotek]

Hi,
I tried, but there is no way to create test configuration on 3.0 an 3.7
branches. 
Project generated by generate-idea-files or generate-eclipse-files only
contains launcher for Cassandra deamon, not unit test. 
And way you described does not deal with JVM settings some unit tests
needs.
Only way to run unit tests from IDE is via Ant, that does not allow
debuging. 
I could create launcher manually and replicate all JVM settings from
Ant runner, but that seems a bit too complex:
https://github.com/apache/cassandra/blob/trunk/build.xml#L1201
Jan
  
On Fri, 2016-06-10 at 09:55 +0800, Mahdi Mohammadi wrote:
> You can create JUnit run configurations in IDEA and run/debug a
> single unit test. You will need to specify class name of the test.
> 
> 
> Best Regards
> 
> On Fri, Jun 10, 2016 at 12:53 AM, Josh McKenzie  > wrote:
> > Does ant generate-idea-files
> > <https://wiki.apache.org/cassandra/RunningCassandraInIDEA> cover
> > this?
> > 
> > On Thu, Jun 9, 2016 at 11:26 AM, Jan Kotek 
> > wrote:
> > 
> > > Hi,
> > >
> > > I would like to run and debug unit tests directly in Intellij
> > Idea.
> > > Current Ant runner does not work for me.
> > >
> > > I studied the problem a bit. Tests use java-agent that
> > complicates it a
> > > bit. Also I would like to make minimal changes. So far my
> > approach is:
> > >
> > > - Run special test case under regular Ant
> > >
> > > - It saves JVM parameters generated by Ant (properties, JVM args,
> > java
> > > agents, classpath) into text file
> > >
> > > - Invoke Ant target to convert this text file into Run
> > Configuration
> > > for IDE
> > >
> > > - Click Run in Idea... :-)
> > >
> > > I would be happy to polish it a bit more. Would you merge such
> > patch?
> > >
> > > Regards,
> > > Jan Kotek
> > >
> > 

Re: SS Tables Implementation

[Jan Kesten]

Hi Karl,

what do you want to do? If you need a high performant, scaleable, 
redundant in-memory data grid I think you may be better off using 
Hazelcast, which is available as enterprise grade and open source software.


To rather short describe what it does, Hazelcast provides distributed 
collections compatible to java.util.Collections. There are also 
executors, replication and so on. Of course there is many more to say 
about it, but if you have a question just ask.


http://hazelcast.org/

I wrote this off-list as I think this is something off-topic.

Hope to help,
Jan

Am 03.02.2015 um 14:45 schrieb Karl Kröber:

Hello Cassandra Devs,

First of all, forgive me if I break some mailing-list rules, this is the first 
time I’m actually writing to an open source project in this manner.

We’re interested in using the SS Tables implementation of Cassandra for fast 
and efficient memory mapped / in-memory data access with generic keys and 
values. The reason is, that we couldn’t find any other native Java 
implementation. Sadly, the current code is tightly interwoven with the rest of 
Cassandra which makes it very hard to split it apart from the rest of the 
project.

Do you guys have some pointers on how we could achieve this?
What classes do we need (currently I’m thinking the complete io pacakge)?
How do we separate the whole Cassandra overhead like CFMetaData and stuff like 
CellName?

My current implementation looks like this: 
https://gist.github.com/kroeber/6170be56fcbbdfb9861e 
<https://gist.github.com/kroeber/6170be56fcbbdfb9861e>
Sadly this doesn’t work and fails with the exception I’ve posted in comment to 
that gist.

Any pointers would be very helpful! Thanks for your engagement!

~Karl



--
i.A. Jan Kesten Systemadministration enercast GmbH Friedrich - Ebert - 
Straße 104 D–34119 Kassel Tel.: +49 561 / 4739664-0 Fax: 
(+49)561/4739664-9 mailto: j.kes...@enercast.de http://www.enercast.de 
AG Kassel HRB 15471 Thomas Landgraf Geschäftsführer 
t.landg...@enercast.de Tel.: (+49)561/4739664-0 FAX: -9 Mobil: 
(+49)172/6565087 enercast GmbH Friedrich-Ebert-Str. 104 D-34119 Kassel 
HRB15471 http://www.enercast.de Online-Prognosen für erneuerbare 
Energien Geschäftsführung: Thomas Landgraf (CEO), Bernd Kratz (CTO), 
Philipp Rinder (CSO) Diese E-Mail und etwaige Anhänge können 
vertrauliche und/oder rechtlich geschützte Informationen enthalten. 
Falls Sie nicht der angegebene Empfänger sind oder falls diese E-Mail 
irrtümlich an Sie adressiert wurde, benachrichtigen Sie uns bitte sofort 
durch Antwort-E-Mail und löschen Sie diese E-Mail nebst etwaigen Anlagen 
von Ihrem System. Ebenso dürfen Sie diese E-Mail oder ihre Anlagen nicht 
kopieren oder an Dritte weitergeben. Vielen Dank. This e-mail and any 
attachment may contain confidential and/or privileged information. If 
you are not the named addressee or if this transmission has been 
addressed to you in error, please notify us immediately by reply e-mail 
and then delete this e-mail and any attachment from your system. Please 
understand that you must not copy this e-mail or any attachment or 
disclose the contents to any other person. Thank you for your cooperation.

Problem with upgrade to 2.1.3

[Jan Kesten]

Hi all,

this is my first post on the development list of cassandra - but I think 
this belongs here.


I'm running cassandra for a while now, and on testing I ran in many of 
2.1.2s issues. So I upgraded my test cluster from 2.1.2 to 2.1.3. I 
updated with the debian packages, diffed my cassandra.yaml and applied 
our changes to the new cassandra.yaml. Those changes are:


- cluster_name
- seeds
- listen_interface instead of listen_address
- rpc_interface instead of rpc_address

Nothing really spectacular - interface is nice as all nodes have the 
same cassandra.yaml. But trying to start 2.1.3 gives me an NPE:


ERROR [main] 2015-02-20 07:50:09,661 DatabaseDescriptor.java:144 - Fatal 
error during configuration loading

java.lang.NullPointerException: null
at 
org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:411) 
~[apache-cassandra-2.1.3.jar:2.1.3]
at 
org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:133) 
~[apache-cassandra-2.1.3.jar:2.1.3]
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:110) 
[apache-cassandra-2.1.3.jar:2.1.3]
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465) 
[apache-cassandra-2.1.3.jar:2.1.3]
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554) 
[apache-cassandra-2.1.3.jar:2.1.3]


I also tried the .tar.gz distribution with the same result. I looked at 
the code of DatabaseDescriptor, and if someone uses rpc_interface this 
piece of code is executed:


/* Local IP, hostname or interface to bind RPC server to */
if(conf.rpc_address !=null&& conf.rpc_interface !=null)
{
throw newConfigurationException("Set rpc_address OR rpc_interface, not 
both");
}
else if(conf.rpc_address !=null)
{
try
{
rpcAddress = InetAddress.getByName(conf.rpc_address);
}
catch(UnknownHostException e)
{
throw newConfigurationException("Unknown host in rpc_address "+ 
conf.rpc_address);
}
}
else if(conf.rpc_interface !=null)
{
listenAddress = 
getNetworkInterfaceAddress(conf.rpc_interface,"rpc_interface");
}
else
{
rpcAddress = FBUtilities.getLocalAddress();
}


I think that listenAddress in the second else block is an error. In my 
case rpc_interface is eth0, so listenAddress gets set, and rpcAddress 
remains unset. The result is NPE in line 411:


if(rpcAddress.isAnyLocalAddress())

After changing rpc_interface to rpc_address everything works as expected.

Best regards,
Jan

Re: Problem with upgrade to 2.1.3

[Jan Kesten]

Hi,
I put this into an JIRA issue:CASSANDRA-8839 

use of static method/variables in the codebase

[Asif Jan]

Hi

Being a novice to the cassandra project, I have a question. I see a  
lots of code in cassandra uses static methods/members. Was/Is this a  
design choice ?


The only implication for users like me is that it is slightly  
difficult to debug the code specially when you want to follow the  
interaction between different components etc.


cheers

asif

Re: use of static method/variables in the codebase

[Asif Jan]

thanks a million

asif
On Sep 8, 2010, at 10:32 AM, roger schildmeijer wrote:

The best I can come up with gdusbabek's talk from the cassandra  
summit.


slides:
http://www.slideshare.net/gdusbabek/getting-to-know-the-cassandra-codebase
video: http://blip.tv/file/4015751

On Wed, Sep 8, 2010 at 10:11 AM, Asif Jan  wrote:



Hi

Being a novice to the cassandra project, I have a question. I see a  
lots of
code in cassandra uses static methods/members. Was/Is this a design  
choice ?


The only implication for users like me is that it is slightly  
difficult to
debug the code specially when you want to follow the interaction  
between

different components etc.

cheers

asif