I see now there is a package org.apache.cassandra.cql3.statements, with 
BatchStatement class. Is this what I should use?

-----Original Message-----
From: Leonid Ilyevsky [mailto:lilyev...@mooncapital.com]
Sent: Tuesday, July 10, 2012 11:45 AM
To: user@cassandra.apache.org
Subject: RE: Dynamic CF

I see. I actually tried it, and it consistently throws an exception. Below is 
my test code. I have two tests; test1 is for the composite key case, and test2 
is for the simple key. The test2 works fine, while test1 gives me:

Exception in thread "main" InvalidRequestException(why:Not enough bytes to read 
value of component 0)
        at 
org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:20253)
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
        at 
org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:922)
        at 
org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:908)
        at com.moon.cql.BatchTest.test1(BatchTest.java:99)
        at com.moon.cql.BatchTest.main(BatchTest.java:45)


So you suggest to use BATCH statement. Since I do it from Java, it means 
creating a huge string (I may need to update thousands records at once), and 
executing it. Does it even make sense? Why is this going to be any better than 
simply execute prepared statement multiple times? The only thing it does is 
reduce number of calls to the server, but I have to figure out if this is the 
bottle neck I need to optimize.
Or maybe I need to break all my updates in a number of batches.
By the way, can a batch statement be prepared? With thousands of question marks 
in it?


============================================================

public class BatchTest {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) throws TTransportException,
            InvalidRequestException, TException, UnavailableException,
            TimedOutException {

        String host = args[0];
        int port = Integer.parseInt(args[1]);

        test1(host, port);
        //test2(host, port);
    }

    private static void test1(String host, int port) throws TTransportException,
            InvalidRequestException, TException, UnavailableException,
            TimedOutException {
        TTransport transport =
                new TFramedTransport(new org.apache.thrift.transport.TSocket(
                host, port));
        transport.open();
        TProtocol protocol = new TBinaryProtocol(transport);
        Cassandra.Client client = new Cassandra.Client(protocol);
        client.set_cql_version("3.0.0");
        client.set_keyspace("test");

        Map<ByteBuffer, Map<String, List<Mutation>>> mutationMap =
                new HashMap();

        Map<String, List<Mutation>> mutations = new HashMap();
        List<Mutation> columnsMutations = new ArrayList();

        // key
        ByteBuffer keyBuffer = AsciiType.instance.decompose("KEY1");

        // key1 as column
        Column key1 = new Column();
        key1.setName("key1".getBytes());
        key1.setValue(LongType.instance.decompose(System.nanoTime()));
        key1.setTimestamp(System.currentTimeMillis());
        ColumnOrSuperColumn cc = new ColumnOrSuperColumn();
        cc.setColumn(key1);
        Mutation m = new Mutation();
        m.setColumn_or_supercolumn(cc);
        columnsMutations.add(m);

        // value column
        Column value = new Column();
        value.setName("value".getBytes());
        value.setValue(DoubleType.instance.decompose(5.3));
        value.setTimestamp(System.currentTimeMillis());
        cc = new ColumnOrSuperColumn();
        cc.setColumn(value);
        m = new Mutation();
        m.setColumn_or_supercolumn(cc);
        columnsMutations.add(m);

        // Inner mutation map
        mutations.put("testtable1", columnsMutations);

        // outer map : use the partition key
        mutationMap.put(keyBuffer, mutations);

        // Execute
        client.batch_mutate(mutationMap, ConsistencyLevel.ANY);
    }

      private static void test2(String host, int port) throws 
TTransportException,
            InvalidRequestException, TException, UnavailableException,
            TimedOutException {
        TTransport transport =
                new TFramedTransport(new org.apache.thrift.transport.TSocket(
                host, port));
        transport.open();
        TProtocol protocol = new TBinaryProtocol(transport);
        Cassandra.Client client = new Cassandra.Client(protocol);
        client.set_cql_version("3.0.0");
        client.set_keyspace("test");

        Map<ByteBuffer, Map<String, List<Mutation>>> mutationMap =
                new HashMap();

        Map<String, List<Mutation>> mutations = new HashMap();
        List<Mutation> columnsMutations = new ArrayList();

        // key
        ByteBuffer keyBuffer = AsciiType.instance.decompose("KEY1");

        // value column
        Column value = new Column();
        value.setName("value".getBytes());
        value.setValue(DoubleType.instance.decompose(5.3));
        value.setTimestamp(System.currentTimeMillis());
        ColumnOrSuperColumn cc = new ColumnOrSuperColumn();
        cc.setColumn(value);
        Mutation m = new Mutation();
        m.setColumn_or_supercolumn(cc);
        columnsMutations.add(m);

        // Inner mutation map
        mutations.put("testtable2", columnsMutations);

        // outer map : use the partition key
        mutationMap.put(keyBuffer, mutations);

        // Execute
        client.batch_mutate(mutationMap, ConsistencyLevel.ANY);
    }
}

============================================================

-----Original Message-----
From: Sylvain Lebresne [mailto:sylv...@datastax.com]
Sent: Tuesday, July 10, 2012 10:37 AM
To: user@cassandra.apache.org
Subject: Re: Dynamic CF

On Tue, Jul 10, 2012 at 4:17 PM, Leonid Ilyevsky
<lilyev...@mooncapital.com> wrote:
> So I guess, in the batch_mutate call, in the map that I pass to it, only the 
> first element of the composite key should be used as a key (because it is the 
> real key), and the other parts of the key should be passed as regular 
> columns? Is this correct? While I am waiting for your confirmation, I am 
> going to try it.

I would really advise you to use the BATCH statement of CQL3 rather
than the thrift batch_mutate call. If only because until
https://issues.apache.org/jira/browse/CASSANDRA-4377 is resolved it
won't work at all, but also because the whole point of CQL3 is to hide
that kind of complexity.

--
Sylvain

>
> -----Original Message-----
> From: Sylvain Lebresne [mailto:sylv...@datastax.com]
> Sent: Tuesday, July 10, 2012 8:24 AM
> To: user@cassandra.apache.org
> Subject: Re: Dynamic CF
>
> On Fri, Jul 6, 2012 at 10:49 PM, Leonid Ilyevsky
> <lilyev...@mooncapital.com> wrote:
>> At this point I am really confused about what direction Cassandra is going. 
>> CQL 3 has the benefit of composite keys, but no dynamic columns.
>> I thought, the whole point of Cassandra was to provide dynamic tables.
>
> CQL3 absolutely provide "dynamic tables"/wide rows, the syntax is just
> different. The typical example for wide rows is a time serie, for
> instance keeping all the events for a given event_kind in the same C*
> row ordered by time. You declare that in CQL3 using:
>   CREATE TABLE events (
>     event_kind text,
>     time timestamp,
>     event_name text,
>     event_details text,
>     PRIMARY KEY (event_kind, time)
>   )
>
> The important part in such definition is that one CQL row (i.e a given
> event_kind, time, event_name, even_details) does not map to an internal
> Cassandra row. More precisely, all events sharing the same event_kind will be
> in the same internal row. This is a wide row/dynamic table in the sense of
> thrift.
>
>
>> I need to have a huge table to store market quotes, and be able to query it 
>> by name and timestamp (t1 <= t <= t2), therefore I wanted the composite key.
>> Loading data to such table using prepared statements (CQL 3-based) was very 
>> slow, because it makes a server call for each row.
>
> You should use a BATCH statement which is the equivalent to batch_mutate.
>
> --
> Sylvain
>
> This email, along with any attachments, is confidential and may be legally 
> privileged or otherwise protected from disclosure. Any unauthorized 
> dissemination, copying or use of the contents of this email is strictly 
> prohibited and may be in violation of law. If you are not the intended 
> recipient, any disclosure, copying, forwarding or distribution of this email 
> is strictly prohibited and this email and any attachments should be deleted 
> immediately.  This email and any attachments do not constitute an offer to 
> sell or a solicitation of an offer to purchase any interest in any investment 
> vehicle sponsored by Moon Capital Management LP ("Moon Capital"). Moon 
> Capital does not provide legal, accounting or tax advice. Any statement 
> regarding legal, accounting or tax matters was not intended or written to be 
> relied upon by any person as advice. Moon Capital does not waive 
> confidentiality or privilege as a result of this email.

This email, along with any attachments, is confidential and may be legally 
privileged or otherwise protected from disclosure. Any unauthorized 
dissemination, copying or use of the contents of this email is strictly 
prohibited and may be in violation of law. If you are not the intended 
recipient, any disclosure, copying, forwarding or distribution of this email is 
strictly prohibited and this email and any attachments should be deleted 
immediately.  This email and any attachments do not constitute an offer to sell 
or a solicitation of an offer to purchase any interest in any investment 
vehicle sponsored by Moon Capital Management LP ("Moon Capital"). Moon Capital 
does not provide legal, accounting or tax advice. Any statement regarding 
legal, accounting or tax matters was not intended or written to be relied upon 
by any person as advice. Moon Capital does not waive confidentiality or 
privilege as a result of this email.

This email, along with any attachments, is confidential and may be legally 
privileged or otherwise protected from disclosure. Any unauthorized 
dissemination, copying or use of the contents of this email is strictly 
prohibited and may be in violation of law. If you are not the intended 
recipient, any disclosure, copying, forwarding or distribution of this email is 
strictly prohibited and this email and any attachments should be deleted 
immediately.  This email and any attachments do not constitute an offer to sell 
or a solicitation of an offer to purchase any interest in any investment 
vehicle sponsored by Moon Capital Management LP ("Moon Capital"). Moon Capital 
does not provide legal, accounting or tax advice. Any statement regarding 
legal, accounting or tax matters was not intended or written to be relied upon 
by any person as advice. Moon Capital does not waive confidentiality or 
privilege as a result of this email.

Reply via email to