Oleg, note that the unofficial recommendation is to use microsec, not mili.
As jonathan notes, although there isn't a real way to get microsec in java,
at the very least you should take the mili and multiply it by 1000. If you
use hector then just use Keyspace.createTimestamp() (
http://github.com/rantav/hector/blob/master/src/main/java/me/prettyprint/cassandra/service/Keyspace.java#L236
)
On Sun, Jul 25, 2010 at 8:54 AM, Oleg Tsvinev <oleg.tsvi...@gmail.com>wrote:
> Thank you guys for your help!
>
> Yes, I am using System.currentTimeMillis() in my CRUD test. Even though I'm
> still using it my tests now run as expected. I do not use cassandra-cli
> anymore.
>
> @Ran great job on Hector, I wish there was more documentation but I
> managed.
>
> @Jonathan, what is the recommended time source? I use batch_mutation to
> insert and update multiple columns atomically. Do I have to use
> the batch_mutation for deletion, too?
>
> On Sat, Jul 24, 2010 at 2:36 PM, Jonathan Shook <jsh...@gmail.com> wrote:
>
>> Just to clarify, microseconds may be used, but they provide the same
>> behavior as milliseconds if they aren't using a higher time resolution
>> underneath. In some cases, the microseconds are generated simply as
>> milliseconds * 1000, which doesn't actually fix any sequencing bugs.
>>
>> On Sat, Jul 24, 2010 at 3:46 PM, Ran Tavory <ran...@gmail.com> wrote:
>> > Hi Oleg, I didn't follow up the entire thread, but just to let you know
>> that
>> > the 0.6.* version of the CLI uses microsec as the time unit for
>> timestamps.
>> > Hector also uses micros to match that, however, previous versions of
>> hector
>> > (as well as the CLI) used milliseconds, not micro.
>> > So if you're using hector version 0.6.0-11 or earlier, or by any chance
>> in
>> > some other ways are mixing milisec in your app (are you using
>> > System.currentTimeMili() somewhere?) then the behavior you're seeing is
>> > expected.
>> >
>> > On Sat, Jul 24, 2010 at 1:06 AM, Jonathan Shook <jsh...@gmail.com>
>> wrote:
>> >>
>> >> I think you are getting it.
>> >>
>> >> As far as what means what at which level, it's really about using them
>> >> consistently in every case. The [row] key (or [row] key range) is a
>> >> top-level argument for all of the operations, since it is the "key" to
>> >> mapping the set of responsible nodes. The key is the part of the name
>> >> of any column which most affects how the load is apportioned in the
>> >> cluster, so it is used very early in request processing.
>> >>
>> >>
>> >> On Fri, Jul 23, 2010 at 4:22 PM, Peter Minearo
>> >> <peter.mine...@reardencommerce.com> wrote:
>> >> > Consequentially the remove should look like:
>> >> >
>> >> >
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> > cp1.setSuper_column("Best Western".getBytes());
>> >> >
>> >> > client.remove(KEYSPACE,
>> >> > "hotel",
>> >> > cp1,
>> >> > System.currentTimeMillis(),
>> >> > ConsistencyLevel.ONE);
>> >> >
>> >> > ColumnPath cp2 = new ColumnPath("Super2");
>> >> > cp2.setSuper_column("Econolodge".getBytes());
>> >> >
>> >> > client.remove(KEYSPACE,
>> >> > "hotel",
>> >> > cp2,
>> >> > System.currentTimeMillis(),
>> >> > ConsistencyLevel.ONE);
>> >> >
>> >> >
>> >> > -----Original Message-----
>> >> > From: Peter Minearo [mailto:peter.mine...@reardencommerce.com]
>> >> > Sent: Fri 7/23/2010 2:17 PM
>> >> > To: user@cassandra.apache.org
>> >> > Subject: RE: CRUD test
>> >> >
>> >> > CORRECTION:
>> >> >
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> > cp1.setSuper_column("Best Western".getBytes());
>> >> > cp1.setColumn("name".getBytes());
>> >> > client.insert(KEYSPACE, "hotel", cp1, "Best Western of
>> SF".getBytes(),
>> >> > System.currentTimeMillis(), ConsistencyLevel.ALL);
>> >> >
>> >> >
>> >> > -----Original Message-----
>> >> > From: Peter Minearo [mailto:peter.mine...@reardencommerce.com]
>> >> > Sent: Friday, July 23, 2010 2:14 PM
>> >> > To: user@cassandra.apache.org
>> >> > Subject: RE: CRUD test
>> >> >
>> >> > Interesting!! Let me rephrase to make sure I understood what is going
>> >> > on:
>> >> >
>> >> > When Inserting data via the "insert" function/method:
>> >> >
>> >> > void insert(string keyspace, string key, ColumnPath column_path,
>> binary
>> >> > value, i64 timestamp, ConsistencyLevel consistency_level)
>> >> >
>> >> > The "key" parameter is the actual Key to the "Row", which contains
>> >> > SuperColumns. The 'ColumnPath' gives the path within the Key.
>> >> >
>> >> >
>> >> >
>> >> > INCORRECT:
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> > cp1.setSuper_column("hotel".getBytes());
>> >> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> >> > "name", cp1, "Best Western of SF".getBytes(),
>> System.currentTimeMillis(),
>> >> > ConsistencyLevel.ALL);
>> >> >
>> >> >
>> >> > CORRECT:
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> > cp1.setSuper_column("name".getBytes());
>> >> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> >> > "hotel", cp1, "Best Western of SF".getBytes(),
>> System.currentTimeMillis(),
>> >> > ConsistencyLevel.ALL);
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > -----Original Message-----
>> >> > From: Jonathan Shook [mailto:jsh...@gmail.com]
>> >> > Sent: Friday, July 23, 2010 1:49 PM
>> >> > To: user@cassandra.apache.org
>> >> > Subject: Re: CRUD test
>> >> >
>> >> > Correct.
>> >> >
>> >> > After the initial insert,
>> >> >
>> >> > cassandra> get Keyspace1.Super2['name']
>> >> > => (super_column=hotel,
>> >> > (column=Best Western, value=Best Western of SF,
>> >> > timestamp=1279916772571)
>> >> > (column=Econolodge, value=Econolodge of SF,
>> >> > timestamp=1279916772573)) Returned 1 results.
>> >> >
>> >> > ... and ...
>> >> >
>> >> > cassandra> get Keyspace1.Super2['hotel']
>> >> > Returned 0 results.
>> >> >
>> >> >
>> >> >
>> >> > On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo
>> >> > <peter.mine...@reardencommerce.com> wrote:
>> >> >> The Model Should look like:
>> >> >>
>> >> >>
>> >> >> Super2 = {
>> >> >> hotel: {
>> >> >> Best Western: {name: "Best Western of SF"}
>> >> >> Econolodge: {name: "Econolodge of SF"}
>> >> >> }
>> >> >> }
>> >> >>
>> >> >> Are the CRUD Operations not referencing this correctly?
>> >> >>
>> >> >>
>> >> >>
>> >> >> -----Original Message-----
>> >> >> From: Jonathan Shook [mailto:jsh...@gmail.com]
>> >> >> Sent: Friday, July 23, 2010 1:34 PM
>> >> >> To: user@cassandra.apache.org
>> >> >> Subject: Re: CRUD test
>> >> >>
>> >> >> There seem to be data consistency bugs in the test. Are "name" and
>> >> >> "hotel" being used in a pair-wise way?
>> >> >> Specifically, the first test is using creating one and checking for
>> the
>> >> >> other.
>> >> >>
>> >> >> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <
>> oleg.tsvi...@gmail.com>
>> >> >> wrote:
>> >> >>> Johathan,
>> >> >>> I followed your suggestion. Unfortunately, CRUD test still does not
>> >> >>> work for me. Can you provide a simplest CRUD test possible that
>> works?
>> >> >>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <jsh...@gmail.com
>> >
>> >> >>> wrote:
>> >> >>>>
>> >> >>>> I suspect that it is still your timestamps.
>> >> >>>> You can verify this with a fake timestamp generator that is simply
>> >> >>>> incremented on each getTimestamp().
>> >> >>>>
>> >> >>>> 1 millisecond is a long time for code that is wrapped tightly in a
>> >> >>>> test. You are likely using the same logical time stamp for
>> multiple
>> >> >>>> operations.
>> >> >>>>
>> >> >>>>
>> >> >>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>> >> >>>> <peter.mine...@reardencommerce.com> wrote:
>> >> >>>> > I am able to reproduce his problem. If you take the default
>> >> >>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>> >> >>>> > the code below. You will see that the data is not getting
>> created
>> >> >>>> > once you run the delete. It seems to not allow you to create
>> data
>> >> >>>> > via Thrift. HOWEVER, data can be created via the command line
>> >> >>>> > tool.
>> >> >>>> >
>> >> >>>> > import java.io.UnsupportedEncodingException;
>> >> >>>> > import java.util.List;
>> >> >>>> >
>> >> >>>> > import org.apache.cassandra.thrift.Cassandra;
>> >> >>>> > import org.apache.cassandra.thrift.Column;
>> >> >>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>> >> >>>> > import org.apache.cassandra.thrift.ColumnParent;
>> >> >>>> > import org.apache.cassandra.thrift.ColumnPath;
>> >> >>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>> >> >>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>> >> >>>> > import org.apache.cassandra.thrift.NotFoundException;
>> >> >>>> > import org.apache.cassandra.thrift.SlicePredicate;
>> >> >>>> > import org.apache.cassandra.thrift.SliceRange;
>> >> >>>> > import org.apache.cassandra.thrift.SuperColumn;
>> >> >>>> > import org.apache.cassandra.thrift.TimedOutException;
>> >> >>>> > import org.apache.cassandra.thrift.UnavailableException;
>> >> >>>> > import org.apache.thrift.TException; import
>> >> >>>> > org.apache.thrift.protocol.TBinaryProtocol;
>> >> >>>> > import org.apache.thrift.protocol.TProtocol;
>> >> >>>> > import org.apache.thrift.transport.TSocket;
>> >> >>>> > import org.apache.thrift.transport.TTransport;
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > public class CrudTest {
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > private static final String KEYSPACE = "Keyspace1";
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > public static void main(String[] args) {
>> >> >>>> > CrudTest client = new CrudTest();
>> >> >>>> >
>> >> >>>> > try {
>> >> >>>> > client.run();
>> >> >>>> > } catch (Exception e) {
>> >> >>>> > e.printStackTrace();
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > public void run() throws TException,
>> >> >>>> > InvalidRequestException, UnavailableException,
>> >> >>>> > UnsupportedEncodingException, NotFoundException,
>> TimedOutException
>> >> >>>> > {
>> >> >>>> > TTransport tr = new TSocket("localhost", 9160);
>> >> >>>> > TProtocol proto = new TBinaryProtocol(tr);
>> >> >>>> > Cassandra.Client client = new
>> >> >>>> > Cassandra.Client(proto);
>> >> >>>> > tr.open();
>> >> >>>> >
>> >> >>>> > System.out.println("******** CREATING DATA
>> >> >>>> > *********");
>> >> >>>> > createData(client);
>> >> >>>> > getData(client);
>> >> >>>> > System.out.println();
>> >> >>>> > System.out.println("******** DELETING DATA
>> >> >>>> > *********");
>> >> >>>> > deleteData(client);
>> >> >>>> > getData(client);
>> >> >>>> > System.out.println();
>> >> >>>> > System.out.println("******** CREATING DATA
>> >> >>>> > *********");
>> >> >>>> > createData(client);
>> >> >>>> > getData(client);
>> >> >>>> >
>> >> >>>> > tr.close();
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > private void createData(Cassandra.Client client) throws
>> >> >>>> > InvalidRequestException, UnavailableException,
>> TimedOutException,
>> >> >>>> > TException {
>> >> >>>> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> >>>> > cp1.setSuper_column("hotel".getBytes());
>> >> >>>> > cp1.setColumn("Best Western".getBytes());
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > client.insert(KEYSPACE,
>> >> >>>> > "name",
>> >> >>>> > cp1,
>> >> >>>> > "Best Western of SF".getBytes(),
>> >> >>>> > System.currentTimeMillis(),
>> >> >>>> > ConsistencyLevel.ALL);
>> >> >>>> >
>> >> >>>> > ColumnPath cp2 = new ColumnPath("Super2");
>> >> >>>> > cp2.setSuper_column("hotel".getBytes());
>> >> >>>> > cp2.setColumn("Econolodge".getBytes());
>> >> >>>> >
>> >> >>>> > client.insert(KEYSPACE,
>> >> >>>> > "name",
>> >> >>>> > cp2,
>> >> >>>> > "Econolodge of SF".getBytes(),
>> >> >>>> > System.currentTimeMillis(),
>> >> >>>> > ConsistencyLevel.ALL);
>> >> >>>> >
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > private void deleteData(Cassandra.Client client) throws
>> >> >>>> > InvalidRequestException, UnavailableException,
>> TimedOutException,
>> >> >>>> > TException {
>> >> >>>> >
>> >> >>>> > client.remove(KEYSPACE,
>> >> >>>> > "hotel",
>> >> >>>> > new ColumnPath("Super2"),
>> >> >>>> > System.currentTimeMillis(),
>> >> >>>> > ConsistencyLevel.ONE);
>> >> >>>> >
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > private void getData(Cassandra.Client client) throws
>> >> >>>> > InvalidRequestException, UnavailableException,
>> TimedOutException,
>> >> >>>> > TException {
>> >> >>>> > SliceRange sliceRange = new SliceRange();
>> >> >>>> > sliceRange.setStart(new byte[] {});
>> >> >>>> > sliceRange.setFinish(new byte[] {});
>> >> >>>> >
>> >> >>>> > SlicePredicate slicePredicate = new
>> >> >>>> > SlicePredicate();
>> >> >>>> > slicePredicate.setSlice_range(sliceRange);
>> >> >>>> >
>> >> >>>> > getData(client, slicePredicate);
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > private void getData(Cassandra.Client client,
>> >> >>>> > SlicePredicate
>> >> >>>> > slicePredicate) throws InvalidRequestException,
>> >> >>>> > UnavailableException, TimedOutException, TException {
>> >> >>>> > List<ColumnOrSuperColumn> coscList =
>> >> >>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>> >> >>>> > slicePredicate, ConsistencyLevel.ALL);
>> >> >>>> >
>> >> >>>> > if (coscList.isEmpty()) {
>> >> >>>> > System.out.println("Column Or Super
>> Column
>> >> >>>> > is EMPTY");
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> > for (ColumnOrSuperColumn cosc: coscList) {
>> >> >>>> >
>> >> >>>> > if (cosc == null) {
>> >> >>>> > System.out.println("NULL RETURN
>> >> >>>> > VALUE");
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> > SuperColumn superColumn =
>> >> >>>> > cosc.getSuper_column();
>> >> >>>> >
>> >> >>>> > if (superColumn == null) {
>> >> >>>> > System.out.println("Super Column
>> is
>> >> >>>> > NULL");
>> >> >>>> > }
>> >> >>>> > else {
>> >> >>>> > showSuperColumnInfo(superColumn);
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > private void showSuperColumnInfo(SuperColumn superColumn)
>> {
>> >> >>>> > System.out.println("######## Super Columns
>> >> >>>> > ###########");
>> >> >>>> > System.out.println("Super Column Name = " + new
>> >> >>>> > String(superColumn.getName()));
>> >> >>>> > List<Column> columnList =
>> superColumn.getColumns();
>> >> >>>> >
>> >> >>>> > System.out.println("--------- Start Columns
>> >> >>>> > -----------");
>> >> >>>> > for (Column column: columnList) {
>> >> >>>> > System.out.println("Column Name = " + new
>> >> >>>> > String(column.getName()));
>> >> >>>> > System.out.println("Column Value = " +
>> new
>> >> >>>> > String(column.getValue()));
>> >> >>>> > }
>> >> >>>> > System.out.println("--------- End Columns
>> >> >>>> > -----------");
>> >> >>>> >
>> >> >>>> > System.out.println("##################################");
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > -----Original Message-----
>> >> >>>> > From: Oleg Tsvinev [mailto:oleg.tsvi...@gmail.com]
>> >> >>>> > Sent: Thu 7/22/2010 1:56 PM
>> >> >>>> > To: user@cassandra.apache.org
>> >> >>>> > Subject: Re: CRUD test
>> >> >>>> >
>> >> >>>> > Yes, and that was the issue. But now after I delete a row from
>> >> >>>> > cassandra-cli, I cannot insert anything back with my code.
>> Insert
>> >> >>>> > code works does not throw any exceptions but when I read just
>> >> >>>> > inserted columns I get NotFoundException at the last line:
>> >> >>>> >
>> >> >>>> > client = borrowClient();
>> >> >>>> > Keyspace keyspace = client.getKeyspace(KEYSPACE,
>> >> >>>> > CONSISTENCY_LEVEL);
>> >> >>>> > ColumnPath cp = new ColumnPath(application);
>> >> >>>> > cp.setSuper_column(uuid.getBytes());
>> >> >>>> > SuperColumn sc = keyspace.getSuperColumn(category,
>> cp);
>> >> >>>> >
>> >> >>>> > It makes me think that once I remove supercolumn it cannot be
>> >> >>>> > created again.
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <zodiac...@gmail.com>
>> >> >>>> > wrote:
>> >> >>>> >
>> >> >>>> > Have you checked the timestamp you're using for the subsequent
>> >> >>>> > inserts is higher than that used in the delete?
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>> >> >>>> > <oleg.tsvi...@gmail.com>
>> >> >>>> > wrote:
>> >> >>>> >> Hi there,
>> >> >>>> >> I'm try...
>> >> >>>> > --
>> >> >>>> > Maybe she awoke to see the roommate's boyfriend swinging from
>> the
>> >> >>>> > chandelier wearing a boar's head.
>> >> >>>> >
>> >> >>>> > Something which you, I, and everyone else would call "Tuesday",
>> of
>> >> >>>> > course.
>> >> >>>> >
>> >> >>>> >
>> >> >>>
>> >> >>>
>> >> >>
>> >> >
>> >> >
>> >
>> >
>>
>
>
