Where was the 09_09 column inserted from? Are you sure whatever did the insert is doing a local_quorum on the same DC the cli is in? It may return before all the nodes get response back (ie 2 of the 3 in local DC) which report not having the data. After all the nodes respond it will check the digests from all the responses, see theres an inconsistency and do a read repair. Which would explain it showing up following queries.
Chris On Jun 26, 2014, at 10:06 AM, Ravikumar Govindarajan <ravikumar.govindara...@gmail.com> wrote: > I ran the following set of commands via CLI in our servers. There is a > data-discrepancy that I encountered as below during gets... > > We are running 1.2.4 version with replication-factor=3 (DC1) & 2 (DC2). Reads > and writes are at LOCAL_QUORUM > > create column family TestCF with key_validation_class=AsciiType AND > comparator = 'CompositeType(AsciiType,LongType)' AND > compression_options={sstable_compression:SnappyCompressor, > chunk_length_kb:64}; > > [default@Sample] consistencylevel AS LOCAL_QUORUM; > Consistency level is set to 'LOCAL_QUORUM'. > > [default@Sample] get TestCF [ascii('1773221000000008001')] > ['1773221000004550009_:1773221000004560008']; > => (column=1773221000004550009_:1773221000004560008, > value=31373733323231303030303034353530303039, timestamp=1397743374931) > Elapsed time: 8.64 msec(s). > > //Do a full row dump which shows the above column > [default@Sample] get TestCF [ascii('1773221000000008001')]; > ........................... > => (column=1773221000004547019_:1773221000004560001, > value=31373733323231303030303034353437303139, timestamp=1397743139121) > => (column=1773221000004550009_:1773221000004560008, > value=31373733323231303030303034353530303039, timestamp=1397743374931) > => (column=1773221000004560003_:1773221000004560005, > value=31373733323231303030303034353630303033, timestamp=1397743323261) > => (column=1773221000004562001_:1773221000004564003, > value=31373733323231303030303034353632303031, timestamp=1397749523707) > --------------------------- > Returned 4771 results. > Elapsed time: 518 msec(s). > //Try again > [default@Sample] get TestCF[ascii('1773221000000008001')] > ['1773221000004550009_:1773221000004560008']; > => (column=1773221000004550009_:1773221000004560008, > value=31373733323231303030303034353530303039, timestamp=1397743374931) > Elapsed time: 8.03 msec(s). > > //Here CLI flipped showing value as not found > [default@Sample] get TestCF[ascii('1773221000000008001')] > ['1773221000004550009_:1773221000004550009']; > Value was not found > Elapsed time: 12 msec(s). > > //Query again, it shows as value found > [default@Sample] get TestCF[ascii('1773221000000008001')] > ['1773221000004550009_:1773221000004550009']; > => (column=1773221000004550009_:1773221000004550009, > value=31373733323231303030303034353530303039, timestamp=1397743374931) > Elapsed time: 23 msec(s). > > Is this just limited to CLI bug or some-thing deeper is brewing? Our app > faced a serious issue in code involving this query. Is it a known issue? > > Any help is much appreciated > > -- > Ravi