Looking at this closer. I noticed the following in the SSTableImport Class:
if (col.isDeleted) {
cfamily.addColumn(path, hexToBytes(col.value), new
TimestampClock(col.timestamp));
} else {
cfamily.addTombstone(path, hexToBytes(col.value), new
TimestampClock(col.timestamp));
}
This appears to be backwards.
On Tue, Nov 16, 2010 at 4:03 PM, CassUser CassUser <[email protected]>wrote:
> Looked at how DatabaseDescriptor is loading the yaml file. Using that
> approach solves the problem with the column_families mapping exception.
>
> The problems we are running into currently is regarding a known dataset not
> being loaded into our test instance correctly.
>
> Steps:
> 1. Create temp directory to host cassandra test instance
> 2. Create keyspace directories
> 3. Update yaml file, and copy to test location
> 4. Add schema from yaml using DatabaseDescriptor.
> readTablesFromYaml() & DatabaseDescriptor.
> setTableDefinition
> 4. Use SSTableImport.importJson to setup a known dataset. We have JSON
> files converted to 0.7 compliant (byte [] rows)
> 5. Start Cassandra instance using the EmbeddedCassandraService class
>
> Everything appears to work from the log messages. I get the message "
> Sampling index for ..", and finally "Listening for thrift clients...". When
> i use a client to query the data in test instance, I notice keys with no
> columns/values stored (via keyrange slice). Using a open ended column slice
> for a known key I'm getting zero results.
>
> I'm currently running cassandra 0.7. beta2. Is the steps I've outlined
> above suppose to work in 0.7? We did something similar in 0.6.4 without any
> problems. Is there known bugs I can look into?
>
> Thanks.
>
>
>
> On Tue, Nov 16, 2010 at 12:24 PM, Aaron Morton <[email protected]>wrote:
>
>> I've not used the embedded service.
>>
>> The code in o.a.c.service.EmbeddedCassandraService says it will read the
>> yaml file. If the cluster does not have a schema stored I think it will load
>> the one from yaml.
>>
>> Have you tried starting it up with an empty system data dir ? Does it
>> pickup the schema from the yaml?
>>
>> Aaron
>>
>> On 17 Nov, 2010,at 09:17 AM, CassUser CassUser <[email protected]>
>> wrote:
>>
>> Loading yaml file like so:
>> FileInputStream yamlInputStream = new FileInputStream(
>> configTemplateFile);
>> Constructor constructor = new Constructor(Config.class);
>> Yaml yaml = new Yaml(new Loader(constructor));
>> Config conf = (Config) yaml.load(yamlInputStream);
>>
>> Fails on the last line.
>>
>> Although if i have the CFs defined like this:
>> keyspaces:
>> - column_families:
>> - !!org.apache.cassandra.config.RawColumnFamily
>> column_metadata: []
>> column_type: null
>> comment: null
>> compare_subcolumns_with: null
>> compare_with: BytesType
>> default_validation_class: null
>> gc_grace_seconds: 864000
>> keys_cached: 200000.0
>> max_compaction_threshold: 32
>> min_compaction_threshold: 4
>> name: Similarity
>> preload_row_cache: false
>> read_repair_chance: 1.0
>> rows_cached: 0.0
>>
>>
>> it appears to startup. Any idea whats going on here?
>>
>>
>> On Tue, Nov 16, 2010 at 11:58 AM, CassUser CassUser
>> <[email protected]>wrote:
>>
>>> This is embedded for testing cassandra 0.7 beta2. using
>>> EmbeddedCassandraService.
>>>
>>> and manually adding schema programmatically using:
>>> for (KSMetaData table : DatabaseDescriptor.readTablesFromYaml())
>>> {
>>> for (CFMetaData cfm : table.cfMetaData().values()) {
>>> CFMetaData.map(cfm);
>>> }
>>> DatabaseDescriptor.setTableDefinition(table,
>>> DatabaseDescriptor.getDefsVersion());
>>> }
>>>
>>> Is this the correct way to start up a test server, with the schema
>>> loaded?
>>>
>>>
>>>
>>>
>>> On Tue, Nov 16, 2010 at 11:41 AM, Aaron Morton
>>> <[email protected]>wrote:
>>>
>>>> AFAIK the ArrayStoreException is similar to a type mismatch. Is it
>>>> possible you have something mixed up in your class path or source code if
>>>> you built from source?
>>>>
>>>> It looks like the column family info was deserialised into a
>>>> o.a.c.config.RawColumnFamily but when that object was added to the
>>>> RawColumnFamily[] array on o.a.c.config.RawKeyspace if was the wrong type.
>>>>
>>>> Have you tried a clean build ? There are some things in the call stack
>>>> which look custom, are you starting from the command line or is this
>>>> embedded for testing?
>>>>
>>>> Hope that helps.
>>>> Aaron
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 17 Nov, 2010,at 08:09 AM, CassUser CassUser <[email protected]>
>>>> wrote:
>>>>
>>>>
>>>> Here is the yaml:
>>>>
>>>> # Cassandra YAML generated from previous config
>>>> # Configuration wiki:
>>>> http://wiki.apache.org/cassandra/StorageConfiguration
>>>> authenticator: org.apache.cassandraauth.AllowAllAuthenticator
>>>>
>>>> auto_bootstrap: false
>>>> binary_memtable_throughput_in_mb: 256
>>>> cluster_name: Test Cluster
>>>> column_index_size_in_kb: 64
>>>> commitlog_rotation_threshold_in_mb: 128
>>>> commitlog_sync: periodic
>>>> commitlog_sync_period_in_ms: 10000
>>>> compaction_thread_priority: 1
>>>> concurrent_reads: 8
>>>> concurrent_writes: 32
>>>> disk_access_mode: auto
>>>> dynamic_snitch: false
>>>> endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch
>>>> request_scheduler: org.apache.cassandra.scheduler.RoundRobinScheduler
>>>> request_scheduler_id: keyspace
>>>> hinted_handoff_enabled: true
>>>> in_memory_compaction_limit_in_mb: 128
>>>> index_interval: 128
>>>> keyspaces:
>>>> - name: myKeyspace
>>>> replica_placement_strategy:
>>>> orgapache.cassandra.locator.SimpleStrategy
>>>>
>>>>
>>>> replication_factor: 1
>>>> column_families:
>>>> - name: Standard1
>>>> rows_cached: 100
>>>> keys_cached: 1
>>>> compare_with: UTF8Type
>>>> memtable_flush_after_mins: 60
>>>> memtable_operations_in_millions: 0.3
>>>> memtable_throughput_in_mb: 64
>>>> partitioner: org.apache.cassandra.dht.RandomPartitioner
>>>> phi_convict_threshold: 8
>>>> rpc_keepalive: true
>>>> rpc_port: 9160
>>>> rpc_timeout_in_ms: 10000
>>>> sliced_buffer_size_in_kb: 64
>>>> snapshot_before_compaction: false
>>>> storage_port: 7000
>>>> thrift_framed_transport_size_in_mb: 15
>>>> thrift_max_message_length_in_mb: 16
>>>>
>>>> and here is the exception i'm receiving:
>>>>
>>>> Caused by: Can't construct a java object for
>>>> tag:yaml.org,2002:org.apache.cassandra.config.Config;
>>>> exception=Cannot create property=keyspaces for Java
>>>> bean=org.apache.cassandra.config.con...@12e43f1; Cannot create
>>>> property=column_families for
>>>> javabean=org.apache.cassandra.config.rawkeysp...@1a8bd74;
>>>> null
>>>> in "<reader>", line 3, column 1:
>>>> authenticator: org.apache.cassan ...
>>>> ^
>>>>
>>>> at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:372)
>>>> at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177)
>>>> at
>>>> org.yaml.snakeyamlconstructor.BaseConstructorconstructDocument(BaseConstructor.java:136)
>>>>
>>>> at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:122)
>>>> at org.yaml.snakeyaml.Loader.load(Loader.java:52)
>>>> at org.yaml.snakeyaml.Yaml.load(Yaml.java:166)
>>>> at
>>>> com.atsid.cassandra.testutils.CassandraTestRunner.updateYamlConfig(CassandraTestRunner.java:134)
>>>> at
>>>> com.atsid.cassandra.testutils.CassandraTestRunner.init(CassandraTestRunner.java:119)
>>>> at
>>>> com.atsid.cassandra.testutils.CassandraTestRunner.main(CassandraTestRunner.java:272)
>>>> ... 6 more
>>>> Caused by: org.yaml.snakeyaml.error.YAMLException: Cannot create
>>>> property=keyspaces for javabean=org.apache.cassandra.config.con...@12e43f1;
>>>> Cannot cr
>>>> eate property=column_families for
>>>> javabean=org.apache.cassandra.config.rawkeysp...@1a8bd74; null
>>>> at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:305)
>>>> at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:184)
>>>> at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:370)
>>>> ... 14 more
>>>> Caused by: org.yaml.snakeyaml.error.YAMLException: Cannot create
>>>> property=column_families for
>>>> javabean=org.apache.cassandra.config.rawkeysp...@1a8bd74
>>>> ; null
>>>> at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:305)
>>>> at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:184)
>>>> at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177)
>>>> at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructSequenceStep2(BaseConstructor.java:239)
>>>> at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructSequence(BaseConstructor.java:233)
>>>> at
>>>> org.yaml.snakeyaml.constructorConstructor$ConstructSequence.construct(Constructor.java:543)
>>>>
>>>> at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177)
>>>> at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:298)
>>>> ... 16 more
>>>> Caused by: java.lang.ArrayStoreException
>>>> at java.lang.System.arraycopy(Native Method)
>>>> at java.util.Arrays.copyOf(Arrays.java:2763)
>>>> at java.util.ArrayList.toArray(ArrayList.java:305)
>>>> at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:301)
>>>> ... 23 more
>>>>
>>>>
>>>
>>
>
