Thanks for looking into that. I created a jira issue for it: https://issues.apache.org/jira/browse/CASSANDRA-1753
On Wed, Nov 17, 2010 at 11:28 AM, Aaron Morton <aa...@thelastpickle.com>wrote: > That does look like a bug > http://www.mail-archive.com/d...@cassandra.apache.org/msg01167.html > > <http://www.mail-archive.com/d...@cassandra.apache.org/msg01167.html>Aaron > > > On 17 Nov, 2010,at 03:55 PM, Aaron Morton <aa...@thelastpickle.com> wrote: > > It's certainly looks suspect. I've had a look at the code around > SSTableImport and SSTableExport and the isDeleted value for the col is based > on IColumn.isMarkedForDelete read when the data was exported. I'll try to > have a look tonight, or if someone is still up in the states they may help. > The current 0.7 branch has the same code. > > Are there any other errors in the log? > > The code sample you've got there uses the TimestampClocks that were once a > part of the 0.7 development, but have been removed. So your data sample may > not be usable going forwards. Can you jump to beta3? > > Aaron > > > > On 17 Nov, 2010,at 02:03 PM, CassUser CassUser <cassu...@gmail.com> wrote: > > Looking at this closer. I noticed the following in the SSTableImport > Class: > > if (col.isDeleted) { > cfamily.addColumn(path, hexToBytes(col.value), new > TimestampClock(col.timestamp)); > } else { > cfamily.addTombstone(path, hexToBytes(col.value), new > TimestampClock(col.timestamp)); > } > > This appears to be backwards. > > > On Tue, Nov 16, 2010 at 4:03 PM, CassUser CassUser <cassu...@gmail.com>wrote: > >> Looked at how DatabaseDescriptor is loading the yaml file. Using that >> approach solves the problem with the column_families mapping exception. >> >> The problems we are running into currently is regarding a known dataset >> not being loaded into our test instance correctly. >> >> Steps: >> 1. Create temp directory to host cassandra test instance >> 2. Create keyspace directories >> 3 Update yaml file, and copy to test location >> 4. Add schema from yaml using DatabaseDescriptor. >> readTablesFromYaml() & DatabaseDescriptor. >> setTableDefinition >> 4. Use SSTableImport.importJson to setup a known dataset. We have JSON >> files converted to 0.7 compliant (byte [] rows) >> 5. Start Cassandra instance using the EmbeddedCassandraService class >> >> Everything appears to work from the log messages. I get the message " >> Sampling index for ..", and finally "Listening for thrift clients...". When >> i use a client to query the data in test instance, I notice keys with no >> columns/values stored (via keyrange slice). Using a open ended column slice >> for a known key I'm getting zero results. >> >> I'm currently running cassandra 0.7. beta2. Is the steps I've outlined >> above suppose to work in 0.7? We did something similar in 0.6.4 without any >> problems. Is there known bugs I can look into? >> >> Thanks. >> >> >> >> >> On Tue, Nov 16, 2010 at 12:24 PM, Aaron Morton >> <aa...@thelastpickle.com>wrote: >> >>> I've not used the embedded service. >>> >>> The code in o.a.c.service.EmbeddedCassandraService says it will read the >>> yaml file. If the cluster does not have a schema stored I think it will load >>> the one from yaml. >>> >>> Have you tried starting it up with an empty system data dir ? Does it >>> pickup the schema from the yaml? >>> >>> Aaron >>> >>> >>> On 17 Nov, 2010,at 09:17 AM, CassUser CassUser <cassu...@gmail.com> >>> wrote: >>> >>> >>> Loading yaml file like so: >>> FileInputStream yamlInputStream = new FileInputStream( >>> configTemplateFile); >>> Constructor constructor = new Constructor(Config.class); >>> Yaml yaml = new Yaml(new Loader(constructor)); >>> Config conf = (Config) yaml.load(yamlInputStream); >>> >>> Fails on the last line. >>> >>> Although if i have the CFs defined like this: >>> keyspaces: >>> - column_families: >>> - !!org.apache.cassandra.config.RawColumnFamily >>> column_metadata: [] >>> column_type: null >>> comment: null >>> compare_subcolumns_with: null >>> compare_with: BytesType >>> default_validation_class: null >>> gc_grace_seconds: 864000 >>> keys_cached: 200000.0 >>> max_compaction_threshold: 32 >>> min_compaction_threshold: 4 >>> name: Similarity >>> preload_row_cache: false >>> read_repair_chance: 1.0 >>> rows_cached: 0.0 >>> >>> >>> it appears to startup. Any idea whats going on here? >>> >>> >>> >>> On Tue, Nov 16, 2010 at 11:58 AM, CassUser CassUser >>> <cassu...@gmail.com>wrote: >>> >>>> >>>> This is embedded for testing cassandra 0.7 beta2. using >>>> EmbeddedCassandraService. >>>> >>>> and manually adding schema programmatically using: >>>> for (KSMetaData table : DatabaseDescriptor.readTablesFromYaml()) >>>> { >>>> for (CFMetaData cfm : table.cfMetaData().values()) { >>>> CFMetaData.map(cfm); >>>> } >>>> DatabaseDescriptor.setTableDefinition(table, >>>> DatabaseDescriptorgetDefsVersion()); >>>> } >>>> >>>> Is this the correct way to start up a test server, with the schema >>>> loaded? >>>> >>>> >>>> >>>> >>>> >>>> On Tue, Nov 16, 2010 at 11:41 AM, Aaron Morton <aa...@thelastpickle.com >>>> > wrote: >>>> >>>>> >>>>> AFAIK the ArrayStoreException is similar to a type mismatch Is it >>>>> possible you have something mixed up in your class path or source code if >>>>> you built from source? >>>>> >>>>> It looks like the column family info was deserialised into a >>>>> o.a.c.config.RawColumnFamily but when that object was added to the >>>>> RawColumnFamily[] array on o.a.c.config.RawKeyspace if was the wrong type. >>>>> >>>>> Have you tried a clean build ? There are some things in the call stack >>>>> which look custom, are you starting from the command line or is this >>>>> embedded for testing? >>>>> >>>>> Hope that helps. >>>>> Aaron >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 17 Nov, 2010,at 08:09 AM, CassUser CassUser <cassu...@gmail.com> >>>>> wrote: >>>>> >>>>> >>>>> >>>>> Here is the yaml: >>>>> >>>>> # Cassandra YAML generated from previous config >>>>> # Configuration wiki: >>>>> http://wiki.apache.org/cassandra/StorageConfiguration >>>>> authenticator: org.apache.cassandraauth.AllowAllAuthenticator >>>>> >>>>> >>>>> auto_bootstrap: false >>>>> binary_memtable_throughput_in_mb: 256 >>>>> cluster_name: Test Cluster >>>>> column_index_size_in_kb: 64 >>>>> commitlog_rotation_threshold_in_mb: 128 >>>>> commitlog_sync: periodic >>>>> commitlog_sync_period_in_ms: 10000 >>>>> compaction_thread_priority: 1 >>>>> concurrent_reads: 8 >>>>> concurrent_writes: 32 >>>>> disk_access_mode: auto >>>>> dynamic_snitch: false >>>>> endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch >>>>> request_scheduler: org.apache.cassandra.scheduler.RoundRobinScheduler >>>>> request_scheduler_id: keyspace >>>>> hinted_handoff_enabled: true >>>>> in_memory_compaction_limit_in_mb: 128 >>>>> index_interval: 128 >>>>> keyspaces: >>>>> - name: myKeyspace >>>>> >>>>> replica_placement_strategy: >>>>> orgapache.cassandra.locator.SimpleStrategy >>>>> >>>>> >>>>> replication_factor: 1 >>>>> column_families: >>>>> - name: Standard1 >>>>> rows_cached: 100 >>>>> keys_cached: 1 >>>>> compare_with: UTF8Type >>>>> memtable_flush_after_mins: 60 >>>>> memtable_operations_in_millions: 0.3 >>>>> memtable_throughput_in_mb: 64 >>>>> partitioner: org.apache.cassandra.dht.RandomPartitioner >>>>> phi_convict_threshold: 8 >>>>> rpc_keepalive: true >>>>> rpc_port: 9160 >>>>> rpc_timeout_in_ms: 10000 >>>>> sliced_buffer_size_in_kb: 64 >>>>> snapshot_before_compaction: false >>>>> storage_port: 7000 >>>>> thrift_framed_transport_size_in_mb: 15 >>>>> thrift_max_message_length_in_mb: 16 >>>>> >>>>> and here is the exception i'm receiving: >>>>> >>>>> Caused by: Can't construct a java object for >>>>> tag:yaml.org,2002:org.apachecassandra.config.Config; >>>>> exception=Cannot create property=keyspaces for Java >>>>> bean=org.apache.cassandra.config.con...@12e43f1; Cannot create >>>>> property=column_families for >>>>> javabean=org.apache.cassandra.config.rawkeysp...@1a8bd74; >>>>> null >>>>> in "<reader>", line 3, column 1: >>>>> authenticator: org.apache.cassan ... >>>>> ^ >>>>> >>>>> at >>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:372) >>>>> at >>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructorjava:177) >>>>> at >>>>> org.yaml.snakeyamlconstructor.BaseConstructorconstructDocument(BaseConstructor.java:136) >>>>> >>>>> >>>>> at >>>>> org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:122) >>>>> at org.yaml.snakeyaml.Loader.load(Loader.java:52) >>>>> at org.yaml.snakeyaml.Yaml.load(Yaml.java:166) >>>>> at >>>>> com.atsid.cassandra.testutils.CassandraTestRunner.updateYamlConfig(CassandraTestRunner.java:134) >>>>> at >>>>> com.atsid.cassandra.testutils.CassandraTestRunner.init(CassandraTestRunner.java:119) >>>>> at >>>>> com.atsid.cassandra.testutils.CassandraTestRunner.main(CassandraTestRunner.java:272) >>>>> ... 6 more >>>>> Caused by: org.yaml.snakeyaml.error.YAMLException: Cannot create >>>>> property=keyspaces for javabean=org.apache.cassandraconfig.con...@12e43f1; >>>>> Cannot cr >>>>> eate property=column_families for >>>>> javabean=org.apache.cassandra.config.rawkeysp...@1a8bd74; null >>>>> at >>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:305) >>>>> at >>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:184) >>>>> at >>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:370) >>>>> ... 14 more >>>>> Caused by: org.yaml.snakeyaml.error.YAMLException: Cannot create >>>>> property=column_families for >>>>> javabean=org.apache.cassandra.config.rawkeysp...@1a8bd74 >>>>> ; null >>>>> at >>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:305) >>>>> at >>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:184) >>>>> at >>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177) >>>>> at >>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructSequenceStep2(BaseConstructor.java:239) >>>>> at >>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructSequence(BaseConstructor.java:233) >>>>> at >>>>> org.yaml.snakeyaml.constructorConstructor$ConstructSequence.construct(Constructor.java:543) >>>>> >>>>> at >>>>> org.yaml.snakeyaml.constructor.BaseConstructorconstructObject(BaseConstructor.java:177) >>>>> at >>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:298) >>>>> ... 16 more >>>>> Caused by: java.lang.ArrayStoreException >>>>> at java.lang.System.arraycopy(Native Method) >>>>> at java.util.Arrays.copyOf(Arrays.java:2763) >>>>> at java.util.ArrayList.toArray(ArrayList.java:305) >>>>> at >>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:301) >>>>> ... 23 more >>>>> >>>>> >>>> >>> >> >
