Hi Mark,
I recently did a Proof-of-Concept with Ignite and Cassandra. There are a number
of challenges you need to look out for:
1) In order to do SQL queries, you need to have all the data in Ignite. It is
not practical to include other persistence layers in the query process.
The exception is Ignite Native Persistence.
2) You need to deploy the POJO for the objects (and keys if they are not basic
types) in Ignite. This may become unnecessary in a near-future release.
3) Preloading data is not particularly efficient. Each Ignite node sends the
preload query. Each Ignite node ignores the data it is not primary or backup
for. It may be possible to optimize this for the case where the partitioning is
the same in Ignite and Cassandra.
We wanted to dynamically load data from Cassandra, and thus were sensitive to
this point.
4) Things get a bit trickier when the primary keys (partitioning) are not the
same in Ignite and Cassandra. Only one key POJO can be deployed in Ignite, and
it needs to handle the Ignite and Cassandra use cases.
Data Source is configured at the top level:
<!-- Cassandra Data Source -->
<bean id="cassandraDataSource"
class="org.apache.ignite.cache.store.cassandra.datasource.DataSource">
<property name="user" value="cassandra"></property>
<property name="password" value="cassandra"></property>
<property name="contactPoints"
value="10.24.51.190,10.24.51.187,10.24.51.150,10.24.51.240"/>
<property name="port" value="9042"/>
</bean>
Persistence settings for each cache are also at the top level. Here is an
example that has a composite primary key and some field name mappings. Note the
CDATA section.
<!-- Persistence settings for PortStats cache -->
<bean id="PortStatsCachePersistenceSettings"
class="org.apache.ignite.cache.store.cassandra.persistence.KeyValuePersistenceSettings">
<constructor-arg type="java.lang.String">
<value><![CDATA[
<persistence keyspace="test_11" table="port_stats">
<keyPersistence
class="com.abc.poc.icpoc.model.PortStatsKey" strategy="POJO">
<partitionKey>
<field name="day"/>
<field name="group"/> <!-- hash of switch-id -->
</partitionKey>
<clusterKey>
<field name="dateTime" column="date_time"/>
<field name="portId" column="port_id"/>
</clusterKey>
</keyPersistence>
<valuePersistence class="com.abc.poc.icpoc.model.PortStats"
strategy="POJO">
<field name="portId" column="port_id"/>
<field name="dateTime" column="date_time"/>
<field name="portType" column="port_type"/>
<field name="switchId" column="switch_id"/>
<field name="rxUtil" column="rx_util"/>
<field name="txUtil" column="tx_util"/>
</valuePersistence>
</persistence>
]]></value>
</constructor-arg>
</bean>
Lastly, in the cache config, you need to reference the persistence settings. We
did not use write-through or write-behind.
<bean
class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="PortStatsCache"/>
<property name="cacheMode" value="PARTITIONED"/>
<property name="backups" value="1"/>
<!-- for Cassandra backing store -->
<property name="readThrough" value="true"/>
<property name="cacheStoreFactory">
<bean
class="org.apache.ignite.cache.store.cassandra.CassandraCacheStoreFactory">
<property name="dataSourceBean"
value="cassandraDataSource"/>
<property name="persistenceSettingsBean"
value="PortStatsCachePersistenceSettings"/>
</bean>
</property>
Hope this helps.
Roger
From: Ilya Kasnacheev [mailto:[email protected]]
Sent: Thursday, August 31, 2017 8:56 AM
To: [email protected]
Subject: Re: Ignite as Cache and Query layer for Cassandra.
Hello Mark,
I have found some documentation on how to do what the article describes:
"Apache Ignite can be inserted between Apache Cassandra and an existing
application layer with no changes to the Cassandra data and only minimal
changes to the application"
https://apacheignite-mix.readme.io/docs/ignite-with-apache-cassandra<https://urldefense.proofpoint.com/v2/url?u=https-3A__apacheignite-2Dmix.readme.io_docs_ignite-2Dwith-2Dapache-2Dcassandra&d=DwMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=1esZO0r0bYS90lcsaLA6N4AFxuNo6lzauhETGwdJQoQ&m=V1yhk9HVShi4pdJWzPqkw3L6T-IdasiuF8bYdpMPp7w&s=JEswLCWrb1oQBPKMEjakE1kWv0rr5KARYVMUphxYciI&e=>
On the left you can see there are config examples, tests, AWS configuration,
etc.
Hope it's good enough.
--
Ilya Kasnacheev
2017-08-31 16:15 GMT+03:00 Mark Farnan
<[email protected]<mailto:[email protected]>>:
Howdy,
I'm currently evaluating Ignite for a project, and am looking for some guidance
and project sample for using Ignite over Cassandra as a cache and query layer.
Note: The Cassandra database and schema already exists and is mature.
In a Recent Infoworld post,
(https://www.infoworld.com/article/3191895/application-development/light-a-fire-under-cassandra-with-apache-ignite.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.infoworld.com_article_3191895_application-2Ddevelopment_light-2Da-2Dfire-2Dunder-2Dcassandra-2Dwith-2Dapache-2Dignite.html&d=DwMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=1esZO0r0bYS90lcsaLA6N4AFxuNo6lzauhETGwdJQoQ&m=V1yhk9HVShi4pdJWzPqkw3L6T-IdasiuF8bYdpMPp7w&s=9B2fqNQEKRiN4vS_MCJyh1K_6SCV1xVaPiLVEoqUqUQ&e=>)
it stated
"No remodeling of Cassandra data
Apache Ignite reads from Apache Cassandra and other NoSQL databases, so moving
Cassandra data into Ignite requires no data modification. The data schema can
also be migrated directly into Ignite as is. "
However, I can't find in any documentation, or online articles, guidance on
just how to do this and there are no links in the article.
- The Ignite docs /cassandra section appear to only cover using Cassandra as
a persistence layer, not for going over and caching / querying an existing
schema. (unless I'm missing something obvious !)
- The Youtube video cuts out to blank at the end, so no links are shown.
The links on the slideshare document appear to be broken.
- Can't find any git hub for examples on how to do this (especially not for
the couple you tube vids)
Can anyone please point me to some working sample code and/or documentation on
how to do this ?
I'm especially interested in not having to migrate any schema's, nor having to
pre-load the data into the cache. (lazy load), which the articles imply.
Also some of the data would be too large to load it all into Ram anyway.
Any assistance greatly appreciated.
Regards
Mark.
