Upgrading Ignite Persistence from 2.1 to 2.3

[Josephine Barboza]

Hi,

I am currently using version 2.1 of ignite and want to upgrade to latest 
version 2.3. My application uses Ignite 
Persistence(org.apache.ignite.configuration. PersistentStoreConfiguration) to 
persist data. As part of upgrading to version 2.3 do I need to change this to 
DataStoreConfiguration or can I continue using PersistentStoreConfiguration. 
What is recommended? Also, if I do need to change to DataStoreConfiguration how 
do I migrate the existing data?

Please help.

Thanks,
Josephine
IMPORTANT NOTICE: This email and any files transmitted with it are confidential 
and intended solely for the use of the individual or entity to whom they are 
addressed. If you have received this email in error, please notify the system 
manager and/or the sender immediately.

Re: How to identify if the data returned from cache is partial data due to multiple node failures in partitioned cache

[Evgenii Zhuravlev]

Hi,

If all nodes, that hold partition left topology, will be thrown event
EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST. You can listen to it using
IgniteEvents.localListen(lsnr, EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST)

Regards,
Evgenii

2018-01-11 9:01 GMT+03:00 aMark :

> Hi,
>
> We are using Ignite cache version 2.3 . We are using persistent cache in
> Partitioned Mode having 4 cluster node running.   We have configured caches
> to have 1 backup.
>
> I understand that if there are more than one node failure at a time, then
> data present in the live cluster may not be complete data for a given
> cache.
>
> In the above setup, when all four nodes are running, I get close to ~650K
> key value pair for a cache. But if I bring down three nodes then I get
> close
> to ~300K key values pair for the same cache.
>
> If I dont have initial count of entries in the the cache, I dont know if
> the
> entries returned is a full set of partial set.
>
> Is there an API/configuration in Ignite to identify that cache might not
> have complete data in the cluster for the time being (due to any reason) ?
>
>
>
> Thanks,
>
>
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: 3rd Party Persistence and two phase commit

[ALEKSEY KUZNETSOV]

Hi, Andrey!

Actually, data could be persisted not on tx initiating node, but on 
primary(I.e. we have partitioned cache and local cache) . 

Additionally, data would be persisted on backup node if you enable the 
corresponding flag.

> 11 янв. 2018 г., в 10:12, Andrey Nestrogaev  
> написал(а):
> 
> Hi Denis, 
> 
> Yes, I have already read the article you mentioned.
> 
> But it shows an example where the primary data being changed is located on
> the same node, at least I understand it so.
> In my original understanding it was that each node creates its own
> connection to the 3rd database. But perhaps this works only in the case of
> Read-Through, and with distributed transactions, another approach works.
> 
> 
> 
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Apache Ignite best practice

[Borisov Sergey]

Hello Ilya!

Thanks for the advice with "JOIN TABLE".

Can you recommend something for the configuration and diagnostics of the
Apache Ignite cluster?
For example, what metrics should I look for when the response time starts to
grow?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: String as value problem ?

[Mikael]

Hi!

Forget about it, it was my fault, when you use a String as a value in a 
cache (and use indexing), it requires that you have Lucene jar's in the 
classpath, I didn't have that.


Mikael


Den 2018-01-11 kl. 00:11, skrev vkulichenko:

Mikael,

First of all, the trace should contain the cause with more details. What
does it tell? If this doesn't help to figure our the reason of the failure,
please show the cache configuration.

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite Persistent Cache DotNet

[Hemasundara Rao]

Hi Team,
I am testing Ignite cache persistent configuration on version 2.30.
I am experiencing strange behavior and it is not consistent behavior.
Could you please suggest me how to configure and test persistent  behavior?


Thanks and Regards,
Hemasundar.

Is the default memory region always there ?

[Mikael]

If I use something like the configuration example from the documentation:

|  name="dataStorageConfiguration"> class="org.apache.ignite.configuration.DataStorageConfiguration"> 
   class="org.apache.ignite.configuration.DataRegionConfiguration">   
   value="#{500L * 1024 * 1024}"/>
  Will this replace the default memory region 
with only one called "500MB_Region" or will the default region still be 
there ? Mikael |

Re: 3rd Party Persistence and two phase commit

[Andrey Nestrogaev]

Hi Aleksey, thanks for info,

"/Actually, data could be persisted not on tx initiating node, but on
primary(I.e. we have partitioned cache and local cache)/"
Ok, but no matter where the data is persisted, there will always be only 1
database connection within the transaction, no matter how many nodes are
involved in the transaction.
Right?

"/Additionally, data would be persisted on backup node if you enable the
corresponding flag./"
Would be persisted on backup node with using the same database connection as
for primary node?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite Upgrade from 1.9 to 2.3

[Tejashwa Kumar Verma]

Hi All,

We are upgrading from 1.9 to 2.3 and here we are facing some issue

In 2.3 we are configuring Data Region as below


 


   
  
 `
  
 * *
*  *

   

 


Here we are trying to configure swapPath and *persistenceEnabled is false.*
So i was expecting that once cache with 5GB_SWAP_Region_Swapping region
will be full then data should go to  swapPath. But we are seeing some weird
behavior here and entire data is going in *SWAP. *Can anyone help me here
if i am doing some wrong configuration or my expectations are not correct.



Thanks & Regards
Tejas

Re: Ignite Persistent Cache DotNet

[Pavel Tupitsyn]

Hi Hemasundar,

Can you provide more details? What is not consistent? Is there a reproducer?

Thanks,
Pavel

On Thu, Jan 11, 2018 at 12:53 PM, Hemasundara Rao <
hemasundara@travelcentrictechnology.com> wrote:

> Hi Team,
> I am testing Ignite cache persistent configuration on version 2.30.
> I am experiencing strange behavior and it is not consistent behavior.
> Could you please suggest me how to configure and test persistent  behavior?
>
>
> Thanks and Regards,
> Hemasundar.
>
>

Re: Ignite Upgrade from 1.9 to 2.3

[Alexey Kukushkin]

Ignite native persistence replaces Swapping feature from Ignite 1.x. Use
native persistence instead of swapping (remove "swapPath" configuration and
enable persistence).

"swapPath" setting in Ignite 2.x has different semantics: specifying a
"swapPath" makes Ignite use memory mapped file for the data region instead
of off-heap RAM. This is why you see all data go to the file.

Re: Is the default memory region always there ?

[Alexey Kukushkin]

Default memory region is always there even if you add more user-defined
data regions. If you are not going to use it at all and not happy with the
memory still allocated to the default region you can specify a smaller
amount of memory allocated for the default region as described here
.

Re: Upgrading Ignite Persistence from 2.1 to 2.3

[Andrey Mashenkov]

Hi,

In 2.3 some configuration methods are deprecated and will be removed in
next major release (ignite-3.0).
You can safely continue to use old-style configuration, but of course we
recommend to switch to new-style if possible to be able to use new features.

I'm not sure 2.3 can support old storage out of box, but you can try.
Also, please, note the default pagesize was changed in between 2.2 and 2.3
releases from 2048 to 4096. So, you may need to specify it explicitly in
your config.


On Thu, Jan 11, 2018 at 11:23 AM, Josephine Barboza <
josephine.barb...@nviz.com> wrote:

> Hi,
>
>
>
> I am currently using version 2.1 of ignite and want to upgrade to latest
> version 2.3. My application uses Ignite 
> Persistence(org.apache.ignite.configuration.
> PersistentStoreConfiguration) to persist data. As part of upgrading to
> version 2.3 do I need to change this to DataStoreConfiguration or can I
> continue using PersistentStoreConfiguration. What is recommended? Also, if
> I do need to change to DataStoreConfiguration how do I migrate the existing
> data?
>
>
>
> Please help.
>
>
>
> Thanks,
>
> Josephine
> IMPORTANT NOTICE: This email and any files transmitted with it are
> confidential and intended solely for the use of the individual or entity to
> whom they are addressed. If you have received this email in error, please
> notify the system manager and/or the sender immediately.
>



-- 
Best regards,
Andrey V. Mashenkov

Re: 3rd Party Persistence and two phase commit

[ALEKSEY KUZNETSOV]

Local store means store, that resides only on one node. No other nodes see
it.

If you don't have local stores in cluster(only distributed ones), then it
will be the only db connection within transaction opened.
But If you have local stores, then nodes *could open* their own connections
to local stores(i.e. in replicated cache nodes could open connections to
local stores, if any).

Only local stores could be filled with data from backup nodes, therefore
new connection must be opened(cannot reuse old one from primary node).


чт, 11 янв. 2018 г. в 13:48, Andrey Nestrogaev :

> Hi Aleksey, thanks for info,
>
> "/Actually, data could be persisted not on tx initiating node, but on
> primary(I.e. we have partitioned cache and local cache)/"
> Ok, but no matter where the data is persisted, there will always be only 1
> database connection within the transaction, no matter how many nodes are
> involved in the transaction.
> Right?
>
> "/Additionally, data would be persisted on backup node if you enable the
> corresponding flag./"
> Would be persisted on backup node with using the same database connection
> as
> for primary node?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>


-- 

*Best Regards,*

*Kuznetsov Aleksey*

RE: Upgrading Ignite Persistence from 2.1 to 2.3

[Josephine Barboza]

Hi Andrey,
Thanks for the info. Could you also let me know how to migrate the data from 
2.1 to 2.3. Are there any APIs/feature which I can use to do that?

From: Andrey Mashenkov [mailto:andrey.mashen...@gmail.com]
Sent: Thursday, January 11, 2018 5:04 PM
To: user@ignite.apache.org
Subject: Re: Upgrading Ignite Persistence from 2.1 to 2.3

Hi,

In 2.3 some configuration methods are deprecated and will be removed in next 
major release (ignite-3.0).
You can safely continue to use old-style configuration, but of course we 
recommend to switch to new-style if possible to be able to use new features.

I'm not sure 2.3 can support old storage out of box, but you can try.
Also, please, note the default pagesize was changed in between 2.2 and 2.3 
releases from 2048 to 4096. So, you may need to specify it explicitly in your 
config.


On Thu, Jan 11, 2018 at 11:23 AM, Josephine Barboza 
mailto:josephine.barb...@nviz.com>> wrote:
Hi,

I am currently using version 2.1 of ignite and want to upgrade to latest 
version 2.3. My application uses Ignite 
Persistence(org.apache.ignite.configuration. PersistentStoreConfiguration) to 
persist data. As part of upgrading to version 2.3 do I need to change this to 
DataStoreConfiguration or can I continue using PersistentStoreConfiguration. 
What is recommended? Also, if I do need to change to DataStoreConfiguration how 
do I migrate the existing data?

Please help.

Thanks,
Josephine
IMPORTANT NOTICE: This email and any files transmitted with it are confidential 
and intended solely for the use of the individual or entity to whom they are 
addressed. If you have received this email in error, please notify the system 
manager and/or the sender immediately.



--
Best regards,
Andrey V. Mashenkov

Re: Adding custom processor in Ignite Node

[Alexey Popov]

Hi,

I am not sure that user-list will be able to help you with such questions.

You should discuss the implementation options/details at dev-list
(http://apache-ignite-developers.2346864.n4.nabble.com/).
Probably, you don't need a custom processor class at all.

Anyway:
1. Please have a look at the thin clients' server-side code (starting from
ClientListenerNioListener class).
Maybe you can just add one more client processing code here.

2. Or you can look at GridRestProcessor code as a reference for building
your own processor.

Thank you,
Alexey



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: IgniteOutOfMemoryException when using putAll instead of put

[Alexey Popov]

Hi,

Can you share your configuration for
1) cache
2) memory region?

I see "evicts=0" in your stats that looks very strange. Are you sure you
have a configured eviction policy in the data region (policyName=RefData)?

Does this cache work fine (evicts some data) without Cache Store enabled?

Thank you,
Alexey



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Problem enabling read-through on cache - no suitable driver

[Thomas Isaksen]

I am trying to enable read-through on my cache as follows:









   
   
   



Once I activate my cluster I get this error message:

SEVERE: Failed to initialize cache. Will try to rollback cache start routine. 
[cacheName=testCache]
class org.apache.ignite.IgniteCheckedException: Failed to start component: 
javax.cache.CacheException: Failed access to metadata for detect database 
dialect.
at 
org.apache.ignite.internal.util.IgniteUtils.startLifecycleAware(IgniteUtils.java:8757)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1344)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1799)
at 
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:751)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClusterStateChangeRequest(GridDhtPartitionsExchangeFuture.java:817)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:583)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2279)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.cache.CacheException: Failed access to metadata for detect 
database dialect.
at 
org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.resolveDialect(CacheAbstractJdbcStore.java:273)
at 
org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.start(CacheAbstractJdbcStore.java:305)
at 
org.apache.ignite.internal.util.IgniteUtils.startLifecycleAware(IgniteUtils.java:8753)
... 8 more
Caused by: java.sql.SQLException: No suitable driver found for
at 
org.h2.jdbcx.JdbcDataSource.getJdbcConnection(JdbcDataSource.java:190)
at org.h2.jdbcx.JdbcDataSource.getConnection(JdbcDataSource.java:159)
at 
org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.openConnection(CacheAbstractJdbcStore.java:325)
at 
org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.resolveDialect(CacheAbstractJdbcStore.java:268)
... 10 more


I can see that there is a message "No suitable driver found for .." but 
shouldn't the H2 driver already be on the Ignite classpath already? If not, how 
do I fix it?

Regards
Thomas Isaksen

No user-defined default MemoryPolicy found

[Josephine Barboza]

Hi,

In spite of setting a defaultMemoryPocliySize I am getting the below warning on 
starting the node. Is there any other configuration required?

WARNING: No user-defined default MemoryPolicy found; system default of 1GB size 
will be used.




  


  

  
  
  

  

  
   

Thanks.
IMPORTANT NOTICE: This email and any files transmitted with it are confidential 
and intended solely for the use of the individual or entity to whom they are 
addressed. If you have received this email in error, please notify the system 
manager and/or the sender immediately.

Re: IgniteOutOfMemoryException when using putAll instead of put

[Larry Mark]

Here are the configurations

DataRegionConfiguration =
(new DataRegionConfiguration)
  .setName("RefData")
  .setInitialSize(21 * 1024 * 1024)
  .setMaxSize(21 * 1024 * 1024)
  .setPersistenceEnabled(false)
  .setPageEvictionMode(DataPageEvictionMode.RANDOM_LRU)
  .setMetricsEnabled(true)
  .setEvictionThreshold(.9)

I then created a cache that uses that data region.

  val cfg = new CacheConfiguration[Int, String]
  cfg.setName("testCache")
  .setCacheMode(CacheMode.PARTITIONED)
  .setAtomicityMode(CacheAtomicityMode.ATOMIC)
  .setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(Durati
on.ETERNAL))
// .setIndexedTypes(keytag.runtimeClass, valtag.runtimeClass) uncommenting
this causes the error
  .setDataRegionName("RefData"))
  .setStatisticsEnabled(true)

val cacheFactory = FactoryBuilder.factoryOf("CacheConstantString")
cfg.setCacheStoreFactory(cacheFactory)
  .setReadThrough(readThrough)

   val myCache = ignite.getOrCreateCache(cfg)

// This is my stubbed read through
class CacheConstantString[K,V] extends CacheStoreAdapter[K, V]  with
Logging {
  override def load(key: K): V = {
"foo".asInstanceOf[V]
  }
}

It is the combination of both cache enabled and setting indexed types that
is the problem, it works fine without either, but I need the read through.
I never see any value in the metrics except evicts=0 and I know it must be
evicting pages because i am writing 35,000 keys into a data region that
only has enough space for 8000 or so.  This is a cache metric, is it
showing page evicts or key evicts?   Because I have an expiry of Eternal,
so I dont expect to see key evicts


On Thu, Jan 11, 2018 at 8:40 AM, Alexey Popov  wrote:

> Hi,
>
> Can you share your configuration for
> 1) cache
> 2) memory region?
>
> I see "evicts=0" in your stats that looks very strange. Are you sure you
> have a configured eviction policy in the data region (policyName=RefData)?
>
> Does this cache work fine (evicts some data) without Cache Store enabled?
>
> Thank you,
> Alexey
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: No user-defined default MemoryPolicy found

[Alexey Popov]

Hi,

I see the same message with 2.1 release. 2.3 does not have it.

I will check 2.1 source code later.

Do you use 2.1 release?

Thanks,
Alexey



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: No user-defined default MemoryPolicy found

[Alexey Popov]

ok. I see you are at 2.1. (migrating from 2.1 to 2.3)



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Problem enabling read-through on cache - no suitable driver

[slava.koptilin]

Hi Thomas,

Please make sure that your h2 database is started and required tables are
created.
You can use the following test as example how to do that (please see,
CacheJdbcPojoStoreAbstractSelfTest#beforeTest)
https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/cache/store/jdbc/CacheJdbcPojoStoreAbstractSelfTest.java

Cache configuration should be modified as well.

1. First of all, correct DB connection URL should be provided. For instance,






2. secondly, you need to configure types mapping within the cache
configuration. Let's assume that you have a simple cache
IgniteCache























































additional example(s) can be found here:
https://github.com/apache/ignite/blob/master/modules/spring/src/test/config/jdbc-pojo-store-builtin.xml

Thanks,
Slava.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite Upgrade from 1.9 to 2.3

[Tejashwa Kumar Verma]

Thanks Alexey for quick response.


-
Tejas

On Thu, Jan 11, 2018 at 4:51 PM, Alexey Kukushkin  wrote:

> Ignite native persistence replaces Swapping feature from Ignite 1.x. Use
> native persistence instead of swapping (remove "swapPath" configuration and
> enable persistence).
>
> "swapPath" setting in Ignite 2.x has different semantics: specifying a
> "swapPath" makes Ignite use memory mapped file for the data region instead
> of off-heap RAM. This is why you see all data go to the file.
>

Invitation: [Webinar] Scale Out and Conquer: Architectural Decisions ... @ Wed Jan 17, 2018 11am - 12pm (PST) (user@ignite.apache.org)

[dmagda]

BEGIN:VCALENDAR
PRODID:-//Google Inc//Google Calendar 70.9054//EN
VERSION:2.0
CALSCALE:GREGORIAN
METHOD:REQUEST
BEGIN:VEVENT
DTSTART:20180117T19Z
DTEND:20180117T20Z
DTSTAMP:20180112T010357Z
ORGANIZER;CN=dma...@gridgain.com:mailto:dma...@gridgain.com
UID:7cur73ncgtvjh8v004do9fa...@google.com
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;RSVP=TRUE
 ;CN=dma...@gridgain.com;X-NUM-GUESTS=0:mailto:dma...@gridgain.com
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=
 TRUE;CN=d...@ignite.apache.org;X-NUM-GUESTS=0:mailto:d...@ignite.apache.org
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=
 TRUE;CN=user@ignite.apache.org;X-NUM-GUESTS=0:mailto:user@ignite.apache.org
CREATED:20180112T010357Z
DESCRIPTION:Igniters\,Join me to learn on the subj! Low-level st
 uff prepared by Vladimir Ozerov and Yakov Zhdanov\, not for newbies!RSVP: \;https://www.gridgain.com/resources/webin
 ars/scale-out-and-conquer-architectural-decisions-behind-distributed-in-mem
 oryDistributed platforms like Apache® Ignite™ rely on a horizontal “
 scale-out” architecture where you dynamically add more machines to achieve 
 near-linear\, elastic scalability. But how does it really work? What are it
 s limits? And how can you optimize performance and scalability?In th
 is webinar\, we will cover the challenges engineers face when designing dis
 tributed systems\, and the tips and tricks for optimizing Apache Ignite inc
 luding:When to use different data-sharding algorithmsH
 ow to design effective data models that scale in a distributed in-memory sy
 stemHow to optimize distributed synchronization and coordinationHow to solve the most common scalability challenges with clustered n
 odes\n\n-::~:~::~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~
 :~:~:~:~:~:~:~:~:~:~:~::~:~::-\nPlease do not edit this section of the desc
 ription.\n\nView your event at https://www.google.com/calendar/event?action
 =VIEW&eid=N2N1cjczbmNndHZqaDh2MDA0ZG85ZmFkamogdXNlckBpZ25pdGUuYXBhY2hlLm9yZ
 w&tok=MTkjZG1hZ2RhQGdyaWRnYWluLmNvbTIwMjg1MjcxMTM5NTExNTFjZGYxYmRhNzhiNmFiN
 TFmZmI4NmM2YzE&ctz=America/Los_Angeles&hl=en.\n-::~:~::~:~:~:~:~:~:~:~:~:~:
 ~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~::~:~::-
LAST-MODIFIED:20180112T010357Z
LOCATION:https://www.gridgain.com/resources/webinars/scale-out-and-conquer-
 architectural-decisions-behind-distributed-in-memory
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:[Webinar] Scale Out and Conquer: Architectural Decisions Behind Dis
 tributed In-Memory Systems
TRANSP:OPAQUE
END:VEVENT
END:VCALENDAR


invite.ics
Description: application/ics

RE: No user-defined default MemoryPolicy found

[Josephine Barboza]

Hi Alexey,

Is there any way I can check the total memory being used by the off heap cache? 
I wanted to confirm if the warning is valid.
Sorry that I forgot to mention the version I am currently using. It is 2.1.


-Original Message-
From: Alexey Popov [mailto:tank2.a...@gmail.com] 
Sent: Thursday, January 11, 2018 9:38 PM
To: user@ignite.apache.org
Subject: Re: No user-defined default MemoryPolicy found

ok. I see you are at 2.1. (migrating from 2.1 to 2.3)



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Use SQL to query IgniteRDD with scala at Zeppelin

[itriA40453]

Hi,
Below is my code and result where I am trying to query IgniteRDD through SQL 
statement with scala at Zeppelin.
First I use restful API to put a pair of data(key=10, val=45) into Ignite cache.

%python
import requests
requests.put("http://127.0.0.1:8080/ignite?cmd=put&key=10&val=45&cacheName=zeppelincache";)

Result:



Then I execute the code below.

%spark
import org.apache.ignite.configuration._
import org.apache.ignite.spark._

val ic = new IgniteContext(sc,()=>new IgniteConfiguration())
val sharedRDD2 = ic.fromCache[Integer,Integer]("zeppelincache")
sharedRDD2.savePairs(sc.parallelize(1 to 3, 3).map(i => (i, i*2)))

sharedRDD2.collect
sharedRDD2.sql("select * from Integer").show()

Result:
import org.apache.ignite.configuration._
import org.apache.ignite.spark._
ic: org.apache.ignite.spark.IgniteContext = 
org.apache.ignite.spark.IgniteContext@7a017337
sharedRDD2: org.apache.ignite.spark.IgniteRDD[Int,Int] = IgniteRDD[136] at RDD 
at IgniteAbstractRDD.scala:32
res129: Array[(Int, Int)] = Array((1,2), (2,4), (3,6), (10,45))
+++
|_KEY|_VAL|
+++
|   1|2|
|   2|4|
|   3|6|
+++



1.  Why can the collect command get the pair (10,45) but SQL statement 
can’t?

2.  How to get the pair (10,45) through SQL statement?

Thanks.



--
本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain 
confidential information. Please do not use or disclose it in any way and 
delete it if you are not the intended recipient.

Re: Adding custom processor in Ignite Node

[kotamrajuyashasvi]

Hi

Thanks for your response.
As you mentioned will try to discuss/take help from the dev-list.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: No user-defined default MemoryPolicy found

[Alexey Popov]

Hi,

Please have a look at topic below to get an estimation of memory usage:

http://apache-ignite-users.70518.x6.nabble.com/off-heap-memory-usage-tc19282.html

I will check the warning later, it looks strange to me.

Thanks,
Alexey



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/