Re: Migrating from Oracle to Apache Ignite.
Hi Denis,
My cluster node still cannot distribute the data.
Here is my experiment:
*My Client Apps*:
Client using JDBC Thin Ignite.
Connection conn =
DriverManager.getConnection("jdbc:ignite:thin://10.1.12.67");
And
Create PrepareStatement for Execute query "Insert ABC".
*Check 3 Node*:
Using Application DBeaver to connect to 3 Cluster Node
(10.1.12.67, 10.1.12.68, 10.1.12.69).
Data 'ABC' only stored at Cluster Node (10.1.12.67).
The other node didn`t get the data.
*Here is my Ignite Configuration, for 3 machine/IP Address (10.1.12.67,
10.1.12.68, 10.1.12.69)* :
/IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setIgniteInstanceName("BrandCluster");
cfg.setClientMode(false);
TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
ipFinder.setAddresses(Arrays.asList("127.0.0.1:47500..47510"));
TcpDiscoverySpi discovery = new TcpDiscoverySpi();
discovery.setLocalAddress("127.0.0.1");
discovery.setLocalPort(47500);
discovery.setIpFinder(ipFinder);
cfg.setDiscoverySpi(discovery);/
Would you mind to show me IgniteConfiguration for multi cluster (different
ip address)?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: Problem enabling read-through on cache - no suitable driver
Hi Slava, Thank you so much for your reply it is most helpful! I'm a little confused by all of this because I have been connecting with DBeaver to jdbc:ignite:thin://127.0.0.1/ and I can create tables there if I use the predefined templates PARTITIONED or REPLICATED, however, when I try using my own templates I simply get SQL Error[5] "Cache doesn't exist!" - not sure why this is happening but maybe I'm not connecting to the right DB? Is the jdbc:ignite:thin connection not a connection to the H2 instance used by Ignite internally? Is it not "legit" to use this instance to create my tables or do I have to use another database? ./t -Original Message- From: slava.koptilin [mailto:slava.kopti...@gmail.com] Sent: torsdag 11. januar 2018 17.26 To: user@ignite.apache.org Subject: Re: Problem enabling read-through on cache - no suitable driver Hi Thomas, Please make sure that your h2 database is started and required tables are created. You can use the following test as example how to do that (please see, CacheJdbcPojoStoreAbstractSelfTest#beforeTest) https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/cache/store/jdbc/CacheJdbcPojoStoreAbstractSelfTest.java Cache configuration should be modified as well. 1. First of all, correct DB connection URL should be provided. For instance, 2. secondly, you need to configure types mapping within the cache configuration. Let's assume that you have a simple cache IgniteCache additional example(s) can be found here: https://github.com/apache/ignite/blob/master/modules/spring/src/test/config/jdbc-pojo-store-builtin.xml Thanks, Slava. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Apache Ignite & unixODBC and truncating text
Well, yes, it's a bug. I hope to close it by the end of the next week, so the fix could be included in the Ignite 2.4 Best Regards, Igor On Wed, Jan 10, 2018 at 3:42 PM, bagsiur wrote: > Ok, thank you very much for your time nad reply. > > So, if I understend corectly, this is bug of Apache Ignite. How much time > will take to resolve and fix this problem? > > I will track progress :) > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: IgniteOutOfMemoryException when using putAll instead of put
Hi,
You are right, "evicts=0" is related to cache evictions for on-heap caching
[1]. It should be always 0 for you.
I tried your case (with the same configs as you) and page evictions work
fine with cache store enabled and indexed types. It seems that you have some
misconfiguration.
What are you trying to achieve by adding
.setIndexedTypes(keytag.runtimeClass, valtag.runtimeClass) to String-value
cache? and what is keytag.runtimeClass and valtag.runtimeClass?
Could you please try with DummyClass with valid indexes enabled as below:
/**
* DummyClass
*/
public class DummyClass {
/** Dummy string. */
public String dummyStr;
/** Dummy int. */
@QuerySqlField(index = true)
public Integer dummyInt;
public DummyClass(Integer dummyInt) {
this.dummyInt = dummyInt;
this.dummyStr = StringUtils.rightPad(dummyInt.toString(), 1024,
'*');
}
}
CacheConfiguration cacheCfg = new
CacheConfiguration(CACHE_NAME)
.setCacheMode(CacheMode.PARTITIONED)
.setAtomicityMode(CacheAtomicityMode.ATOMIC)
.setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(Duration.ETERNAL))
.setDataRegionName(REG_NAME)
.setStatisticsEnabled(true)
.setCacheStoreFactory(FactoryBuilder.factoryOf(DummyStoreFromAdapter.class))
.setReadThrough(true)
.setIndexedTypes(Integer.class, DummyClass.class);
Thanks,
Alexey
[1] https://apacheignite.readme.io/docs/evictions#section-java-heap-cache
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: off heap memory usage
Hi Colin, You should use TotalAllocatedPages if you have persistence disabled. There is an know issue with PhysicalMemoryPages: https://issues.apache.org/jira/browse/IGNITE-6963 Thank you, Alexey -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: failed to find sql table for type:
Hi, Do you still have this issue? Can you share your code? Ignite.NET solution has a QueryExample.cs, you can use it as a reference code. Thank you, Alexey From: kenn_thomp...@qat.com Sent: Wednesday, January 10, 2018 5:56 PM To: user@ignite.apache.org Subject: failed to find sql table for type: Ignite.NET 2.3.0 I'm pulling data from another DB and loading into Ignite node started under .NET Core console app. After converting the datarows into binary objects and putting them into the cache, I both do a count as well as debug and see the objects in the cache with the correct data. I try pulling the objects out of the cache using SqlQuery, and get the exception. I'm not sure what to check, but at one point yesterday I was able to do it successfully. I'm not sure what I changed and have been unable to walk whatever change I thought I made back. Where should I dig to get this resolved? This is all running on a vanilla ignite node. Exception has occurred: CLR/Apache.Ignite.Core.Common.IgniteException An unhandled exception of type 'Apache.Ignite.Core.Common.IgniteException' occurred in Apache.Ignite.Core.dll: 'Failed to find SQL table for type: trebuchetsettings' Inner exceptions found, see $exception in variables window for more details. Innermost exception Apache.Ignite.Core.Common.JavaException : class org.apache.ignite.IgniteCheckedException: Failed to find SQL table for type: trebuchetsettings at org.apache.ignite.internal.processors.platform.utils.PlatformUtils.unwrapQueryException(PlatformUtils.java:519) at org.apache.ignite.internal.processors.platform.cache.PlatformCache.runQuery(PlatformCache.java:1220) at org.apache.ignite.internal.processors.platform.cache.PlatformCache.processInStreamOutObject(PlatformCache.java:874) at org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutObject(PlatformTargetProxyImpl.java:79) Caused by: javax.cache.CacheException: class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to find SQL table for type: trebuchetsettings at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:597) at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:368) at org.apache.ignite.internal.processors.platform.cache.PlatformCache.runQuery(PlatformCache.java:1214) ... 2 more Caused by: class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to find SQL table for type: trebuchetsettings at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.queryDistributedSql(IgniteH2Indexing.java:1248) at org.apache.ignite.internal.processors.query.GridQueryProcessor$8.applyx(GridQueryProcessor.java:2068) at org.apache.ignite.internal.processors.query.GridQueryProcessor$8.applyx(GridQueryProcessor.java:2066) at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36) at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2445) at org.apache.ignite.internal.processors.query.GridQueryProcessor.queryDistributedSql(GridQueryProcessor.java:2065) at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySql(GridQueryProcessor.java:2045) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:582) ... 4 more -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Use SQL to query IgniteRDD with scala at Zeppelin
Hi, Please note that REST protocol puts keys as String. So you have 3 Integer keys and 1 String key in a cache. That is the reason for your results. You can find details in topic [1] [1] http://apache-ignite-users.70518.x6.nabble.com/Rest-API-PUT-command-syntax-tc19158.html Thank you, Alexey -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Migrating from Oracle to Apache Ignite.
Hello!
> ipFinder.setAddresses(Arrays.asList("127.0.0.1:47500..47510"));
> discovery.setLocalAddress("127.0.0.1");
It seems that you don't have a cluster of 3 nodes, but rather 3 different
localhost-only clusters. No wonder data doesn't get distributed.
I would expect something between the lines of:
ipFinder.setAddresses(Arrays.asList("10.1.12.67:47500..47510",
"10.1.12.68:47500..47510", "10.1.12.69:47500..47510"));
And I'd remove setLocalAddress() assignment.
Make sure that nodes report topology snapshot of 3 servers after they start
up.
Regards,
--
Ilya Kasnacheev
2018-01-12 11:14 GMT+03:00 rizal123 :
> Hi Denis,
>
> My cluster node still cannot distribute the data.
>
> Here is my experiment:
> *My Client Apps*:
> Client using JDBC Thin Ignite.
> Connection conn =
> DriverManager.getConnection("jdbc:ignite:thin://10.1.12.67");
> And
> Create PrepareStatement for Execute query "Insert ABC".
>
> *Check 3 Node*:
> Using Application DBeaver to connect to 3 Cluster Node
> (10.1.12.67, 10.1.12.68, 10.1.12.69).
> Data 'ABC' only stored at Cluster Node (10.1.12.67).
> The other node didn`t get the data.
>
> *Here is my Ignite Configuration, for 3 machine/IP Address (10.1.12.67,
> 10.1.12.68, 10.1.12.69)* :
>
> /IgniteConfiguration cfg = new IgniteConfiguration();
> cfg.setIgniteInstanceName("BrandCluster");
> cfg.setClientMode(false);
>
> TcpDiscoveryVmIpFinder ipFinder = new
> TcpDiscoveryVmIpFinder();
> ipFinder.setAddresses(Arrays.asList("127.0.0.1:47500..
> 47510"));
>
> TcpDiscoverySpi discovery = new TcpDiscoverySpi();
> discovery.setLocalAddress("127.0.0.1");
> discovery.setLocalPort(47500);
> discovery.setIpFinder(ipFinder);
>
> cfg.setDiscoverySpi(discovery);/
>
> Would you mind to show me IgniteConfiguration for multi cluster (different
> ip address)?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: No user-defined default MemoryPolicy found
I checked the code
Actually, you can ignore this warning ("WARNING: No user-defined default
MemoryPolicy found; system default of 1GB size will be used.").
Apache Ignite applies your defaultMemoryPolicySize
Or you can change your config to avoid such warning:
Thanks,
Alexey
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Apache Ignite best practice
Hi Sergey, There could not be an exact answer to your question. It depends mostly on your use-case. 1. first of all, you should look at CPU/mem/network usage 2. and then you should check SQL debugging guide, starting from EXPLAIN for your query. Please see [1] for details. 3. you could enable dataRegionMetrics and dataStorageMetrics as it described at [2] and compare values with small/huge load/data capacity 4. you could also enable Cache metrics [3] [1] https://apacheignite-sql.readme.io/docs/performance-and-debugging [2] https://apacheignite.readme.io/docs/memory-metrics [3] https://apacheignite.readme.io/docs/cache-metrics Thank you, Alexey -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ignite.sh spring xml file secret.properties file not found error
Hi, I have specified secret.properties file location in the spring xml file and when it start ignite.sh, I see the below error as its not able to find the secret.properties file. - Where does ignite.sh look for this file? & how do I specify the right path for this file. $bin/ignite.sh ganesh.xml class org.apache.ignite.IgniteException: Failed to instantiate Spring XML application context [springUrl=file:/opt/vdp/ignite/config/ganesh.xml, err=Could not load properties; nested exception is java.io.FileNotFoundException: class path resource [secret.properties] cannot be opened because it does not exist] at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:966) at org.apache.ignite.Ignition.start(Ignition.java:350) at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302) Caused by: class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML application context [springUrl=file:/opt/vdp/ignite/config/ganesh.xml, err=Could not load properties; nested exception is java.io.FileNotFoundException: class path resource [secret.properties] cannot be opened because it does not exist] -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Exception Occuredjava.sql.BatchUpdateException: ORA-00001: unique constraint
Hi,
We were doing a POC(Ignite 2.3 - oracle) to check how cache Persistence
Store works using writeBehind, We are updating data as a batch rather than
updating one by one record.
though there are no duplication record but still getting "unique
constraint" exception.
We couldnt find any root cause for this not sure is any issue if this is a
possible ignite bug as this error is coming randomly.
Also in another POC we have seen that it's not inserting all the data to
the database.
For example we have pushed 10K but in Database we can only see 9800, though
there is no error or exception.
could you please shade some lights on these issues ?
Sample Code:
for(int i=1;i<=10;i++)
{
frstnm="test1name"+i;
scndnm="test2name";
p1 = new Person(i, frstnm, scndnm);
cache.put((long)i, p1);
}
insert into PERSON (id, firstname, lastname) values (?, ?, ?)
*inside write: *
st.setLong(1, val.getId());
st.setString(2, val.getFirstName());
st.setString(3, val.getLastName());
st.addBatch();
if(counter==100)
{
System.out.println("Counter "+ counter);
st.executeBatch();
counter=0;
}
got exception after 11K records
Exception Occuredjava.sql.BatchUpdateException: ORA-1: unique
constraint
--
Regards,
Rajarshi Pain
Re: ignite.sh spring xml file secret.properties file not found error
Ganesh, ignite.sh itself doesn't look for any properties files, it's your XML that has a reference to one. Looks like currently it's looking for a classpath resource. Make sure it's available there, or fix the path in the XML is location should be different. -Val -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: IgniteOutOfMemoryException when using putAll instead of put
Alexey,
The runtime class is used so I can have a common method to create any cache
type and index the key and value types of the cache.
To simplify things, attached is a tar file that is a small program that
throws an OOM exception for me. I get the OOM when loading from the cache
store on misses. I would expect it to just evict memory pages when it
inserts a new value, and not care if the insert is because of a put or a
read from cache.
If you comment out line 79 in the IgniteConfigGenerator class then it does
not get the OOM.
This is a simple test program, in my production code the value in the cache
will be an object, and I am calling setIndexedTypes to create a table for
that value so I can use sql query calls.
Can you let me know if you get the same results, and if so why we get the
OOM?
Thanks,
Larry
On Fri, Jan 12, 2018 at 6:53 AM, Alexey Popov wrote:
> Hi,
>
> You are right, "evicts=0" is related to cache evictions for on-heap caching
> [1]. It should be always 0 for you.
>
> I tried your case (with the same configs as you) and page evictions work
> fine with cache store enabled and indexed types. It seems that you have
> some
> misconfiguration.
>
> What are you trying to achieve by adding
> .setIndexedTypes(keytag.runtimeClass, valtag.runtimeClass) to String-value
> cache? and what is keytag.runtimeClass and valtag.runtimeClass?
>
> Could you please try with DummyClass with valid indexes enabled as below:
>
> /**
> * DummyClass
> */
> public class DummyClass {
> /** Dummy string. */
> public String dummyStr;
>
> /** Dummy int. */
> @QuerySqlField(index = true)
> public Integer dummyInt;
>
> public DummyClass(Integer dummyInt) {
> this.dummyInt = dummyInt;
> this.dummyStr = StringUtils.rightPad(dummyInt.toString(), 1024,
> '*');
> }
> }
>
> CacheConfiguration cacheCfg = new
> CacheConfiguration(CACHE_NAME)
> .setCacheMode(CacheMode.PARTITIONED)
> .setAtomicityMode(CacheAtomicityMode.ATOMIC)
>
> .setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(Duration.ETERNAL))
> .setDataRegionName(REG_NAME)
> .setStatisticsEnabled(true)
>
> .setCacheStoreFactory(FactoryBuilder.factoryOf(
> DummyStoreFromAdapter.class))
> .setReadThrough(true)
> .setIndexedTypes(Integer.class, DummyClass.class);
>
> Thanks,
> Alexey
>
> [1] https://apacheignite.readme.io/docs/evictions#section-java-heap-cache
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
fail.tar
Description: Unix tar archive
ignite c++ client CacheEntryEventFilter has no effect!
Hello Everyone, I use c++ client library to connect to ignite server node as a client node（），then run continuous-query-example，I find the RangeFilter（RangeFilter : event::CacheEntryEventFilter） doesn't have any effect, because Listener （Listener : public event::CacheEntryEventListener） print all the elements I have put。And I find the ignite server node print following errors： *[16:22:06,841][SEVERE][sys-stripe-4-#5][query] CacheEntryEventFilter failed: class o.a.i.IgniteException: Platforms are not available [nodeId=9ff3f363-c3bc-4a3a-a38d-8bc8655b8682] (Use Apache.Ignite.Core.Ignition.Start() or Apache.Ignite.exe to start Ignite.NET nodes; ignite::Ignition::Start() or ignite.exe to start Ignite C++ nodes).* -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
