RE: Best practice for class versioning: marshaller error

2018-07-16 Thread dkarachentsev
Hi Calvin, > Can I assume that BinaryMarshaller won't be used for any object embedded > inside GridCacheQueryResponse? Yes, because Binary can fallback to Optimized, but not vice versa. > If I am correct, do you have any suggestion on how I can avoid this type > of issue? Probably you need t

Re: Exception while running sql inside ignite transaction

2018-07-16 Thread dkarachentsev
Hi Akash, First of all SQL is not transactional yet, this feature will be available only since 2.7 [1]. Your exception might be caused if query was canceled or node stopped. [1] https://issues.apache.org/jira/browse/IGNITE-5934 Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6

Re: ALTER TABLE ... NOLOGGING

2018-08-03 Thread dkarachentsev
Hi, It might be an issue with deactivation. Try update to 2.6 or wait 2.7. Right now just skip cluster deactivation. Once you formed a baseline topology and finished loading data, just enable WAL log for all caches. When log enabled successfully, you can safely stop nodes. On next time when all b

Re: Additional field problems occurred in ignite2.6

2018-08-03 Thread dkarachentsev
Hi, Rules of field naming defined in BinaryIdMapper interface. By default used BinaryBasicIdMapper implementation that is by default converts all field names to lower case. So Ignite doesn't support the same field names in different cases as it will treat them as same field. But you can configure

Re: SYSTEM_WORKER_TERMINATION (Item Not found)

2018-08-03 Thread dkarachentsev
Hi, TTL fixes are not included in 2.6 as it was an emergency release. You'll need to wait for 2.7. https://issues.apache.org/jira/browse/IGNITE-5874 https://issues.apache.org/jira/browse/IGNITE-8503 https://issues.apache.org/jira/browse/IGNITE-8681 https://issues.apache.org/jira/browse/IGNITE-865

Re: Question

2018-08-07 Thread dkarachentsev
Hi, It defines by AffinityFunction [1]. By default 1024 partitions, affinity automatically calculates nodes that will keep required partitions and minifies rebalancing when topology changes (nodes arrive or quit). [1]https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/affini

Re: Partitions distribution across nodes

2018-08-08 Thread dkarachentsev
Hi Akash, How do you measure partition distribution? Can you provide code for that test? I can assume that you get partitions before exchange process if finished. Try to use delay in 5 sec after all nodes are started and check again. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.705

Re: SYSTEM_WORKER_TERMINATION (Item Not found)

2018-08-08 Thread dkarachentsev
Hi, I'm not sure that nightly builds are updates regularly, but you should a try. The biggest impact that nightly build could have some bugs that will be fixed on release. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Partitions distribution across nodes

2018-08-08 Thread dkarachentsev
Hi Akash, 1) Actually exchange is a short-time process when nodes remap partitions. But Ignite uses late affinity assignment, that means affinity distribution will be switched after rebalance completed. In other words after rebalance it will atomically switch partition distribution. But you don't

Re: When using CacheMode.LOCAL, OOM

2018-08-13 Thread dkarachentsev
Hi, I've opened a ticket for this [1]. It seems LOCAL cache keeps all entries on-heap. If you use only one node - switch to PARTITIONED, if more than one - PARTITIONED + node filter [2] [1] https://issues.apache.org/jira/browse/IGNITE-9257 [2] https://ignite.apache.org/releases/latest/javadoc/or

Re: security question - custom plugin

2018-08-13 Thread dkarachentsev
Hi, 1) You need to add jetbrains annotation in compile-time [1]. 2) Imports depend on what are you using :) It's hard to say if your imports enough. Add ignite-core to your plugin dependencies. I don't think that there are other examples besides that blog post. [1] https://mvnrepository.com/arti

Re: Partition distribution across nodes

2018-08-13 Thread dkarachentsev
Hi, Ignite by default uses Rendezvous hashing algorithm [1] and RendezvoudAffinityFunction is an implementation that responsible of partition distribution [2]. This allows significantly reduce traffic on partiton rebalancing. [1] https://en.wikipedia.org/wiki/Rendezvous_hashing [2] https://ignite

Re: Free Network Segmentation (aka split brain) plugin available

2018-08-13 Thread dkarachentsev
Hi, Nice work, thank you! I'm sure it will be very useful. Looking forward for your contributions in Apache Ignite project ;) Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Spark to Ignite Data load, Ignite node crashashing

2018-08-13 Thread dkarachentsev
Hi, Looks like it was killed by kernel. Check logs for OOM Killer: grep -i 'killed process' /var/log/messages If process was killed by Linux, correct your config, you might be set too much memory for Ignite paged memory, set to lower values [1] If not, try to find in logs by PID, maybe it was ki

Re: System cache's DataRegion size is configured to 40 MB.

2018-08-13 Thread dkarachentsev
Hi, Yes, you're right, it was missed during refactoring. I've created a ticket [1], you may fix it and contribute to Apache Ignite :) [1] https://issues.apache.org/jira/browse/IGNITE-9259 Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Transaction Throughput in Data Streamer

2018-08-13 Thread dkarachentsev
Hi, It looks like the most of the time transactions in receiver are waiting for locks. Any lock adds serialization for parallel code. And in your case I don't think it's possible to tune throughput with settings, because ten transactions could wait when one finish. You need to change algorithm.

Re: what are the alternative for IgniteQueue for FIFO transactional, reliable, low-latency messaging

2018-08-13 Thread dkarachentsev
Hi, I think the best way here would be to read items directly from kafka, process and store in cache and rememeber in another cache kafka stream offset. If node crashes, your service could start from the last point (offset). Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nab

Re: Eviction Policy on Dirty data

2018-08-14 Thread dkarachentsev
Hi, Could you please explain how do you update database? Do you use CacheStore with writeThrough or manually save? Anyway, you can update data with custom eviction policy: cache.withExpiryPolicy(policy) [1] [1] https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html#

Re: Some problems when using Ignite

2018-08-14 Thread dkarachentsev
Hi, Dynamic schema chages is available only via SQL/JDBC [1]. BTW caches created via SQL could be accessed from java API if you add SQL_PUBLIC_ to table. For example: ignite.cache(SQL_PUBLIC_TABLENAME). [1] https://apacheignite-sql.readme.io/docs/ddl Thanks! -Dmitry -- Sent from: http://apac

Re: distributed-ddl extended-parameters section showing 404 page not found

2018-08-14 Thread dkarachentsev
Hi, Where did you find it? It might be a broken link. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: What is the precise definition of classes eligible for P2P-classloading?

2018-08-31 Thread dkarachentsev
Hi, There are no such limitations on peer class loading, but it was designed and works for compute jobs, remote filters or queries only. All unknown classes from tasks or queries will be deployed in cluster with dependencies according to deployment mode [1]. Actually with job Ignite sends deployme

Re: Connection between servers.

2018-08-31 Thread dkarachentsev
Hi, Usually it's enough to open ports for communication and discovery, thair default values: 47500 and 47100. If you run more than one node per pachine, you'll need to open a port range: 47500..47509 and 47100...47109. You always can configure other values [1, 2] [1] https://ignite.apache.org/re

Re: Grid state check before it's completely caught up

2018-08-31 Thread dkarachentsev
Hi, You can, for example, set SYNC rebalance mode for your replicated cache [1]. In that case all cache operations will be blocked unless rebalance is finished, and when it's done you'll get a fully replicated cache. But this will block cache on each topology change. [1] https://ignite.apache.or

Re: Load balancing ignite get requests

2018-08-31 Thread dkarachentsev
Hi, get() operation from client always go to the primary node. If you run compute task on other nodes, where each will do get() request for that key, it will read local value. REPLICATED has many other optimizations, for example for SQL queries. Thanks! -Dmitry -- Sent from: http://apache-igni

Re: Ignite SQL function questions

2017-10-11 Thread dkarachentsev
Hi, You will have only one copy of data, tables are needed for correct H2 work. When do some query, H2 builds a query plan that will be delegated to Ignite, which underneath gets data from cache. Thanks! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: REST API Secret key not working

2017-10-11 Thread dkarachentsev
Hi, You heed to hash not just secret key, but "timestamp:secret_key", f.e. 1507726963290:LuM57LVuM3aN4tEjHF6XgkHo0fU= where hash got from "1507726963290:test". Thanks! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: REST API Secret key not working

2017-10-13 Thread dkarachentsev
Ankit, Thanks for pointing to mistake in documentation. I've suggested edits for it. Thanks! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Security question

2017-10-13 Thread dkarachentsev
Hi Franck, Yes, here is used client-side security, looks like it was made to allow connect of different clients with different permissions. But it depends on GridSecurityProcessor. For example, it may have a node validation logic that will not accept nodes with unapproved security processor. In

Re: custom restful service using jetty server on top of ignite grid to perform crud operations and configure cache store

2017-10-16 Thread dkarachentsev
Hi, @IgniteInstanceResource annotation is a correct and the best way to get Ignite instance in service. Thanks! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Error with ScanQuery

2017-10-16 Thread dkarachentsev
Hi Raymond, Could you please attach full log and config for failed node? Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Security question

2017-10-16 Thread dkarachentsev
Franck, You're definitely right, but this is more like client roles than regular security. On "they have a number of connected clients with actual applications" I meant that user's application is connected to the grid via clients with their local permissions. But end user cannot access the grid d

Re: Error with ScanQuery

2017-10-16 Thread dkarachentsev
Raymond, Without logs I see just that deserialization failed by some reason. Actually I more interested in exceptions that come from Ignite's java part if any. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: DML sql transaction

2017-10-16 Thread dkarachentsev
Hi Alisher, This issue is under active development: https://issues.apache.org/jira/browse/IGNITE-3478 Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Benchmark results questions

2017-10-16 Thread dkarachentsev
Hi Ray, Could you please attach dstat and GC logs for client and servers for 4 and 12 configurations (put operations would be just enough)? dstat --top-mem -msgdtc --fs --top-io Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Benchmark results questions

2017-10-24 Thread dkarachentsev
Hi Ray, I've got the same results on my environment and checking what happens. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: custom restful service using jetty server on top of ignite grid to perform crud operations and configure cache store

2017-10-25 Thread dkarachentsev
Hi, In your code Ignite could not inject it's instance, because you have two instances of your class: one in Ignite as a service, another one is object that processes requests in Jersey. So when you're doing http query, it goes to jersey instance. How do you start Ignite? You may get ignite with

Re: Benchmark results questions

2017-11-02 Thread dkarachentsev
Hi Ray, I wasn't able to run benchmarks quickly, but I've got following results for atomic put throughput (numbers are a bit lower that could be, because of profiling): Throughput Cluster 264930 1c4s 513775 2c4s 968475 4c4s 281425 1c8s 530597

Re: Benchmark results questions

2017-11-09 Thread dkarachentsev
Hi Ray, I've finally got results of query benchmarks: 4s1c 80725.80 80725.80 4s2c 78797.90 157595.80 4s4c 54029.70 216118.80 8s1c 64185.60 64185.60 8s2c 61058.10 122116.20 8s4c 34792.70 139170.80 First column - cluster configuration (in 8 server variant 2 nodes per machine), second - average thr

Re: what did this log indicate?

2017-11-16 Thread dkarachentsev
Hi, Try to enable paired connections [1]. [1] https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/spi/communication/tcp/TcpCommunicationSpi.html#setUsePairedConnections(boolean) Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: what did this log indicate?

2017-11-16 Thread dkarachentsev
Hi again, This looks quite similar to your issue, and it was fixed in 2.3 [1]. Check it out. [1] https://issues.apache.org/jira/browse/IGNITE-6071 Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Group By Query is slow : Apache Ignite 2.3.0

2017-12-19 Thread dkarachentsev
Hi, How many nodes do you have and how do you measure that 70 ms? Is it first query or average time? Please show your EXPLAIN of the query. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite 2.1.0 - transaction restrictions in multi-threading environment

2017-12-19 Thread dkarachentsev
Hi, First of all, Ignite object represents an Ignite node. Each such node may run more than one transaction and this object is thread safe. You may start only one transaction in thread, but Ignite object could be safely shared between your threads. Each transaction is bound to thread that it's st

Re: Is it possible to import existing mysql database from file in console?

2017-12-19 Thread dkarachentsev
Hi, You have a few options here: 1) Write code that scans all tables in MySQL and loads data to grid with IgniteDataStreamer [1]. 2) Write code that parses MySQL CSV and using IgniteDataStreamer loads to grid. 3) Use existing CacheJdbcStore to preload data from MySQL (check out screen casts [2] th

Re: Ignite service method cannot invoke for third time

2017-12-19 Thread dkarachentsev
Hi, Please attach thread dumps from all cluster nodes. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Group By Query is slow : Apache Ignite 2.3.0

2017-12-20 Thread dkarachentsev
Hi, How many records your query returns without LIMIT? How long does it take to select all records without grouping? Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: “Failed to communicate with Ignite cluster" error when using JDBC Thin driver

2017-12-20 Thread dkarachentsev
Hi, Is it possible that version of thin driver is different from version of cluster nodes? Does it happen on concrete queries or it could be on any one? Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: List of running Continuous queries or CacheEntryListener per cache or node

2017-12-20 Thread dkarachentsev
Hi, Currently it's not possible. What's for do you need such possibility? Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Group By Query is slow : Apache Ignite 2.3.0

2017-12-20 Thread dkarachentsev
Hi Indranil, These measurements are not fully correct, for example select count(*) might use only index and in select * was not actually invoked, because you need to run over cursor. Also by default query is not parallelized on one node, and scan with grouping is going sequentially in one thread.

Re: Ignite service method cannot invoke for third time

2017-12-21 Thread dkarachentsev
Hi, It looks like anonymous EntryProcessor gets excess data in context. Try to make it inner static class and check logs for exceptions on all nodes. Thanks! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Server node is frequently getting dropped from cluster.

2017-12-21 Thread dkarachentsev
Hi, Is it possible that firewall configured to block DDoS breaks connection to client node? Because I see here two possible cases: 1) STW pause on client, but we should see connection timeout exception; 2) Firewall rejects connections with a large traffic, and now you're getting connection refuse

Re: “Failed to communicate with Ignite cluster" error when using JDBC Thin driver

2017-12-21 Thread dkarachentsev
Is there any case that you're using Connection in more than one thread? It's not thread safe for now. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite cluster configure (2.3.0)

2017-12-21 Thread dkarachentsev
Hi Dmitriy, 1. You may use node filter [1] and specifically org.apache.ignite.util.AttributeNodeFilter that could be configured in XML without writing code. 2. Yes you can. You need to configure data regions and set persistenceEnabled flag. After that you may apply cachesh to that regions. [2]

Re: Ignite service method cannot invoke for third time

2017-12-21 Thread dkarachentsev
Sure, I meant you need to create your own inner class: private static class WorkflowEntryProcessor extends EntryProcessor { @Override public Object process(MutableEntry entry, Object... arguments) throws EntryProcessorException { System.out.println("EntryProcessor started");

Re: Ignite service method cannot invoke for third time

2017-12-21 Thread dkarachentsev
Glad to hear that it was helpful! I wrote the example just in email, so didn't have a compiler to check it :) Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Is it possible to import existing mysql database from file in console?

2017-12-22 Thread dkarachentsev
Hi, There are few options: 1) You need to have backups to survive node loss. [1] 2) You may enable persistence to survive grid restart and store more data that available in memory. [2] 3) Checkout nohup command [3] [1] https://apacheignite.readme.io/docs/primary-and-backup-copies [2] https://apac

Re: When do we get this error - Unknown pair [platformId=0, typeId=1078091073]]

2017-12-22 Thread dkarachentsev
Hi, Looks like not on all nodes exist your classes. Please check if all classes that you're using in cache are available on all nodes. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Reconnect after cluster shutdown fails

2017-12-24 Thread dkarachentsev
Hi, Please attach thread dumps from all nodes taken at the moment of hang. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: “Failed to communicate with Ignite cluster" error when using JDBC Thin driver

2017-12-26 Thread dkarachentsev
Hi, It's hard to say why it happens. I'm not familiar with mybatis and actually don't know if it shares jdbc connection between threads. It would be great if you could provide some reproducible example that will help to debug the issue. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.

Re: Reconnect after cluster shutdown fails

2017-12-26 Thread dkarachentsev
Hi, Discovery events are processed in a single thread, and cache creation uses discovery custom messages. Trying to create cache in discovery thread will lead to deadlock, because discovery thread will wait in your lambda instead of processing messages. To avoid it just start another thread in yo

Re: Ignite service method cannot invoke for third time

2017-12-26 Thread dkarachentsev
Hi, Anonymous and inner classes have link to outer class object and might bring it to marshaller. When you set it inner static or separate class you're explicitly saying that you don't need such links. In thread dumps you need to lookup for waiting or blocked threads. In your case in service node

Re: Question about persisting stream processing results

2018-01-23 Thread dkarachentsev
Hi Svonn, I'm not sure that I properly understand your issue. Could you please provide a problematic code snipped? > is the policy also deleting the Map Yes, if it was stored as a value. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Memory usage by ignite nodes

2018-01-24 Thread dkarachentsev
Hi Ranjit, That metrics should be correct, you also may check [1], because Ignite anyway keeps data in offheap. But if enabled on-heap, it caches entries in java heap. [1] https://apacheignite.readme.io/docs/memory-metrics Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabb

Re: setNodeFilter throwing a CacheException

2018-01-25 Thread dkarachentsev
Hi Sharavya, This exception means that client node is disconnected from cluster and tries to reconnect. You may get reconnect future on it (IgniteClientDisconnectedException.reconnectFuture().get()) and wait when client will be reconnected. So it looks like you're trying to create cache on stoppe

Re: setNodeFilter throwing a CacheException

2018-01-29 Thread dkarachentsev
Hi Shravya, To understand what's going on in your cluster I need full logs from all nodes. Please, share all files, if it's possible. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to identify stale ignite client in case of data grid restart and auto reconnect to cluster

2018-02-06 Thread dkarachentsev
Hi, This exception says that client node was stopped, but by default it should wait for servers. In other words, wait for reconnect, in this case it throws IgniteClientDisconnectedException that contains future on which you may wait for reconnect event. You may locally listen for EventType.EVT_CLI

Re: Design help implementing custom counter on ignite

2018-02-06 Thread dkarachentsev
Hi, Transaction here might be a not optimal solution, as it by default optimistic and may throw optimistic transaction exception. I believe the best solution would be to use EntryProcessor [1], it will atomically modify entry as on TRANSACTIONAL as on ATOMIC cache on affinity data node (that actua

Re: Text Query question

2018-02-07 Thread dkarachentsev
Hi Jet, Full text search creates Lucene in-memory indexes and after restart they are not available, so you cannot use it with persistence. @QuerySqlField enables DB indexes that are able to work with persisted data, and probably no way to rebuild them for now. Thanks! -Dmitry -- Sent from: htt

Re: continuous query - changes from local server only

2018-02-08 Thread dkarachentsev
Hi, You may fuse filter for that, for example: ContinuousQuery qry = new ContinuousQuery<>(); final Set nodes = new HashSet<>(client.cluster().forDataNodes("cache") .forHost(client.cluster().localNode()).nodes()); qry.setRemoteFilterFactory(new Factory>() {

Re: Autowire in CacheStore implementation

2018-02-12 Thread dkarachentsev
Hi Prasad, If you started Ignite with IgniteSpringBean or IgniteSpring try @SpringApplicationContextResource [1] annotation. Ignite's resource injector will use spring context to set a dependency annotated by it. But I'm not sure that this will work with CacheStore, it should be rechecked. [1] ht

Re: Text Query question

2018-02-12 Thread dkarachentsev
Jet, Yep, this should work, but meanwhile this ticket remains unresolved [1]. [1] https://issues.apache.org/jira/browse/IGNITE-5371 Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Distributed transaction (Executing task on client as well as on key owner node)

2018-02-12 Thread dkarachentsev
Hi Prasad, This approach will work with multiple keys if they are collocated on the same node and you start/stop transaction in the same thread/task. There no other workaround. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Kubernetes discovery with readinessProbe

2018-02-15 Thread dkarachentsev
Hi Bryan, You need to use StatefulSet [1], Kubernetes will start nodes one-by-one when each comes in a ready state. [1] https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/ Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Exception while using select query

2018-03-16 Thread dkarachentsev
Hi Anshu, This looks like a bug that was fixed in 2.4, try to upgrade [1]. [1] https://ignite.apache.org/download.cgi Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Strange node fail

2018-04-10 Thread dkarachentsev
Hi Ray, If your JVM process consumes more memory, then started swapping may cause JVM freeze, and as a consequence, throwing it out from the cluster. Check your free memory, disable swapping, if possible, or increase IgniteConfiguration.failureDetectionTimeout. To check that guess you may use dst

Re: Strange node fail

2018-04-10 Thread dkarachentsev
Duplicates http://apache-ignite-users.70518.x6.nabble.com/Strange-node-fail-td21078.html. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Delete SQL is failing with IN clause for a table which has composite key

2018-04-10 Thread dkarachentsev
Hi Naveen, Unfortunately I'm unable to reproduce that error. Could you please attach simple code/project that fails with specified exception? Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite Eviction Policy

2018-04-11 Thread dkarachentsev
Hi, If you have enabled read through mode for cache, entry will be loaded on next IgniteCache.get() operation, or when IgniteCache.loadCache() was called. Next time entry will be evicted according to your eviction policy. Please note that entry will not be counted in SQL queries if it was evicte

Re: How to set Expiry Policies when using Dataframe API to save data to Ignite?

2018-04-11 Thread dkarachentsev
Hi Ray, I think the only way to do it is to use IgniteDataFrameSettings.OPTION_CONFIG_FILE and set path to xml configuration with all settings you need. Here is a nice article about this [1] [1] https://medium.com/hashmapinc/apache-ignite-using-a-memory-grid-for-distributed-computation-frameworks

Re: Issue IGNITE-3471

2018-04-11 Thread dkarachentsev
Hi Prasad, This issue could not be completed in 2.5 as it's done in a low priority. As a workaround, you can wrap your executeEntryProcessorTransaction() method into affinity run [1], and no additional value transferring will happen. [1] https://apacheignite.readme.io/docs/collocate-compute-and-d

Re: JMX-MBean Reports OffHeapAllocatedSize as zero

2018-04-11 Thread dkarachentsev
Hi Christoph, This metric is not implemented because of complexity. But you may get to know now much of space your cache or cashes consumes with DataRegionMetrics: DataRegionMetrics drm = ignite.dataRegionMetrics("region_name"); long used = (long)(drm.getPhysicalMemorySize() * drm.getPagesFillFac

Re: NullPointerException in GridCacheTtlManager.expire

2018-04-12 Thread dkarachentsev
Hi Dome, Could you please attach full logs? Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite Data Streamer Hung after a period

2018-04-13 Thread dkarachentsev
Hi, Blocked threads show only the fact that there are no tasks to process in pool. Do you use persistence and/or indexing? Could you please attach your configs and logs from all nodes? Please take few sequential thread dumps when throughput is low. Thanks! -Dmitry -- Sent from: http://apache-

Re: The thread which is inserting data into Ignite is hung

2018-04-16 Thread dkarachentsev
Hi Praveen, Stack traces only show that thread is waiting for response, to get the full picture, please attach full logs and thread dumps at the moment of hang from all nodes. I need from all nodes, because actual issue happened on remote node. Also, according to last exception, there might be c

Re: Ignite query statement reused/cached

2018-04-16 Thread dkarachentsev
Hi, For sure Ignite caches queries, that's why first request runs much longer than rest ones. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Do we require to set MaxDirectMemorySize JVM parameter?

2018-04-17 Thread dkarachentsev
Hi Ankit, No, Ignite uses sun.misc.Unsafe for offheap memory. Direct memory may be used in DirectBuffers used for intercommunication. Usually defaults quite enough. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Issue IGNITE-3471

2018-04-20 Thread dkarachentsev
Hi, Yes, for complex transaction this workaround will not work. So you need either wait for fix or avoid using EntryProcessor for now. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Messages and topics

2018-06-01 Thread dkarachentsev
Hi, Yes, Ignite will send messages to all nodes, but you may use filter: ignite.message(ignite.cluster().forAttribute("topic1", Boolean.TRUE)); In this case messages would be sent to all nodes from the cluster group, in this example - only nodes with set attribute "topic1" [1]. [1] https://igni

Re: Ignite Cluster getting stuck when new node Join or release

2018-06-04 Thread dkarachentsev
Hi, It's hard to get what's going wrong from your question. Please attach full logs and thread dumps from all server nodes. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite Backup doubts

2018-06-08 Thread dkarachentsev
Hi, 1. By default get() will read backups if node, on which it's invoked is affinity node. In other words, if current node has backups, Ignite prefer to read local data from backup rather requesting primary node over network. This can be changed by setting CacheConfiguration.setReadFromBackup(fals

Re: ClusterTopologyServerNotFoundException

2018-06-11 Thread dkarachentsev
Hi, Could you please provide a reproducer? I don't get such exception. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: "WAL segment tail is reached", Is it a problem ?

2018-06-11 Thread dkarachentsev
Hi Mikael, Please share your Ignite settings and logs. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite opens/close 5000 sockets in every 5mins after NODE_FAILED event

2018-06-12 Thread dkarachentsev
Hi, TcpDiscoveryMulticastIpFinder produces such a big number of connections. I'd recommend to switch to TcpDiscoveryVmIpFinder with static set of addresses. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to use Affinity Function to Map a set of Keys to a particular node in a cluster?

2018-06-12 Thread dkarachentsev
Hi, Make sure that your keys are go to specific partition. Only one node could keep that partition at a time (except backups, of course). To do that, you may use @AffinityKeyMapped annotation [1]. Additionally you can implement your own AffinityFunction that will assign partitions that you need t

Re: How to use Affinity Function to Map a set of Keys to a particular node in a cluster?

2018-06-12 Thread dkarachentsev
Normally (without @AffinityKeyMapped) Ignite will use CustomerKey hash code (not object hashCode()) to find a partition. Ignite will colsult with AffinityFunction (partition() method) and to what partition put key and with assignPartitions find concrete node that holds that partition. In other han

Re: How to use Affinity Function to Map a set of Keys to a particular node in a cluster?

2018-06-12 Thread dkarachentsev
There are various possible ways, but use one partition per node is definitely a bad idea, because you're loosing scaling possibilities. If you have 5 partitions and 5 nodes, then 6 node will be empty. It's much better if you in AffinityFunction.partition() method will calculate node according to

Re: How to use Affinity Function to Map a set of Keys to a particular node in a cluster?

2018-06-12 Thread dkarachentsev
1. Affinity knows that, because it does assignments. Method assignPartitions() returns that assignments. Please read the javadoc [1]. 2. I just described how keys could be assigned to partition. For example: @Override public int partition(Object key) { if (key instanceof Integer)

RE: Ignite opens/close 5000 sockets in every 5mins after NODE_FAILED event

2018-06-12 Thread dkarachentsev
There is no difference on how do you start/stop your node. Node on start will examine all connections specified in address list: it takes one address and port and tries to connect to it. If not successfull, get another address and port. For instance if you have address 1.2.3.4:47500..47509, node w

Re: Transactional cache

2018-06-12 Thread dkarachentsev
Hi, Ignite keeps Tx cached values on-heap. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Baseline topology issue when restarting server nodes one by one

2018-06-12 Thread dkarachentsev
Hi, What IgniteConfiguration do you use? Could you please share it? Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to use Affinity Function to Map a set of Keys to a particular node in a cluster?

2018-06-13 Thread dkarachentsev
Hi, I totally agree with Val that implementing own AffinityFunction is quite complex way. Requirement that you described is named affinity co-location as I wrote before. Let me explain in more details what to do and what are the drawbacks. 1. Use use @AffinityKeyMapped for all your keys. For exa

  1   2   3   4   >