According to the configuration you shared, you don’t have backups. I would expect to see something like:
CacheConfiguration<Integer, Department> deptCacheConfig = new CacheConfiguration<>(DEPT_CACHE); deptCacheConfig.setCacheMode(CacheMode.PARTITIONED); deptCacheConfig.setBackups(1); IgniteCache<Integer, Department> deptCache = ignite.getOrCreateCache(deptCacheConfig); The default is zero backups, so loss of a single node results in lost partitions. Note, that since you have persistence enabled, you’ll have to drop and recreate the table for the change to take effect. > On 31 Jan 2022, at 11:16, Surinder Mehra <redni...@gmail.com> wrote: > > Hi, thanks for pointing out the problem of why data is being stored on the > same machine. > The 2nd point: > As I mentioned in steps, we are getting the same exception even with backups. > Could you please point out other problems with configuration as you mentioned > > > On Mon, Jan 31, 2022 at 4:35 PM Stephen Darlington > <stephen.darling...@gridgain.com <mailto:stephen.darling...@gridgain.com>> > wrote: > There’s a lot to unpack here, but there are (at least) two problems with your > configuration. > > First, you should not be activating your cluster in your startup script. > That’s an operation that needs to be completed only once, not every time the > cluster (or node) starts. This is probably why all the data is being stored > on a single machine. > > Second, you create a cache but without any backups. That means that when a > node dies, you lose access to all the data. > > > On 31 Jan 2022, at 10:57, Surinder Mehra <redni...@gmail.com > > <mailto:redni...@gmail.com>> wrote: > > > > > > Hi Guys, > > I observed below behavior with persistence enabled. Could you please check > > and confirm if it's a bug or I missed some configuration. > > > > Steps: > > 1. Start ignite node1 and 2 with persistence enabled > > 2. Step 1 will write few entries to ignite cache > > 3. All entries go to one node > > 4. Stop node containing the data > > 5. Run a query to get data, below exception will be thrown. I guess this is > > expected because the node containing the data is down > > 6. Restart killed node, exception shouldn’t be thrown because data is back > > in caches > > 7. Run a query to get data, below exception will be thrown again. This > > looks weird. > > 8. Kill node2 and try to query again, still the same exception > > 9. Restart only the left node(data owning node). After restart, the query > > starts working again. > > 10. I checked the cache directory under node1, it didn't have part0.bin > > ever. files start from part1 and so on. > > 11. From the exception, it looks like it is trying to read non existent file > > 12. Tried setting consistent Id for each node, still same exception. > > > > > > Another observation : When backup is enabled, shouldn't the backup node > > serve the query. Why it throws same below exception when node1(primary) is > > down > > > > This exception is only thrown if data containing node goes down > > > > Caused by: class > > org.apache.ignite.internal.processors.cache.CacheInvalidStateException: > > Failed to execute query because cache partition has been lostParts > > [cacheName=deptCache, part=0] > > > > Node2{ > > > > DataStorageConfiguration storageCfg = new DataStorageConfiguration(); > > storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(true); > > IgniteConfiguration cfg = new IgniteConfiguration(); > > cfg.setDataStorageConfiguration(storageCfg); > > > > try (Ignite ignite = Ignition.start(cfg)) { > > ignite.cluster().state(ClusterState.ACTIVE); > > CacheConfiguration<Integer, Department> deptCacheConfig = new > > CacheConfiguration<>(DEPT_CACHE); > > deptCacheConfig.setCacheMode(CacheMode.PARTITIONED); > > IgniteCache<Integer, Department> deptCache = > > ignite.getOrCreateCache(deptCacheConfig); > > > > Department d1 = new Department(1, "CS"); > > Department d2 = new Department(2, "ECE"); > > Department d3 = new Department(3, "CIVIL"); > > > > if(deptCache.size(CachePeekMode.ALL) == 0){ > > System.out.println("Adding dept data to cache"); > > deptCache.put(d1.getDeptId(), d1); > > deptCache.put(d2.getDeptId(), d2); > > deptCache.put(d3.getDeptId(), d3); > > } > > > > } > > > > } > > > > Node1{ > > > > DataStorageConfiguration storageCfg = new DataStorageConfiguration(); > > storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(true); > > IgniteConfiguration cfg = new IgniteConfiguration(); > > cfg.setDataStorageConfiguration(storageCfg); > > > > try (Ignite ignite = Ignition.start(cfg)) { > > ignite.cluster().state(ClusterState.ACTIVE); > > CacheConfiguration<Integer, Department> deptCacheConfig = new > > CacheConfiguration<>(DEPT_CACHE); > > deptCacheConfig.setCacheMode(CacheMode.PARTITIONED); > > IgniteCache<Integer, Department> deptCache = > > ignite.getOrCreateCache(deptCacheConfig); > > > > Department d1 = new Department(1, "CS"); > > Department d2 = new Department(2, "ECE"); > > Department d3 = new Department(3, "CIVIL"); > > > > if(deptCache.size(CachePeekMode.ALL) == 0){ > > System.out.println("Adding dept data to cache"); > > deptCache.put(d1.getDeptId(), d1); > > deptCache.put(d2.getDeptId(), d2); > > deptCache.put(d3.getDeptId(), d3); > > } > > > > } > > } > > > > Client{ > > > > IgniteConfiguration cfg = new IgniteConfiguration(); > > cfg.setDataStorageConfiguration(storageCfg); > > > > try (Ignite ignite = Ignition.start(cfg)) { > > IgniteCache<Integer, Department> deptCache = > > ignite.getOrCreateCache(DEPT_CACHE); > > List<Cache.Entry<Object, Object>> depts = deptCache.query(new > > ScanQuery<>()).getAll(); > > depts.stream().forEach(entry -> > > System.out.println(entry.getValue())); > > } > > } >
