Hi Team,
We are trying to start another Ignite cluster by taking a copy
of the running cluster's datastore (source cluster's datastore is getting
modified in parallel). So, when we try to start the server node with copied
datastore, it gives error as below. Also, giving cluster configuration for
reference:
pageSize=#{4 * 1024}
walMode=LOG_ONLY
walFlushFrequency=60000
rebalanceThreadPoolSize=8
rebalanceThrottle=100
rebalanceBatchSize=#{32 * 1024 * 1024}
storagePath=/datastore/datastore
walPath=/datastore1/wal
walArchivePath=/datastore1/archive
metadataWorkDir=/datastore/metadataWorkDir
[2019-06-05T12:21:52,943][INFO ][main][GridCacheDatabaseSharedManager] Read
checkpoint status [startMarker=null, endMarker=null]
[2019-06-05T12:21:52,967][INFO ][main][PageMemoryImpl] Started page memory
[memoryAllocated=128.0 MiB, pages=31744, tableSize=2.5 MiB,
checkpointBuffer=100.0 MiB]
[2019-06-05T12:21:52,968][INFO ][main][GridCacheDatabaseSharedManager] Checking
memory state [lastValidPos=FileWALPointer [idx=0, fileOff=0, len=0],
lastMarked=FileWALPointer [idx=0, fileOff=0, len=0],
lastCheckpointId=00000000-0000-0000-0000-000000000000]
[2019-06-05T12:21:52,973][ERROR][main][IgniteKernal%EDIFCustomer_DR] Exception
during start processors, node will be stopped and close connections
org.apache.ignite.IgniteCheckedException: Failed to start processor:
GridProcessorAdapter []
at
org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1742)
~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:980)
[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2014)
[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1723)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1151)
[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1069)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:955)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:854)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:724)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:693)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.Ignition.start(Ignition.java:352)
[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)
[ignite-core-2.6.0.jar:2.6.0]
Caused by: org.apache.ignite.IgniteCheckedException: WAL history is too short
[descs=[org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@4d6c2],
start=FileWALPointer [idx=0, fileOff=0, len=0]]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.init(FileWriteAheadLogManager.java:3009)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:2960)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:2896)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:799)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1968)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:574)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.start0(GridCacheDatabaseSharedManager.java:525)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.start(GridCacheSharedManagerAdapter.java:61)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.start(GridCacheProcessor.java:700)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1739)
~[ignite-core-2.6.0.jar:2.6.0]
... 11 more
[2019-06-05T12:21:52,978][ERROR][main][IgniteKernal%EDIFCustomer_DR] Got
exception while starting (will rollback startup routine).
org.apache.ignite.IgniteCheckedException: Failed to start processor:
GridProcessorAdapter []
at
org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1742)
~[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:980)
[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2014)
[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1723)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1151)
[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1069)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:955)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:854)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:724)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:693)
[ignite-core-2.6.0.jar:2.6.0]
at org.apache.ignite.Ignition.start(Ignition.java:352)
[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)
[ignite-core-2.6.0.jar:2.6.0]
Caused by: org.apache.ignite.IgniteCheckedException: WAL history is too short
[descs=[org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@4d6c2],
start=FileWALPointer [idx=0, fileOff=0, len=0]]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.init(FileWriteAheadLogManager.java:3009)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:2960)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:2896)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:799)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1968)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:574)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.start0(GridCacheDatabaseSharedManager.java:525)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.start(GridCacheSharedManagerAdapter.java:61)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.start(GridCacheProcessor.java:700)
~[ignite-core-2.6.0.jar:2.6.0]
at
org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1739)
~[ignite-core-2.6.0.jar:2.6.0]
So, is there any way to start this cluster with copied data
store forcefully? This scenario may also arrive if WAL disk gets failed. How
can we atleast start the cluster with minimum data loss ?
Any help would be highly appreciated !
Thanks and Regards,
Kamlesh Joshi
"Confidentiality Warning: This message and any attachments are intended only
for the use of the intended recipient(s).
are confidential and may be privileged. If you are not the intended recipient.
you are hereby notified that any
review. re-transmission. conversion to hard copy. copying. circulation or other
use of this message and any attachments is
strictly prohibited. If you are not the intended recipient. please notify the
sender immediately by return email.
and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure
no viruses are present in this email.
The company cannot accept responsibility for any loss or damage arising from
the use of this email or attachment."
