I just saw this "Ignite node crashed in the middle of checkpoint" on my development machine with the latest Ignite 2.1.4 - it appeared when activating a single node cluster with persistence enabled but no data to preload at all. I will also look into it after I complete my current tasks. Best regards, Alexey
On Tuesday, August 15, 2017, 3:39:57 AM GMT+3, Roger Fischer (CW) <[email protected]> wrote: #yiv8195883399 #yiv8195883399 -- _filtered #yiv8195883399 {panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv8195883399 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv8195883399 {font-family:Tahoma;panose-1:2 11 6 4 3 5 4 4 2 4;}#yiv8195883399 #yiv8195883399 p.yiv8195883399MsoNormal, #yiv8195883399 li.yiv8195883399MsoNormal, #yiv8195883399 div.yiv8195883399MsoNormal {margin:0in;margin-bottom:.0001pt;font-size:12.0pt;}#yiv8195883399 a:link, #yiv8195883399 span.yiv8195883399MsoHyperlink {color:blue;text-decoration:underline;}#yiv8195883399 a:visited, #yiv8195883399 span.yiv8195883399MsoHyperlinkFollowed {color:purple;text-decoration:underline;}#yiv8195883399 span.yiv8195883399EmailStyle17 {color:#1F497D;}#yiv8195883399 .yiv8195883399MsoChpDefault {} _filtered #yiv8195883399 {margin:1.0in 1.0in 1.0in 1.0in;}#yiv8195883399 div.yiv8195883399WordSection1 {}#yiv8195883399 Hi Alex, there were no other relevant logs than what I already listed in the first email. <beans xmlns="http://www.springframework.org/schema/beans"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd";> <bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration"> <property name="discoverySpi"> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> <property name="ipFinder"> <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder"> <property name="addresses"> <list> <value>10.24.51.190</value> <value>10.24.51.187</value> <value>10.24.51.150</value> </list> </property> </bean> </property> </bean> </property> <property name="persistentStoreConfiguration"> <bean class="org.apache.ignite.configuration.PersistentStoreConfiguration"/> </property> <property name="cacheConfiguration"> <list> <bean class="org.apache.ignite.configuration.CacheConfiguration"> <property name="name" value="FcPortStatsCache"/> <property name="cacheMode" value="PARTITIONED"/> <property name="backups" value="1"/> <property name="queryEntities"> <list> <bean class="org.apache.ignite.cache.QueryEntity"> <property name="keyType" value="com.abc.poc.inppoc.model.FcPortStatsKey"/> <property name="valueType" value="com.abc.poc.inppoc.model.FcPortStats"/> <property name="fields"> <map> <entry key="dateTime" value="java.util.Date"/> <entry key="portId" value="java.util.UUID"/> <entry key="portType" value="java.lang.Integer"/> <entry key="switchId" value="java.util.UUID"/> <entry key="fabricId" value="java.util.UUID"/> <entry key="rxUtil" value="java.lang.Float"/> <entry key="txUtil" value="java.lang.Float"/> <entry key="higherUtil" value="java.lang.Float"/> <entry key="lowerUtil" value="java.lang.Float"/> <entry key="rxRate" value="java.lang.Float"/> <entry key="txRate" value="java.lang.Float"/> <entry key="higherRate" value="java.lang.Float"/> <entry key="lowerRate" value="java.lang.Float"/> <entry key="c3Discards" value="java.lang.Float"/> <entry key="crcErrors" value="java.lang.Float"/> </map> </property> <property name="indexes"> <list> <bean class="org.apache.ignite.cache.QueryIndex"> <constructor-arg> <list> <value>dateTime</value> <value>portId</value> </list> </constructor-arg> <constructor-arg value="SORTED"/> </bean> <bean class="org.apache.ignite.cache.QueryIndex"> <constructor-arg> <list> <value>portId</value> <value>dateTime</value> </list> </constructor-arg> <constructor-arg value="SORTED"/> </bean> <bean class="org.apache.ignite.cache.QueryIndex"> <constructor-arg> <list> <value>switchId</value> <value>dateTime</value> </list> </constructor-arg> <constructor-arg value="SORTED"/> </bean> </list> </property> </bean> </list> </property> </bean> </list> </property> </bean> </beans> All 3 servers (and the client) are on VMs on the same host. No network hop latency. But all 3 VMs use the same physical disk (on the host). Servers have 16 GB of RAM. Data on disk (work/db) was about 35 GB per mode. About 36M objects. Please also notehttp://apache-ignite-users.70518.x6.nabble.com/Strange-problems-with-Ignite-native-Persistence-when-Data-exceeds-Memory-td16187.html. There were some odd problems at time that may have affected the activation. Roger From: afedotov [mailto:[email protected]] Sent: Monday, August 14, 2017 11:05 AM To: [email protected] Subject: Re: Activation: slow and: Ignite node crashed in the middle of checkpoint. Hi, Could you please share the logs and configuration? Actually, the activation time depends on the amount of data, network connectivity and other variables. Kind regards, Alex. On Sat, Aug 12, 2017 at 12:39 AM, Roger Fischer (CW) [via Apache Ignite Users] <[hidden email]> wrote: Hello, I am wondering if the following behavior is typical, or if it represents a concern. I have a 3 node cluster with native persistence. Each node as 4 CPU and 16 GB of RAM. Each node has ~45 GB used in work/db. Total across the 3 nodes is about 36.5 M objects. I am using SQL queries, and there are 3 indexes. The servers start up normally and join the cluster, as expected. When I start the client, which calls active(), all 3 servers report the following: [12:41:28] Topology snapshot [ver=5, servers=3, clients=1, CPUs=16, heap=4.8GB] [12:41:29] Default checkpoint page buffer size is too small, setting to an adjusted value: 2.0 GiB [12:41:29] Ignite node crashed in the middle of checkpoint. Will restore memory state and enforce checkpoint on node start. 1) Should I worry about the “crashed” log? The activation takes more than 30 minutes (until active() returns). 2) Is that normal for activate to take that long? ver. 2.1.0#20170720-sha1:a6ca5c8a OS: Linux 3.10.0-514.el7.x86_64 amd64 Thanks… Roger If you reply to this email, your message will be added to the discussion below: http://apache-ignite-users.70518.x6.nabble.com/Activation-slow-and-Ignite-node-crashed-in-the-middle-of-checkpoint-tp16144.html To start a new topic under Apache Ignite Users, email[hidden email] To unsubscribe from Apache Ignite Users, click here. NAML View this message in context: Re: Activation: slow and: Ignite node crashed in the middle of checkpoint. Sent from the Apache Ignite Users mailing list archive at Nabble.com.
