Hello guys,
I got a node in joining status for almost more than 12 hours even if the
streaming seems to be over.
Checking the logs I found this error
ERROR [main] 2021-12-15 19:39:43,886 InsightsModule.java:166 - Unable to
initialize insights client due to error
org.apache.cassandra.exceptions.IsBootstrappingException: Cannot read from
a bootstrapping node
at
org.apache.cassandra.service.StorageProxy.checkNotBootstrappingOrSystemQuery(StorageProxy.java:1295)
at
org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1306)
at
org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:1239)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:371)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:579)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:312)
at
com.datastax.bdp.util.QueryProcessorUtil.processPreparedSelect(QueryProcessorUtil.java:96)
at
com.datastax.bdp.util.QueryProcessorUtil.processPreparedSelect(QueryProcessorUtil.java:84)
at
com.datastax.bdp.insights.storage.config.DseLocalInsightsRuntimeConfigAccess.selectConfigValue(DseLocalInsightsRuntimeConfigAccess.java:239)
at
com.datastax.bdp.insights.storage.config.InsightsRuntimeConfigManager.populateCachedConfigWithLocalConfig(InsightsRuntimeConfigManager.java:97)
at
com.datastax.bdp.insights.storage.config.InsightsRuntimeConfigManager.getRuntimeConfig(InsightsRuntimeConfigManager.java:328)
at
com.datastax.bdp.insights.InsightsModule.insightsClient(InsightsModule.java:154)
Before I got some errors in the streaming
ERROR [STREAM-IN-/10.17.74.94:7000] 2021-12-15 10:10:58,181
StreamSession.java:650 - [Stream #840346f0-5daf-11ec-a92c-2728dd47bba2]
Streaming error occurred on session with peer x.x.x.x
java.lang.NullPointerException: null
at
org.apache.cassandra.utils.ByteSource.optionalSignedFixedLengthNumber(ByteSource.java:111)
at
org.apache.cassandra.db.marshal.LongType.asByteComparableSource(LongType.java:53)
at
org.apache.cassandra.db.ClusteringComparator$ClusteringSource.next(ClusteringComparator.java:284)
at
org.apache.cassandra.utils.ByteSource$Separator.next(ByteSource.java:572)
at
org.apache.cassandra.io.tries.IncrementalTrieWriterBase.add(IncrementalTrieWriterBase.java:72)
at
org.apache.cassandra.io.sstable.format.trieindex.RowIndexWriter.add(RowIndexWriter.java:73)
at
org.apache.cassandra.io.sstable.format.trieindex.PartitionWriter.addIndexBlock(PartitionWriter.java:139)
at
org.apache.cassandra.io.sstable.format.trieindex.PartitionWriter.finish(PartitionWriter.java:190)
at
org.apache.cassandra.io.sstable.format.trieindex.PartitionWriter.writePartition(PartitionWriter.java:107)
at
org.apache.cassandra.io.sstable.format.trieindex.TrieIndexSSTableWriter.append(TrieIndexSSTableWriter.java:218)
at
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:47)
at
org.apache.cassandra.io.sstable.format.RangeAwareSSTableWriter.append(RangeAwareSSTableWriter.java:105)
at
org.apache.cassandra.streaming.StreamReader.writePartition(StreamReader.java:168)
at
org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:102)
at
org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:52)
at
org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:41)
at
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:97)
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:318)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
WARN [Thread-1400] 2021-12-15 10:10:58,220 CompressedInputStream.java:232
- Error while reading compressed input stream.
java.nio.channels.ClosedChannelException: null
at
sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:254)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:297)
at
sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:219)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at
java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at
org.apache.cassandra.streaming.compress.CompressedInputStream$Reader.runMayThrow(CompressedInputStream.java:221)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
ERROR [main] 2021-12-15 19:39:40,756 StorageService.java:1889 - Error
while waiting on bootstrap to complete. Bootstrap will have to be restarted.
java.util.concurrent.ExecutionException:
org.apache.cassandra.streaming.StreamException: Stream failed
at
com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at
com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at
org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1882)
at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1115)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:689)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:613)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:402)
at com.datastax.bdp.server.DseDaemon.setup(DseDaemon.java:527)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:702)
After using nodetool bootstrap resume, I got the same error with
INFO [OptionalTasks:1] 2021-12-16 04:41:34,078
CassandraRoleManager.java:432 - Setup task failed with error, rescheduling
org.apache.cassandra.exceptions.IsBootstrappingException: Cannot read from
a bootstrapping node
Thanks
Jean Carlo
"The best way to predict the future is to invent it" Alan Kay
