robertzych opened a new pull request, #12323: URL: https://github.com/apache/pinot/pull/12323
To reproduce an `IndexOutOfBoundException` like the one mentioned in [12301](https://github.com/apache/pinot/issues/12301) run `UpsertCompactionMinionClusterIntegrationTest.testRecompaction`. This test is failing on `waitForAllDocsLoaded(30_000L, 2)` because the `IndexOutOfBoundException` is preventing segment replacement after compaction. Here is the error and stack trace from the test: ``` 15:27:47.728 ERROR [HelixTask] [HelixTaskExecutor-message_handle_USER_DEFINE_MSG] Exception while executing a message. java.lang.RuntimeException: Caught exception while replacing segment: mytable__0__0__20240125T2321Z, table: mytable_REALTIME msgId: fd0cf62e-f17e-4b64-a1e6-ec226f95ab6c type: USER_DEFINE_MSG java.lang.RuntimeException: Caught exception while replacing segment: mytable__0__0__20240125T2321Z, table: mytable_REALTIME at org.apache.pinot.segment.local.upsert.BasePartitionUpsertMetadataManager.doReplaceSegment(BasePartitionUpsertMetadataManager.java:431) ~[classes/:?] at org.apache.pinot.segment.local.upsert.BasePartitionUpsertMetadataManager.replaceSegment(BasePartitionUpsertMetadataManager.java:396) ~[classes/:?] at org.apache.pinot.core.data.manager.realtime.RealtimeTableDataManager.handleUpsert(RealtimeTableDataManager.java:564) ~[classes/:?] at org.apache.pinot.core.data.manager.realtime.RealtimeTableDataManager.addSegment(RealtimeTableDataManager.java:491) ~[classes/:?] at org.apache.pinot.core.data.manager.BaseTableDataManager.addOrReplaceSegment(BaseTableDataManager.java:537) ~[classes/:?] at org.apache.pinot.server.starter.helix.HelixInstanceDataManager.addOrReplaceSegment(HelixInstanceDataManager.java:502) ~[classes/:?] at org.apache.pinot.server.starter.helix.SegmentMessageHandlerFactory$SegmentRefreshMessageHandler.handleMessage(SegmentMessageHandlerFactory.java:103) ~[classes/:?] at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97) [helix-core-1.3.1.jar:1.3.1] at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49) [helix-core-1.3.1.jar:1.3.1] at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) [?:?] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) [?:?] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?] at java.base/java.lang.Thread.run(Thread.java:832) [?:?] Caused by: java.lang.IndexOutOfBoundsException at java.base/java.nio.Buffer.checkIndex(Buffer.java:737) ~[?:?] at java.base/java.nio.DirectByteBuffer.getInt(DirectByteBuffer.java:846) ~[?:?] at org.apache.pinot.segment.spi.memory.PinotByteBuffer.getInt(PinotByteBuffer.java:143) ~[classes/:?] at org.apache.pinot.segment.local.io.util.FixedByteValueReaderWriter.getInt(FixedByteValueReaderWriter.java:38) ~[classes/:?] at org.apache.pinot.segment.local.segment.index.readers.sorted.SortedIndexReaderImpl.getDictId(SortedIndexReaderImpl.java:83) ~[classes/:?] at org.apache.pinot.segment.local.segment.index.readers.sorted.SortedIndexReaderImpl.getDictId(SortedIndexReaderImpl.java:33) ~[classes/:?] at org.apache.pinot.segment.local.segment.readers.PinotSegmentColumnReader.getValue(PinotSegmentColumnReader.java:96) ~[classes/:?] at org.apache.pinot.segment.local.upsert.UpsertUtils.getValue(UpsertUtils.java:153) ~[classes/:?] at org.apache.pinot.segment.local.upsert.UpsertUtils$PrimaryKeyReader.getPrimaryKey(UpsertUtils.java:130) ~[classes/:?] at org.apache.pinot.segment.local.upsert.UpsertUtils$RecordInfoReader.getRecordInfo(UpsertUtils.java:101) ~[classes/:?] at org.apache.pinot.segment.local.upsert.UpsertUtils$1.next(UpsertUtils.java:55) ~[classes/:?] at org.apache.pinot.segment.local.upsert.UpsertUtils$1.next(UpsertUtils.java:45) ~[classes/:?] at org.apache.pinot.segment.local.upsert.ConcurrentMapPartitionUpsertMetadataManager.addOrReplaceSegment(ConcurrentMapPartitionUpsertMetadataManager.java:75) ~[classes/:?] at org.apache.pinot.segment.local.upsert.BasePartitionUpsertMetadataManager.replaceSegment(BasePartitionUpsertMetadataManager.java:473) ~[classes/:?] at org.apache.pinot.segment.local.upsert.BasePartitionUpsertMetadataManager.doReplaceSegment(BasePartitionUpsertMetadataManager.java:428) ~[classes/:?] ... 13 more ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
