xiangfu0 opened a new issue, #12762:
URL: https://github.com/apache/pinot/issues/12762
Found this issue while adding a new column and trying to fill it up using a
transform function.
Segment reload/ingestion may fail due to the null value.
```
2024/03/30 21:37:46.805 ERROR [BaseDefaultColumnHandler]
[HelixTaskExecutor-message_handle_thread_39] Caught exception while creating
derived column: int_converted_day1_date with transform function:
DATETIMECONVERT(day1_date, '1:DAYS:SIMPLE_DATE_FORMAT:yyyy-MM-dd',
'1:DAYS:EPOCH', '1:DAYS')
java.lang.RuntimeException: Caught exception while executing function:
dateTimeConvert(day1_date,'1:DAYS:SIMPLE_DATE_FORMAT:yyyy-MM-dd','1:DAYS:EPOCH','1:DAYS')
at
org.apache.pinot.segment.local.function.InbuiltFunctionEvaluator$FunctionExecutionNode.execute(InbuiltFunctionEvaluator.java:262)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.segment.local.function.InbuiltFunctionEvaluator.evaluate(InbuiltFunctionEvaluator.java:119)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.segment.local.segment.index.loader.defaultcolumn.BaseDefaultColumnHandler.createDerivedColumnV1Indices(BaseDefaultColumnHandler.java:580)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.segment.local.segment.index.loader.defaultcolumn.BaseDefaultColumnHandler.createColumnV1Indices(BaseDefaultColumnHandler.java:405)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.segment.local.segment.index.loader.defaultcolumn.V3DefaultColumnHandler.updateDefaultColumn(V3DefaultColumnHandler.java:58)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.segment.local.segment.index.loader.defaultcolumn.BaseDefaultColumnHandler.updateDefaultColumns(BaseDefaultColumnHandler.java:165)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.segment.local.segment.index.loader.SegmentPreProcessor.process(SegmentPreProcessor.java:99)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader.preprocessSegment(ImmutableSegmentLoader.java:267)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader.preprocess(ImmutableSegmentLoader.java:142)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.core.data.manager.BaseTableDataManager.tryLoadExistingSegment(BaseTableDataManager.java:849)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.core.data.manager.BaseTableDataManager.addOrReplaceSegment(BaseTableDataManager.java:495)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.server.starter.helix.HelixInstanceDataManager.addOrReplaceSegment(HelixInstanceDataManager.java:505)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeOnlineFromOffline(SegmentOnlineOfflineStateModelFactory.java:176)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
~[?:?]
at java.lang.reflect.Method.invoke(Method.java:580) ~[?:?]
at
org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:350)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:278)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
~[?:?]
at java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: java.lang.IllegalStateException: Caught exception while invoking
method: public java.lang.Object
org.apache.pinot.common.function.scalar.DateTimeConvert.dateTimeConvert(java.lang.String,java.lang.String,java.lang.String,java.lang.String)
with arguments: [null, 1:DAYS:SIMPLE_DATE_FORMAT:yyyy-MM-dd, 1:DAYS:EPOCH,
1:DAYS]
at
org.apache.pinot.common.function.FunctionInvoker.invoke(FunctionInvoker.java:142)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.segment.local.function.InbuiltFunctionEvaluator$FunctionExecutionNode.execute(InbuiltFunctionEvaluator.java:260)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
... 22 more
Caused by: java.lang.reflect.InvocationTargetException
at
jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:118)
~[?:?]
at java.lang.reflect.Method.invoke(Method.java:580) ~[?:?]
at
org.apache.pinot.common.function.FunctionInvoker.invoke(FunctionInvoker.java:139)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.segment.local.function.InbuiltFunctionEvaluator$FunctionExecutionNode.execute(InbuiltFunctionEvaluator.java:260)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
... 22 more
Caused by: java.lang.IllegalArgumentException: Invalid format: "null"
at
org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:186)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:834)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.spi.data.DateTimeFormatSpec.fromFormatToMillis(DateTimeFormatSpec.java:310)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.common.function.scalar.DateTimeConvert.dateTimeConvert(DateTimeConvert.java:47)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
~[?:?]
at java.lang.reflect.Method.invoke(Method.java:580) ~[?:?]
at
org.apache.pinot.common.function.FunctionInvoker.invoke(FunctionInvoker.java:139)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
at
org.apache.pinot.segment.local.function.InbuiltFunctionEvaluator$FunctionExecutionNode.execute(InbuiltFunctionEvaluator.java:260)
~[startree-pinot-all-1.1.0-ST.19-jar-with-dependencies.jar:1.1.0-ST.19-538c2536602ade97dea3c22dbcb5715092d0823e]
... 22 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
