huaxingao commented on code in PR #395:
URL: https://github.com/apache/datafusion-comet/pull/395#discussion_r1602271900
##########
common/src/main/java/org/apache/comet/vector/CometPlainVector.java:
##########
@@ -111,7 +115,12 @@ public UTF8String getUTF8String(int rowId) {
byte[] result = new byte[length];
Platform.copyMemory(
null, valueBufferAddress + offset, result,
Platform.BYTE_ARRAY_OFFSET, length);
- return UTF8String.fromBytes(result);
+
+ if (!isUuid) {
+ return UTF8String.fromBytes(result);
+ } else {
+ return UTF8String.fromString(convertToUuid(result).toString());
+ }
Review Comment:
I tried
```
val schemaStr =
"""
|message root {
| required int32 id;
| required FIXED_LEN_BYTE_ARRAY(16) uuid (UUID);
|}
""".stripMargin
```
I got
```
Fixed Binary size 36 does not match field type length 16
java.lang.IllegalArgumentException: Fixed Binary size 36 does not match
field type length 16
at
org.apache.parquet.column.values.plain.FixedLenByteArrayPlainValuesWriter.writeBytes(FixedLenByteArrayPlainValuesWriter.java:56)
at
org.apache.parquet.column.impl.ColumnWriterBase.write(ColumnWriterBase.java:241)
at
org.apache.parquet.io.MessageColumnIO$MessageColumnIORecordConsumer.addBinary(MessageColumnIO.java:476)
at
org.apache.parquet.example.data.simple.BinaryValue.writeValue(BinaryValue.java:45)
at
org.apache.parquet.example.data.simple.SimpleGroup.writeValue(SimpleGroup.java:233)
at
org.apache.parquet.example.data.GroupWriter.writeGroup(GroupWriter.java:51)
at
org.apache.parquet.example.data.GroupWriter.write(GroupWriter.java:37)
at
org.apache.parquet.hadoop.example.GroupWriteSupport.write(GroupWriteSupport.java:85)
at
org.apache.parquet.hadoop.example.GroupWriteSupport.write(GroupWriteSupport.java:35)
at
org.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:138)
at org.apache.parquet.hadoop.ParquetWriter.write(ParquetWriter.java:310)
at
org.apache.spark.sql.CometTestBase.$anonfun$createParquetFileWithIDAndUUID$1(CometTestBase.scala:482)
```
It has to be `FIXED_LEN_BYTE_ARRAY(16) uuid (UUID)`, otherwise I will get
```
UUID can only annotate FIXED_LEN_BYTE_ARRAY(16)
java.lang.IllegalStateException: UUID can only annotate
FIXED_LEN_BYTE_ARRAY(16)
at org.apache.parquet.Preconditions.checkState(Preconditions.java:196)
at
org.apache.parquet.schema.Types$BasePrimitiveBuilder$1.checkFixedPrimitiveType(Types.java:556)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
