andygrove commented on issue #1115:
URL:
https://github.com/apache/datafusion-comet/issues/1115#issuecomment-2495541883
I refactored the code to make separate FFI calls for exporting schema and
array and measured the time of each:
```scala
val arrowSchema = ArrowSchema.wrap(schemaAddrs(index))
val arrowArray = ArrowArray.wrap(arrayAddrs(index))
val export = getFieldVector(valueVector, "export")
// export schema
val t1 = System.nanoTime()
Data.exportField(allocator, export.getField, provider, arrowSchema)
val t2 = System.nanoTime()
// export array
Data.exportVector(allocator, export, provider, arrowArray)
val t3 = System.nanoTime()
// scalastyle:off println
println(s"Exported schema in ${t2 - t1} ns and array in ${t3 - t2}
ns")
```
Exporting the schema seems to be more expensive than exporting the data. It
seems like this would be worth optimizing.
```
Exported schema in 1773 ns and array in 421 ns
Exported schema in 1402 ns and array in 401 ns
Exported schema in 1704 ns and array in 891 ns
Exported schema in 1884 ns and array in 351 ns
Exported schema in 1232 ns and array in 1072 ns
Exported schema in 1553 ns and array in 450 ns
Exported schema in 1923 ns and array in 1032 ns
Exported schema in 1392 ns and array in 1663 ns
Exported schema in 481 ns and array in 561 ns
Exported schema in 1022 ns and array in 551 ns
Exported schema in 1082 ns and array in 942 ns
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
