leekeiabstraction opened a new pull request, #2707: URL: https://github.com/apache/fluss/pull/2707
### Purpose Linked issue: close #2706 Fix Lance writer to emit Arrow FixedSizeList for array columns to enable native vector search. Querying with pylance native vector search fails because pylance expects embedding column to be of fixedSizeList type instead of variable size list. This PR fixes the conversion from Fluss' arrow data to Lance's arrow data by using FixedSizeList if `<column>.arrow.fixed-size-list.size` is defined, similar to how Spark SQL does it ref: https://lance.org/integrations/spark/operations/ddl/create-table/#creating-large-string-columns ### Tests Added unit test - Tested manually that pylance successfully performed a vector search on the tiered data -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
