Hi Shengkai, Thanks for the FLIP, this will be a great addition to flink AI capabilities. +1 for this feature.
Best, Yash Anand On Tue, Jul 29, 2025 at 7:23 PM Jacky Lau <liuyong...@gmail.com> wrote: > Hi Shengkai, > > Thanks for the FLIP and enhancement for AI capabilities in Flink. +1 for > this feature > > Best, > Jacky Lau > > Hao Li <h...@confluent.io.invalid> 于2025年7月30日周三 01:03写道： > > > Hi Shengkai, > > > > Thanks for the FLIP and enhancement for AI capabilities in Flink. +1. > > > > Thanks, > > Hao > > > > On Tue, Jul 29, 2025 at 2:16 AM Shengkai Fang <fskm...@gmail.com> wrote: > > > > > Hi, > > > I'd like to start a discussion of FLIP-540: Support VECTOR_SEARCH in > > Flink > > > SQL[1]. > > > > > > In FLIP-437/FLIP-525, Apache Flink has initially integrated Large > > Language > > > Model (LLM) capabilities, enabling semantic understanding and real-time > > > processing of streaming data pipelines. This integration has been > > > technically validated in scenarios such as log classification and > > real-time > > > question-answering systems. However, the current architecture allows > > Flink > > > to only use embedding models to convert unstructured data (e.g., text, > > > images) into high-dimensional vector features, which are then persisted > > to > > > downstream storage systems (e.g., Milvus, Mongodb). It lacks real-time > > > online querying and similarity analysis capabilities for vector spaces. > > To > > > address this limitation, we propose introducing the VECTOR_SEARCH > > function > > > in this FLIP, enabling users to perform streaming vector similarity > > > searches and real-time context retrieval (e.g., Retrieval-Augmented > > > Generation, RAG) directly within Flink. > > > > > > Looking forward to comments and suggestions for improvements! > > > > > > Best, > > > Shengkai > > > > > > [1] > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-540%3A+Support+VECTOR_SEARCH+in+Flink+SQL > > > > > >
