Hello, An update, please. I am working through persistence of Lucene index using Ignite Dictionary, and will be asking some questions soon.
I had one doubt - - where does this change go? Ignite 3? Also, I know we want to build native support for text searches in Ignite 3. Is the work I am proposing here part of that, or will that be a separate effort? On Mon, 28 Jun 2021, 19:20 Ilya Kasnacheev, <ilya.kasnach...@gmail.com> wrote: > Hello! > > I think that number one is the most important one, then maybe it will see > more use and other deficiencies become more apparent, leading to more > tickets and visibility. > > Maybe 2. and 3. will even use a different approach when persistence is > implemented. > > Regards, > -- > Ilya Kasnacheev > > > пн, 28 июн. 2021 г. в 14:34, Atri Sharma <a...@apache.org>: > > > Hello Again! > > > > I have been looking into the aforementioned and here are my follow up > > thoughts: > > > > 1. Support persistence of Lucene indexes. > > 2. https://issues.apache.org/jira/browse/IGNITE-12401 (Needs fixing of > > moving partitions first) > > 3. Figure out how to return scores from nodes and use them as sort > > parameters on the coordinator node > > (https://issues.apache.org/jira/browse/IGNITE-12291) > > > > Please let me know if this looks ok to make text queries functional? > > > > Atri > > > > On Mon, Jun 21, 2021 at 2:49 PM Alexei Scherbakov > > <alexey.scherbak...@gmail.com> wrote: > > > > > > Hi. > > > > > > One of the biggest issues with text queries is a lack of support for > > lucene > > > indices persistence, which makes this functionality useless if a > > > persistence is enabled. > > > > > > I would first take care of it. > > > > > > пн, 21 июн. 2021 г. в 12:16, Maksim Timonin <timonin.ma...@gmail.com>: > > > > > > > Hi, Atri! > > > > > > > > You're right, Actually there is a lack of support for TextQueries. > For > > the > > > > last ticket I'm doing I see some obvious issues with them (no page > size > > > > support, for example). I'm glad that somebody wants to maintain this > > > > functionality. Thanks a lot! > > > > > > > > For the MergeSort algorithm there is already a patch for that [1]. > It's > > > > currently on review. This patch introduces an abstract reducer for > > > > CacheQueries with 2 implementations (unordered, merge-sort). Then > > TextQuery > > > > leverages on MergeSort to order results from multiple nodes by score. > > This > > > > patch also fixes the pageSize issue, I've mentioned before. Could you > > > > please check if it fully matches your idea? Any issues or comments > are > > > > welcome. > > > > > > > > I've prepared this ticket, because I need the MergeSort algorithm for > > the > > > > new type of queries I'm implementing (IndexQuery, it should also > > provide > > > > ordered results over multiple nodes). Currently I'm not planning to > go > > > > further with TextQuery, so if you're going to support this it'll be a > > great > > > > contribution, I think. > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-14703 > > > > [2] https://github.com/apache/ignite/pull/9081 > > > > > > > > > > > > On Mon, Jun 21, 2021 at 11:11 AM Atri Sharma <a...@apache.org> > wrote: > > > > > > > > > Hi All, > > > > > > > > > > I have been looking into our text queries support and see that it > has > > > > > limited community support. > > > > > > > > > > Therefore, I volunteer to be the maintainer of the module and work > on > > > > > enhancing it further. > > > > > > > > > > First goal would be to move to Lucene 8.x, then work on sorted > reduce > > > > > - merge across nodes. Fundamentally, this is doable since Lucene > > ranks > > > > > documents according to their score, and documents are returned in > the > > > > > order of their score. Since the scoring function is homogeneous, > this > > > > > means that across nodes, we can compare scores and merge sort. > > > > > > > > > > Please let me know if I can take this up. > > > > > > > > > > Atri > > > > > > > > > > -- > > > > > Regards, > > > > > > > > > > Atri > > > > > Apache Concerted > > > > > > > > > > > > > > > > > > -- > > > > > > Best regards, > > > Alexei Scherbakov > > > > -- > > Regards, > > > > Atri > > Apache Concerted > > >
