Vladimir, Is my understanding right you suggest to create a custom Lucene codec which will be able to store index in ignite page memory?
Lucene javadoc has complete description of codec API [1] and looks like it is possible to implement custom one. [1] https://lucene.apache.org/core/5_5_2/core/org/apache/lucene/codecs/lucene54/package-summary.html#package_description On Wed, Aug 2, 2017 at 12:02 AM, Denis Magda <dma...@apache.org> wrote: > Andrey, > > What I’m trying to say is that presently both text and geo-spatial indexes > are unsupported NOT only for the persistence layer but for the durable > (page) memory in general - as you properly pointed out the text indexes are > stored in the old off-heap memory but has to be moved to be handled in page > memory’s off-heap region directly. > > Makes sense? > > — > Denis > > > On Aug 1, 2017, at 1:05 PM, Andrey Mashenkov <andrey.mashen...@gmail.com> > wrote: > > > > Denis, > > > > I see we still use GridUnsafeMemory in LuceneIndex [1] in master. > > Do you mean a wrapper to new memory manager is used here? > > > > [1] > > https://github.com/apache/ignite/blob/master/modules/ > indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/ > GridLuceneIndex.java > > > > On Tue, Aug 1, 2017 at 10:28 PM, Denis Magda <dma...@apache.org> wrote: > > > >> Andrey, > >> > >> I’ve already wiped the old off-heap memory out of my mind. Sure I > assumed > >> the off-heap space managed by the new memory architecture. > >> > >> — > >> Denis > >> > >>> On Aug 1, 2017, at 11:14 AM, Andrey Mashenkov < > >> andrey.mashen...@gmail.com> wrote: > >>> > >>> Denis, > >>> > >>> Lucene fullText index stores off-heap in old way, not in page memory. > >>> Therefore it is not persistent. > >>> > >>> There is a second issue related to FullText index as it is different > kind > >>> of index and it has own query type and it can't be use in SQL queries. > >>> Looks like it make sense to integrate FullText indices in our SQL layer > >> as > >>> well. > >>> > >>> AFAIK, H2 has some support of FullText indices based on Lucene, so we > can > >>> get a hint how we can integrate it. > >>> > >>> > >>> > >>> On Tue, Aug 1, 2017 at 8:09 PM, Denis Magda <dma...@apache.org> wrote: > >>> > >>>> Vladimir, > >>>> > >>>> We need to consider that these two types of indexes are not stored > >>>> off-heap either. > >>>> > >>>> It expands the task a bit — the indexes have to be fully integrated > with > >>>> the new durable memory architecture supporting both off-heap and > >>>> persistence layers. > >>>> > >>>> — > >>>> Denis > >>>> > >>>>> On Aug 1, 2017, at 3:26 AM, Vladimir Ozerov <voze...@gridgain.com> > >>>> wrote: > >>>>> > >>>>> Guys, > >>>>> > >>>>> AFAIK these two index types are not supported with enabled > persistence > >> at > >>>>> the moment, neither they stored on the disk anyhow. Can someone help > >> with > >>>>> estimates on how difficult would it be to implement these indexes > over > >>>>> page-memory architecture? > >>>>> > >>>>> Looks like we will have to write our own implementation of these > >> indexes, > >>>>> instead of relying on Lucene and H2. Am I right? > >>>>> > >>>>> Vladimir. > >>>> > >>>> > >>> > >>> > >>> -- > >>> Best regards, > >>> Andrey V. Mashenkov > >> > >> > > > > > > -- > > Best regards, > > Andrey V. Mashenkov > > -- Best regards, Andrey V. Mashenkov
