To be clear, the "field" is indeed tokenized, which is accompanied with a SortedDocValueField so that it is sortable too. Am I making the wrong assumption here ?
On Wednesday, July 6, 2016, Sheng <sheng...@gmail.com> wrote: > Hi Eric, > > I am refactoring a legacy system. One of the most annoying things is I > have to keep the old feature even though it makes little sense. In this > case, we have to index a particular data structure which has bunch of > fields and each of them is promised to be searchable and search-sortable to > the user. Turns out one field is notoriously large. I think the old > implementation uses some quite clumsy way to make it happen. But since we > decide to refactor the system with all the goodies from Lucene, we want to > do the sorting right, and here we are at this issue... :-( > > On Wednesday, July 6, 2016, Erick Erickson <erickerick...@gmail.com > <javascript:_e(%7B%7D,'cvml','erickerick...@gmail.com');>> wrote: > >> Is this an "XY" problem? Meaning, why do you need DV fields larger than >> 32K? >> >> You can't search it as text as it's not tokenized. Faceting and sorting >> by a 32K >> field doesn't seem very useful. You may have a perfectly valid reason, >> but it's >> not obvious what use-case you're serving from this thread so far.... >> >> Nobody has yet put forth a compelling use-case for such large fields, >> perhaps >> this would be one. >> >> Best, >> Erick >> >> On Wed, Jul 6, 2016 at 2:24 PM, Sheng <sheng...@gmail.com> wrote: >> > Mike - Thanks for the prompt response. Is there a way to bypass this >> > constraint for SortedDocValueField ? Or we have to live with it, >> meaning no >> > fix even in future release? >> > >> > On Wednesday, July 6, 2016, Michael McCandless < >> luc...@mikemccandless.com> >> > wrote: >> > >> >> I believe only binary DVs can be larger than 32K bytes. >> >> >> >> Mike McCandless >> >> >> >> http://blog.mikemccandless.com >> >> >> >> On Wed, Jul 6, 2016 at 10:31 AM, Sheng <sheng...@gmail.com >> <javascript:;>> >> >> wrote: >> >> >> >> > Hi, >> >> > >> >> > I am getting an IAE indicating one of the SortedDocValueField is too >> >> large, >> >> > > 32k >> >> > >> >> > I googled a bit, and it seems like #Lucene-4583 has addressed this >> issue >> >> in >> >> > 4.5 and 6.0, while I am currently using Lucene 6.1. Do I miss or >> >> > misunderstand anything ? >> >> > >> >> > Thanks, >> >> > >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >>
