Just re-read my post and I don't think it was clear. The algorithm I ended up doing is:
for all daily data gather hourly ids in a set build map with placeholders for hourly values get hourly documents from set for all daily data insert hourly data from documents fetched in placeholders So the method I showed below is the one to "get hourly documents from set" François François Eric wrote: > > Hello, > > I have a performance problem and would need expert advice on how to go > about fixing it: > > I currently have 2 indexes: Daily and Hourly. The Daily index contains > about 1,000,000 documents and my Hourly index approximately: 24,000,000 > documents. My Daily index contains many fields and some of them are IDs > to my Hourly Index. > What I want to do is fetch data in one request (if possible). > Right now I do it in many requests: > 1- Get the matching Daily documents (say it returns 500 documents) > 2- For each of these documents, locate the Hourly Index Id and fetch it. > > Therefore I make 501 requests to lucene. This causes some performance > issues I guess because of the overhead to making a request to Lucene. > > Is it possible to do this in 1 request? I'm thinking no because I'm not > sure what the result set would be but maybe I'm missing something. > > If not I guess it would be possible to build a query with my 500 hourly > ids and make a OR between them to make it in 2 requests....but then I > have to find the matching documents. Will this overflow if I have 50000 > ids in my query? > > Anyway, I just want advice on how one would address this situation. > > Thank you very much, > > François > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > -- View this message in context: http://old.nabble.com/Query-joining-2-indexes-tp26843980p26848479.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org