Re: Aligning Shards from different Collections on the same Solr server based on Date Range

2021-07-14 Thread Matt Kuiper
Thanks Joel! I will give this a try. That is quite a performance boost. Matt On Tue, Jul 13, 2021 at 9:14 AM Joel Bernstein wrote: > The optimized join was added in Solr 8.8: > https://issues.apache.org/jira/browse/SOLR-15049 > > It kicks in when you use the join qparser plugin in the followi

Re: Aligning Shards from different Collections on the same Solr server based on Date Range

2021-07-13 Thread Joel Bernstein
The optimized join was added in Solr 8.8: https://issues.apache.org/jira/browse/SOLR-15049 It kicks in when you use the join qparser plugin in the following scenario: 1) Do not specify a fromIndex. This is because the to and from index are the same. 2) The to and from fields are the same. 3) The

Re: Aligning Shards from different Collections on the same Solr server based on Date Range

2021-07-12 Thread Matt Kuiper
Hi Joel, I reviewed a few options with my team, and your recommendation is at the top of the list. I believe it will work for our use case. You mentioned that if this approach worked, you would be willing to share more details on an "optimized self join." I would enjoy hearing more. Thanks, Ma

Re: Aligning Shards from different Collections on the same Solr server based on Date Range

2021-07-09 Thread Joel Bernstein
Block join is another option. If that works for you, from an indexing standpoint, it's the most performant query time join. If block indexing doesn't work for you then the optimized self join is almost as fast. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jul 9, 2021 at 11:31 AM Matt K

Re: Aligning Shards from different Collections on the same Solr server based on Date Range

2021-07-09 Thread Matt Kuiper
Thanks Joel! On my list is to investigate Block Joins and Nested Child docs. https://solr.apache.org/guide/8_8/other-parsers.html#block-join-query-parsers https://solr.apache.org/guide/8_8/indexing-nested-documents.html#indexing-nested-documents However, it looks like you are not suggesting usi

Re: Aligning Shards from different Collections on the same Solr server based on Date Range

2021-07-09 Thread Joel Bernstein
Can you solve this problem by adding all documents into the same collection and performing self joins. You could add a field called rec_type to differentiate between the records. There are two good reasons for wanting to do this. 1) This allows you to route by the join key and easily co-locate re

Re: Aligning Shards from different Collections on the same Solr server based on Date Range

2021-07-02 Thread Matt Kuiper
After some research, it appears the following approach may help in this situation and relieve the requirement of collocating indexes for Joins. It appears one drawback maybe the types of fields supported for the JOIN field. https://solr.apache.org/guide/8_8/other-parsers.html#cross-collection-joi

Aligning Shards from different Collections on the same Solr server based on Date Range

2021-06-30 Thread Matt Kuiper
Hi Solr Group, I am not sure the following is a viable use-case, welcoming input and any implementation recommendations. I would like to perform joins over two sharded collections. Where docs are routed to specific shards based on a date range and are the same for shards in each collection. I u