Thanks so much, David! I’d be happy to submit a PR if you point me in the right direction. And we’ll work on setting up a separate requestHandler for our advanced search in the meantime.
-Jane From: David Smiley <david.w.smi...@gmail.com> Date: Tuesday, August 1, 2023 at 10:52 AM To: users@solr.apache.org <users@solr.apache.org> Subject: Re: JSON boolean query syntax with edismax as default QueryParser I filed https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSOLR-16916&data=05%7C01%7Cjs7389%40princeton.edu%7C37b82c5c00014b65d45008db92b81c9a%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638265091687761654%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KXRIt3w61BIbUmnzdrtiMrir4z4EP8qIWJdSZ0TmyXc%3D&reserved=0<https://issues.apache.org/jira/browse/SOLR-16916> And I also see in a debugger what's going on; I can advise you on an internal fix if you wish to submit a PR. It'll be a one-liner, plus whatever is needed for a test. Until this is resolved, simply use another request handler just for JSON query DSL that does not change defType from it's default "lucene" value. ~ David On Tue, Aug 1, 2023 at 9:09 AM Jane Sandberg <js7...@princeton.edu> wrote: > defType=edismax is what we have historically used in our application. > We’re only trying to use the JSON query DSL for one feature (an advanced > search form), and are planning to continue using edismax for all the rest. > > -Jane > > From: David Smiley <david.w.smi...@gmail.com> > Date: Monday, July 31, 2023 at 8:53 PM > To: users@solr.apache.org <users@solr.apache.org> > Subject: Re: JSON boolean query syntax with edismax as default QueryParser > Why did you set defType=edismax in the /query handler in your > solrconfig.xml? > > edismax is for handling user queries -- text directly from a text box. > JSON Query DSL obviously isn't that. > > ~ David > > > On Mon, Jul 31, 2023 at 3:09 PM Jane Sandberg <js7...@princeton.edu> > wrote: > > > Hi David, > > > > Thanks for looking into this, and for the security fix. > > > > My colleague and I put together a small repository to reproduce the > > issue. It has a configset, a docker-compose file, and a README with the > > steps to reproduce it on solr 8.4: > > > https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpulibrary%2Fedismax-json-queries%2F&data=05%7C01%7Cjs7389%40princeton.edu%7C37b82c5c00014b65d45008db92b81c9a%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638265091687761654%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=psWXR5qOZ8KlZdS6U70jlonu7TTBGUL0vQZM%2FwCN10Y%3D&reserved=0<https://github.com/pulibrary/edismax-json-queries/> > <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpulibrary%2Fedismax-json-queries%2F&data=05%7C01%7Cjs7389%40princeton.edu%7C37b82c5c00014b65d45008db92b81c9a%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638265091687761654%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=psWXR5qOZ8KlZdS6U70jlonu7TTBGUL0vQZM%2FwCN10Y%3D&reserved=0<https://github.com/pulibrary/edismax-json-queries/>> > > > > Appreciatively, > > > > -Jane > > > > From: David Smiley <david.w.smi...@gmail.com> > > Date: Monday, July 31, 2023 at 5:58 AM > > To: users@solr.apache.org <users@solr.apache.org> > > Subject: Re: JSON boolean query syntax with edismax as default > QueryParser > > Hi Jane, > > > > That change in 7.2 does look like it's correlated, and I'm the one who > > implemented it in the name of security. > > > > Could you walk me through a short series of steps to show the problem > with > > one of Solr's "example" setups like techproducts? Step one is run it, > step > > two is you sending the query via curl. Use whatever Solr version you > want. > > > > ~ David > > > > > > On Fri, Jul 28, 2023 at 10:33 AM Jane Sandberg <js7...@princeton.edu> > > wrote: > > > > > Hi Solr colleagues, > > > > > > On Solr 8.4.1, we’ve noticed that the following types of JSON DSL > queries > > > work if our luceneMatchVersion is 7.1 or lower, or if our default query > > > parser is set to lucene: > > > > > > > > > > > > {"query":{"bool":{"must":[{"lucene":{"query":"plasticity","df":"title_a_index"}}]}}} > > > > > > However, if the query parser is set to edismax and the > luceneMatchVersion > > > is 7.2 or higher, the parsed query visible with debug=true becomes a > > > complete mess, searching for the terms “bool” and “must”, rather than > the > > > terms we actually want to search for: > > > > > > > > > +(DisjunctionMaxQuery(((author_main_unstem_search:bool)^1000.0 | > > > (local_subject_unstem_search:bool)^15.0 | > > (author_unstem_search:bool)^40.0 > > > […] > > > > > > Also while debug=true, we noticed that the JSON DSL queries get > converted > > > into a querystring with local params: ”{!bool must=$_tt1 }”. So I am > > > suspecting these two changes in Solr 7.2 as the reason we can’t use > > Boolean > > > JSON queries with edismax and a recent luceneMatchVersion: > > > > > > https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsolr.apache.org%2Fdocs%2F7_2_0%2Fchanges%2FChanges.html%23v7.2.0.upgrade_notes&data=05%7C01%7Cjs7389%40princeton.edu%7C37b82c5c00014b65d45008db92b81c9a%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638265091687761654%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ez4YRZ2PUg1CXqPb3yc97iWhXCaXFFsSrndb3%2FO2sFo%3D&reserved=0<https://solr.apache.org/docs/7_2_0/changes/Changes.html#v7.2.0.upgrade_notes> > < > https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsolr.apache.org%2Fdocs%2F7_2_0%2Fchanges%2FChanges.html%23v7.2.0.upgrade_notes&data=05%7C01%7Cjs7389%40princeton.edu%7C37b82c5c00014b65d45008db92b81c9a%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638265091687761654%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ez4YRZ2PUg1CXqPb3yc97iWhXCaXFFsSrndb3%2FO2sFo%3D&reserved=0<https://solr.apache.org/docs/7_2_0/changes/Changes.html#v7.2.0.upgrade_notes> > > > > < > > > https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsolr.apache.org%2Fdocs%2F7_2_0%2Fchanges%2FChanges.html%23v7.2.0.upgrade_notes&data=05%7C01%7Cjs7389%40princeton.edu%7C37b82c5c00014b65d45008db92b81c9a%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638265091687761654%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ez4YRZ2PUg1CXqPb3yc97iWhXCaXFFsSrndb3%2FO2sFo%3D&reserved=0<https://solr.apache.org/docs/7_2_0/changes/Changes.html#v7.2.0.upgrade_notes> > < > https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsolr.apache.org%2Fdocs%2F7_2_0%2Fchanges%2FChanges.html%23v7.2.0.upgrade_notes&data=05%7C01%7Cjs7389%40princeton.edu%7C37b82c5c00014b65d45008db92b81c9a%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638265091687761654%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ez4YRZ2PUg1CXqPb3yc97iWhXCaXFFsSrndb3%2FO2sFo%3D&reserved=0<https://solr.apache.org/docs/7_2_0/changes/Changes.html#v7.2.0.upgrade_notes> > > > > >. > > > Does that seem correct? > > > > > > Also, could this be related to the question Benjamin Armintor asked on > > > June 23 (subject: Changes to JSON query API/syntax in Solr 9.x?)? I’m > > > specifically curious about whether a luceneMatchVersion of 7.1 or lower > > > still works in Solr 9? > > > > > > Thanks for your insights, > > > > > > -Jane > > > > > > -- > > > Jane Sandberg (she/her) > > > Library Software Engineer, Discovery and Access Services > > > > > >
