Hi Ere Thanks for the help on this. I have raised SOLR-15246 to cover this.
Many thanks Matthew Matthew Flowerday | Consultant | ULEAF Unisys | 01908 774830| matthew.flower...@unisys.com Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes | MK17 8LX THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all devices. -----Original Message----- From: Ere Maijala <ere.maij...@helsinki.fi> Sent: 04 March 2021 10:20 To: solr-u...@lucene.apache.org Subject: Re: Potential Slow searching for unified highlighting on Solr 8.8.0/8.8.1 EXTERNAL EMAIL - Be cautious of all links and attachments. Hi, Solr uses JIRA for issue tickets. You can find it here: https://issues.apache.org/jira/browse/SOLR I'd suggest filing a new bug issue in the SOLR project (note that several other projects also use this JIRA installation). Here's an example of an existing highlighter issue for reference: https://issues.apache.org/jira/browse/SOLR-14019. See also some brief documentation: https://cwiki.apache.org/confluence/display/solr/HowToContribute#HowToContri bute-JIRAtips(ourissue/bugtracker) Regards, Ere Flowerday, Matthew J kirjoitti 1.3.2021 klo 14.58: > Hi Ere > > Please to be of service! > > No I have not filed a JIRA ticket. I am new to interacting with the > Solr Community and only beginning to 'find my legs'. I am not too sure > what JIRA is I am afraid! > > Regards > > Matthew > > Matthew Flowerday | Consultant | ULEAF Unisys | 01908 774830| > matthew.flower...@unisys.com Address Enigma | Wavendon Business Park | > Wavendon | Milton Keynes | MK17 8LX > > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE > PROPRIETARY MATERIAL and is for use only by the intended recipient. If > you received this in error, please contact the sender and delete the > e-mail and its attachments from all devices. > > > -----Original Message----- > From: Ere Maijala <ere.maij...@helsinki.fi> > Sent: 01 March 2021 12:53 > To: solr-u...@lucene.apache.org > Subject: Re: Potential Slow searching for unified highlighting on Solr > 8.8.0/8.8.1 > > EXTERNAL EMAIL - Be cautious of all links and attachments. > > Hi, > > Whoa, thanks for the heads-up! You may just have saved me from a whole > lot of trouble. Did you file a JIRA ticket already? > > Thanks, > Ere > > Flowerday, Matthew J kirjoitti 1.3.2021 klo 14.00: >> Hi There >> >> I just came across a situation where a unified highlighting search >> under solr 8.8.0/8.8.1 can take over 20 mins to run and eventually >> times > out. >> I resolved it by a config change but it can catch you out. Hence >> this email. >> >> With solr 8.8.0 a new unified highlighting parameter >> &hl.fragAlignRatio was implemented which if not set defaults to 0.5. >> This attempts to improve the high lighting so that highlighted text >> does not appear right at the left. This works well but if you have a >> search result with numerous occurrences of the word in question >> within the record performance goes right down! >> >> 2021-02-27 06:45:03.151 INFO (qtp762476028-20) [ x:uleaf] >> o.a.s.c.S.Request [uleaf] webapp=/solr path=/select >> params={hl.snippets=2&q=test&hl=on&hl.maxAnalyzedChars=1000000&fl=id, >> d >> escription,specification,score&start=20&hl.fl=*&rows=10&_=16144051191 >> 3 >> 4} >> hits=57008 status=0 QTime=1414320 >> >> 2021-02-27 06:45:03.245 INFO (qtp762476028-20) [ x:uleaf] >> o.a.s.s.HttpSolrCall Unable to write response, client closed >> connection or we are shutting down => >> org.eclipse.jetty.io.EofException >> >> at >> org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279) >> >> org.eclipse.jetty.io.EofException: null >> >> at >> org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279) >> ~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102] >> >> at >> org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422) >> ~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102] >> >> at >> org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:378 >> ) ~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102] >> >> when I set &hl.fragAlignRatio=0.25 results came back much quicker >> >> 2021-02-27 14:59:57.189 INFO (qtp1291367132-24) [ x:holmes] >> o.a.s.c.S.Request [holmes] webapp=/solr path=/select >> params={hl.weightMatches=false&hl=on&fl=id,description,specification, >> s >> core&start=1&hl.fragAlignRatio=0.25&rows=100&hl.snippets=2&q=test&hl. >> m >> axAnalyzedChars=1000000&hl.fl=*&hl.method=unified&timeAllowed=90000&_ >> = >> 1614430061690} >> hits=136939 status=0 QTime=87024 >> >> And &hl.fragAlignRatio=0.1 >> >> 2021-02-27 15:18:45.542 INFO (qtp1291367132-19) [ x:holmes] >> o.a.s.c.S.Request [holmes] webapp=/solr path=/select >> params={hl.weightMatches=false&hl=on&fl=id,description,specification, >> s >> core&start=1&hl.fragAlignRatio=0.1&rows=100&hl.snippets=2&q=test&hl.m >> a >> xAnalyzedChars=1000000&hl.fl=*&hl.method=unified&timeAllowed=90000&_= >> 1 >> 614430061690} >> hits=136939 status=0 QTime=69033 >> >> And &hl.fragAlignRatio=0.0 >> >> 2021-02-27 15:20:38.194 INFO (qtp1291367132-24) [ x:holmes] >> o.a.s.c.S.Request [holmes] webapp=/solr path=/select >> params={hl.weightMatches=false&hl=on&fl=id,description,specification, >> s >> core&start=1&hl.fragAlignRatio=0.0&rows=100&hl.snippets=2&q=test&hl.m >> a >> xAnalyzedChars=1000000&hl.fl=*&hl.method=unified&timeAllowed=90000&_= >> 1 >> 614430061690} >> hits=136939 status=0 QTime=2841 >> >> I left our setting at 0.0 this presumably how it was in 7.7.1 >> (fully left aligned). I am not too sure as to how many time a word >> has to occur in a record for performance to go right down but if >> too many it can have a BIG impact. >> >> I also noticed that setting &timeAllowed=90000 did not break out of >> the query until it finished. Perhaps because the query finished >> quickly and what took the time was the highlighting. It might be an >> idea to get &timeAllowed to also cover any highlighting so that the >> query does not run until the jetty timeout is hit. The machine 100% >> one core for about >> 20 mins!. >> >> Hope this helps. >> >> Regards >> >> Matthew >> >> *Matthew Flowerday*| Consultant | ULEAF >> >> Unisys | 01908 774830| matthew.flower...@unisys.com >> <mailto:matthew.flower...@unisys.com> >> >> Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes | >> MK17 8LX >> >> unisys_logo <http://www.unisys.com/> >> >> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE >> PROPRIETARY MATERIAL and is for use only by the intended recipient. >> If you received this in error, please contact the sender and delete >> the e-mail and its attachments from all devices. >> >> Grey_LI <http://www.linkedin.com/company/unisys>Grey_TW >> <http://twitter.com/unisyscorp>Grey_YT >> <http://www.youtube.com/theunisyschannel>Grey_FB >> <http://www.facebook.com/unisyscorp>Grey_Vimeo >> <https://vimeo.com/unisys>Grey_UB <http://blogs.unisys.com/> >> > > -- > Ere Maijala > Kansalliskirjasto / The National Library of Finland > -- Ere Maijala Kansalliskirjasto / The National Library of Finland
smime.p7s
Description: S/MIME cryptographic signature
