commitWithin UpdateProcessor for existing docs
Hi, We have a requirement to update existing docs live within say 30s. New docs updated depending on solrconfig autoSoftCommit. As our dev team are finding this difficult to implement within our middleware, I was thinking of writing an update processor as a 'post-processor' to add commitWithin (unless already set) so it runs on shard leader/replicas to add commitWithin as described above. It seems like a reasonable idea to have existing updates visible within X secs, wondering whether others have implemented something similar, or if this is inherently a flawed idea? Many thanks, Dan
Rollback or transation in SolrJ
Hi, We want to use rollback in SolrJ while autoSoftCommit and autoCommit are open ,and we expect it works like a transaction in databases,but we found all non-committed documents are rolled back after SolrClient#rollback called.Although the rolled back documents can not be searched after rollback ,after another document is added,they can be retrieved again. Does solr supports transations like databases? Many thanks, Tony
Reg Apache Solr 8.4.1 error log for Delta Import Handling functionality via REST command.
Hi , I am employing Java Web Project to incorporate Delta Import Handling functionality of Solr with my Collection data objects. I could do full-import of data to the configured collection via Solr UI, but let the Java Web App do a routine Delta-Import functionality instructed via Automated Java Thread, so as to make Collection data availability to downstream apps to function in an automated manner. But when I hit one of the collection by - *http://<1.2.3.4>:8983/solr/OBJECT/dataimport?command=delta-import&clean=false&commit=true&start=0&rows=10* But it now I am hitting this error - 2021-12-14 10:53:03.969 ERROR (qtp1546693040-33) [ x:customer] o.a.s.s.HttpSolrCall null:java.lang.ClassCastException: java.util.Arrays$ArrayList cannot be cast to java.lang.String at org.apache.solr.handler.dataimport.RequestInfo.(RequestInfo.java:65) at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:135) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2596) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:799) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:578) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:419) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:505) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917) at java.lang.Thread.run(Thread.java:748) Kindly share your thoughts/exploration on the above error. Is it an FATAL error or kind of any WARNING to neglect upon. Thanks, GPS
Is the embedded Solr ( also affected by the log4j2 vulnerability?
Is the embedded Solr also affected by the log4j2 vulnerability? If yes: does starting the embedded Solr server ( in a tomcat ) with -Dlog4j2.formatMsgNoLookups=true mitigate the issue alike? Our current Solr version is 8.8.2 Thx Clemens
Re: Is the embedded Solr ( also affected by the log4j2 vulnerability?
Hi, As long as you have configured your application having the embedded Solr, to use log4j for logging, and you pass user-entered queries to embedded Solr, then yes, you are vulnerable. And yes, setting that property in the JVM running (embedded) Solr should help. If your application uses another log framework, so that Solr logging is bridged through slf4j to e.g. Logback, then you may not be vulnerable. Look for log4j-core jar file. If you have a vulnerable log4j-core jar in your application, then upgrade log4j directly. Jan > 14. des. 2021 kl. 14:47 skrev clemens...@mysign.ch: > > Is the embedded Solr also affected by the log4j2 vulnerability? If yes: does > starting the embedded Solr server ( in a tomcat ) with > -Dlog4j2.formatMsgNoLookups=true mitigate the issue alike? > > Our current Solr version is 8.8.2 > > Thx > Clemens
Question Apache Solr 7.7.0, 8.7 and 8.9 - log4j vulnerability
Hello all We are using Apache Solr 7.7.0, 8.7 and 8.9 on Windows and Linux environment. What mitigation option do we need to take for this vulnerability? Thank you in advance. Regards Manisha Confidentiality Notice This email message, including any attachments, is for the sole use of the intended recipient and may contain confidential and privileged information. Any unauthorized view, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Anju Software, Inc. 4500 S. Lakeshore Drive, Suite 620, Tempe, AZ USA 85282.
Re: Question Apache Solr 7.7.0, 8.7 and 8.9 - log4j vulnerability
> On Dec 14, 2021, at 9:00 AM, Manisha Rahatadkar > wrote: > > We are using Apache Solr 7.7.0, 8.7 and 8.9 on Windows and Linux > environment. What mitigation option do we need to take for this vulnerability? https://solr.apache.org/security.html#apache-solr-affected-by-apache-log4j-cve-2021-44228
Re: Question Apache Solr 7.7.0, 8.7 and 8.9 - log4j vulnerability
When start Solr add to your env: SOLR_OPTS="$SOLR_OPTS -Dlog4j2.formatMsgNoLookups=true" On Tue, Dec 14, 2021 at 4:07 PM Andy Lester wrote: > > > > On Dec 14, 2021, at 9:00 AM, Manisha Rahatadkar > wrote: > > > > We are using Apache Solr 7.7.0, 8.7 and 8.9 on Windows and Linux > environment. What mitigation option do we need to take for this > vulnerability? > > > > https://solr.apache.org/security.html#apache-solr-affected-by-apache-log4j-cve-2021-44228 -- Vincenzo D'Amore
running Solr 6.6 with OpenJDK 11?
Hi list, we are running SolrCloud 6.6 and changing server (hardware). Now I'm thinking about upgrading from Oracle Java 8 to OpenJDK 11. Does Solr 6.6 run with OpenJDK 11, are there any known problems? Regards Bernd
Re: running Solr 6.6 with OpenJDK 11?
As far as I know changing the Java version does not solve the problem. https://twitter.com/Laughing_Mantis/status/1470412026119798786 On Tue, Dec 14, 2021 at 4:09 PM Bernd Fehling < bernd.fehl...@uni-bielefeld.de> wrote: > Hi list, > > we are running SolrCloud 6.6 and changing server (hardware). > Now I'm thinking about upgrading from Oracle Java 8 to OpenJDK 11. > > Does Solr 6.6 run with OpenJDK 11, are there any known problems? > > Regards Bernd > -- Vincenzo D'Amore
RE: Question Apache Solr 7.7.0, 8.7 and 8.9 - log4j vulnerability
Hello all We are using Apache Solr 7.7.0, 8.7 and 8.9 on Windows and Linux environment. What mitigation option do we need to take for this vulnerability? Where to get the log4j2? Can we just replace the log4j* files in solr-8.7.0\server\lib\ext folder? Will it work? https://solr.apache.org/security.html [cid:image001.png@01D7F0CC.79E0F6E0] Thank you in advance. Regards Manisha Confidentiality Notice This email message, including any attachments, is for the sole use of the intended recipient and may contain confidential and privileged information. Any unauthorized view, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Anju Software, Inc. 4500 S. Lakeshore Drive, Suite 620, Tempe, AZ USA 85282.
Re: Question Apache Solr 7.7.0, 8.7 and 8.9 - log4j vulnerability
You can download log4j at https://logging.apache.org/log4j/2.x/download.html When replacing the jar files, you will also need to restart your services. On Tue, Dec 14, 2021 at 9:30 AM Manisha Rahatadkar < manisha.rahatad...@anjusoftware.com> wrote: > Hello all > > > > We are using Apache Solr 7.7.0, 8.7 and 8.9 on Windows and Linux > environment. What mitigation option do we need to take for this > vulnerability? > > Where to get the log4j2? Can we just replace the log4j* files in > solr-8.7.0\server\lib\ext folder? Will it work? > > > > https://solr.apache.org/security.html > > > > > > Thank you in advance. > > > > Regards > > Manisha > > > > > > *Confidentiality Notice This email message, including > any attachments, is for the sole use of the intended recipient and may > contain confidential and privileged information. Any unauthorized view, > use, disclosure or distribution is prohibited. If you are not the intended > recipient, please contact the sender by reply email and destroy all copies > of the original message. Anju Software, Inc. 4500 S. Lakeshore Drive, Suite > 620, Tempe, AZ USA 85282.* >
Re: Rollback or transation in SolrJ
Rollback is not implemented for Solr Cloud clusters. We were using it for our master/slave clusters, but had to rewrite to use a different approach when we moved to Solr Cloud. No, Solr does not have transactions. The rollback just cancels all submitted updates (from any client) that have not been committed. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Dec 14, 2021, at 4:13 AM, zhta...@163.com wrote: > > Hi, >We want to use rollback in SolrJ while autoSoftCommit and autoCommit are > open ,and we expect it works like a transaction in databases,but we found all > non-committed documents are rolled back after SolrClient#rollback > called.Although the rolled back documents can not be searched after rollback > ,after another document is added,they can be retrieved again. > > Does solr supports transations like databases? > > > > Many thanks, > Tony
Re: running Solr 6.6 with OpenJDK 11?
On 2021-12-14 9:08 AM, Bernd Fehling wrote: Hi list, we are running SolrCloud 6.6 and changing server (hardware). Now I'm thinking about upgrading from Oracle Java 8 to OpenJDK 11. Does Solr 6.6 run with OpenJDK 11, are there any known problems? IIRC it wasn't recommended back when oracle EOL'ed Java 8 and we had to update (Solr 6.5). I forget whether it's untested or unsupported, but I ended up installing amazon coretto 1.8. Dima
Re: commitWithin UpdateProcessor for existing docs
On 12/14/21 3:46 AM, Dan Rosher wrote: We have a requirement to update existing docs live within say 30s. New docs updated depending on solrconfig autoSoftCommit. As our dev team are finding this difficult to implement within our middleware, I was thinking of writing an update processor as a 'post-processor' to add commitWithin (unless already set) so it runs on shard leader/replicas to add commitWithin as described above. It seems like a reasonable idea to have existing updates visible within X secs, wondering whether others have implemented something similar, or if this is inherently a flawed idea? It would be better to rely on autoSoftCommit than writing and incorporating custom code. Normally I would not suggest configuring autoSoftCommit with a maxTime so low, but if your index is small enough, and your server has enough memory, and you eliminate index/cache warming, you might be able to get commit speeds fast enough that a low autoSoftCommit interval will not cause problems. Generally speaking, 30 seconds is much too aggressive a goal for change visibility. But it might be achievable, depending on the use case. Thanks, Shawn
Re: running Solr 6.6 with OpenJDK 11?
On 12/14/21 8:08 AM, Bernd Fehling wrote: we are running SolrCloud 6.6 and changing server (hardware). Now I'm thinking about upgrading from Oracle Java 8 to OpenJDK 11. Does Solr 6.6 run with OpenJDK 11, are there any known problems? It probably will not work without at least some changes to the solr or solr.cmd script. Solr 7.0 was the first version to be qualified as compatible with Java 9, which was still new at the time 7.0 was released. The newer Java versions do things a little differently than Java 8 did, and I would expect Solr 6.x to have problems with anything newer than Java 8. If you want guaranteed compatibility with Java 11, your best bet is to upgrade to Solr 8.x. The 6.x versions are quite old. I know that such an upgrade is probably difficult to get approved. What I would recommend is installing something like OpenJDK 8. One of the replies mentioned Coretto. I have no experience with that, but it would probably work too. Thanks, Shawn
Re: Zookeeper and Solr and CVE-2021-44228
unsubscribe On Mon, Dec 13, 2021 at 8:53 AM Walter Underwood wrote: > Zookeeper 3.5.7 uses log4j 1.x, so is not vulnerable. I checked. > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > > > On Dec 13, 2021, at 6:20 AM, Michael Conrad wrote: > > > > I presume this also needs fixing for zookeeper nodes? > > > > On 12/10/21 13:44, Walter Underwood wrote: > >> Does all Solr logging go through slf4j? If so, that should protect > against this vulnerability. > >> > >> If not, who has tested Solr with log4j 2.15.1? > >> > >> We are running 8.8.2. > >> > >> wunder > >> Walter Underwood > >> wun...@wunderwood.org > >> http://observer.wunderwood.org/ (my blog) > >> > >> > >
Re: running Solr 6.6 with OpenJDK 11?
Hi Vincenzo, my question had nothing to do with log4j. Thanks, Bernd Am 14.12.21 um 16:14 schrieb Vincenzo D'Amore: As far as I know changing the Java version does not solve the problem. https://twitter.com/Laughing_Mantis/status/1470412026119798786 On Tue, Dec 14, 2021 at 4:09 PM Bernd Fehling < bernd.fehl...@uni-bielefeld.de> wrote: Hi list, we are running SolrCloud 6.6 and changing server (hardware). Now I'm thinking about upgrading from Oracle Java 8 to OpenJDK 11. Does Solr 6.6 run with OpenJDK 11, are there any known problems? Regards Bernd
Re: running Solr 6.6 with OpenJDK 11?
Hi Shawn, unfortunately upgrading from Solr 6.x to 8.x is currently no solution. I have several extensions and plugins developed like thesaurus for 26 European languages as selectable query extension, query pre- and post-processing and so on. Nevertheless, upgrading from Java 8 to OpenJDK 11 has an upgrade of Solr from 6.x to 8.x at a later time already in mind. Yes, Solr 6.x is quite old but rock solid and fast. And there were some complains on this list about degration of search speed with Solr 8.x. This would be also something which I have to check and do some testing with measurements. To get away from the Oracle License by switching from Java 8 to OpenJDK 8, do you have any observations or measurements? Thanks for your hints, Bernd Am 14.12.21 um 19:30 schrieb Shawn Heisey: On 12/14/21 8:08 AM, Bernd Fehling wrote: we are running SolrCloud 6.6 and changing server (hardware). Now I'm thinking about upgrading from Oracle Java 8 to OpenJDK 11. Does Solr 6.6 run with OpenJDK 11, are there any known problems? It probably will not work without at least some changes to the solr or solr.cmd script. Solr 7.0 was the first version to be qualified as compatible with Java 9, which was still new at the time 7.0 was released. The newer Java versions do things a little differently than Java 8 did, and I would expect Solr 6.x to have problems with anything newer than Java 8. If you want guaranteed compatibility with Java 11, your best bet is to upgrade to Solr 8.x. The 6.x versions are quite old. I know that such an upgrade is probably difficult to get approved. What I would recommend is installing something like OpenJDK 8. One of the replies mentioned Coretto. I have no experience with that, but it would probably work too. Thanks, Shawn
