lines in the past:
6.0.0 announced on 2016/04/08
7.0.0 announced on 2017/09/20
8.0.0 announced on 2019/03/13
9.0.0 announced on 2022/05/12
Which means there isn't a lot of predictability to major releases. The
last one was 3 years, but the previous two were only about 1.5 years.
Thanks,
Shawn
side code,
not included in SolrJ.
In almost all cases, the data types used in SolrJ code should be the
most primitive class or variable type you can find to represent the data
... String, int, Integer, etc.
Thanks,
Shawn
configset.
3) Upload an entire configset.
Thanks,
Shawn
resulting log entries will end up in solr.log with the
rest of Solr's logging.
Thanks,
Shawn
lity and/or multiple replicas, this isn't recommended.
Thanks,
Shawn
On 8/22/2024 17:55, Shawn Heisey wrote:
Then you will need these commands, starting from the root of the checkout:
ant ivy-bootstrap
cd solr
ant clean package
I can confirm that Java 17 does NOT work.
I installed Java 11 (it was missing on my Windows 11 machine) and added
this command
On 8/22/2024 17:40, Shawn Heisey wrote:
On 8/22/2024 11:38, Raju Vaddeh wrote:
Below are the steps we have performed on the local solr environment
for the patch install.
*1. Clone the solr repo*
git clone GitHub - apache/solr: Apache Solr open-source search
software <https://github.
if not the latest 9.x
version. Version 8.1.1 was released five years ago.
Thanks,
Shawn
cured had exception logging. Solr has a lot of
code without any logging. Now, the cause of the OOME is always logged
in Java's error file.
That said, you are free to implement your own auto-restart mechanism.
Thanks,
Shawn
te project.
Thanks,
Shawn
ncer, as such a client always knows the state
of the whole cluster.
Thanks,
Shawn
nd haproxy. Solr is on port 8983 without TLS, and
this port is not accessible via the Internet. The haproxy config only
lets a very small number of public IP addresses talk to solr on 443.
Solr is running on an AWS instance.
Thanks,
Shawn
o. It would also mean that most of the
host memory is being allocated to other programs and so the host memory
might simply be too small for the workload it runs. As this system
grows, you will probably need to increase the host memory beyond 4gb.
Thanks,
Shawn
t from the docker host, you would use
"http://localhost:32774"; to reach the admin UI.
Thanks,
Shawn
configured correctly even if it
does start ... and chances are that it won't even start.
The installation script creates the solr user without a working
password, not even a blank password.
Thanks,
Shawn
at any given moment. ZK is used as
a central database to coordinate the cluster, it does not handle requests.
Thanks,
Shawn
match an asterisk
character, you have to escape it.
Thanks,
Shawn
', version:
'10+'
implementation group: 'org.eclipse.jetty', name: 'jetty-client',
version: '10+'
implementation group: 'org.eclipse.jetty', name: 'jetty-alpn-client',
version: '10+'
implementation group: 'org.eclipse.jetty', name:
'jetty-alpn-java-client', version: '10+'
}
I don't know if using vanilla SolrJ is even going to be possible in
Spring Boot 3 until Solr gets upgraded to Jetty 12. That is a question
you should ask Spring.
Thanks,
Shawn
might live.
Note that when you DO upgrade Solr, you're not going to be able to use
your existing index in 6.x and later. It will need to be built again
from scratch.
Thanks,
Shawn
.. but 9.x listens on localhost by default.
Doing it this way, a ZK server will be run in the same JVM as Solr and
you won't have to worry about it.
Thanks,
Shawn
On 4/25/24 00:02, Anuj Bhargava wrote:
Out of memory Error - Solr version 8.11.2
*The oom_solr.sh file contains the following* -
You did not ask a question and did not provide any kind of error message.
What can we help you with?
Thanks,
Shawn
tems using replication, so it will be
difficult for me to verify what I am saying.
Thanks,
Shawn
instead of going through a load balancer. The cloud
client is instantly aware of any cluster changes, and can talk to all
the servers in the cluster. Even though defining the client uses
zookeeper info, the actual communication with Solr uses http2, with URLs
obtained from ZK.
Thanks,
Shawn
and the schema.
Without seeing the config, it's difficult to answer that question. If
the config is not using anything that is deprecated in 8.11.3, it SHOULD
work just fine in 9.x. Anything that is deprecated in 8.x will be
removed in 9.x.
Thanks,
Shawn
surprised if
they are.
It is always a good idea to recompile custom jars when upgrading Solr to
ensure that they are referencing correct library versions.
Thanks,
Shawn
ile named "lang/contractions_it.txt" and that file is not found with
the config. Most likely you just copied solrconfig.xml and the schema
and did not copy the rest of the files in the conf directory, and some
of those files ARE needed.
Thanks,
Shawn
the target index will be copied, and it will
delete any files that do not exist in the source.
Thanks,
Shawn
good
replicas of the shard online, but DO have a copy of the data directory
for that shard.
Thanks,
Shawn
ml
Thanks,
Shawn
we cannot
advise you on how to upgrade.
Thanks,
Shawn
pagination? Using the start and rows
parameter will be slow once you get a few pages in.
Thanks,
Shawn
will ever
happen.
Although technically we are supporting all 9.x versions, vulnerabilities
in older minor versions (currently 9.4.x and earlier) are only likely to
be fixed in a new point release in the latest minor version (currently
9.5.x) or a new minor version.
Thanks,
Shawn
g list ate your attachments, so we can't see those.
Thanks,
Shawn
On 2/1/24 08:09, Shawn Heisey wrote:
To verify that what I say is true, manually kick an optimize off when
you're NOT doing an import. If your index is named XXX, this command
would do it:
curl "http://hostname:port/solr/XXX/update?optimize=true";
That command is not wo
do it:
curl "http://hostname:port/solr/XXX/update?optimize=true";
An optimize is a slow operation.
Thanks,
Shawn
tored to
false, docValues to true, and useDocValuesAsStored to true. Not all
fieldtype classes support docValues, though -- in particular TextField
does not.
Thanks,
Shawn
cValues on fields that you use for things other than
searching (one example is sorting) can help.
Thanks,
Shawn
m is on the network using something
like NFS or SMB, something that does not have all the same locking
capabilities as a locally mounted filesystem. The default of "native"
tends to be the best option.
Thanks,
Shawn
y adds the "all documents" before the negative query so the
effective query is what I suggested above.
Solr is only able to detect a purely negative query if it is not mixed
in with other clauses.
Thanks,
Shawn
"curl" button highlighted. Right
next to that is a "SolrJ" button ... click on that, and you'll see how
to do it in SolrJ.
Thanks,
Shawn
}
}
}
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter'
implementation group: 'org.apache.solr', name: 'solr-solrj', version:
'9+'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
tasks.named('test') {
useJUnitPlatform()
}
Thanks,
Shawn
multiple servers. Optionally you could have that virtual IP address
live on a set of machines that operate a load balancer sitting in front
of your SolrCloud cluster.
Things I have seen that can do this include:
pacemaker
keepalived
ucarp
Thanks,
Shawn
is no advantage to a 31GB heap compared to 32GB when
using ZGC.
Thanks,
Shawn
, that probably
does require a restart, because when Solr is not in cloud mode, it is
far less dynamic.
Thanks,
Shawn
ality provides a value for
search results by default.
I do not know why you are running into a discrepancy on the _version_
field. It seems very odd.
Thanks,
Shawn
st is sent to all shards so it won't matter what shard it's
in or what _version_ it has.
Thanks,
Shawn
On 12/14/2023 13:41, Walter Underwood wrote:
Running this with Corretto 17 on my Apple Silicon MacBook:
java -XX:+PrintFlagsFinal
I see this line in the 500+ lines of output:
bool UseZGC = false
{product} {default}
I
On 12/14/23 13:04, Shawn Heisey wrote:
On 12/14/23 12:58, Shawn Heisey wrote:
On closer read, that list for Java 17 is applicable to Java itself,
not necessarily ZGC.
And on a third read, I figured out that the list DOES apply to ZGC.
Apologies for the noise!
They do have a nice clear
On 12/14/23 12:58, Shawn Heisey wrote:
On closer read, that list for Java 17 is applicable to Java itself, not
necessarily ZGC.
And on a third read, I figured out that the list DOES apply to ZGC.
Apologies for the noise!
Thanks,
Shawn
On 12/14/23 12:47, Shawn Heisey wrote:
On 12/14/23 08:37, Walter Underwood wrote:
Thanks for the recommendation. Are you running this on Intel or ARM64?
We’ve mostly moved to ARM64. —wunder
This is on x64.
When ZGC was first released, it only worked on Linux/x64.
In Java 17, the supported
to switch to ZGC, in
which case you can use the GC_TUNE that I provided elsewhere in this thread.
Thanks,
Shawn
Linux/AArch64
macOS/x64
macOS/AArch64
Windows/x64
Windows/AArch64
Thanks,
Shawn
versions before 9.2.0 may not log that exception. In 9.2.0 the
OOME handling was updated to something that ALWAYS logs the reason for
the OOME.
I do not see any information here about what version of Solr is running.
Thanks,
Shawn
hatever mechanisms are already in place and gain access.
Thanks,
Shawn
not a lot of info here. Starting in version 9.0.0, Solr only
listens on localhost by default. To listen on all interfaces, set the
SOLR_JETTY_HOST environment variable to 0.0.0.0 in the include script.
Thanks,
Shawn
tart with Java 11, look in solr.log and
solr--console.log for info about why it failed.
Thanks,
Shawn
exing program rather than using DIH.
I have an idea for a multi-threaded database->solr indexing program, but
haven't had much time to spend on it. If I can ever get it done, it
will be freely available.
On the entity, "rows" is not a valid attribute. To control how many DB
rows are fetched at a time, set batchSize on the dataSource element.
The default batchSize is 500.
Thanks,
Shawn
On 11/22/23 08:57, Vince McMahon wrote:
Shawn,
The guy left the job. And the pass word is gone.
As I said before, if the schema with those ENTITY definitions works,
then the files ARE on the system somewhere.
If it's Linux or another UNIX-like system, running this as root should
l
live on each node? How big is each
shard replica's on-disk index? How many total nodes are there? Are you
running multiple Solr nodes per server?
Thanks,
Shawn
schemas, the string fieldType is defined like this:
sortMissingLast="true" docValues="true" />
Thanks,
Shawn
x27;s the nature
of the beast. With the compositeId router, a deleteById could be
directed to a specific shard, but not a DBQ.
Thanks,
Shawn
h log.
Once you know the reason for the OOME, we can proceed. There are
exactly two ways to deal with OOME:
1) Increase the amount available of the depleted resource.
2) Change the configuration or fix the software so it needs less of that
resource.
Thanks,
Shawn
and it should fix the UI.
The fix for SOLR-16621, which is included in 9.2.1, should have
eliminated this problem. Are you absolutely sure you're running 9.2.1
and not 9.0.x or 9.1.x?
Thanks,
Shawn
query
to get the list of uniqueKey values, and then use that information to
execute DeleteById updates, then the "interfering with ongoing indexing"
that DBQ can cause will not happen.
Thanks,
Shawn
On 11/11/2023 17:30, Vince McMahon wrote:
From the Solr UI, how can I tell the number of threads are set for indexing
?
If you are not using threads (or multiple processes) in your program to
send data to Solr, then the indexing is single-threaded.
Thanks,
Shawn
milliseconds.
Thanks,
Shawn
-threaded and it would do about 1000 docs per second. Indexing
with multiple threads is the secret to making Solr index quickly.
Thanks,
Shawn
pears that ZGC does clean both generations on its incremental runs.
Thanks,
Shawn
On 11/4/2023 10:22, Shawn Heisey wrote:
java -Dc=techproducts -Dauto -classpath
server\solr-webapp\webapp\WEB-INF\lib\solr-core-9.4.0.jar
org.apache.solr.cli.SimplePostTool example\exampledocs\*
This command also works:
bin\solr post -url http://localhost:8983/solr/techproducts/update
on 9.4.0. I have not checked other release versions.
Thanks,
Shawn
ry object will contain a billion values. It is
quite slow.
On my tiny Solr install with emails indexed from dovecot (currently at
306811 docs), a query of "*" (df is set to the body field) had a QTime
of 31 milliseconds. A query of "*:*" only took 11 milliseconds.
Thanks,
Shawn
olr's default limit for a POST body is 2 megabytes. It is highly
unlikely that calls to the Schema API will exceed this.
https://solr.apache.org/guide/solr/latest/indexing-guide/schema-api.html
Thanks,
Shawn
vel from the default of INFO, Solr's
logfile will be almost completely silent. When I was asking for the
solr.log above, I am assuming that it is still at INFO. If it's at WARN
or ERROR it will not have any useful info.
Thanks,
Shawn
hose
warming queries complete quickly.
Generally, I would probably run with useColdSearcher set to true, even
you have a number of Solr servers behind a load balancer. That way the
searcher will be usable immediately, even if it is not warmed yet by the
queries in the firstSearcher listener.
Thanks,
Shawn
their
guarantee.
The latest stable release of ZK right now is 3.8.3, that is the version
I would recommend for the server.
Thanks,
Shawn
lso
need it to match a value of abctestxyz? If it's a distinctive word, you
might be better off with a relatively standard analysis chain on a
fieldType of TextField and no wildcards.
Thanks,
Shawn
o
change the context without breaking things.
Alternately you could just edit the solr.xml, but it is cleaner to use
the system properties. If you edit the file directly, then every server
would have a different solr.xml config.
Thanks,
Shawn
noted before, a SolrCloud collection is composed of one or more
shards. Each shard is composed of one or more replicas, each of which
is a Solr core. One Solr instance can host many cores. I would
recommend NOT running multiple Solr instances on each machine.
Thanks,
Shawn
hat wasn't user error or a Solr bug.
Thanks,
Shawn
(which is /solr in my example) is encouraged just in
case you might want to use your ZK install to coordinate software other
than Solr or for multiple SolrCloud clusters. The Solr reference guide
has info about how to create the chroot with a 'bin/solr zk' command.
Thanks,
Shawn
new Solr install, especially if you want
fault tolerance, should run SolrCloud, not standalone mode.
Thanks,
Shawn
command finishes, and then use the
"cat" command to show the entire log before pasting it here.
Thanks,
Shawn
Verified. The _version_ field is not being sent in update requests.
On October 12, 2023 4:29:41 PM MDT, Shawn Heisey
wrote:
>On 10/12/23 13:26, Chris Hostetter wrote:
>> i think you're missunderstanding rules of _version_ and how '0' is
>> handled.
>
>
On 10/12/23 13:26, Chris Hostetter wrote:
i think you're missunderstanding rules of _version_ and how '0' is
handled.
I'm about 90 percent sure the update requests are not being sent with
the _version_ field populated. I am verifying this right as quickly as
I can.
Thanks,
Shawn
ike dropbox or imgur and give us a link.
Thanks,
Shawn
ess of
that resource.
The second option is not always possible, and whether you can do the
first option depends on the hardware.
Thanks,
Shawn
t.
See this URL:
https://cwiki.apache.org/confluence/display/solr/solrperformanceproblems#SolrPerformanceProblems-Askingforhelponamemory/performanceissue
Thanks,
Shawn
,
Shawn
://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems
Thanks,
Shawn
attachments to the mailing list will not work. You will
have to either paste the config in the body of your message or place the
config on a file sharing site and share the URL for it in the email.
Thanks,
Shawn
especially if it's not
running in cloud mode.
Thanks,
Shawn
heir developers would have actually
intended to do the things that have been observed.
Thanks,
Shawn
.
This is client behavior. The server behavior is the same whether it's
Solr 4 or Solr 9.
Which behavior is correct? I would guess SolrJ 9, but I want to make
sure so I know whether to open an issue or not.
Thanks,
Shawn
completely wrong, and I
am surprised that it even worked. They got VERY lucky.
Thanks,
Shawn
the second
update comes in.
Thanks,
Shawn
On 9/25/23 08:53, Dmitri Maziuk wrote:
On 9/25/23 08:24, Shawn Heisey wrote:
...
Solr 4.7 handles this as the customer wants, but Solr 9.1 doesn't.
This sounds like a race condition, are you sure it's Solr doing the
ordering and not e.g. the network stack?
I have seen the reque
exing operation that takes longer is actually
applied after the one that happens faster ... while 9.1 is applying the
updates in the order they were received.
Is my idea correct? Is Lucene 9 or Solr 9 smarter than 4 about order of
operations?
Thanks,
Shawn
ve not verified this. I do not know much about any other filesystem
types on Linux.
If your instance uses LVM, then you have to extend the PV, VG, and LV
after extending the partition and before doing the filesystem. My EC2
instance is not using LVM.
Thanks,
Shawn
appens VERY quickly.
Thanks,
Shawn
ning, then the delta
between the replicas might exceed the threshold for transaction log
recovery. Solr may be doing a full replication to the cores on the
restarted node. But I would expect that to only affect the shard
leaders, which are the source for the replicated data.
Thanks,
Shawn
1 - 100 of 701 matches
Mail list logo
