Apologies I misread the embedded part so you can ignore the first line - the
rest is still hopefully useful to you though.
From: Karl Stoney
Date: Tuesday, 22 July 2025 at 07:24
To: users@solr.apache.org
Subject: Re: Annoying problem when running SolrCloud fully containerized.
* Running a
* Running a containerized SolrCloud with embedded ZooKeepers seems to be
impossible.
Most certainly not true, but yes it does require some engineering effort. We
(Autotrader UK) run solr and zookeeper on Kubernetes and have done for circa 7
years without any major issues. We are a high read an
delete -c YOUR_DUPLICATA
> ? I am noob and I am just asking.
>
> Le mer. 7 mai 2025 à 09:07, Karl Stoney
> a écrit :
>
> > Ah no worries, thanks for the reply.
> > I already have a customer operator that watches the statefulsets and does
> > some admin type stuff, gu
have collection-A-shard-1-replica-2" case, the
best way would be to first move all replicas away from the node that will
upgrade disk, and then move them back again.
Jan
> 6. mai 2025 kl. 13:55 skrev Karl Stoney
> :
>
> Hi,
> I run solr cloud on GKE; and I’m trying to move m
Hi,
I run solr cloud on GKE; and I’m trying to move my pods to a new disk type. In
doing so the disk will be brand new. I’ve landed in a position that I’m unsure
how to recover from, where the new node is not syncing data from the leader.
To explain exactly what’s happening, lets say I have tw
Hi All,
We use Solr Cloud across multiple AZ’s in GCP.
It’s by far the biggest £ cost of cross-zone egress when replicating data from
the primary, so I’ve been trying to explore if there are any ways for us to
reduce this. We are a heavy write setup.
In non-solr cloud there’s some configuration
on solr9 when using a binary field
Hello Karl.
Please answer a few question in JIRA. Thanks
On Thu, Apr 18, 2024 at 11:25 AM Karl Stoney
wrote:
> Hi Thomas,
> It does thanks, and actually I’m stumbled across that too.
> I’ve raised a Jira issue and referenced the Jira issue from th
3D%7C0%7C%7C%7C&sdata=zOoU0uV7qToy08sHGbZN0WVfhFpmK3KkiD%2F3aKp4sxE%3D&reserved=0<https://lists.apache.org/thread/b717gcmjj92vxgq48g4ho8wct3zm4rct>
Thomas
Op do 18 apr 2024 08:34 schreef Karl Stoney
:
> Hi Thomas,
> That link doesn’t seem to work for me (might be that my corp
-10653?filter=-3 which seems
to imply it was fixed in 9.5.0 but doesn’t seem to be.
Certainly feels like a bug so I’ll raise a new ticket.
Karl
From: Karl Stoney
Date: Thursday, 18 April 2024 at 07:36
To: users@solr.apache.org
Cc: Michael Eves , Craig Wrigglesworth
Subject: Re: Invalid JSON on
field>
Thomas
Op wo 17 apr 2024 om 16:55 schreef Karl Stoney
:
> Hi,
> I’m currently looking into the solr 8 -> solr 9 upgrade, and we’ve noticed
> that if a document is using a binary field, the json returned is invalid
> because the value is not
Hi,
I’m currently looking into the solr 8 -> solr 9 upgrade, and we’ve noticed that
if a document is using a binary field, the json returned is invalid because the
value is not quoted:
For example on solr 8 we’d see:
```
"STOCK_ITEM_BINARY_FIELD":”OikKAfqEdXJzSWRjNTEwMj…”
```
But on solr 8 we
duction traffic queries there to help warmup caches before
zk gets it as active shard solves it. Thank you all for the suggestions.
On Thu, Sep 14, 2023 at 5:03 AM Karl Stoney
wrote:
> Just noticed this thread and thought I’d chime in with my experiences.
>
> I run Solr on Kubernetes and
Just noticed this thread and thought I’d chime in with my experiences.
I run Solr on Kubernetes and it’s pretty high throughput (it powers the search
for autotrader.co.uk).
During node rollouts, which happen for a variety of reasons (solr upgrades,
Kubernetes upgrades, etc) we experience those
Hi,
Just discussing here before I raise a bug.
Updated our cluster to 8.11.0 in testing this morning and noticed that the
“file” explorer is broken when viewing a file.
To recreate:
* Select a collection
* Select files
* Click on an XML file
It won’t load, and in the console you se
running solr in cloud & who has used
this option to speak up, as I can't help but feel I'm missing something
critical here as it almost seems too good to be true.
Thanks
Karl
On 09/04/2021, 10:49, "Karl Stoney" wrote:
We've lowered our autoCommit from 1 min to
t uncomfortable (at 3mins
under peak write load they're about 1.5gb each).
On 08/04/2021, 19:31, "Karl Stoney" wrote:
The documents are pretty large yes, 650 fields, circa 20kb/document so at
peak (300/sec) that's circa 6meg/sec. ramBufferSizeMB is 512 so we'd
The documents are pretty large yes, 650 fields, circa 20kb/document so at peak
(300/sec) that's circa 6meg/sec. ramBufferSizeMB is 512 so we'd be averaging 1
segment every 90 seconds (ish)?
>This means you never explicitly commits from the client? But You
> autoCommit openSearcher=false ev
Update rate, and how you do commits?
Jan
> 1. apr. 2021 kl. 13:43 skrev Karl Stoney
:
>
> Hi all.
> I’m looking for some opinions on how to best configure the Merges to run
optimally on GCP SSD’s (network attached). For context; we have a 9 node NRT
8.8.1 So
Appears to be a bug as the same request without async works fine; raised at
https://issues.apache.org/jira/browse/SOLR-15311
On 03/04/2021, 12:36, "Karl Stoney" wrote:
Further to this I decided to experiment and did a MODIFYCOLLECTION, I get
the following error in the solr log
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
However if I look in the solr interface, the collection has been updated to the
new config so _seems_ OK?
On 02/04/2021, 18:05, "Karl Stoney&quo
Hi all,
I’m trying to move away from using `zkcli` and using more first-class APIs like
the configset one.
However I am unsure how to replicate the `linkconfig` command, eg:
/solr/script/zkcli.sh -cmd linkconfig -collection $COLLECTION_NAME -confname
$NEW_CONFIG_NAME -zkhost ${ZK_HOST}
Would it
Hi all.
I’m looking for some opinions on how to best configure the Merges to run
optimally on GCP SSD’s (network attached). For context; we have a 9 node NRT
8.8.1 Solr Cloud cluster, each node has an index which is between 25 and 35gb
in size, depending on the current merge state / deleted doc
Hi,
Lucene is detecting our SSDs as Spinning disks, as seen through the admin
metrics endpoint:
* CONTAINER.fs.coreRoot.spins: true
* CONTAINER.fs.spins: true
However we can see in the `system.properties` endpoint we are overriding it:
* lucene.cms.override_spins: "false"
Does an
of
the nodes, which is odd.
Feels like a restart may well solve it, but I’m hesitant to do so in case this
is an opportunity to collect more information for a bug report?
Thanks
From: Karl Stoney
Date: Monday, 29 March 2021 at 20:22
To: "solr-u...@lucene.apache.org"
Subject: Re:
To add to this, the request succeeds on 7/9 nodes, but 2/9 fail with this error
From: Karl Stoney
Date: Monday, 29 March 2021 at 20:07
To: "solr-u...@lucene.apache.org"
Subject: Failure to create collection: Lock Held
Hi,
I’ve got an odd problem where I’m unable to create a collecti
Hi,
I’ve got an odd problem where I’m unable to create a collection as it’s
claiming a lock is held at /solr/solr-data/data/tech-data-001/index/write.lock,
but that location (/solr/solr-data/data/tech-data-001) doesn’t exist on any of
the nodes on the cluster.
Does anyone know what might be cau
Haha thanks for the spot! __
On 26/03/2021, 01:26, "Koji Sekiguchi" wrote:
It seems the reference guide has typo for the parameter. Try
query.queryResultMaxDocsCached.
Koji
On 2021/03/25 22:12, Karl Stoney wrote:
> Hey,
>
https://eur03.safelinks.protec
Hey,
https://solr.apache.org/guide/8_8/config-api.html states the field is editable,
however I get a 400 back from solr:
‘query.queryResultMaxDocCached' is not an editable property
Any ideas?
Can change other fields fine.
Unless expressly stated otherwise in this email, this e-mail is sent on b
s the daemon is using GET and
hitting the same url limit (but the error is being swallowed).
Significantly increasing the max http header length in jetty resolved the issue
so this feels like a bit of a bug?
On 22/03/2021, 15:37, "Karl Stoney"
wrote:
So for context we have
wMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6Sj5I7Zi9N63ON73luJsiKo%2FKUk63RunO7UDDdQDUw0%3D&reserved=0
>
>
> On Mon, Mar 22, 2021 at 9:01 AM Karl Stoney
> wrote:
>
> > Hi,
> > Sorry for all the questions recently…
Hi,
Sorry for all the questions recently…
So as per https://solr.apache.org/guide/8_0/reindexing.html; we’re trying to
remove a load of fields. Subsequently we’ve created a new collection with the
new schema and we’re attempting to reindex from old to new.
There’s about 216 fields in total bein
Hi,
I’m aware I can check the status of a reindx, if I know both the source and
destination cluster, or I can check the progress of the async request via the
async API.
However, if I know neither of these, and I just want to check if there are any
REINDEX’s running on the cluster at any given t
Hi all,
I'm trying to add some tests to the pipeline which deploys our solr config, and
right now i'm working with bash/xmlint/xmlstarlet, and it's a bit crude.
Does anyone have any tips/pointers on how best to validate solrconfig.xml and
schema.xml. Is there published schemas for specific solr
uld disable via configAPI cache autoWarming on the index for
the reindex duration?
____
From: Karl Stoney
Sent: 18 March 2021 10:21
To: solr-u...@lucene.apache.org ;
users@solr.apache.org
Subject: Re: REINDEXCOLLECTION gradually slows
A bit more data on this to help...
*
maxTime: 90
}
}
}
}
```
This in turn seems to be causing cache warming, which obviously gets
increasingly slower and increasingly adds load.
____
From: Karl Stoney
Sent: 17 March 2021 19:16
To: solr-u...@lucene.apache.org
Subject: REINDEXCOLLECTION gradually slow
From: Karl Stoney
Sent: 17 March 2021 22:20
To: users@solr.apache.org
Subject: Re: Disable commits during a REINDEXCOLLECTION
That's the source collection, I'm referring to the destination.
@David - seeing as REINDEX creates the target collection, I had to start the
process and then
puts the source collection in read-only mode,
am I wrong? Doesn’t that alsı disable commits?
Sent from my iPhone
> On 18 Mar 2021, at 00:28, Karl Stoney
> wrote:
>
> Can you believe I had never come across that before! Thanks!
>
> Every day’s a school day
>
&g
M%3D&reserved=0
On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
wrote:
> Hi all,
> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
> aggressive autoCommit interval configured by default (intentionally).
>
> Ideally I'd like to be able to disabl
Hey,
So we're trying to use REINDEXCOLLECTION again (solr 8.8) and similar to the
last time we tried it (8.1) we see it gradually slow down; as you can see from
the logs below.
I terminated it early but you can see the trend in the processingRatePerSecond.
I'm wondering if anyone has any idea's
Hi all,
We're wanting to use REINDEXCOLLECTION, but our config has a relatively
aggressive autoCommit interval configured by default (intentionally).
Ideally I'd like to be able to disable hard commits for the duration of the
reindex, but can't see a way to do that without pushing a whole new co
OK this makes sense, thanks for the reply Shawn.
On 13/03/2021, 21:34, "Shawn Heisey" wrote:
On 3/13/2021 11:36 AM, Karl Stoney wrote:
> Apologies if this is a silly question, I just can't find anything
explaining the benefits online.
>
> Would anyon
Hi,
Apologies if this is a silly question, I just can't find anything explaining
the benefits online.
Would anyone be able to tell me why you would use a documentCache, if you have
sufficient RAM on your machine that the OS disk cache is effectively caching
all the documents anyway?
Thanks!
Un
42 matches
Mail list logo
