Re: [PR] SOLR-17635: unmarshalling Map to SimpleOrderedMap if key i of type St… [solr]
renatoh commented on PR #3163: URL: https://github.com/apache/solr/pull/3163#issuecomment-2670744975 > FYI I'm _very_ eager to examine this in detail but am on vacation this week. No worries, enjoy your vacation, I will off next week and limited to my phone -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[I] Zk-setup error [solr-operator]
idjemaoune opened a new issue, #759: URL: https://github.com/apache/solr-operator/issues/759 https://github.com/apache/solr-operator/blob/0ffddb6a4be25ffb3fdad6cb3ffe759d89c5fcf2/controllers/util/solr_security_util.go#L243 Zk setup container produce this error ```grep: ]: No such file or directory``` when authentication is activated. I face this error after upgrading from 0.8.1 to 0.9.0 operator version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17447 : Support to early terminate a search based on maxHits per collector. [solr]
sijuv commented on PR #2960: URL: https://github.com/apache/solr/pull/2960#issuecomment-2673329627 @dsmiley @gus-asf pls relook -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17679) Request for Documentation on Hybrid Lexical and Vector Search with Score Breakdown and Cutoff Logic
[
https://issues.apache.org/jira/browse/SOLR-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Khaled Alkhouli updated SOLR-17679:
---
Attachment: Screenshot from 2025-02-20 16-31-48.png
Description:
Hello Apache Solr team,
I am building a hybrid search engine that combines lexical search (traditional
keyword-based search) and vector search (semantic search using embeddings) in a
single request. I’m aiming to achieve the following in one request:
# *Lexical Search:* Using edismax with specified fields and weights.
# *Vector Search:* Using K-Nearest Neighbors (KNN) based on embeddings.
# *Hybrid Score Combination:* The final score is the sum of the normalized
lexical score and the vector search score. If a document appears in only one
search, the other score should be treated as zero.
I have implemented the following logic using Python:
{code:java}
def hybrid_search(query, top_k=10):
embedding = np.array(embed([query]), dtype=np.float32
embedding = list(embedding[0])
lxq= rf"""{{!type=edismax
qf='text'
q.op=OR
tie=0.1
bq=''
bf=''
boost=''
}}({query})"""
solr_query = {"params": {
"q": "{!bool filter=$retrievalStage must=$rankingStage}",
"rankingStage":
"{!func}sum(query($normalisedLexicalQuery),query($vectorQuery))",
"retrievalStage":"{!bool should=$lexicalQuery should=$vectorQuery}", #
Union
"normalisedLexicalQuery": "{!func}scale(query($lexicalQuery),0,1)",
"lexicalQuery": lxq,
"vectorQuery": f"{{!knn f=all_v512 topK={top_k}}}{embedding}",
"fl": "text",
"rows": top_k,
"fq": [""],
"rq": "{!rerank reRankQuery=$rqq reRankDocs=100 reRankWeight=3}",
"rqq": "{!frange l=$cutoff}query($rankingStage)",
"sort": "score desc",
}}
response = requests.post(SOLR_URL, headers=HEADERS, json=solr_query)
response = response.json()
return response {code}
The response returns documents with a combined score, which I assume is the
addition of:
* *Lexical Search Score:* Normalized between 0 and 1.
* *Vector Search Score:* Already bounded between 0 and 1.
If a document is present in one search but not the other, the score from the
missing part is added as zero. Attached is an image of the current output.
h3. *Request:*
I would like documentation or guidance on the following:
# {*}View and Return Individual Scores:{*}{*}{*}1.1 Lexical search score
1.2 Vector search score
1.3 Final combined score (already retrieved)
I would like to display all three scores in the response together for each
document.
# *Cutoff Logic:*
I am using a Python function to calculate a cutoff threshold based on the
scores. Is it possible to implement this cutoff directly in Solr so that only
documents that pass a certain threshold are returned? If so, how can I achieve
this within Solr’s query syntax, without relying on external Python logic?
How can I retrieve the following scores in the same request?
*
I appreciate any help or documentation that can assist with:
* Returning separate scores for lexical and vector queries.
* Implementing cutoff logic natively in Solr.
Thank you!
was:
Hello Apache Solr team,
I am building a hybrid search engine that combines lexical search (traditional
keyword-based search) and vector search (semantic search using embeddings) in a
single request. I’m aiming to achieve the following in one request:
# *Lexical Search:* Using edismax with specified fields and weights.
# *Vector Search:* Using K-Nearest Neighbors (KNN) based on embeddings.
# *Hybrid Score Combination:* The final score is the sum of the normalized
lexical score and the vector search score. If a document appears in only one
search, the other score should be treated as zero.
I have implemented the following logic using Python:
{{}}
{code:java}
def hybrid_search(query, top_k=10):
embedding = np.array(embed([query]), dtype=np.float32) embedding =
list(embedding[0]) lxq = rf"""{{!type=edismax qf='all_txt'
q.op=OR tie=0.1
}}({query_terms})""" solr_query = { "params": { "q":
"{!bool filter=$retrievalStage must=$rankingStage}",
"rankingStage":
"{!func}sum(query($normalisedLexicalQuery),query($vectorQuery))",
"retrievalStage":"{!bool should=$lexicalQuery should=$vectorQuery}", # Union
"normalisedLexicalQuery": "{!func}scale(query($lexicalQuery),0,1)",
"lexicalQuery": lxq, "vectorQuery": f"{{!knn f=all_v512
topK={top_k}}}{embedding}", "fl": "post_id,all_txt,score",
"rows": top_k, "fq": [""], "rq": "{!rerank
reRankQuery=$rqq reRankDocs=100 reRankWeight=3}", "rqq": "{!frange
l=$cutoff}query($rankingStage
[jira] [Created] (SOLR-17679) Request for Documentation on Hybrid Lexical and Vector Search with Score Breakdown and Cutoff Logic
Khaled Alkhouli created SOLR-17679:
--
Summary: Request for Documentation on Hybrid Lexical and Vector
Search with Score Breakdown and Cutoff Logic
Key: SOLR-17679
URL: https://issues.apache.org/jira/browse/SOLR-17679
Project: Solr
Issue Type: Task
Components: search
Affects Versions: 9.6.1
Reporter: Khaled Alkhouli
Hello Apache Solr team,
I am building a hybrid search engine that combines lexical search (traditional
keyword-based search) and vector search (semantic search using embeddings) in a
single request. I’m aiming to achieve the following in one request:
# *Lexical Search:* Using edismax with specified fields and weights.
# *Vector Search:* Using K-Nearest Neighbors (KNN) based on embeddings.
# *Hybrid Score Combination:* The final score is the sum of the normalized
lexical score and the vector search score. If a document appears in only one
search, the other score should be treated as zero.
I have implemented the following logic using Python:
{{}}
{code:java}
def hybrid_search(query, top_k=10):
embedding = np.array(embed([query]), dtype=np.float32) embedding =
list(embedding[0]) lxq = rf"""{{!type=edismax qf='all_txt'
q.op=OR tie=0.1
}}({query_terms})""" solr_query = { "params": { "q":
"{!bool filter=$retrievalStage must=$rankingStage}",
"rankingStage":
"{!func}sum(query($normalisedLexicalQuery),query($vectorQuery))",
"retrievalStage":"{!bool should=$lexicalQuery should=$vectorQuery}", # Union
"normalisedLexicalQuery": "{!func}scale(query($lexicalQuery),0,1)",
"lexicalQuery": lxq, "vectorQuery": f"{{!knn f=all_v512
topK={top_k}}}{embedding}", "fl": "post_id,all_txt,score",
"rows": top_k, "fq": [""], "rq": "{!rerank
reRankQuery=$rqq reRankDocs=100 reRankWeight=3}", "rqq": "{!frange
l=$cutoff}query($rankingStage)", "sort": "score desc",
"cutoff": f"{cutoff_ratio}" } } response =
requests.post(SOLR_URL, headers=HEADERS, json=solr_query) response =
response.json() return response
{code}
The response returns documents with a combined score, which I assume is the
addition of:
* *Lexical Search Score:* Normalized between 0 and 1.
* *Vector Search Score:* Already bounded between 0 and 1.
If a document is present in one search but not the other, the score from the
missing part is added as zero.
h3. *Request:*
I would like documentation or guidance on the following:
# *View and Return Individual Scores:*
How can I retrieve the following scores in the same request?
** Lexical search score
** Vector search score
** Final combined score (already retrieved)
I would like to display all three scores in the response together for each
document.
# *Cutoff Logic:*
I am using a Python function to calculate a cutoff threshold based on the
scores. Is it possible to implement this cutoff directly in Solr so that only
documents that pass a certain threshold are returned? If so, how can I achieve
this within Solr’s query syntax, without relying on external Python logic?
I appreciate any help or documentation that can assist with:
* Returning separate scores for lexical and vector queries.
* Implementing cutoff logic natively in Solr.
Thank you!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [I] Text to Vector Filter for Indexing [solr-operator]
epugh commented on issue #758: URL: https://github.com/apache/solr-operator/issues/758#issuecomment-2671659642 Is the correct repo? Did you mean `apache/solr`? For issues in the core Solr project we use JIRA. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17679) Request for Documentation on Hybrid Lexical and Vector Search with Score Breakdown and Cutoff Logic
[
https://issues.apache.org/jira/browse/SOLR-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Khaled Alkhouli updated SOLR-17679:
---
Description:
Hello Apache Solr team,
I was able to implement a hybrid search engine that combines *lexical search
(edismax)* and *vector search (KNN-based embeddings)* within a single request.
The idea is simple:
* *Lexical Search* retrieves results based on text relevance.
* *Vector Search* retrieves results based on semantic similarity.
* *Hybrid Scoring* sums both scores, where a missing score (if a document
appears in only one search) should be treated as zero.
This approach is working, but *there is a critical lack of documentation* on
how to properly return individual score components of lexical search (score1)
vs. vector search (score2 from cosine similarity). Right now, Solr only returns
the final combined score, but there is no clear way to see {*}how much of that
score comes from lexical search vs. vector search{*}. This is essential for
debugging and for fine-tuning ranking strategies.
I have implemented the following logic using Python:
{code:java}
def hybrid_search(query, top_k=10):
embedding = np.array(embed([query]), dtype=np.float32
embedding = list(embedding[0])
lxq= rf"""{{!type=edismax
qf='text'
q.op=OR
tie=0.1
bq=''
bf=''
boost=''
}}({query})"""
solr_query = {"params": {
"q": "{!bool filter=$retrievalStage must=$rankingStage}",
"rankingStage":
"{!func}sum(query($normalisedLexicalQuery),query($vectorQuery))",
"retrievalStage":"{!bool should=$lexicalQuery should=$vectorQuery}", #
Union
"normalisedLexicalQuery": "{!func}scale(query($lexicalQuery),0,1)",
"lexicalQuery": lxq,
"vectorQuery": f"{{!knn f=all_v512 topK={top_k}}}{embedding}",
"fl": "text",
"rows": top_k,
"fq": [""],
"rq": "{!rerank reRankQuery=$rqq reRankDocs=100 reRankWeight=3}",
"rqq": "{!frange l=$cutoff}query($rankingStage)",
"sort": "score desc",
}}
response = requests.post(SOLR_URL, headers=HEADERS, json=solr_query)
response = response.json()
return response {code}
h3. *Issues & Missing Documentation*
# *No Way to Retrieve Individual Scores in a Hybrid Search*
There is no clear documentation on how to return:
** The *lexical search score* separately.
** The *vector search score* separately.
** The *final combined score* (which Solr already provides).
Right now, we’re left guessing whether the sum of these scores works as
expected, making debugging and tuning unnecessarily difficult.
# *No Clear Way to Implement Cutoff Logic in Solr*
In a hybrid search, I need to filter out results that don’t meet a {*}minimum
score threshold{*}. Right now, I have to implement this in Python, {*}which
defeats the purpose of using Solr for ranking in the first place{*}.
** How can we enforce a {*}score-based cutoff directly in Solr{*}, without
external filtering?
** The {{{!frange}}} function is mentioned in the documentation but lacks
{*}clear examples on how to apply it to hybrid search{*}.
h3. *Feature Request / Documentation Improvement*
* *Provide a way to return individual scores for lexical and vector search in
the response.* This should be as simple as adding fields like
{{{}fl=score,lexical_score,vector_score{}}}.
* *Clarify how to apply cutoff logic in a hybrid search.* This is an essential
ranking mechanism, and yet, there’s little guidance on how to do this
efficiently within Solr itself.
Looking forward to a response.
was:
Hello Apache Solr team,
I am building a hybrid search engine that combines lexical search (traditional
keyword-based search) and vector search (semantic search using embeddings) in a
single request. I’m aiming to achieve the following in one request:
# *Lexical Search:* Using edismax with specified fields and weights.
# *Vector Search:* Using K-Nearest Neighbors (KNN) based on embeddings.
# *Hybrid Score Combination:* The final score is the sum of the normalized
lexical score and the vector search score. If a document appears in only one
search, the other score should be treated as zero.
I have implemented the following logic using Python:
{code:java}
def hybrid_search(query, top_k=10):
embedding = np.array(embed([query]), dtype=np.float32
embedding = list(embedding[0])
lxq= rf"""{{!type=edismax
qf='text'
q.op=OR
tie=0.1
bq=''
bf=''
boost=''
}}({query})"""
solr_query = {"params": {
"q": "{!bool filter=$retrievalStage must=$rankingStage}",
"rankingStage":
"{!func}sum(query($normalisedLexicalQuery),query($vectorQuery))",
"retrievalStage":"{!bool should=$lexicalQuery should=$vect
[jira] [Updated] (SOLR-17679) Request for Documentation/Feature Improvement on Hybrid Lexical and Vector Search with Score Breakdown and Cutoff Logic
[
https://issues.apache.org/jira/browse/SOLR-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Khaled Alkhouli updated SOLR-17679:
---
Summary: Request for Documentation/Feature Improvement on Hybrid Lexical
and Vector Search with Score Breakdown and Cutoff Logic (was: Request for
Documentation on Hybrid Lexical and Vector Search with Score Breakdown and
Cutoff Logic)
> Request for Documentation/Feature Improvement on Hybrid Lexical and Vector
> Search with Score Breakdown and Cutoff Logic
> ---
>
> Key: SOLR-17679
> URL: https://issues.apache.org/jira/browse/SOLR-17679
> Project: Solr
> Issue Type: Improvement
> Components: search
>Affects Versions: 9.6.1
>Reporter: Khaled Alkhouli
>Priority: Major
> Labels: hybrid-search, search, solr, vector-based-search
> Attachments: Screenshot from 2025-02-20 16-31-48.png
>
>
> Hello Apache Solr team,
> I was able to implement a hybrid search engine that combines *lexical search
> (edismax)* and *vector search (KNN-based embeddings)* within a single
> request. The idea is simple:
> * *Lexical Search* retrieves results based on text relevance.
> * *Vector Search* retrieves results based on semantic similarity.
> * *Hybrid Scoring* sums both scores, where a missing score (if a document
> appears in only one search) should be treated as zero.
> This approach is working, but *there is a critical lack of documentation* on
> how to properly return individual score components of lexical search (score1)
> vs. vector search (score2 from cosine similarity). Right now, Solr only
> returns the final combined score, but there is no clear way to see {*}how
> much of that score comes from lexical search vs. vector search{*}. This is
> essential for debugging and for fine-tuning ranking strategies.
>
> I have implemented the following logic using Python:
> {code:java}
> def hybrid_search(query, top_k=10):
> embedding = np.array(embed([query]), dtype=np.float32
> embedding = list(embedding[0])
> lxq= rf"""{{!type=edismax
> qf='text'
> q.op=OR
> tie=0.1
> bq=''
> bf=''
> boost=''
> }}({query})"""
> solr_query = {"params": {
> "q": "{!bool filter=$retrievalStage must=$rankingStage}",
> "rankingStage":
> "{!func}sum(query($normalisedLexicalQuery),query($vectorQuery))",
> "retrievalStage":"{!bool should=$lexicalQuery should=$vectorQuery}",
> # Union
> "normalisedLexicalQuery": "{!func}scale(query($lexicalQuery),0,1)",
> "lexicalQuery": lxq,
> "vectorQuery": f"{{!knn f=all_v512 topK={top_k}}}{embedding}",
> "fl": "text",
> "rows": top_k,
> "fq": [""],
> "rq": "{!rerank reRankQuery=$rqq reRankDocs=100 reRankWeight=3}",
> "rqq": "{!frange l=$cutoff}query($rankingStage)",
> "sort": "score desc",
> }}
> response = requests.post(SOLR_URL, headers=HEADERS, json=solr_query)
> response = response.json()
> return response {code}
> h3. *Issues & Missing Documentation*
> # *No Way to Retrieve Individual Scores in a Hybrid Search*
> There is no clear documentation on how to return:
> *
> ** The *lexical search score* separately.
> ** The *vector search score* separately.
> ** The *final combined score* (which Solr already provides).
> Right now, we’re left guessing whether the sum of these scores works as
> expected, making debugging and tuning unnecessarily difficult.
> # *No Clear Way to Implement Cutoff Logic in Solr*
> In a hybrid search, I need to filter out results that don’t meet a {*}minimum
> score threshold{*}. Right now, I have to implement this in Python, {*}which
> defeats the purpose of using Solr for ranking in the first place{*}.
> *
> ** How can we enforce a {*}score-based cutoff directly in Solr{*}, without
> external filtering?
> ** The \{!frange} function is mentioned in the documentation but lacks
> {*}clear examples on how to apply it to hybrid search{*}.
> h3. *Feature Request / Documentation Improvement*
> * *Provide a way to return individual scores for lexical and vector search
> in the response.* This should be as simple as adding fields like
> {{{}fl=score,lexical_score,vector_score{}}}.
> * *Clarify how to apply cutoff logic in a hybrid search.* This is an
> essential ranking mechanism, and yet, there’s little guidance on how to do
> this efficiently within Solr itself.
> Looking forward to a response.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additiona
[jira] [Updated] (SOLR-17679) Request for Documentation on Hybrid Lexical and Vector Search with Score Breakdown and Cutoff Logic
[
https://issues.apache.org/jira/browse/SOLR-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Khaled Alkhouli updated SOLR-17679:
---
Description:
Hello Apache Solr team,
I was able to implement a hybrid search engine that combines *lexical search
(edismax)* and *vector search (KNN-based embeddings)* within a single request.
The idea is simple:
* *Lexical Search* retrieves results based on text relevance.
* *Vector Search* retrieves results based on semantic similarity.
* *Hybrid Scoring* sums both scores, where a missing score (if a document
appears in only one search) should be treated as zero.
This approach is working, but *there is a critical lack of documentation* on
how to properly return individual score components of lexical search (score1)
vs. vector search (score2 from cosine similarity). Right now, Solr only returns
the final combined score, but there is no clear way to see {*}how much of that
score comes from lexical search vs. vector search{*}. This is essential for
debugging and for fine-tuning ranking strategies.
I have implemented the following logic using Python:
{code:java}
def hybrid_search(query, top_k=10):
embedding = np.array(embed([query]), dtype=np.float32
embedding = list(embedding[0])
lxq= rf"""{{!type=edismax
qf='text'
q.op=OR
tie=0.1
bq=''
bf=''
boost=''
}}({query})"""
solr_query = {"params": {
"q": "{!bool filter=$retrievalStage must=$rankingStage}",
"rankingStage":
"{!func}sum(query($normalisedLexicalQuery),query($vectorQuery))",
"retrievalStage":"{!bool should=$lexicalQuery should=$vectorQuery}", #
Union
"normalisedLexicalQuery": "{!func}scale(query($lexicalQuery),0,1)",
"lexicalQuery": lxq,
"vectorQuery": f"{{!knn f=all_v512 topK={top_k}}}{embedding}",
"fl": "text",
"rows": top_k,
"fq": [""],
"rq": "{!rerank reRankQuery=$rqq reRankDocs=100 reRankWeight=3}",
"rqq": "{!frange l=$cutoff}query($rankingStage)",
"sort": "score desc",
}}
response = requests.post(SOLR_URL, headers=HEADERS, json=solr_query)
response = response.json()
return response {code}
h3. *Issues & Missing Documentation*
# *No Way to Retrieve Individual Scores in a Hybrid Search*
There is no clear documentation on how to return:
*
** The *lexical search score* separately.
** The *vector search score* separately.
** The *final combined score* (which Solr already provides).
Right now, we’re left guessing whether the sum of these scores works as
expected, making debugging and tuning unnecessarily difficult.
# *No Clear Way to Implement Cutoff Logic in Solr*
In a hybrid search, I need to filter out results that don’t meet a {*}minimum
score threshold{*}. Right now, I have to implement this in Python, {*}which
defeats the purpose of using Solr for ranking in the first place{*}.
*
** How can we enforce a {*}score-based cutoff directly in Solr{*}, without
external filtering?
** The \{!frange} function is mentioned in the documentation but lacks
{*}clear examples on how to apply it to hybrid search{*}.
h3. *Feature Request / Documentation Improvement*
* *Provide a way to return individual scores for lexical and vector search in
the response.* This should be as simple as adding fields like
{{{}fl=score,lexical_score,vector_score{}}}.
* *Clarify how to apply cutoff logic in a hybrid search.* This is an essential
ranking mechanism, and yet, there’s little guidance on how to do this
efficiently within Solr itself.
Looking forward to a response.
was:
Hello Apache Solr team,
I was able to implement a hybrid search engine that combines *lexical search
(edismax)* and *vector search (KNN-based embeddings)* within a single request.
The idea is simple:
* *Lexical Search* retrieves results based on text relevance.
* *Vector Search* retrieves results based on semantic similarity.
* *Hybrid Scoring* sums both scores, where a missing score (if a document
appears in only one search) should be treated as zero.
This approach is working, but *there is a critical lack of documentation* on
how to properly return individual score components of lexical search (score1)
vs. vector search (score2 from cosine similarity). Right now, Solr only returns
the final combined score, but there is no clear way to see {*}how much of that
score comes from lexical search vs. vector search{*}. This is essential for
debugging and for fine-tuning ranking strategies.
I have implemented the following logic using Python:
{code:java}
def hybrid_search(query, top_k=10):
embedding = np.array(embed([query]), dtype=np.float32
embedding = list(embedding[0])
lxq= rf"""{{!type=edismax
qf='text'
q.op=OR
tie=0.1
bq=''
[jira] [Commented] (SOLR-17677) {!join} in delete-by-query throws ClassCastException and closes IndexWriter
[
https://issues.apache.org/jira/browse/SOLR-17677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17928868#comment-17928868
]
Jason Gerlowski commented on SOLR-17677:
A few small updates:
First, this is pretty trivial to reproduce with the following:
{code}
bin/solr start -c -e techproducts
curl -ilk -X GET "http://localhost:8983/solr/techproducts/update?commit=true";
-H "Content-type: application/xml" -d '{!join from=manu_id_s
to=manu_id_s v="id:123"}'
{code}
Second, the issue here isn't specific to JoinQuery - a handful of Solr Query
implementations run into similar issues. The list I've found so far includes:
* Join
* Graph
* Collapse
* HashRange
* "IGainTerms"
* SignificantTerms
* "TextLogisticRegression"
(It's hard to imagine anyone running some of these in a DBQ, but still worth
safeguarding nonetheless I suppose. Join and Graph definitely seem like the
two most likely scenarios this might arise for a user.)
> {!join} in delete-by-query throws ClassCastException and closes IndexWriter
> ---
>
> Key: SOLR-17677
> URL: https://issues.apache.org/jira/browse/SOLR-17677
> Project: Solr
> Issue Type: Improvement
>Affects Versions: 9.8
>Reporter: Jason Gerlowski
>Priority: Minor
>
> Solr's JoinQuery implementation explicitly casts the provided "IndexSearcher"
> to a "SolrIndexSearcher". In most contexts this assumption bears out, but
> not always.
> One counter-example is Solr's "Delete By Query" codepath, which runs the
> deletion query using a "raw" Lucene IndexSearcher. (Presumably this is
> because the new searcher has just been opened?). Any DBQ containing a join
> query will throw a ClassCastException, which then bubbles up to the
> IndexWriter as a "tragic" Lucene exception, force-closing the IndexWriter and
> throwing the surrounding SolrCore in to a bad state:
> {code}
> 2025-02-18 19:39:25.339 ERROR (qtp1426725223-177-localhost-73)
> [c:techproducts s:shard2 r:core_node3 x:techproducts_shard2_replica_n1
> t:localhost-73] o.a.s.h.RequestHandlerBase Server exception =>
> org.apache.solr.common.SolrException: this IndexWriter is closed
> at
> org.apache.solr.common.SolrException.wrapLuceneTragicExceptionIfNecessary(SolrException.java:218)
> org.apache.solr.common.SolrException: this IndexWriter is closed
> at
> org.apache.solr.common.SolrException.wrapLuceneTragicExceptionIfNecessary(SolrException.java:218)
> ~[?:?]
> at
> org.apache.solr.handler.RequestHandlerBase.normalizeReceivedException(RequestHandlerBase.java:272)
> ~[?:?]
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:238)
> ~[?:?]
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2880) ~[?:?]
> at
> org.apache.solr.servlet.HttpSolrCall.executeCoreRequest(HttpSolrCall.java:890)
> ~[?:?]
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:576)
> ~[?:?]
> at
> org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:241)
> ~[?:?]
> at
> org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilterRetry$0(SolrDispatchFilter.java:198)
> ~[?:?]
> at
> org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:227)
> ~[?:?]
> at
> org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:197)
> ~[?:?]
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilterRetry(SolrDispatchFilter.java:192)
> ~[?:?]
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
> ~[?:?]
> at javax.servlet.http.HttpFilter.doFilter(HttpFilter.java:97)
> ~[jetty-servlet-api-4.0.6.jar:?]
> at
> org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:210)
> ~[jetty-servlet-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
> ~[jetty-servlet-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)
> ~[jetty-servlet-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)
> ~[jetty-server-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:598)
> ~[jetty-security-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
> ~[jetty-server-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)
> ~[jetty-server-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1580)
> ~[jetty-server-10.0.22.jar:10.
Re: [PR] SOLR-17632: Text to Vector Update Request Processor [solr]
alessandrobenedetti commented on code in PR #3151:
URL: https://github.com/apache/solr/pull/3151#discussion_r1964081310
##
solr/modules/llm/src/java/org/apache/solr/llm/textvectorisation/update/processor/TextToVectorUpdateProcessorFactory.java:
##
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.llm.textvectorisation.update.processor;
+
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.params.RequiredSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.llm.textvectorisation.model.SolrTextToVectorModel;
+import
org.apache.solr.llm.textvectorisation.store.rest.ManagedTextToVectorModelStore;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.solr.schema.DenseVectorField;
+import org.apache.solr.schema.FieldType;
+import org.apache.solr.schema.IndexSchema;
+import org.apache.solr.schema.SchemaField;
+import org.apache.solr.update.processor.UpdateRequestProcessor;
+import org.apache.solr.update.processor.UpdateRequestProcessorFactory;
+
+/**
+ * This class implements an UpdateProcessorFactory for the Text To Vector
Update Processor.
+ */
+public class TextToVectorUpdateProcessorFactory extends
UpdateRequestProcessorFactory {
+private static final String INPUT_FIELD_PARAM = "inputField";
+private static final String OUTPUT_FIELD_PARAM = "outputField";
+private static final String MODEL_NAME = "model";
+
+private String inputField;
+private String outputField;
+private String modelName;
+private SolrParams params;
+
+
+@Override
+public void init(final NamedList args) {
+params = args.toSolrParams();
+RequiredSolrParams required = params.required();
+inputField = required.get(INPUT_FIELD_PARAM);
+outputField = required.get(OUTPUT_FIELD_PARAM);
+modelName = required.get(MODEL_NAME);
+}
+
+@Override
+public UpdateRequestProcessor getInstance(SolrQueryRequest req,
SolrQueryResponse rsp, UpdateRequestProcessor next) {
+IndexSchema latestSchema = req.getCore().getLatestSchema();
+if(!latestSchema.hasExplicitField(inputField)){
+throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
"undefined field: \"" + inputField + "\"");
+}
+if(!latestSchema.hasExplicitField(outputField)){
+throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
"undefined field: \"" + outputField + "\"");
+}
+
+final SchemaField outputFieldSchema =
latestSchema.getField(outputField);
+assertIsDenseVectorField(outputFieldSchema);
+
+ManagedTextToVectorModelStore modelStore =
ManagedTextToVectorModelStore.getManagedModelStore(req.getCore());
+SolrTextToVectorModel textToVector = modelStore.getModel(modelName);
Review Comment:
I have the same assumption, I debugged it a few times and the manage
resource keeps track of models in a map so should work as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17632: Text to Vector Update Request Processor [solr]
alessandrobenedetti commented on code in PR #3151:
URL: https://github.com/apache/solr/pull/3151#discussion_r1964083376
##
solr/modules/llm/src/test/org/apache/solr/llm/textvectorisation/search/TextToVectorQParserTest.java:
##
@@ -29,6 +30,11 @@ public static void init() throws Exception {
loadModel("dummy-model.json");
}
+ @AfterClass
+ public static void cleanup() throws Exception {
+afterTest();
+ }
Review Comment:
That's the part that is giving me headache and nasty leftovers in tests (as
detailed in the initial comment).
The reason for that was to index once before all the tests, do the tests and
then do the cleanup at the end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17632: Text to Vector Update Request Processor [solr]
alessandrobenedetti commented on code in PR #3151:
URL: https://github.com/apache/solr/pull/3151#discussion_r1964064479
##
solr/modules/llm/src/java/org/apache/solr/llm/texttovector/update/processor/TextToVectorUpdateProcessor.java:
##
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.llm.texttovector.update.processor;
+
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrInputDocument;
+import org.apache.solr.common.SolrInputField;
+import org.apache.solr.llm.texttovector.model.SolrTextToVectorModel;
+import
org.apache.solr.llm.texttovector.store.rest.ManagedTextToVectorModelStore;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.update.AddUpdateCommand;
+import org.apache.solr.update.processor.UpdateRequestProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.List;
+
+
+class TextToVectorUpdateProcessor extends UpdateRequestProcessor {
+private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+private final String inputField;
+private final String outputField;
+private final String model;
+private SolrTextToVectorModel textToVector;
+private ManagedTextToVectorModelStore modelStore = null;
+
+public TextToVectorUpdateProcessor(
+String inputField,
+String outputField,
+String model,
+SolrQueryRequest req,
+UpdateRequestProcessor next) {
+super(next);
+this.inputField = inputField;
+this.outputField = outputField;
+this.model = model;
+this.modelStore =
ManagedTextToVectorModelStore.getManagedModelStore(req.getCore());
+}
+
+/**
+ * @param cmd the update command in input containing the Document to
process
+ * @throws IOException If there is a low-level I/O error
+ */
+@Override
+public void processAdd(AddUpdateCommand cmd) throws IOException {
+this.textToVector = modelStore.getModel(model);
+if (textToVector == null) {
+throw new SolrException(
+SolrException.ErrorCode.BAD_REQUEST,
+"The model requested '"
++ model
++ "' can't be found in the store: "
++ ManagedTextToVectorModelStore.REST_END_POINT);
+}
+
+SolrInputDocument doc = cmd.getSolrInputDocument();
+SolrInputField inputFieldContent = doc.get(inputField);
+if (!isNullOrEmpty(inputFieldContent, doc, inputField)) {
+String textToVectorise =
inputFieldContent.getValue().toString();//add null checks and
+float[] vector = textToVector.vectorise(textToVectorise);
+List vectorAsList = new ArrayList(vector.length);
+for (float f : vector) {
+vectorAsList.add(f);
+}
+doc.addField(outputField, vectorAsList);
+}
+super.processAdd(cmd);
+}
+
+protected boolean isNullOrEmpty(SolrInputField inputFieldContent,
SolrInputDocument doc, String fieldName) {
Review Comment:
addressed, resolving this, feel free to open a new comment if necessary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17632: Text to Vector Update Request Processor [solr]
alessandrobenedetti commented on code in PR #3151:
URL: https://github.com/apache/solr/pull/3151#discussion_r1964094479
##
solr/modules/llm/src/test/org/apache/solr/llm/textvectorisation/update/processor/TextToVectorUpdateProcessorFactoryTest.java:
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.llm.textvectorisation.update.processor;
+
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.params.MultiMapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.llm.TestLlmBase;
+import org.apache.solr.request.SolrQueryRequestBase;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+
+
+public class TextToVectorUpdateProcessorFactoryTest extends TestLlmBase {
+ private TextToVectorUpdateProcessorFactory factoryToTest =
+ new TextToVectorUpdateProcessorFactory();
+ private NamedList args = new NamedList<>();
Review Comment:
Agreed and pushed the changes!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17632: Text to Vector Update Request Processor [solr]
alessandrobenedetti commented on code in PR #3151:
URL: https://github.com/apache/solr/pull/3151#discussion_r1964100880
##
solr/modules/llm/src/test/org/apache/solr/llm/textvectorisation/update/processor/TextToVectorUpdateProcessorFactoryTest.java:
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.llm.textvectorisation.update.processor;
+
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.params.MultiMapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.llm.TestLlmBase;
+import org.apache.solr.request.SolrQueryRequestBase;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+
+
+public class TextToVectorUpdateProcessorFactoryTest extends TestLlmBase {
+ private TextToVectorUpdateProcessorFactory factoryToTest =
+ new TextToVectorUpdateProcessorFactory();
+ private NamedList args = new NamedList<>();
+
+ @BeforeClass
+ public static void initArgs() throws Exception {
+setupTest("solrconfig-llm.xml", "schema.xml", false, false);
+ }
+
+ @AfterClass
+ public static void after() throws Exception {
+afterTest();
+ }
+
+ @Test
+ public void init_fullArgs_shouldInitAllParams() {
+args.add("inputField", "_text_");
+args.add("outputField", "vector");
+args.add("model", "model1");
+factoryToTest.init(args);
+
+assertEquals("_text_", factoryToTest.getInputField());
+assertEquals("vector", factoryToTest.getOutputField());
+assertEquals("model1", factoryToTest.getModelName());
+ }
+
+ @Test
+ public void init_nullInputField_shouldThrowExceptionWithDetailedMessage() {
+args.add("outputField", "vector");
+args.add("model", "model1");
+
+SolrException e = assertThrows(SolrException.class, () ->
factoryToTest.init(args));
+assertEquals("Missing required parameter: inputField", e.getMessage());
+ }
+
+ @Test
+ public void init_nullOutputField_shouldThrowExceptionWithDetailedMessage() {
+args.add("inputField", "_text_");
+args.add("model", "model1");
+
+SolrException e = assertThrows(SolrException.class, () ->
factoryToTest.init(args));
+assertEquals("Missing required parameter: outputField", e.getMessage());
+ }
+
+ @Test
+ public void init_nullModel_shouldThrowExceptionWithDetailedMessage() {
+args.add("inputField", "_text_");
+args.add("outputField", "vector");
+
+SolrException e = assertThrows(SolrException.class, () ->
factoryToTest.init(args));
+assertEquals("Missing required parameter: model", e.getMessage());
+ }
+
+ /* Following tests depends on a real solr schema */
+ @Test
+ public void
init_notExistentOutputField_shouldThrowExceptionWithDetailedMessage() throws
Exception {
+args.add("inputField", "_text_");
+args.add("outputField", "notExistentOutput");
+args.add("model", "model1");
+
+Map params = new HashMap<>();
+MultiMapSolrParams mmparams = new MultiMapSolrParams(params);
Review Comment:
Thanks! just pushed it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17632: Text to Vector Update Request Processor [solr]
alessandrobenedetti commented on PR #3151: URL: https://github.com/apache/solr/pull/3151#issuecomment-2672286942 I've done another round of cleanup, documentation and improvements. There's still the annoying problem with the beforeClass/afterClass and tests leftovers. I'll be back in a couple of weeks, feel free to contribute any suggestions! I believe we are quite close to finalise this contribution, thanks for all the help! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-17518: Deprecate UpdateRequest.getXml() and replace it with XMLRequestWriter [solr]
psalagnac opened a new pull request, #3200: URL: https://github.com/apache/solr/pull/3200 https://issues.apache.org/jira/browse/SOLR-17518 # Description Update request in XML are written directly by `UpdateRequest`. It would be much cleaner to have a dedicated class, specific to XML, similar to existing `BinaryRequestWriter`. # Solution Add `XMLRequestWriter`. Methods `UpdateRequest.getXML()` and `UpdateRequest.writeXML()` are flagged as deprecated, so the change may be merged in 9x branch. I plan to fully remove them from 10 in another PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-14414) New Admin UI
[
https://issues.apache.org/jira/browse/SOLR-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17928885#comment-17928885
]
Chris M. Hostetter commented on SOLR-14414:
---
The jetbrains issue was just an example of someone saying these types of
problems can be resolved by pinning nodejs - the versions are certainly
different ad older.
I think expecting people to upgrade their entire host OS in order to build solr
– just because one (still experimental) optional sub-project needs nodejs – is
a big ask. It's one thing to say "this project requires jdk21" because you
easily have multiple JDKs on the same bod, but you can't easily have multiple
GLIBC versions.
(and remember, this isn't just me: [we keep seeing jenkins
failures|https://ci-builds.apache.org/job/Solr/job/Solr-Docker-Test-main/630/console#gradle-task-76]
because some jenkins workers aren't up to date with the latest greatest GLIBC
either)
Given how much complexity & Host OS requirements are involved in
building/testing the new UI (you also have to have {{chrome}} installed
somewhere in your PATH to run tests correct?) , I think ideally there should be
a simple gradle property that completely enables/disables all tasks in the
{{:solr:ui}} subproject so they never try to run (and never run any of their
dependency tasks in the top level build (although i'm also not really sure i
understand why {{kotlinNpmInstall}} needs to be a be level task instead of a
task in the {{:solr:ui}} subproject?)
> New Admin UI
>
>
> Key: SOLR-14414
> URL: https://issues.apache.org/jira/browse/SOLR-14414
> Project: Solr
> Issue Type: Improvement
> Components: Admin UI
>Affects Versions: 9.0
>Reporter: Marcus Eagan
>Priority: Major
> Labels: pull-request-available
> Attachments: QueryUX-SolrAdminUIReboot.mov
>
> Time Spent: 10h
> Remaining Estimate: 0h
>
> We have had a lengthy discussion in the mailing list about the need to build
> a modern UI that is both more security and does not depend on deprecated, end
> of life code. In this ticket, I intend to familiarize the community with the
> efforts of the community to do just that that. While we are nearing feature
> parity, but not there yet as many have suggested we could complete this task
> in iterations, here is an attempt to get the ball rolling. I have mostly
> worked on it in weekend nights on the occasion that I could find the time.
> Angular is certainly not my specialty, and this is my first attempt at using
> TypeScript besides a few brief learning exercises here and there. However, I
> will be engaging experts in both of these areas for consultation as our
> community tries to pull our UI into another era.
> Many of the components here can improve. One or two them need to be
> rewritten, and there are even at least three essential components to the app
> missing, along with some tests. A couple other things missing are the V2 API,
> which I found difficult to build with in this context because it is not
> documented on the web. I understand that it is "self-documenting," but the
> most easy-to-use APIs are still documented on the web. Maybe it is entirely
> documented on the web, and I had trouble finding it. Forgive me, as that
> could be an area of assistance. Another area where I need assistance is
> packaging this application as a Solr package. I understand this app is not in
> the right place for that today, but it can be. There are still many
> improvements to be made in this Jira and certainly in this code.
> The project is located in {{lucene-solr/solr/webapp2}}, where there is a
> README for information on running the app.
> The app can be started from the this directory with {{npm start}} for now. It
> can quickly be modified to start as a part of the typical start commands as
> it approaches parity. I expect there will be a lot of opinions. I welcome
> them, of course. The community input should drive the project's success.
> Discussion in mailing list:
> https://mail-archives.apache.org/mod_mbox/lucene-dev/202004.mbox/%3CCAF76exK-EB_tyFx0B4fBiA%3DJj8gH%3Divn2Uo6cWvMwhvzRdA3KA%40mail.gmail.com%3E
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17518) Refactor out a XmlRequestWriter so that RequestWriter is abstract
[ https://issues.apache.org/jira/browse/SOLR-17518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SOLR-17518: -- Labels: newdev pull-request-available (was: newdev) > Refactor out a XmlRequestWriter so that RequestWriter is abstract > - > > Key: SOLR-17518 > URL: https://issues.apache.org/jira/browse/SOLR-17518 > Project: Solr > Issue Type: Task > Components: SolrJ >Reporter: David Smiley >Priority: Minor > Labels: newdev, pull-request-available > Fix For: main (10.0) > > Time Spent: 10m > Remaining Estimate: 0h > > RequestWriter writes XML; some subclasses write other things. This is > terrible API design; the XML choice should be a subclass, RequestWriter > should be abstract (or an interface). > While we're at this, the XML generation is kind of split into multiple > places; it should be consolidated: UpdateRequest & ClientUtils have XML > generation methods. Those methods should move to the new XmlRequestWriter. > BinaryRequestWriter should probably be final and/or ensure the CBOR one does > not subclass it, which is weird since CBOR != "javabin". > Be sure to note this refactoring in the update notes since SolrJ users will > be impacted. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17518: Deprecate UpdateRequest.getXml() and replace it with XMLRequestWriter [solr]
epugh commented on code in PR #3200:
URL: https://github.com/apache/solr/pull/3200#discussion_r1964253721
##
solr/solrj/src/java/org/apache/solr/client/solrj/impl/XMLRequestWriter.java:
##
@@ -0,0 +1,216 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.client.solrj.impl;
+
+import java.io.BufferedWriter;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.io.OutputStreamWriter;
+import java.io.Writer;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.request.RequestWriter;
+import org.apache.solr.client.solrj.request.UpdateRequest;
+import org.apache.solr.client.solrj.util.ClientUtils;
+import org.apache.solr.common.SolrInputDocument;
+import org.apache.solr.common.params.ShardParams;
+import org.apache.solr.common.util.ContentStream;
+import org.apache.solr.common.util.XML;
+
+public class XMLRequestWriter extends RequestWriter {
+
+ /**
+ * Use this to do a push writing instead of pull. If this method returns
null {@link
+ *
org.apache.solr.client.solrj.request.RequestWriter#getContentStreams(SolrRequest)}
is invoked
+ * to do a pull write.
+ */
+ @Override
+ public RequestWriter.ContentWriter getContentWriter(SolrRequest req) {
+if (req instanceof UpdateRequest updateRequest) {
+ if (isEmpty(updateRequest)) return null;
+ return new RequestWriter.ContentWriter() {
+@Override
+public void write(OutputStream os) throws IOException {
+ OutputStreamWriter writer = new OutputStreamWriter(os,
StandardCharsets.UTF_8);
+ writeXML(updateRequest, writer);
+ writer.flush();
+}
+
+@Override
+public String getContentType() {
+ return ClientUtils.TEXT_XML;
+}
+ };
+}
+return req.getContentWriter(ClientUtils.TEXT_XML);
+ }
+
+ @Override
+ public Collection getContentStreams(SolrRequest req)
throws IOException {
+if (req instanceof UpdateRequest) {
+ return null;
+}
+return req.getContentStreams();
+ }
+
+ @Override
+ public void write(SolrRequest request, OutputStream os) throws
IOException {
+if (request instanceof UpdateRequest updateRequest) {
+ BufferedWriter writer =
+ new BufferedWriter(new OutputStreamWriter(os,
StandardCharsets.UTF_8));
+ writeXML(updateRequest, writer);
+ writer.flush();
+}
+ }
+
+ @Override
+ public String getUpdateContentType() {
+return ClientUtils.TEXT_XML;
+ }
+
+ public void writeXML(UpdateRequest request, Writer writer) throws
IOException {
+List>> getDocLists =
getDocLists(request);
+
+for (Map> docs : getDocLists) {
+
+ if (docs != null && !docs.isEmpty()) {
+Map.Entry> firstDoc =
+docs.entrySet().iterator().next();
+Map map = firstDoc.getValue();
+Integer cw = null;
Review Comment:
seems odd to have both `cw` and later `commitWithin`, and since we don't
normally abbriveate variables, do `commitWithin` and `overwrite` everywhere?
##
solr/solrj/src/java/org/apache/solr/client/solrj/request/UpdateRequest.java:
##
@@ -368,147 +368,39 @@ public void setDeleteQuery(List deleteQuery) {
// --
// --
+ /**
+ * @deprecated Method will be removed in Solr 10.0. Use {@link
XMLRequestWriter} instead.
+ */
+ @Deprecated(since = "9.9")
Review Comment:
thanks for the `since`, I like seeing those to help remind us when we can
clean them up. Plus, you added a reminder in the docs!
##
solr/solrj/src/java/org/apache/solr/client/solrj/impl/XMLRequestWriter.java:
##
@@ -0,0 +1,216 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for ad
[jira] [Commented] (SOLR-14414) New Admin UI
[
https://issues.apache.org/jira/browse/SOLR-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17928898#comment-17928898
]
Christos Malliaridis commented on SOLR-14414:
-
I completely agree with your points. An "up-to-date" system is nothing we can
just expect from everyone, especially not in a project that is out there for
many years now. And updating an OS is also not a simple task, as it affects
both devs and CI servers.
My current concern is that NodeJs v16 has reached end of life in 2023,
downgrading doesn't seem like the way to go. And since the next LTS version,
which happens to cause the same issue, is also about to [reach
EOL|https://nodejs.org/en/about/previous-releases#release-schedule], I believe
it is not a bad idea to go with the latest LTS version which is Node 22.
However, your point about disabling the new UI is something we may consider.
{{kotlinNpmInstall}} is a top-level task probably because the Kotlin
Multiplatform plugin is added at project level, even though it is enabled at
module level only. Perhaps it would be different if we would add and configure
it only at module level, but I am not sure if that works.
We would still have to disable the entire UI module, which is possible from
what I know. But we would not have any guarantee about the new stuff that is
being implemented and eventually added in releases. And that also does not
resolve the usage of the EOL node version we are currently using in our project.
I wanted to discuss these topics in the last meetup, but we did not get much
opinions on that, so I am considering a mailing thread next.
> New Admin UI
>
>
> Key: SOLR-14414
> URL: https://issues.apache.org/jira/browse/SOLR-14414
> Project: Solr
> Issue Type: Improvement
> Components: Admin UI
>Affects Versions: 9.0
>Reporter: Marcus Eagan
>Priority: Major
> Labels: pull-request-available
> Attachments: QueryUX-SolrAdminUIReboot.mov
>
> Time Spent: 10h
> Remaining Estimate: 0h
>
> We have had a lengthy discussion in the mailing list about the need to build
> a modern UI that is both more security and does not depend on deprecated, end
> of life code. In this ticket, I intend to familiarize the community with the
> efforts of the community to do just that that. While we are nearing feature
> parity, but not there yet as many have suggested we could complete this task
> in iterations, here is an attempt to get the ball rolling. I have mostly
> worked on it in weekend nights on the occasion that I could find the time.
> Angular is certainly not my specialty, and this is my first attempt at using
> TypeScript besides a few brief learning exercises here and there. However, I
> will be engaging experts in both of these areas for consultation as our
> community tries to pull our UI into another era.
> Many of the components here can improve. One or two them need to be
> rewritten, and there are even at least three essential components to the app
> missing, along with some tests. A couple other things missing are the V2 API,
> which I found difficult to build with in this context because it is not
> documented on the web. I understand that it is "self-documenting," but the
> most easy-to-use APIs are still documented on the web. Maybe it is entirely
> documented on the web, and I had trouble finding it. Forgive me, as that
> could be an area of assistance. Another area where I need assistance is
> packaging this application as a Solr package. I understand this app is not in
> the right place for that today, but it can be. There are still many
> improvements to be made in this Jira and certainly in this code.
> The project is located in {{lucene-solr/solr/webapp2}}, where there is a
> README for information on running the app.
> The app can be started from the this directory with {{npm start}} for now. It
> can quickly be modified to start as a part of the typical start commands as
> it approaches parity. I expect there will be a lot of opinions. I welcome
> them, of course. The community input should drive the project's success.
> Discussion in mailing list:
> https://mail-archives.apache.org/mod_mbox/lucene-dev/202004.mbox/%3CCAF76exK-EB_tyFx0B4fBiA%3DJj8gH%3Divn2Uo6cWvMwhvzRdA3KA%40mail.gmail.com%3E
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17309: Enhance certificate based authentication plugin with flexible cert principal resolution [solr]
epugh commented on PR #3029: URL: https://github.com/apache/solr/pull/3029#issuecomment-2672563684 thanks for the ref guide improvments! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17309: Enhance certificate based authentication plugin with flexible cert principal resolution [solr]
laminelam commented on PR #3029: URL: https://github.com/apache/solr/pull/3029#issuecomment-2672567020 > thanks for the ref guide improvments! Attached is a PDF version for better review experience [CertificateAuthenticationPlugin _ RefGuide_PDF.pdf](https://github.com/user-attachments/files/18894177/CertificateAuthenticationPlugin._.RefGuide_PDF.pdf) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] Feature/update web dependencies [solr]
malliaridis opened a new pull request, #3201: URL: https://github.com/apache/solr/pull/3201 # Description With NodeJS v16 reaching EOL in 2023, we should upgrade to the latest LTS. # Solution This PR updates the ref-guide dependencies (web), including NodeJS to v22 (latest LTS), as the current version (16) has reached EOL about two years ago. The PR also aligns the UI module's node version with the one from the `libs.versions.toml`, so that we can maintain the same version across the project. **Note that this upgrade requires GLIBC v2.28**, which is currently not present in our build servers and causes other tasks (like `:kotlinNpmInstall`) to fail as well. # Tests No tests have been added or removed._ GitHub workflows are likely not affected by version issues and may run successfully, but our jenkins jobs may (continue to) fail. # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [x] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [x] I have developed this patch against the `main` branch. - [x] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-14414) New Admin UI
[
https://issues.apache.org/jira/browse/SOLR-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christos Malliaridis updated SOLR-14414:
Labels: new-ui pull-request-available ui (was: pull-request-available)
> New Admin UI
>
>
> Key: SOLR-14414
> URL: https://issues.apache.org/jira/browse/SOLR-14414
> Project: Solr
> Issue Type: Improvement
> Components: Admin UI
>Affects Versions: 9.0
>Reporter: Marcus Eagan
>Priority: Major
> Labels: new-ui, pull-request-available, ui
> Attachments: QueryUX-SolrAdminUIReboot.mov
>
> Time Spent: 10h
> Remaining Estimate: 0h
>
> We have had a lengthy discussion in the mailing list about the need to build
> a modern UI that is both more security and does not depend on deprecated, end
> of life code. In this ticket, I intend to familiarize the community with the
> efforts of the community to do just that that. While we are nearing feature
> parity, but not there yet as many have suggested we could complete this task
> in iterations, here is an attempt to get the ball rolling. I have mostly
> worked on it in weekend nights on the occasion that I could find the time.
> Angular is certainly not my specialty, and this is my first attempt at using
> TypeScript besides a few brief learning exercises here and there. However, I
> will be engaging experts in both of these areas for consultation as our
> community tries to pull our UI into another era.
> Many of the components here can improve. One or two them need to be
> rewritten, and there are even at least three essential components to the app
> missing, along with some tests. A couple other things missing are the V2 API,
> which I found difficult to build with in this context because it is not
> documented on the web. I understand that it is "self-documenting," but the
> most easy-to-use APIs are still documented on the web. Maybe it is entirely
> documented on the web, and I had trouble finding it. Forgive me, as that
> could be an area of assistance. Another area where I need assistance is
> packaging this application as a Solr package. I understand this app is not in
> the right place for that today, but it can be. There are still many
> improvements to be made in this Jira and certainly in this code.
> The project is located in {{lucene-solr/solr/webapp2}}, where there is a
> README for information on running the app.
> The app can be started from the this directory with {{npm start}} for now. It
> can quickly be modified to start as a part of the typical start commands as
> it approaches parity. I expect there will be a lot of opinions. I welcome
> them, of course. The community input should drive the project's success.
> Discussion in mailing list:
> https://mail-archives.apache.org/mod_mbox/lucene-dev/202004.mbox/%3CCAF76exK-EB_tyFx0B4fBiA%3DJj8gH%3Divn2Uo6cWvMwhvzRdA3KA%40mail.gmail.com%3E
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17309: Enhance certificate based authentication plugin with flexible cert principal resolution [solr]
laminelam commented on PR #3029: URL: https://github.com/apache/solr/pull/3029#issuecomment-2672611692 Thank @epugh for your time on this. When you're ready to merge let me know will update the CHANGES file (or feel free to do it). I you think we could have another pair of eyes that would be great. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] IOUtils: add back closeQuietly(Closeable) [solr]
dsmiley commented on PR #3199: URL: https://github.com/apache/solr/pull/3199#issuecomment-2672660493 Ported to branch_9_8 so that it gets in 9.8.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] IOUtils: add back closeQuietly(Closeable) [solr]
dsmiley merged PR #3199: URL: https://github.com/apache/solr/pull/3199 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] LogListener: remove no-arg convenience methods [solr]
dsmiley opened a new pull request, #3202: URL: https://github.com/apache/solr/pull/3202 These are hazardous/sloppy. Tests should listen to a specific class/package. Motivated by: https://issues.apache.org/jira/browse/SOLR-17667?focusedCommentId=17928175&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17928175 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17650: Fix tests for unordered buffered updates [solr]
markrmiller commented on code in PR #3197:
URL: https://github.com/apache/solr/pull/3197#discussion_r1964347792
##
solr/test-framework/src/java/org/apache/solr/util/SolrMatchers.java:
##
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.util;
+
+import java.util.List;
+import org.hamcrest.Description;
+import org.hamcrest.Matcher;
+import org.hamcrest.TypeSafeDiagnosingMatcher;
+
+public class SolrMatchers {
+
+ public static Matcher> subListMatches(
Review Comment:
Could add "Matches a segment of a list between two indices against a
provided matcher, useful for checking ordered sub-sequences in a larger list"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] LogListener: remove no-arg convenience methods [solr]
hossman commented on PR #3202:
URL: https://github.com/apache/solr/pull/3202#issuecomment-2672710644
Not really sure i understand why you consider them hazardous?
FWIW: The intent of these was to make it as easy as possible for folks to
write tests asserting that "No code run in this block should cause any code,
anywhere, to log an ERROR (WARN, etc...)" but see no great loss in making
people write `LogListener.error("")` instead of `LogListener.error()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17677) {!join} in delete-by-query throws ClassCastException and closes IndexWriter
[
https://issues.apache.org/jira/browse/SOLR-17677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SOLR-17677:
--
Labels: pull-request-available (was: )
> {!join} in delete-by-query throws ClassCastException and closes IndexWriter
> ---
>
> Key: SOLR-17677
> URL: https://issues.apache.org/jira/browse/SOLR-17677
> Project: Solr
> Issue Type: Improvement
>Affects Versions: 9.8
>Reporter: Jason Gerlowski
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Solr's JoinQuery implementation explicitly casts the provided "IndexSearcher"
> to a "SolrIndexSearcher". In most contexts this assumption bears out, but
> not always.
> One counter-example is Solr's "Delete By Query" codepath, which runs the
> deletion query using a "raw" Lucene IndexSearcher. (Presumably this is
> because the new searcher has just been opened?). Any DBQ containing a join
> query will throw a ClassCastException, which then bubbles up to the
> IndexWriter as a "tragic" Lucene exception, force-closing the IndexWriter and
> throwing the surrounding SolrCore in to a bad state:
> {code}
> 2025-02-18 19:39:25.339 ERROR (qtp1426725223-177-localhost-73)
> [c:techproducts s:shard2 r:core_node3 x:techproducts_shard2_replica_n1
> t:localhost-73] o.a.s.h.RequestHandlerBase Server exception =>
> org.apache.solr.common.SolrException: this IndexWriter is closed
> at
> org.apache.solr.common.SolrException.wrapLuceneTragicExceptionIfNecessary(SolrException.java:218)
> org.apache.solr.common.SolrException: this IndexWriter is closed
> at
> org.apache.solr.common.SolrException.wrapLuceneTragicExceptionIfNecessary(SolrException.java:218)
> ~[?:?]
> at
> org.apache.solr.handler.RequestHandlerBase.normalizeReceivedException(RequestHandlerBase.java:272)
> ~[?:?]
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:238)
> ~[?:?]
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2880) ~[?:?]
> at
> org.apache.solr.servlet.HttpSolrCall.executeCoreRequest(HttpSolrCall.java:890)
> ~[?:?]
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:576)
> ~[?:?]
> at
> org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:241)
> ~[?:?]
> at
> org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilterRetry$0(SolrDispatchFilter.java:198)
> ~[?:?]
> at
> org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:227)
> ~[?:?]
> at
> org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:197)
> ~[?:?]
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilterRetry(SolrDispatchFilter.java:192)
> ~[?:?]
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
> ~[?:?]
> at javax.servlet.http.HttpFilter.doFilter(HttpFilter.java:97)
> ~[jetty-servlet-api-4.0.6.jar:?]
> at
> org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:210)
> ~[jetty-servlet-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
> ~[jetty-servlet-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)
> ~[jetty-servlet-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)
> ~[jetty-server-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:598)
> ~[jetty-security-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
> ~[jetty-server-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)
> ~[jetty-server-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1580)
> ~[jetty-server-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
> ~[jetty-server-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384)
> ~[jetty-server-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
> ~[jetty-server-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)
> ~[jetty-servlet-10.0.22.jar:10.0.22]
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1553)
> ~[jetty-server-10.0.22.jar:10.0.22]
>
Re: [PR] LogListener: remove no-arg convenience methods [solr]
dsmiley commented on PR #3202: URL: https://github.com/apache/solr/pull/3202#issuecomment-2672795343 For the hazard, see what I linked to. Houston agrees. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] LogListener: remove no-arg convenience methods [solr]
madrob commented on PR #3202: URL: https://github.com/apache/solr/pull/3202#issuecomment-2672807785 Maybe we remove the debug (and info?) no-arg versions, leave a comment why, and leave the warn and error versions around? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17309: Enhance certificate based authentication plugin with flexible cert principal resolution [solr]
epugh commented on PR #3029: URL: https://github.com/apache/solr/pull/3029#issuecomment-2672811726 I assigned this to myself to not forget. I will leave this open for another day or so and then merge it. I can take care of the CHANGES.txt as that always seems to have merge challenges! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] LogListener: remove no-arg convenience methods [solr]
hossman commented on PR #3202: URL: https://github.com/apache/solr/pull/3202#issuecomment-2672823452 > For the hazard, see what I linked to. Houston agrees. Ah ok -- yeah, sorry i didn't notice that jira link. The crux of the problem being that in order to listen to the logs at a particular level, we must enable the logs at that level, and enabling something like DEBUG at the ROOT level can really slow things down. Totally valid point. > Maybe we remove the debug (and info?) no-arg versions, leave a comment why, and leave the warn and error versions around? We could, but that functionality is still available by referring to the root level logger by `""` (which is already mentioned as possible in the jdocs) for the rare case where it's useful ... I agree with David: we should "Make the common/safe thing easy, Make the rare/scary thing hard" BTW David: skimming the code i see an out of date TODO that should also be removed.. ``` // TODO: no-arg factory variants that use "" -- simpler syntax for ROOT logger? ``` (don't want to leave a TODO suggesting someone re-add the thing you're removing) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
