Upgrade Jackson / SOLR-16443
Hi, I see SOLR-16443 is being addressed in version 9. Will this jackson-databind update also be applied to 8.11? Thanks, Jay
Re: I cannot get nested objects to index - with image links
Hi Thomas,
I am attempting to define my fields to accept the data that will actually be
sent, it is more deeply nested.
In doing so I have interpreted the results of the previous configuration to not
include (pseudo-)fields. In the JSON below I believe those to be "content" from
the prior example, along with "Page, Body, Modules and Fields" but his gives me
the error shown. How do I handle these other fields. Thank you for your
assistance, it is greatly appreciated.
curl --location --request POST
'http://localhost:8983/solr/talix/update?json.command=false' \
--header 'Content-Type: application/json' \
--data-raw '{
"partner": "88027688-62c4-459a-b4d5-a8ecf9edd1bf",
"command": "add",
"doc_id": "2ff99d1a-a21b-4391-9c47-af2865acb753",
"content": {
"Page": {
"Id": "2ff99d1a-a21b-4391-9c47-af2865acb753",
"Name": "Ronald McDonald House Idaho meals",
"Url":
"/blogs/st-lukes/news-and-community/2021/jan/ronald-mcdonald-house-idaho-meals",
"Date": "2022-10-03T12:30:17.3388537",
"ContentType": "Blog",
"Body": {
"Fields": [
{ "Name": "Heading Background Image", "Type": "Image", "Value": "" },
{ "Name": "Tile Wide Image", "Type": "Image", "Value": "" },
{ "Name": "Specialties", "Type": "Treelist", "Value": "" },
{
"Name": "Blog Post Name",
"Type": "Single-Line Text",
"Value": "Ronald McDonald House, St. Luke’s Children’s find new
ways to help families"
},
{ "Name": "Blog Summary", "Type": "Rich Text", "Value": "" },
{ "Name": "Share Summary", "Type": "Multi-Line Text", "Value": "" }
],
"Modules": {
"Fields": [
{ "Name": "Content", "Type": "Rich Text", "Value": "" },
{ "Name": "Image Position", "Type": "Droptree", "Value": "Right" },
{ "Name": "Image Source", "Type": "Image", "Value": "" },
{
"Name": "Image Content",
"Type": "Rich Text",
"Value": "For more than three decades"
}
]
}
},
"Facets": ["Blogs", "Article"],
"Title": "Ronald McDonald House, St. Luke’s Children’s find new ways to
help families",
"Summary": ""
}
}
}'
Response:
{
"responseHeader":{
"status":400,
"QTime":0},
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","org.apache.solr.common.SolrException"],
"msg":"undefined field: \"content.Page.Id\"",
"code":400}}
From: Matthew Castrigno
Sent: Thursday, October 27, 2022 10:36 AM
To: Thomas Corthals
Cc: users@solr.apache.org
Subject: Re: I cannot get nested objects to index - with image links
Thank you, Thomas! Changing the required to false allowed the document to be
indexed. I did not gleam this information from the SOLR documentation regarding
nested documents. Do you know of any good sources for educational material on
this subject? I cannot change the payloads that I am working with
unfortunately. They make heavy use of deeply nested json object. Thank you
again for your assistance, it is most appreciated!
From: Thomas Corthals
Sent: Thursday, October 27, 2022 1:34 AM
To: Matthew Castrigno
Cc: users@solr.apache.org
Subject: Re: I cannot get nested objects to index - with image links
Hi Matthew All the fields in your schema are required="true". This means they
are required for all documents, whether parents or children (or standalone).
"Required" fields (from a business logic perspective) on parents
ZjQcmQRYFpfptBannerStart
This Message Is From an Untrusted Sender
You have not previously corresponded with this sender.
ZjQcmQRYFpfptBannerEnd
Hi Matthew
All the fields in your schema are required="true". This means they are required
for all documents, whether parents or children (or standalone). "Required"
fields (from a business logic perspective) on parents and children will usually
be different and that can't be enforced in a Solr schema.
The only fields that can be required="true" are the ones that are required for
all types of documents. The uniqueKeyField is definitely one of them. For all
other fields, it might make more sense to handle this constraint in your
application.
Thomas
Op do 27 okt. 2022 om 01:08 schreef Matthew Castrigno
mailto:castr...@slhs.org>>:
Thank you so much for your response, Thomas, I do appreciate it.
I made the changes, but I am getting a different error now. Did I not interpret
your instructions correctly?
https://i.postimg.cc/SNM4t7vs/updated-schema.png
[i.postimg.cc]
https://i.p
Re: I cannot get nested objects to index - with image links
This mail server chopped off my cur request so I am posting it here with the
domain marked out so it just looks like text:
curl --location --request POST '
:8983/solr/talix/update?json.command=false' \
--header 'Content-Type: application/json' \
--data-raw '{
"partner": "88027688-62c4-459a-b4d5-a8ecf9edd1bf",
"command": "add",
"doc_id": "2ff99d1a-a21b-4391-9c47-af2865acb753",
"content": {
"Page": {
"Id": "2ff99d1a-a21b-4391-9c47-af2865acb753",
"Name": "Ronald McDonald House Idaho meals",
"Url":
"/blogs/st-lukes/news-and-community/2021/jan/ronald-mcdonald-house-idaho-meals",
"Date": "2022-10-03T12:30:17.3388537",
"ContentType": "Blog",
"Body": {
"Fields": [
{ "Name": "Heading Background Image", "Type": "Image", "Value": "" },
{ "Name": "Tile Wide Image", "Type": "Image", "Value": "" },
{ "Name": "Specialties", "Type": "Treelist", "Value": "" },
{
"Name": "Blog Post Name",
"Type": "Single-Line Text",
"Value": "Ronald McDonald House, St. Luke’s Children’s find new
ways to help families"
},
{ "Name": "Blog Summary", "Type": "Rich Text", "Value": "" },
{ "Name": "Share Summary", "Type": "Multi-Line Text", "Value": "" }
],
"Modules": {
"Fields": [
{ "Name": "Content", "Type": "Rich Text", "Value": "" },
{ "Name": "Image Position", "Type": "Droptree", "Value": "Right" },
{ "Name": "Image Source", "Type": "Image", "Value": "" },
{
"Name": "Image Content",
"Type": "Rich Text",
"Value": "For more than three decades"
}
]
}
},
"Facets": ["Blogs", "Article"],
"Title": "Ronald McDonald House, St. Luke’s Children’s find new ways to
help families",
"Summary": ""
}
}
}'
From: Matthew Castrigno
Sent: Monday, October 31, 2022 12:27 PM
To: Thomas Corthals
Cc: users@solr.apache.org
Subject: Re: I cannot get nested objects to index - with image links
Hi Thomas, I am attempting to define my fields to accept the data that will
actually be sent, it is more deeply nested. In doing so I have interpreted the
results of the previous configuration to not include (pseudo-)fields. In the
JSON below
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside the St. Luke's email system.
ZjQcmQRYFpfptBannerEnd
Hi Thomas,
I am attempting to define my fields to accept the data that will actually be
sent, it is more deeply nested.
In doing so I have interpreted the results of the previous configuration to not
include (pseudo-)fields. In the JSON below I believe those to be "content" from
the prior example, along with "Page, Body, Modules and Fields" but his gives me
the error shown. How do I handle these other fields. Thank you for your
assistance, it is greatly appreciated.
curl --location --request POST
'https://urldefense.com/v3/__http://localhost:8983/solr/talix/update?json.command=false__;!!FkC3_z_N!PWxH83o-VvWmRGMnhl6_edpBVurmHqqpX-1Hd_Bb3KuXh6tU-eR7XXu2dSYpvb_HgM3r_BxEzve8SA$'
\
--header 'Content-Type: application/json' \
--data-raw '{
"partner": "88027688-62c4-459a-b4d5-a8ecf9edd1bf",
"command": "add",
"doc_id": "2ff99d1a-a21b-4391-9c47-af2865acb753",
"content": {
"Page": {
"Id": "2ff99d1a-a21b-4391-9c47-af2865acb753",
"Name": "Ronald McDonald House Idaho meals",
"Url":
"/blogs/st-lukes/news-and-community/2021/jan/ronald-mcdonald-house-idaho-meals",
"Date": "2022-10-03T12:30:17.3388537",
"ContentType": "Blog",
"Body": {
"Fields": [
{ "Name": "Heading Background Image", "Type": "Image", "Value": "" },
{ "Name": "Tile Wide Image", "Type": "Image", "Value": "" },
{ "Name": "Specialties", "Type": "Treelist", "Value": "" },
{
"Name": "Blog Post Name",
"Type": "Single-Line Text",
"Value": "Ronald McDonald House, St. Luke’s Children’s find new
ways to help families"
},
{ "Name": "Blog Summary", "Type": "Rich Text", "Value": "" },
{ "Name": "Share Summary", "Type": "Multi-Line Text", "Value": "" }
],
"Modules": {
"Fields": [
{ "Name": "Content", "Type": "Rich Text", "Value": "" },
{ "Name": "Image Position", "Type": "Droptree", "Value": "Right" },
{ "Name": "Image Source", "Type": "Image", "Value": "" },
{
"Name": "Image Content",
"Type": "Rich Text",
"Value": "For more than three decades"
}
]
}
},
"Facets": ["Blogs", "Article"],
"Title": "Ronald McDonald House, St. Luke’s Children’s fi
Re: V2 API 404 on all endpoints
Are there any differences between your jetty and solr.xml setup between your local and cloud installation? - Houston On Wed, Oct 26, 2022 at 1:10 PM Stephen Lewis Bianamara < stephen.bianam...@gmail.com> wrote: > Hi Solr Community, > > I have been looking into the v2 API and testing it with solr 9. On a local > installation (single node, cloud mode), I can access the API and it all > works as I would expect. However when I tested the same on a cloud > installation (multi node, multi shard, dedicated overseer, external 3 node > zk cluster) I get a 404 for all endpoints. Even if I query from localhost > (full response and screenshot below). > > I tried looking into the logs, but I couldn't find anything in the solr > logs; the requests themselves don't seem to hit solr (it is logging > requests I make to v1 api but nothing for v2 calls), probably because it's > getting stopped at the jetty level. > > Can anyone suggest a next step for me to look into or have a suspicion > about what the issue might be? > > Thanks, > Stephen > > > > > Error 404 Not Found > > HTTP ERROR 404 Not Found > > URI:/api > STATUS:404 > MESSAGE:Not Found > SERVLET:- > > > > > > > [image: image.png] > >
Re: 8.11 docs "Sending JSON Update Commands" bug?
I don't know if there's a generally accepted name for it (but see https://en.wikipedia.org/wiki/JSON_streaming) -- when you're using JSON to pass around large numbers of objects, it's nice to be able to treat the data as "just a bunch of records" that you can process one by one as they arrive rather than having to read a very large array of objects into memory which you then process. These various kinds of "JSON serialization" see a fair amount of use in the wild, including within Solr. cheers, AC On Fri, Oct 28, 2022 at 6:01 PM dmitri maziuk wrote: > On 2022-10-28 4:26 PM, Mikhail Khludnev wrote: > > Well, Dmitry. Turns out keys in JSON _should be_ but not ought to be > > unique. > > Right, just like a parachute _should_ but not ought to open on your way > down. > > > You can think about streaming writer or reader in any rational > programming > > language. > > Of course I can hand-write the string to be any kind of garbage I want, > and then hand-write a parser to read it. But JSON stands for JavaScript > Object Notation. If a string can't be demarshalled into a valid > JavaScript Object, it goes *splat* on the ground. > > Dima > >
Re: 8.11 docs "Sending JSON Update Commands" bug?
Duplicate keys are somewhat surprising, but absolutely allowed and always have been. From the ECMA JSON spec: "The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs.” https://www.ecma-international.org/publications-and-standards/standards/ecma-404/ wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Oct 31, 2022, at 1:42 PM, Adam Constabaris > wrote: > > I don't know if there's a generally accepted name for it (but see > https://en.wikipedia.org/wiki/JSON_streaming) -- when you're using JSON to > pass around large numbers of objects, it's nice to be able to treat the > data as "just a bunch of records" that you can process one by one as they > arrive rather than having to read a very large array of objects into memory > which you then process. These various kinds of "JSON serialization" see a > fair amount of use in the wild, including within Solr. > > cheers, > > AC > > > On Fri, Oct 28, 2022 at 6:01 PM dmitri maziuk > wrote: > >> On 2022-10-28 4:26 PM, Mikhail Khludnev wrote: >>> Well, Dmitry. Turns out keys in JSON _should be_ but not ought to be >>> unique. >> >> Right, just like a parachute _should_ but not ought to open on your way >> down. >> >>> You can think about streaming writer or reader in any rational >> programming >>> language. >> >> Of course I can hand-write the string to be any kind of garbage I want, >> and then hand-write a parser to read it. But JSON stands for JavaScript >> Object Notation. If a string can't be demarshalled into a valid >> JavaScript Object, it goes *splat* on the ground. >> >> Dima >> >>
Re: 8.11 docs "Sending JSON Update Commands" bug?
Solr does deviate from the 'does not assign any significance to the ordering of name/value pairs' part of that spec though. The order of "add"s and "delete"s within an update request does matter. Thomas Op ma 31 okt. 2022 om 21:50 schreef Walter Underwood : > Duplicate keys are somewhat surprising, but absolutely allowed and always > have been. > > From the ECMA JSON spec: > > "The JSON syntax does not impose any restrictions on the strings used as > names, does not require that name strings be unique, and does not assign > any significance to the ordering of name/value pairs.” > > > https://www.ecma-international.org/publications-and-standards/standards/ecma-404/ > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > > > On Oct 31, 2022, at 1:42 PM, Adam Constabaris > wrote: > > > > I don't know if there's a generally accepted name for it (but see > > https://en.wikipedia.org/wiki/JSON_streaming) -- when you're using JSON > to > > pass around large numbers of objects, it's nice to be able to treat the > > data as "just a bunch of records" that you can process one by one as they > > arrive rather than having to read a very large array of objects into > memory > > which you then process. These various kinds of "JSON serialization" see > a > > fair amount of use in the wild, including within Solr. > > > > cheers, > > > > AC > > > > > > On Fri, Oct 28, 2022 at 6:01 PM dmitri maziuk > > wrote: > > > >> On 2022-10-28 4:26 PM, Mikhail Khludnev wrote: > >>> Well, Dmitry. Turns out keys in JSON _should be_ but not ought to be > >>> unique. > >> > >> Right, just like a parachute _should_ but not ought to open on your way > >> down. > >> > >>> You can think about streaming writer or reader in any rational > >> programming > >>> language. > >> > >> Of course I can hand-write the string to be any kind of garbage I want, > >> and then hand-write a parser to read it. But JSON stands for JavaScript > >> Object Notation. If a string can't be demarshalled into a valid > >> JavaScript Object, it goes *splat* on the ground. > >> > >> Dima > >> > >> > >
Re: 8.11 docs "Sending JSON Update Commands" bug?
On 2022-10-31 4:55 PM, Thomas Corthals wrote: Solr does deviate from the 'does not assign any significance to the ordering of name/value pairs' part of that spec though. The order of "add"s and "delete"s within an update request does matter. Yeah, that's the other problem with using a dict (python-speak) to send a *sequence* of commands: keys of a dict are not intrinsically ordered so the implementation is free to run, say, a commit first, the all the adds, and then the deletes. But since that string cannot be demarshalled into a dict, that makes it all right I guess. Dima
Re: I cannot get nested objects to index - with image links
Hi Matthew
If you set json.command=false your JSON will be treated as custom JSON.
You'll have to remove that parameter to have it treated as Solr JSON. You
can use the full command index:
curl --location --request POST ' :8983/solr/talix/update' \
--header 'Content-Type: application/json' \
--data-raw '
{
"add":{
"doc":{
"doc_id":"parent",
"Name":"Parent",
"child":{
"doc_id":"child",
"Name":"Child"
}
}
}
}'
Or you can wrap your document(s) in a JSON array:
curl --location --request POST ' :8983/solr/talix/update' \
--header 'Content-Type: application/json' \
--data-raw '
[
{
"doc_id":"parent",
"Name":"Parent",
"child":{
"doc_id":"child",
"Name":"Child"
}
}
]'
Thomas
Op ma 31 okt. 2022 om 19:27 schreef Matthew Castrigno :
> Hi Thomas,
> I am attempting to define my fields to accept the data that will actually
> be sent, it is more deeply nested.
> In doing so I have interpreted the results of the previous configuration
> to not include (pseudo-)fields. In the JSON below I believe those to be
> "content" from the prior example, along with "Page, Body, Modules and
> Fields" but his gives me the error shown. How do I handle these other
> fields. Thank you for your assistance, it is greatly appreciated.
>
>
> required="false" multiValued="false" />
> required="false" multiValued="false" />
> required="true" multiValued="false" />
>
> "false" multiValued="false" />
> ="false" multiValued="false" />
> "false" multiValued="false" />
> ="false" multiValued="false" />
> required="false" multiValued="false" />
> required="false" multiValued="false" />
> required="false" multiValued="false" />
> required="false" multiValued="false" />
>
> ="false" multiValued="false" />
> required="false" multiValued="false" />
>
>
>
>
>
>
> docValues="false" />
>
> "_nest_path_" class="solr.NestPathField" />
>
> multiValued="true"/>
>
> curl --location --request POST '
> http://localhost:8983/solr/talix/update?json.command=false' \
> --header 'Content-Type: application/json' \
> --data-raw '{
> "partner": "88027688-62c4-459a-b4d5-a8ecf9edd1bf",
> "command": "add",
> "doc_id": "2ff99d1a-a21b-4391-9c47-af2865acb753",
> "content": {
> "Page": {
> "Id": "2ff99d1a-a21b-4391-9c47-af2865acb753",
> "Name": "Ronald McDonald House Idaho meals",
> "Url":
> "/blogs/st-lukes/news-and-community/2021/jan/ronald-mcdonald-house-idaho-meals",
> "Date": "2022-10-03T12:30:17.3388537",
> "ContentType": "Blog",
> "Body": {
> "Fields": [
> { "Name": "Heading Background Image", "Type": "Image", "Value":
> "" },
> { "Name": "Tile Wide Image", "Type": "Image", "Value": "" },
> { "Name": "Specialties", "Type": "Treelist", "Value": "" },
> {
> "Name": "Blog Post Name",
> "Type": "Single-Line Text",
> "Value": "Ronald McDonald House, St. Luke’s Children’s find
> new ways to help families"
> },
> { "Name": "Blog Summary", "Type": "Rich Text", "Value": "" },
> { "Name": "Share Summary", "Type": "Multi-Line Text", "Value":
> "" }
> ],
> "Modules": {
> "Fields": [
> { "Name": "Content", "Type": "Rich Text", "Value": "" },
> { "Name": "Image Position", "Type": "Droptree", "Value":
> "Right" },
> { "Name": "Image Source", "Type": "Image", "Value": "" },
> {
> "Name": "Image Content",
> "Type": "Rich Text",
> "Value": "For more than three decades"
> }
> ]
> }
> },
> "Facets": ["Blogs", "Article"],
> "Title": "Ronald McDonald House, St. Luke’s Children’s find new ways
> to help families",
> "Summary": ""
> }
> }
> }'
>
> Response:
> {
> "responseHeader":{
> "status":400,
> "QTime":0},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"undefined field: \"content.Page.Id\"",
> "code":400}}
> --
> *From:* Matthew Castrigno
> *Sent:* Thursday, October 27, 2022 10:36 AM
> *To:* Thomas Corthals
> *Cc:* users@solr.apache.org
> *Subject:* Re: I cannot get nested objects to index - with image links
>
> Thank you, Thomas! Changing the required to false allowed the document to
> be indexed. I did not gleam this information from the SOLR documentation
> regarding nested documents. Do you know of any good sources for educational
> material on this subject? I cannot change the payloads that I am working
> with unfortunately. They make heavy use of deeply nested json object. Thank
> you again for your
