Duplicate keys in Luke JSON response
The Luke Request Handler returns duplicate object keys in the JSON response
for multiValued fields. Only the last value survives the trip through a
decoder.
E.g. http://localhost:8983/solr/techproducts/admin/luke?show=doc&id=SP2514N
Snippet of the response:
"cat":{
"type":"string",
"schema":"I-S-UMOF-l",
"flags":"ITS---OF--",
"value":"electronics",
"internal":"electronics",
"docFreq":12},
"cat":{
"type":"string",
"schema":"I-S-UMOF-l",
"flags":"ITS---OF--",
"value":"hard drive",
"internal":"hard drive",
"docFreq":2},
Is there any way to work around this while still using JSON? Or is
switching to XML the only option?
Kind regards
Thomas
About IndexUpgraderTool stored=false fields
Hi, I use only one solr instance. I want to use IndexUpgraderTool to migrate from Solr 7.6 to 8.11.1. Is there any problem when migrating fields with stored=false? Does "stored" have to be "true"? Kind Regards, Cihad Guzel
Re: About IndexUpgraderTool stored=false fields
No, that is not a requirement. If the index was created in 7.6 you should be fine. But do note that if the 7.6 index is a result of an upgrade from an earlier version and you are now upgrading to 8.11.1, Lucene will not allow you and the IndexUpgraderTool can't help you either. -Rahul On Sat, Feb 19, 2022 at 5:56 AM Cihad Guzel wrote: > Hi, > > I use only one solr instance. I want to use IndexUpgraderTool to migrate > from Solr 7.6 to 8.11.1. Is there any problem when migrating fields with > stored=false? Does "stored" have to be "true"? > > Kind Regards, > Cihad Guzel >
migration from 5 to 8
We used the solr Upgrader and were able to migrate from 5 to 6 to 7. For example java -cp lucene-core-7.7.3.jar:lucene-backward-codecs-7.7.3.jar org.apache.lucene.index.IndexUpgrader -delete-prior-commits -verbose $index After each migration we used the CheckIndex to confirm and indeed it reported expected version. For example java -ea:org.apache.lucene... -cp lucene-core-7.7.3.jar:lucene-backward-codecs-7.7.3.jar org.apache.lucene.index.CheckIndex When we tried to upgrade from 7 to 8, that failed with a complaint that the index was still version 6 and updating could only be from the previious (ie 7) version. Indeed running the 8 CheckIndex reported the index was version 6, even though the 7 CheckIndex reported version 7. Oh well, so we're trying another approach. Using the fieldtypes, fields, dynamicfileds, and copyfields API, we copy the schema from 5 to 8. Then we extract the documents as json from 5 and update the json to 8. For example curl -X GET "$SRC/select?q=*%3A*&wt=json&indent=true&start=$start&rows=10" > $tmp 2>/dev/null jq ".response.docs" <$tmp >$tmp2 curl -X POST -H 'Content-type:application/json' --data-binary @$tmp2 "$DST/update/json/docs?commit=true" The documents are showing up in 8. We still need to perform some validation, but are we overlooking some obvious reason, why this might fail? Thanks, Ron
Re: migration from 5 to 8
Doing a full re-index into a clean solr 8 install is the recommended approach. So if you have all fields stored (or docValues) and manage to dump complete documents, then you should be all set. You should be able to page through your entire index with ordinary &start= and &rows= params, but if you have a huge index, look into the more efficient cursorMark instead. Jan > 19. feb. 2022 kl. 22:12 skrev Stanonik, Ronald : > > We used the solr Upgrader and were able to migrate from 5 to 6 to 7. > For example > java -cp lucene-core-7.7.3.jar:lucene-backward-codecs-7.7.3.jar > org.apache.lucene.index.IndexUpgrader -delete-prior-commits -verbose $index > > After each migration we used the CheckIndex to confirm and indeed it reported > expected version. > For example > java -ea:org.apache.lucene... -cp > lucene-core-7.7.3.jar:lucene-backward-codecs-7.7.3.jar > org.apache.lucene.index.CheckIndex > > When we tried to upgrade from 7 to 8, that failed with a complaint that the > index was still version 6 and updating could only be from the previious (ie > 7) version. > Indeed running the 8 CheckIndex reported the index was version 6, even though > the 7 CheckIndex reported version 7. > > Oh well, so we're trying another approach. > > Using the fieldtypes, fields, dynamicfileds, and copyfields API, we copy the > schema from 5 to 8. > > Then we extract the documents as json from 5 and update the json to 8. > For example > curl -X GET "$SRC/select?q=*%3A*&wt=json&indent=true&start=$start&rows=10" > > $tmp 2>/dev/null > jq ".response.docs" <$tmp >$tmp2 > curl -X POST -H 'Content-type:application/json' --data-binary @$tmp2 > "$DST/update/json/docs?commit=true" > > The documents are showing up in 8. We still need to perform some validation, > but are we overlooking some obvious reason, why this might fail? > > Thanks, > > Ron > >
