Re: [External] : Re: [Query] [Solr 8.11.1] Retrieving non stored fields data

[Nagarajan Muthupandian]

Hi Jan,

Sharing the query response.. (“ curl 
'http://localhost:8983/solr/TestCore_1/select?q=*:*&wt=xml&indent=true' ”)
<?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
  <int name="status">0</int>
  <int name="QTime">3</int>
  <lst name="params">
    <str name="q">*:*</str>
    <str name="indent">true</str>
    <str name="wt">xml</str>
  </lst>
</lst>
<result name="response" numFound="3" start="0" numFoundExact="true">
  <doc>
    <str name="id">1</str>
    <str name="_src_">{
 "id": "1",
  "title": "Test 1"
  "test_field_1": "test_field_1_1"
  "test_field_2": "test_field_2_1"
  "test_field_3": "test_field_3_1"
}</str>
    <long name="_version_">1747295048719925248</long></doc>
  <doc>
    <str name="id">2</str>
    <str name="_src_">{
  "id": "2",
  "title": "Test 2"
  "test_field_1": "test_field_1_2"
  "test_field_2": "test_field_2_2"
  "test_field_3": "test_field_3_2"
}</str>
    <long name="_version_">1747295048722022400</long></doc>
  <doc>
    <str name="id">3</str>
    <str name="_src_">{
  "id": "3",
  "title": "Test 3"
  "test_field_1": "test_field_1_3"
  "test_field_2": "test_field_2_3"
  "test_field_3": "test_field_3_3"
}</str>
    <long name="_version_">1747295048722022401</long></doc>
</result>
</response>

Thanks
Rajan


From: Jan Høydahl <jan....@cominvent.com>
Date: Wednesday, 2 November 2022 at 8:13 PM
To: users@solr.apache.org <users@solr.apache.org>
Cc: Nagarajan Muthupandian <nagarajan.muthupand...@oracle.com>
Subject: [External] : Re: [Query] [Solr 8.11.1] Retrieving non stored fields 
data
Hi,

Your screenshot image may not have made it to the list (I did not see it), so 
can you please upload it somewhere and share the link, or paste in your query 
and query results as text.

Jan


2. nov. 2022 kl. 11:55 skrev Nagarajan Muthupandian 
<nagarajan.muthupand...@oracle.com>:

Hi,

I am working on POC of Re-Indexing for Solr Upgrade ( 8 to 9 ).
Idea is to query the existing index data and re-index it to the new version.


3 Important field attributes when we query “q=*:*’ to retrieve the data.

stored
docValues
useDocValuesAsStored



I tried different combinations of 3 important field attributes as below.. And I 
found I was able to retrieve all the fields data in all combinations, which 
contradicts the definition of the fields.

Wanted to understand if I am missing out something.. Can you help me understand 
the below behaviour. Sharing the test example which I tried in my local setup.

Examples…

Field Types :
<fieldType name="test_field_1" class="solr.TextField"
                 indexed="true"
                 stored="false"
                 docValues="true"
                 useDocValuesAsStored="true"
                 omitNorms="true"
                 positionIncrementGap="100">
      <analyzer type="index">
          <tokenizer class="solr.KeywordTokenizerFactory"/>
          <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
      <analyzer type="query">
          <tokenizer class="solr.KeywordTokenizerFactory"/>
          <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
    </fieldType>
   <fieldType name="test_field_2" class="solr.TextField"
                 indexed="true"
                 stored="false"
                 docValues="true"
                 useDocValuesAsStored="false"
                 omitNorms="true"
                 positionIncrementGap="100">
      <analyzer type="index">
          <tokenizer class="solr.KeywordTokenizerFactory"/>
          <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
      <analyzer type="query">
          <tokenizer class="solr.KeywordTokenizerFactory"/>
          <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
    </fieldType>
   <fieldType name="test_field_3" class="solr.TextField"
                 indexed="true"
                 stored="false"
                 docValues="false"
                 useDocValuesAsStored="false"
                 omitNorms="true"
                 positionIncrementGap="100">
      <analyzer type="index">
          <tokenizer class="solr.KeywordTokenizerFactory"/>
          <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
      <analyzer type="query">
          <tokenizer class="solr.KeywordTokenizerFactory"/>
          <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
    </fieldType>

Fields
<field name="test_field_1" type="test_field_1" />
<field name="test_field_2" type="test_field_2" />
<field name="test_field_3" type="test_field_3" />


Upload



curl -X POST -H 'Content-Type: application/json' 
'http://localhost:8983/solr/TestCore_1/update/json/docs<https://urldefense.com/v3/__http:/localhost:8983/solr/TestCore_1/update/json/docs__;!!ACWV5N9M2RV99hQ!JCGp0uVPoPNwlQEfyjAi1kq5b4Txe4okmdhqZK1yTsi44RWYDlCNQGgTzZp1jJR6U5YZuAGrTPSC0e0HJhNHuRTq7-IY$>'
 --data-binary '
{
 "id": "1",
  "title": "Test 1"
  "test_field_1": "test_field_1_1"
  "test_field_2": "test_field_2_1"
  "test_field_3": "test_field_3_1"
}
{
  "id": "2",
  "title": "Test 2"
  "test_field_1": "test_field_1_2"
  "test_field_2": "test_field_2_2"
  "test_field_3": "test_field_3_2"
}
{
  "id": "3",
  "title": "Test 3"
  "test_field_1": "test_field_1_3"
  "test_field_2": "test_field_2_3"
  "test_field_3": "test_field_3_3"
}'


Query..

[cid:image001.png@01D8EED7.BB792E10]



Thanks
Rajan