Thanks Markus, The field does not need to be returned in the response so setting stored to false.
Regards, Ryan On Wed, Jul 13, 2022, 06:02 Markus Jelsma <markus.jel...@openindex.io> wrote: > Hello Ryan, > > Solr should have no issues for indexing these kinds of documents, > retrieving them is another thing. For multiValued a lot of > datastructures are created when serializing the document, that is > probably the problem. > > If you do not need the field returned, do not stored=true the field. > If you do, how about making it not multiValued, but store all values > using a delimiter. Use analysis to split on the delimiter so you can > still search for it. When retrieving the document, it should just spit > out a massive field. > > Such a large field will still take some time, but it should take less > time and eat less memory than that many multiValued values. > Deserializing should also take less time, imagine your browser neatly > showing that many objects in XML format. > > Regards, > Markus > > 2022-07-12 15:23 GMT+02:00, Ryan Yacyshyn <ryan.yacys...@gmail.com>: > > Hi all, > > > > I have a data modelling question. A department of ours is indexing > > companies and including a multivalued field in the doc to store a set of > > tickers related to the company. Sounded fine at first until I saw one > > document where the multivalued field had over 50,000 elements in it. > There > > are other documents that are storing even more than this because Solr > will > > hang and won't return the document. > > > > This is a *_ss field, and their query would be an exact match to one of > the > > values in this field to return the company name, eg: > > ticker_fieldname_ss:"AAA NOE 12/31/21". > > > > I'm wondering what would be the best solution to index this data and > search > > for the ticker information on a company? > > > > One way I'm thinking of is rather than storing the ticker information as > a > > field on the company, have each ticker its own document with the > fieldname > > of the company. > > > > Would love to hear your thoughts on this. > > > > Thanks! > > > > Regards, > > Ryan > > >
