On 9/26/2018 2:39 PM, Terry Steichen wrote:
To the best of my knowledge, I'm not using SolrJ at all. Just Solr-out-of-the-box. In this case, if I understand you below, it "should indicate an error status"
I think you'd know if you were using SolrJ directly. You'd have written the indexing program, or whoever DID write it would likely indicate that they used SolrJ to talk to Solr. I was surprised to learn that SimplePostTool does NOT use SolrJ ... it uses the HTTP capability built into Java.
Let me try to clarify a bit - I'm just using bin/post to index the files in a directory. That indexing process produces a lengthy screen display of files that were indexed. (I realize this isn't production-quality, but I'm not ready for production just yet, so that should be OK.)
If you check your index, are you missing files that bin/post said were indexed? Have you looked in that kind of detail?
The post tool should indicate that an error occurred, and if there was any text in the response about the error, it should be displayed. I was looking at the 7.4 code branch. I didn't see anything about which Solr version you're running.
I have not spent any real time using bin/post. It was part of a class that I attended as part of Lucene Revolution in 2010, but I do not recall what the output was. It was all pre-designed and tested so it was known to work before I received it. No errors occurred when I ran the script included with the class materials.
But no errors are shown (even though there have to be because the totals indexed is less than the directory totals). Are you saying I can't use post (to verify correct indexing), but that I have to write custom software to accomplish that?
If you want errors detected programmatically, you'll need to write the indexing program. The simple post tool won't report errors to anything that calls it, it will just log them.
And that there's no solr variable I can define that will do a kind of "verbose" to show that?
If Solr returned errors during the indexing, then they will show up in the solr.log file, or possibly one of the rotated versions of that logfile. You can also see them in the admin UI Logging tab if Solr hasn't been restarted, but the logfile is generally a better way to find them. If you're not seeing errors there, then maybe something went wrong with bin/post.
I notice in a later message you indicate that you're indexing PDF and DOC files. When those kinds of files are sent with bin/post, they will normally end up in the Extracting Request Handler, also known as SolrCell.
It is highly recommended that the Extracting Request Handler never be used in production. That software embeds Tika inside Solr. Tika is known to explode spectacularly when it gets a file it doesn't know how to handle. PDF files in particular seem to trigger this behavior, but other formats can cause it as well. If Tika is running inside Solr when that happens, Solr will also explode, and then you no longer have a search engine on that machine. A better option is to include Tika in an indexing program that you write, so if it explodes, Solr stays running.
Thanks, Shawn
