epugh opened a new pull request, #4863: URL: https://github.com/apache/solr/pull/4863
# Description Tika 4.x's `tika-inference` module can split a parsed document into semantically-bounded chunks (splitting on Markdown headings/paragraphs) and attach a precomputed embedding vector to each, via a server-side embedding filter (e.g. `openai-embedding-filter`). This PR teaches Solr Cell (`ExtractingRequestHandler`) to read those chunks back out and index them directly. # Solution Mostly AI generated... I need to better understand what Tika 4 views as it's place in the world if we aren't just capturing attributes from a tree structure, and indeed, appear to have dumped the tree concept in favor of a markdown file! Maybe these days, all that rich parsing and extraction we used to have has turned into "just give me markdown", but what about images? What about hOCR? # Tests Please describe the tests you've developed or run to confirm this patch implements the feature or solves the problem. # Checklist Please review the following and check all that apply: - [ ] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [ ] I have developed this patch against the `main` branch. - [ ] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) - [ ] I have added a [changelog entry](https://github.com/apache/solr/blob/main/dev-docs/changelog.adoc) for my change -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
