GitHub user jnioche added a comment to the discussion: Proposal & Feedback: Native Integration with OpenCrawling for High-Throughput RAG & AI Vector Ingestion
hi @OpenPj Thanks for reaching out. Great to hear that you like StormCrawler and want to integrate it in OpenCrawling. The option (1) should be straightforward and a good way to start integrating with OpenCrawling; (2) is more complex and could be done later on. The OpenCrawlingBolt should extend [AbstractIndexerBolt](https://github.com/apache/stormcrawler/blob/main/core/src/main/java/org/apache/stormcrawler/indexing/AbstractIndexerBolt.java) - it functionality is similar to other such Indexer bolts (OpenSearch, SOLR etc...) and logic of handling updates in the StatusUpdaterBolt is already implemented. Hope this makes sense GitHub link: https://github.com/apache/stormcrawler/discussions/2073#discussioncomment-18164242 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
