Re: About Using Hadoop in SolrCloud

Yes; this was shocking to me at first because the implications are big and it's almost a secret. Ideally the ref guide would scream this loudly; users today care *way* more about S3 than HDFS. The "HDFS" Solr module uses the HDFS client API which has a pluggable back-end, and thus you can have it

Re: About Using Hadoop in SolrCloud

David, you made a point. Is it true we can keep indexes to S3? I mean index under use not the backup ? On Fri, Feb 24, 2023 at 1:11 AM David Smiley wrote: > I agree with Eric, but wish to add one point: Separation of compute from > storage to get: better redundancy (HDFS or S3 will do it better

Re: About Using Hadoop in SolrCloud

I agree with Eric, but wish to add one point: Separation of compute from storage to get: better redundancy (HDFS or S3 will do it better, maybe cheaper), better elasticity (since Solr nodes become stateless; easy to add more nodes), better cost? Sacrifice indexing performance and a bit of query.

Re: About Using Hadoop in SolrCloud

;-). > On Feb 23, 2023, at 9:34 AM, Zara Parst wrote: > > I think I was looking for someone to tell me, hey don't do something fancy > unless you must have. I am somehow at ease now. Now I will leave Hadoop for > some other project. > > On Thu, Feb 23, 2023 at 6:16 PM Eric Pugh

Re: About Using Hadoop in SolrCloud

I think I was looking for someone to tell me, hey don't do something fancy unless you must have. I am somehow at ease now. Now I will leave Hadoop for some other project. On Thu, Feb 23, 2023 at 6:16 PM Eric Pugh wrote: > I am replying, but just to the users mailing list, as it’s not appropriate

Re: About Using Hadoop in SolrCloud

I am replying, but just to the users mailing list, as it’s not appropriate for dev@. I think the short answer is that if you are already super into the Hadoop ecosystem, then you already have strong reasons why, and you can answer all of your questions listed already ;-). You then look at Solr