Re: Retain Data Import Handler In Solr9.0

2022-07-23 Thread David Hastings
Guess it depends on how many scripts you want to maintain/things to do by hand, but in any case, it is the best route, multiple indexing services/processes, and skip the DIH all together, it wasnt that great of an idea in the first place. it was super clever, and I appreciate the work that went in

Re: Retain Data Import Handler In Solr9.0

2022-07-22 Thread Andy Lester
> On Jul 22, 2022, at 1:39 PM, Dave wrote: > > Oh look into perls fork manager module, > > https://metacpan.org/pod/Parallel::ForkManager > I’m aware of the numerous tools like that (I’ve been doing Perl since the 90s https://metacpan.or

Re: Retain Data Import Handler In Solr9.0

2022-07-22 Thread Dave
Oh look into perls fork manager module, https://metacpan.org/pod/Parallel::ForkManager . Only trick is each time it spawns a process you have to redeclare the dbh and any stored procedures but it’s a small price to pay for being able to simply adjust the number of parallel jobs it will do

Re: Retain Data Import Handler In Solr9.0

2022-07-22 Thread Dave
Not to mention using dynamic fields on the fly in the indexer, applying code logic to the documents and just having full control over it has a lot of benefits to the point that a DIH was a cute idea when it came out but it reality it was just hand holding > On Jul 22, 2022, at 2:19 PM, dmitri m

Re: Retain Data Import Handler In Solr9.0

2022-07-22 Thread Andy Lester
> On Jul 22, 2022, at 1:19 PM, dmitri maziuk wrote: > >> The DIH does not yet support Solr 9 but I don't think it'll be long before >> it does. > > FWIW I've been gradually migrating our DIH imports to little python scripts; > with all the extra things you can do in those, and less bloat in t

Re: Retain Data Import Handler In Solr9.0

2022-07-22 Thread dmitri maziuk
On 2022-07-22 8:46 AM, David Smiley wrote: The DIH does not yet support Solr 9 but I don't think it'll be long before it does. FWIW I've been gradually migrating our DIH imports to little python scripts; with all the extra things you can do in those, and less bloat in the main JVM, you gotta

Re: Retain Data Import Handler In Solr9.0

2022-07-22 Thread David Smiley
The DIH does not yet support Solr 9 but I don't think it'll be long before it does. https://github.com/rohitbemax/dataimporthandler/issues/32 Note the dubious choice of the word "deprecated" was used because it's no longer a part of Solr. Practically speaking, it *moved* and isn't gone. ~ David

Re: Retain Data Import Handler In Solr9.0

2022-07-22 Thread Charlie Hull
You have two options basically: 1. consider using the externally maintained DIH from https://github.com/rohitbemax/dataimporthandler (there appears to be a 9.x branch) 2. move away from DIH and write new ingestor code to pull data from your sources Cheers Charlie On 22/07/2022 04:38, GAN