Agreed. I like either waiting until the entire thing is done or a modulus of like 1000 if you’re worried about it failing part way through and it’s big. The less commits the better
> On Jan 13, 2023, at 10:49 AM, Thomas Corthals <tho...@klascement.net> wrote: > > Build an indexer that can send updates in batches. It'll be faster than > sending each document in a separate request. > > Op vr 13 jan. 2023 om 16:41 schreef Dave <hastings.recurs...@gmail.com>: > >> Yeah, it’s trivial building your own indexer in any language that can read >> a db. Also I wouldn’t trust the dih on its own even when supported >> >>>> On Jan 13, 2023, at 10:17 AM, Jan Høydahl <jan....@cominvent.com> wrote: >>> >>> I don't think the 3rd party DIH is maintained. >>> >>> Other options are using other 3rd party frameworks like >>> ManifoldCF < >> https://datafari.atlassian.net/wiki/spaces/DATAFARI/pages/2718007297/How+to+crawl+a+database+on+Solr+without+DIH+-+DIH+alternative> >> or Apache NiFI < >> https://www.datainmotion.dev/2020/04/building-search-indexes-with-apache.html> >> which is a more visual approach. >>> Personally I'd write a small Python program. >>> >>> When asking ChatGPT to sketch such a program, this is what I got :) >>> >>> Sure, here is a sample Python program that demonstrates how to import >> data from a MySQL database and index it into Solr using the SolrPy library: >>> >>> import mysql.connector >>> from solr import Solr >>> >>> # Connect to MySQL database >>> cnx = mysql.connector.connect(user='username', >>> password='password', >>> host='hostname', >>> database='database_name') >>> cursor = cnx.cursor() >>> >>> # Execute MySQL query >>> query = "SELECT * FROM table_name" >>> cursor.execute(query) >>> >>> # Fetch all data >>> data = cursor.fetchall() >>> >>> # Connect to Solr >>> solr = Solr('http://localhost:8983/solr/core_name') >>> >>> # Index data into Solr >>> for row in data: >>> solr.add(row) >>> >>> # Commit changes >>> solr.commit() >>> >>> # Close database and Solr connection >>> cursor.close() >>> cnx.close() >>> solr.close() >>> >>> >>> I have not tested it, but with some massage you should be able to fly >>> >>> Jan >>> >>>> 13. jan. 2023 kl. 16:02 skrev Markus Jelsma <markus.jel...@openindex.io >>> : >>>> >>>> Hello, >>>> >>>> DIH was removed in Solr 9.x. You can either build your own importer, or >> use >>>> the, now 3rd party, DIH [1] that is available as a package. >>>> >>>> Regards, >>>> Markus >>>> >>>> [1] https://github.com/rohitbemax/dataimporthandler >>>> >>>>> Op vr 13 jan. 2023 om 14:47 schreef Anuj Bhargava <anujb...@gmail.com >>> : >>>>> >>>>> How do I import data from MySql database in Solr 9.1 >>>>> >>>>> Earlier I was using DIH in version 8.6.2 >>>>> >>>>> Regards, >>>>> >>>>> Anuj >>>>> >>> >>
