As part of migration we converted csv data by creating multiple json files each consisting around 100mb data and then wrote a small shell script to inject these files through solr api in loop .
Just make sure when If you have multiple nodes then it might take some time to get the replication done . On Fri, Feb 10, 2023 at 12:57 PM marc nicole <mk1853...@gmail.com> wrote: > what is a common use case then if it is not the csv type? > how to index massively data into Solr using SolrJ > You can't just read line by line each dataset you want to index. > > Le lun. 30 janv. 2023 à 14:11, Jan Høydahl <jan....@cominvent.com> a > écrit : > > > It's not a common use case for SolrJ to post plain CSV content to Solr. > > SolrJ is used to push SolrInputDocument objects. Maybe there's a way to > do > > it by using some Generic request type and overriding content type.. Can > you > > explain more what you app will do, where that CSV file comes from in the > > first place and why you'd want to use SolrJ to move it to Solr, rather > than > > curl or some other http client lib? > > > > Jan > > > > > 29. jan. 2023 kl. 20:44 skrev marc nicole <mk1853...@gmail.com>: > > > > > > The Java code should perform the post. Any piece of code to show to > > better > > > explain this? > > > > > > thanks > > > > > > Le dim. 29 janv. 2023 à 20:29, Jan Høydahl <jan....@cominvent.com> a > > écrit : > > > > > >> Read csv in your app, create a Solr doc from each line and ingest to > > Solr > > >> in fitting batches. You can use a csv library or just parse each line > > >> yourself if the format is fixed. > > >> > > >> If you need to post csv directly to Solr you’d use a plain http post > > with > > >> content-type csv, but in most cases your app would do that. > > >> > > >> Jan Høydahl > > >> > > >>> 29. jan. 2023 kl. 20:21 skrev marc nicole <mk1853...@gmail.com>: > > >>> > > >>> Hi guys, > > >>> > > >>> I can't find a reference on how to index a dataset.csv file into Solr > > >> using > > >>> SolrJ. > > >>> https://solr.apache.org/guide/6_6/using-solrj.html > > >>> > > >>> Thanks. > > >> > > > > >
