Hello, Mike. First of all DIH left solr and is maintaited at https://github.com/SearchScale/dataimporthandler Personally, I never rely on deltaImport but rather come along with full imports with a special form of SQL SELECTs. Hanging full import is an issue. May you check that heap is enough? Enlarge it if necessary? Look at heapdump after all.
"oracle.net.ns.NetException: Got minus one from a read call " is a driver or system specific issue we can hardly blame DIH for it. Overall DIH might be a slightly outdated ETL solution nowadays. Btw, do you run standalone solr or what we call SolrCloud mode? On Sat, Feb 15, 2025 at 2:42 AM Mike Phillips < m.phill...@prosperodigital.com> wrote: > Summary: > > Performing multiple delta imports causes solr to fail imports until the > solr application is stopped and started. > > Performing multiple full imports causes solr to fail imports until the > solr application is stopped and started. > > Solr behaves normally after stopped and started and Imports are not > performed. > > On the 4th delta impor or 6th full import I see read call exceptions. > > rt.DataImportHandlerException: Unable to execute query: select mds.id, > mds.defn_id, mdef.md_key, mdsddd.id PICK_ID, mdsddd.code > CUSTOM_MD_DEFN_DROP_DOWN FROM custom_md mds LEFT OUTER JOIN > custom_md_defn mdef ON mds.defn_id=mdef.id LEFT OUTER JOIN > custom_md_defn_drop_down mdsddd ON mds.pick_id=mdsddd.id WHERE > mds.pick_id IS NOT null AND asset_id='8' Processing Document # 1 > at > > org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:69) > org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to > execute query: select mds.id, mds.defn_id, mdef.md_key, mdsddd.id > PICK_ID, mdsddd.code CUSTOM_MD_DEFN_DROP_DOWN FROM custom_md mds LEFT > OUTER JOIN custom_md_defn mdef ON mds.defn_id=mdef.id LEFT OUTER JOIN > custom_md_defn_drop_down mdsddd ON mds.pick_id=mdsddd.id WHERE > mds.pick_id IS NOT null AND asset_id='8' Processing Document # 1 > at > org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:69) > > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:326) > > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource.createResultSetIterator(JdbcDataSource.java:287) > > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:282) > > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:51) > > ~[?:?] > at > org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59) > > ~[?:?] > at > org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73) > > ~[?:?] > at > org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:267) > > ~[?:?] > at > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:494) > > ~[?:?] > at > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:535) > > ~[?:?] > at > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:433) > > ~[?:?] > at > org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:348) > > ~[?:?] > at > org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:236) > ~[?:?] > at > org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:424) > > ~[?:?] > at > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483) > > ~[?:?] > at > org.apache.solr.handler.dataimport.DataImporter.lambda$runAsync$0(DataImporter.java:466) > > ~[?:?] > at java.base/java.lang.Thread.run(Thread.java:834) [?:?] > Caused by: java.sql.SQLRecoverableException: IO Error: Got minus one > from a read call > at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:854) > ~[?:?] > at > oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:793) > ~[?:?] > at > oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:57) > > ~[?:?] > at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:747) > ~[?:?] > at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:562) > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:191) > > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:171) > > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:527) > > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:316) > > ~[?:?] > ... 15 more > Caused by: oracle.net.ns.NetException: Got minus one from a read call > at oracle.net.ns.NSProtocolNIO.doSocketRead(NSProtocolNIO.java:557) > ~[?:?] > at oracle.net.ns.NIOPacket.readNIOPacket(NIOPacket.java:403) ~[?:?] > at > oracle.net.ns.NSProtocolNIO.negotiateConnection(NSProtocolNIO.java:127) > ~[?:?] > at oracle.net.ns.NSProtocol.connect(NSProtocol.java:340) ~[?:?] > at > oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1596) ~[?:?] > at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:588) > ~[?:?] > at > oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:793) > ~[?:?] > at > oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:57) > > ~[?:?] > at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:747) > ~[?:?] > at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:562) > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:191) > > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:171) > > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:527) > > ~[?:?] > at > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:316) > > ~[?:?] > ... 15 more > > > I see many of these exceptions. Sometimes when performint the imports > and sometimes when shutting down solr after imports. > > All connections should be closed after they have been used or this is > typical behavior. I have worked on multiple projects where lack of > closing database connections has been a real problem > > 2025-02-14 15:25:53.707 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.711 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.712 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.714 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.715 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.717 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.718 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.719 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.721 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.722 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.723 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.725 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.726 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.728 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.729 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.730 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.731 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.733 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.734 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.736 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.737 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.738 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.739 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.741 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > 2025-02-14 15:25:53.742 ERROR (Finalizer) [c: s: r: x: t:] > o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to > finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! > -- Sincerely yours Mikhail Khludnev
