The database connection count to my database goes
After Full import 1: 25 connections
After Full import 2: 50 connections
After Full import 3: 73 connections
After Full import 4: 90 connections
Exception about reading, that is no doubt related to lack of
cleanup of database connections
When I shut down solr the connections are cleared. Starting solr and
performing 4 full updates
produces the same connection count increase until failure.
This is most probably due to a resource problem due to never releasing
the connections:
"oracle.net.ns.NetException: Got minus one from a read call " is a driver
or system specific issue we can hardly blame DIH for it.
Overall DIH might be a slightly outdated ETL solution nowadays.
Btw, do you run standalone solr or what we call SolrCloud mode?
In contrast during the full import while running 8.4.1 the connection
count goes up to 25 and
then when the status of full import is complete the connections goes
back to zero.
It is clear the connections are not being released and this is
problematic. Who might investigate this
in the data import handler code?
On 2/14/2025 11:30 PM, Mikhail Khludnev wrote:
Hello, Mike.
First of all DIH left solr and is maintaited at
https://github.com/SearchScale/dataimporthandler
Personally, I never rely on deltaImport but rather come along with full
imports with a special form of SQL SELECTs.
Hanging full import is an issue. May you check that heap is enough? Enlarge
it if necessary? Look at heapdump after all.
"oracle.net.ns.NetException: Got minus one from a read call " is a driver
or system specific issue we can hardly blame DIH for it.
Overall DIH might be a slightly outdated ETL solution nowadays.
Btw, do you run standalone solr or what we call SolrCloud mode?
On Sat, Feb 15, 2025 at 2:42 AM Mike Phillips <
m.phill...@prosperodigital.com> wrote:
Summary:
Performing multiple delta imports causes solr to fail imports until the
solr application is stopped and started.
Performing multiple full imports causes solr to fail imports until the
solr application is stopped and started.
Solr behaves normally after stopped and started and Imports are not
performed.
On the 4th delta impor or 6th full import I see read call exceptions.
rt.DataImportHandlerException: Unable to execute query: select mds.id,
mds.defn_id, mdef.md_key, mdsddd.id PICK_ID, mdsddd.code
CUSTOM_MD_DEFN_DROP_DOWN FROM custom_md mds LEFT OUTER JOIN
custom_md_defn mdef ON mds.defn_id=mdef.id LEFT OUTER JOIN
custom_md_defn_drop_down mdsddd ON mds.pick_id=mdsddd.id WHERE
mds.pick_id IS NOT null AND asset_id='8' Processing Document # 1
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:69)
org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
execute query: select mds.id, mds.defn_id, mdef.md_key, mdsddd.id
PICK_ID, mdsddd.code CUSTOM_MD_DEFN_DROP_DOWN FROM custom_md mds LEFT
OUTER JOIN custom_md_defn mdef ON mds.defn_id=mdef.id LEFT OUTER JOIN
custom_md_defn_drop_down mdsddd ON mds.pick_id=mdsddd.id WHERE
mds.pick_id IS NOT null AND asset_id='8' Processing Document # 1
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:69)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:326)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource.createResultSetIterator(JdbcDataSource.java:287)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:282)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:51)
~[?:?]
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
~[?:?]
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
~[?:?]
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:267)
~[?:?]
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:494)
~[?:?]
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:535)
~[?:?]
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:433)
~[?:?]
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:348)
~[?:?]
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:236)
~[?:?]
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:424)
~[?:?]
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
~[?:?]
at
org.apache.solr.handler.dataimport.DataImporter.lambda$runAsync$0(DataImporter.java:466)
~[?:?]
at java.base/java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.sql.SQLRecoverableException: IO Error: Got minus one
from a read call
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:854)
~[?:?]
at
oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:793)
~[?:?]
at
oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:57)
~[?:?]
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:747)
~[?:?]
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:562)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:191)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:171)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:527)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:316)
~[?:?]
... 15 more
Caused by: oracle.net.ns.NetException: Got minus one from a read call
at oracle.net.ns.NSProtocolNIO.doSocketRead(NSProtocolNIO.java:557)
~[?:?]
at oracle.net.ns.NIOPacket.readNIOPacket(NIOPacket.java:403) ~[?:?]
at
oracle.net.ns.NSProtocolNIO.negotiateConnection(NSProtocolNIO.java:127)
~[?:?]
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:340) ~[?:?]
at
oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1596) ~[?:?]
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:588)
~[?:?]
at
oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:793)
~[?:?]
at
oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:57)
~[?:?]
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:747)
~[?:?]
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:562)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:191)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:171)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:527)
~[?:?]
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:316)
~[?:?]
... 15 more
I see many of these exceptions. Sometimes when performint the imports
and sometimes when shutting down solr after imports.
All connections should be closed after they have been used or this is
typical behavior. I have worked on multiple projects where lack of
closing database connections has been a real problem
2025-02-14 15:25:53.707 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.711 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.712 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.714 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.715 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.717 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.718 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.719 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.721 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.722 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.723 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.725 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.726 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.728 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.729 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.730 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.731 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.733 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.734 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.736 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.737 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.738 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.739 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.741 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
2025-02-14 15:25:53.742 ERROR (Finalizer) [c: s: r: x: t:]
o.a.s.h.d.JdbcDataSource JdbcDataSource was not closed prior to
finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
