I found this jira bug which seems suspiciously similar to the issue i'm having with the mariadb jdbc driver. @joelbernstein, what other location should I place the jdbc client jar file to aid its ability to load when using the stream endpoint: https://solr.apache.org/guide/8_11/stream-source-reference.html
Mariadb jdbc driver i'm using: https://dlm.mariadb.com/2346414/Connectors/java/connector-java-3.0.7/mariadb-java-client-3.0.7.jar https://mariadb.com/downloads/connectors/connectors-data-access/java8-connector/ The driver loads/works fine for the DIH endpoint when its placed in the /opt/server/dist folder. However when I use the stream endpoint I get a the jdbc driver failed to load error. On Mon, Sep 12, 2022 at 10:17 AM James Greene <ja...@jamesaustingreene.com> wrote: > The jdbc class/driver loads fine when I use the DIH endpoint. When I use > the same connection string for the stream endpoint I get the class not > found driver error. I also see a msg in solr logs during startup that the > class has been added to the classloader from path 'opt/solr/dist' (where I > have the mariadb jdbc driver jar file). > > On Sun, Sep 11, 2022, 4:05 PM Joel Bernstein <joels...@gmail.com> wrote: > >> I suspect this is a classloader issue. You may have to move the driver >> into >> the lib directory with the Solr core and Solrj libs so that its visible >> to >> both DIH and streaming expressions. >> >> >> Joel Bernstein >> http://joelsolr.blogspot.com/ >> >> >> On Sat, Sep 10, 2022 at 10:18 PM James Greene < >> ja...@jamesaustingreene.com> >> wrote: >> >> > Looking at the solr start logs, I can see that the classes i'm including >> > the in solrconfig.xml are being loaded: >> > >> > 2022-09-11 02:07:05.108 INFO >> > (coreLoadExecutor-21-thread-1-processing-n:172.18.0.5:8983_solr) >> [c:col1 >> > s:shard2 r:core_node4 x:col1_shard2_replica_n2] >> o.a.s.c.SolrResourceLoader >> > Added 1 libs to classloader, from paths: [/opt/solr/dist] >> > >> > Also, the jdbc library loads fine and is usable when I use the DIH >> > endpoint. >> > >> > >> > Query: >> > jdbc(connection="jdbc:mariadb://<host>:3306/<db>",sql="select id from >> tbl", >> > sort="id ASC", driver="org.mariadb.jdbc.Driver", user="<user>", >> > password="<pass>") >> > >> > Error: >> > ""Failed to load JDBC driver for 'org.mariadb.jdbc.Driver'"" >> > >> > solrconfig.xml >> > <lib dir="${solr.install.dir}/dist/" >> regex="mariadb-java-client-.*\.jar" /> >> > <expressible name="jdbc" >> > class="org.apache.solr.client.solrj.io.stream.JDBCStream"/> >> > >> > Cheers, >> > >> > /************************************ >> > * James Austin Greene >> > * www.jamesaustingreene.com >> > * 336-lol-nerd >> > ************************************/ >> > >> > >> > On Fri, Sep 9, 2022 at 5:01 PM Joel Bernstein <joels...@gmail.com> >> wrote: >> > >> > > You may be having classloader issues. I'm not sure where you have the >> > > driver jar, but it may need to be copied into the lib directory with >> core >> > > Solr jars. >> > > >> > > >> > > Joel Bernstein >> > > http://joelsolr.blogspot.com/ >> > > >> > > >> > > On Fri, Sep 9, 2022 at 4:58 PM Joel Bernstein <joels...@gmail.com> >> > wrote: >> > > >> > > > Can you post the stack trace from the log? >> > > > >> > > > It looks like the JDBCStream is registered, but the stack trace >> > > > may provide some more info on why the driver didn't load. >> > > > >> > > > Joel Bernstein >> > > > http://joelsolr.blogspot.com/ >> > > > >> > > > >> > > > On Fri, Sep 9, 2022 at 12:51 PM James Greene < >> > > ja...@jamesaustingreene.com> >> > > > wrote: >> > > > >> > > >> I'm still unable to get the JDBC stream to work. I've verified >> that >> > > >> 'jdbc' >> > > >> shows up in the plugin list: >> > > >> >> > > >> http://localhost:8983/solr/col1/stream?action=PLUGINS >> > > >> >> > > >> When using the DIH I use the jdbc driver: >> > > >> org.mariadb.jdbc.Driver >> > > >> >> > > >> With connection string: >> > > >> jdbc:mysql://{{host}}:{{port}}/{{database}} >> > > >> >> > > >> I am able to get DIH pulling data from mariadb with that connection >> > > >> string. >> > > >> >> > > >> However using the same connection string in the streaming >> expression: >> > > >> jdbc(connection="jdbc:mysql://mariadb:3306/db",sql="select id >> from >> > > tbl >> > > >> order by id asc limit 1", sort="id asc", >> > > driver="org.mariadb.jdbc.Driver") >> > > >> >> > > >> I get the error: >> > > >> "EXCEPTION": "Failed to load JDBC driver for >> > > >> 'org.mariadb.jdbc.Driver'" >> > > >> >> > > >> I tried upgrading jdbc connector to 3x version and using the >> > connection >> > > >> string "jdbc:mariadb://<host>:<port>/<db>" and get the same error >> > > message. >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> On Thu, Sep 8, 2022 at 10:16 PM Joel Bernstein <joels...@gmail.com >> > >> > > >> wrote: >> > > >> >> > > >> > The jdbc stream was removed from the standard expression library >> for >> > > >> > security reasons. It is still in the jars though and can be >> included >> > > >> > through a change in the solrconfig.xml. >> > > >> > >> > > >> > This jira describes how to plugin an expression: >> > > >> > >> > > >> > https://issues.apache.org/jira/browse/SOLR-9103 >> > > >> > >> > > >> > The class is: org.apache.solr.client.solrj.io.stream.JDBCStream >> > > >> > >> > > >> > >> > > >> > >> > > >> > Joel Bernstein >> > > >> > http://joelsolr.blogspot.com/ >> > > >> > >> > > >> > >> > > >> > On Thu, Sep 8, 2022 at 6:54 PM James Greene < >> > > >> ja...@jamesaustingreene.com> >> > > >> > wrote: >> > > >> > >> > > >> > > I'm on solr 8.11.1 and trying to use the 'jdbc' streaming >> > expression >> > > >> as >> > > >> > > documented here: >> > > >> > > >> > > https://solr.apache.org/guide/8_11/stream-source-reference.html#jdbc >> > > >> > > >> > > >> > > When I try to run a jdbc stream expression I get the error: >> > > >> > > 'function 'jdbc' is unknown (not mapped to a valid TupleStream' >> > > >> > > >> > > >> > > Here is an example of the expression i'm trying to run: >> > > >> > > >> > > >> >> > jdbc(connection="jdbc:mysql://mariadb:3306/threatquotient2",sql="select >> > > >> > id >> > > >> > > from data order by id asc limit 1", sort="id asc" >> > > >> > > driver="org.mariadb.jdbc.Driver") >> > > >> > > >> > > >> > > I use the mariadb jdbc driver to get data from our mariadb >> > instance >> > > >> using >> > > >> > > the data import handler so the jdbc driver should be property >> > > loaded. >> > > >> > > >> > > >> > >> > > >> >> > > > >> > > >> > >> >
