I suspect this is a classloader issue. You may have to move the driver into the lib directory with the Solr core and Solrj libs so that its visible to both DIH and streaming expressions.
Joel Bernstein http://joelsolr.blogspot.com/ On Sat, Sep 10, 2022 at 10:18 PM James Greene <ja...@jamesaustingreene.com> wrote: > Looking at the solr start logs, I can see that the classes i'm including > the in solrconfig.xml are being loaded: > > 2022-09-11 02:07:05.108 INFO > (coreLoadExecutor-21-thread-1-processing-n:172.18.0.5:8983_solr) [c:col1 > s:shard2 r:core_node4 x:col1_shard2_replica_n2] o.a.s.c.SolrResourceLoader > Added 1 libs to classloader, from paths: [/opt/solr/dist] > > Also, the jdbc library loads fine and is usable when I use the DIH > endpoint. > > > Query: > jdbc(connection="jdbc:mariadb://<host>:3306/<db>",sql="select id from tbl", > sort="id ASC", driver="org.mariadb.jdbc.Driver", user="<user>", > password="<pass>") > > Error: > ""Failed to load JDBC driver for 'org.mariadb.jdbc.Driver'"" > > solrconfig.xml > <lib dir="${solr.install.dir}/dist/" regex="mariadb-java-client-.*\.jar" /> > <expressible name="jdbc" > class="org.apache.solr.client.solrj.io.stream.JDBCStream"/> > > Cheers, > > /************************************ > * James Austin Greene > * www.jamesaustingreene.com > * 336-lol-nerd > ************************************/ > > > On Fri, Sep 9, 2022 at 5:01 PM Joel Bernstein <joels...@gmail.com> wrote: > > > You may be having classloader issues. I'm not sure where you have the > > driver jar, but it may need to be copied into the lib directory with core > > Solr jars. > > > > > > Joel Bernstein > > http://joelsolr.blogspot.com/ > > > > > > On Fri, Sep 9, 2022 at 4:58 PM Joel Bernstein <joels...@gmail.com> > wrote: > > > > > Can you post the stack trace from the log? > > > > > > It looks like the JDBCStream is registered, but the stack trace > > > may provide some more info on why the driver didn't load. > > > > > > Joel Bernstein > > > http://joelsolr.blogspot.com/ > > > > > > > > > On Fri, Sep 9, 2022 at 12:51 PM James Greene < > > ja...@jamesaustingreene.com> > > > wrote: > > > > > >> I'm still unable to get the JDBC stream to work. I've verified that > > >> 'jdbc' > > >> shows up in the plugin list: > > >> > > >> http://localhost:8983/solr/col1/stream?action=PLUGINS > > >> > > >> When using the DIH I use the jdbc driver: > > >> org.mariadb.jdbc.Driver > > >> > > >> With connection string: > > >> jdbc:mysql://{{host}}:{{port}}/{{database}} > > >> > > >> I am able to get DIH pulling data from mariadb with that connection > > >> string. > > >> > > >> However using the same connection string in the streaming expression: > > >> jdbc(connection="jdbc:mysql://mariadb:3306/db",sql="select id from > > tbl > > >> order by id asc limit 1", sort="id asc", > > driver="org.mariadb.jdbc.Driver") > > >> > > >> I get the error: > > >> "EXCEPTION": "Failed to load JDBC driver for > > >> 'org.mariadb.jdbc.Driver'" > > >> > > >> I tried upgrading jdbc connector to 3x version and using the > connection > > >> string "jdbc:mariadb://<host>:<port>/<db>" and get the same error > > message. > > >> > > >> > > >> > > >> > > >> > > >> On Thu, Sep 8, 2022 at 10:16 PM Joel Bernstein <joels...@gmail.com> > > >> wrote: > > >> > > >> > The jdbc stream was removed from the standard expression library for > > >> > security reasons. It is still in the jars though and can be included > > >> > through a change in the solrconfig.xml. > > >> > > > >> > This jira describes how to plugin an expression: > > >> > > > >> > https://issues.apache.org/jira/browse/SOLR-9103 > > >> > > > >> > The class is: org.apache.solr.client.solrj.io.stream.JDBCStream > > >> > > > >> > > > >> > > > >> > Joel Bernstein > > >> > http://joelsolr.blogspot.com/ > > >> > > > >> > > > >> > On Thu, Sep 8, 2022 at 6:54 PM James Greene < > > >> ja...@jamesaustingreene.com> > > >> > wrote: > > >> > > > >> > > I'm on solr 8.11.1 and trying to use the 'jdbc' streaming > expression > > >> as > > >> > > documented here: > > >> > > > > https://solr.apache.org/guide/8_11/stream-source-reference.html#jdbc > > >> > > > > >> > > When I try to run a jdbc stream expression I get the error: > > >> > > 'function 'jdbc' is unknown (not mapped to a valid TupleStream' > > >> > > > > >> > > Here is an example of the expression i'm trying to run: > > >> > > > > >> > jdbc(connection="jdbc:mysql://mariadb:3306/threatquotient2",sql="select > > >> > id > > >> > > from data order by id asc limit 1", sort="id asc" > > >> > > driver="org.mariadb.jdbc.Driver") > > >> > > > > >> > > I use the mariadb jdbc driver to get data from our mariadb > instance > > >> using > > >> > > the data import handler so the jdbc driver should be property > > loaded. > > >> > > > > >> > > > >> > > > > > >
