Re: Need help with solr 8.11 DataImport
Hello Vince.
I think these numbers speak for themselves:
"Total Requests made to DataSource": "1",
> "Total Rows Fetched": "0",
> "Total Documents Processed": "0",
> "Total Documents Skipped": "0",
Does it point to the right schema? Have you committed SQL INSERT
transaction?
I remember there's a debug parameter in DIH, you may use it.
On Mon, Nov 27, 2023 at 10:33 AM Vince McMahon <
sippingonesandze...@gmail.com> wrote:
> Hi,
>
> I am new to Solr dataimport and am trying to do a full-import from
> Postgres. Both Solr 8.11.2 and Postgres 12 are installed in Win10.
>
> Postgres is connecting using id solrpull and login is fine via Pgadmin.
> Solr web and core are running.
>
> I don't see any document imported and I don't find any error. How to fix
> this?
>
> Raw Status-Output
>
> {
> "responseHeader": {
> "status": 0,
> "QTime": 0
> },
> "initArgs": [
> "defaults",
> [
> "config",
> "db_configs.xml"
> ]
> ],
> "command": "status",
> "status": "idle",
> "importResponse": "",
> "statusMessages": {
> "Time Elapsed": "0:3:21.15",
> "Total Requests made to DataSource": "1",
> "Total Rows Fetched": "0",
> "Total Documents Processed": "0",
> "Total Documents Skipped": "0",
> "Full Dump Started": "2023-11-27 06:38:15",
> "Full Import failed": "2023-11-27 06:38:35"
> }
> }
>
> Configuration
>
>
> driver="org.postgresql.Driver"
> url="jdbc:postgresql://localhost/world"
> user="solrpull"
> password="NotTelling"/>
>
> query="select id,name from city">
>
>
>
>
>
> log4j2
> 11/27/2023, 2:13:00 AM WARN false CoreContainer Not all security plugins
> configured! authentication=disabled authorization=disabled. Solr is only as
> secure as you make it. Consider configuring authentication/authorization
> before exposing Solr to users internal or external. See
> https://s.apache.org/solrsecurity for more info
> 11/27/2023, 2:13:00 AM WARN false x:cookbookCSV SolrConfig solrconfig.xml:
> is no longer supported, use solr.xml:/metrics/reporter section
> instead
> 11/27/2023, 2:13:00 AM WARN false x:cookbookCSV WordDelimiterFilterFactory
> Solr
> loaded a deprecated plugin/analysis class
> [solr.WordDelimiterFilterFactory]. Please consult documentation how to
> replace it accordingly.
> 11/27/2023, 2:13:00 AM WARN false x:d
> 11/27/2023, 2:13:00 AM WARN false x:mail SolrConfig solrconfig.xml:
> is no longer supported, use solr.xml:/metrics/reporter section
> instead
> 11/27/2023, 2:13:00 AM WARN false x:db LatLonType Solr loaded a deprecated
> plugin/analysis class [solr.LatLonType]. Please consult documentation how
> to replace it accordingly.
> 11/27/2023, 2:13:01 AM WARN false x:atom DataImportHandler Data Import
> Handler is deprecated as of Solr 8.6. See SOLR-14066 for more details.
> 11/27/2023, 2:13:01 AM WARN false x:mail DataImportHandler Data Import
> Handler is deprecated as of Solr 8.6. See SOLR-14066 for more details.
> 11/27/2023, 2:13:01 AM WARN false x:db DataImportHandler Data Import
> Handler is deprecated as of Solr 8.6. See SOLR-14066 for more details.
> 11/27/2023, 2:13:01 AM WARN false x:solr SolrConfig solrconfig.xml:
> is no longer supported, use solr.xml:/metrics/reporter section
> instead
> 11/27/2023, 2:13:01 AM WARN false x:p SolrConfig solrconfig.xml: is
> no longer supported, use solr.xml:/metrics/reporter section instead
> 11/27/2023, 2:13:01 AM WARN false x:solr DataImportHandler Data Import
> Handler is deprecated as of Solr 8.6. See SOLR-14066 for more details.
> 11/27/2023, 2:13:02 AM WARN false x:tika DataImportHandler Data Import
> Handler is deprecated as of Solr 8.6. See SOLR-14066 for more details.
> 11/27/2023, 2:13:02 AM WARN false x:sample_core SolrConfig solrconfig.xml:
> is no longer supported, use solr.xml:/metrics/reporter section
> instead
> 11/27/2023, 2:13:02 AM WARN false x:f18maf SolrConfig solrconfig.xml:
> is no longer supported, use solr.xml:/metrics/reporter section
> instead
> 11/27/2023, 2:13:02 AM WARN false x:demo SolrConfig solrconfig.xml:
> is no longer supported, use solr.xml:/metrics/reporter section
> instead
> 11/27/2023, 2:13:02 AM WARN false x:p DataImportHandler Data Imp My
> database has data
>
>
> select id,name from city
> "id" "name"
> 1 "Seattle"
> 2 "Japan"
> Successfully run. Total query runtime: 37 msec.
> 2 rows affected.
>
--
Sincerely yours
Mikhail Khludnev
Re: Need help with solr 8.11 DataImport
Hi,
I think it points to the right schema. In the conf/managed-schema file I
added these lines
The SQL INSERT are committed. select * from city returns all rows after
rebooting the computer.
There is a debug check box and
The Raw Debug Response is:
{
"responseHeader": {
"status": 0,
"QTime": 51
},
"initArgs": [
"defaults",
[
"config",
"db_configs.xml"
]
],
"command": "full-import",
"mode": "debug",
"documents": [],
"verbose-output": [],
"status": "idle",
"importResponse": "",
"statusMessages": {
"Time Elapsed": "0:0:0.44",
"Total Requests made to DataSource": "1",
"Total Rows Fetched": "0",
"Total Documents Processed": "0",
"Total Documents Skipped": "0",
"Full Dump Started": "2023-11-27 12:58:12",
"Full Import failed": "2023-11-27 12:58:12"
}
}
On Mon, Nov 27, 2023 at 5:15 AM Mikhail Khludnev wrote:
> Hello Vince.
>
> I think these numbers speak for themselves:
>
> "Total Requests made to DataSource": "1",
> > "Total Rows Fetched": "0",
> > "Total Documents Processed": "0",
> > "Total Documents Skipped": "0",
>
> Does it point to the right schema? Have you committed SQL INSERT
> transaction?
> I remember there's a debug parameter in DIH, you may use it.
>
> On Mon, Nov 27, 2023 at 10:33 AM Vince McMahon <
> sippingonesandze...@gmail.com> wrote:
>
> > Hi,
> >
> > I am new to Solr dataimport and am trying to do a full-import from
> > Postgres. Both Solr 8.11.2 and Postgres 12 are installed in Win10.
> >
> > Postgres is connecting using id solrpull and login is fine via Pgadmin.
> > Solr web and core are running.
> >
> > I don't see any document imported and I don't find any error. How to fix
> > this?
> >
> > Raw Status-Output
> >
> > {
> > "responseHeader": {
> > "status": 0,
> > "QTime": 0
> > },
> > "initArgs": [
> > "defaults",
> > [
> > "config",
> > "db_configs.xml"
> > ]
> > ],
> > "command": "status",
> > "status": "idle",
> > "importResponse": "",
> > "statusMessages": {
> > "Time Elapsed": "0:3:21.15",
> > "Total Requests made to DataSource": "1",
> > "Total Rows Fetched": "0",
> > "Total Documents Processed": "0",
> > "Total Documents Skipped": "0",
> > "Full Dump Started": "2023-11-27 06:38:15",
> > "Full Import failed": "2023-11-27 06:38:35"
> > }
> > }
> >
> > Configuration
> >
> >
> >> driver="org.postgresql.Driver"
> > url="jdbc:postgresql://localhost/world"
> > user="solrpull"
> > password="NotTelling"/>
> >
> > > query="select id,name from city">
> >
> >
> >
> >
> >
> > log4j2
> > 11/27/2023, 2:13:00 AM WARN false CoreContainer Not all security plugins
> > configured! authentication=disabled authorization=disabled. Solr is only
> as
> > secure as you make it. Consider configuring authentication/authorization
> > before exposing Solr to users internal or external. See
> > https://s.apache.org/solrsecurity for more info
> > 11/27/2023, 2:13:00 AM WARN false x:cookbookCSV SolrConfig
> solrconfig.xml:
> > is no longer supported, use solr.xml:/metrics/reporter
> section
> > instead
> > 11/27/2023, 2:13:00 AM WARN false x:cookbookCSV
> WordDelimiterFilterFactory
> > Solr
> > loaded a deprecated plugin/analysis class
> > [solr.WordDelimiterFilterFactory]. Please consult documentation how to
> > replace it accordingly.
> > 11/27/2023, 2:13:00 AM WARN false x:d
> > 11/27/2023, 2:13:00 AM WARN false x:mail SolrConfig solrconfig.xml:
> > is no longer supported, use solr.xml:/metrics/reporter section
> > instead
> > 11/27/2023, 2:13:00 AM WARN false x:db LatLonType Solr loaded a
> deprecated
> > plugin/analysis class [solr.LatLonType]. Please consult documentation how
> > to replace it accordingly.
> > 11/27/2023, 2:13:01 AM WARN false x:atom DataImportHandler Data Import
> > Handler is deprecated as of Solr 8.6. See SOLR-14066 for more details.
> > 11/27/2023, 2:13:01 AM WARN false x:mail DataImportHandler Data Import
> > Handler is deprecated as of Solr 8.6. See SOLR-14066 for more details.
> > 11/27/2023, 2:13:01 AM WARN false x:db DataImportHandler Data Import
> > Handler is deprecated as of Solr 8.6. See SOLR-14066 for more details.
> > 11/27/2023, 2:13:01 AM WARN false x:solr SolrConfig solrconfig.xml:
> > is no longer supported, use solr.xml:/metrics/reporter section
> > instead
> > 11/27/2023, 2:13:01 AM WARN false x:p SolrConfig solrconfig.xml: is
> > no longer supported, use solr.xml:/metrics/reporter section
> instead
> > 11/27/2023, 2:13:01 AM WARN false x:solr DataImportHandler Data Import
> > Handler is deprecated as of Solr 8.6. See SOLR-14066 for more details.
> > 11/27/2023, 2:13:02 AM WARN false x:tika DataImportHandler Data Import
> > Handler is deprecated as of Solr 8.6. See SOLR-14066 for more details.
> > 11/27/2023, 2:13:02 AM WARN false x:sample_core SolrConfig
> solrconfig.xml:
Re: Need help with solr 8.11 DataImport
I start solr by C:\solr-8.11.2\bin\*solr start* When I extract solr the dist, lib folders they are in C:\solr-8.11.2\dist C:\solr-8.11.2\server\*lib* C:\solr-8.11.2\server\solr\configsets\*lib* Where should be the jar files be located and the correct syntax in C:\solr-8.11.2\*myNewCore*\conf\*solrconfig.xml*? Are the following correct for Windows? The jar files are postgresql-42.2.6.jar solr-dataimporthandler-8.11.2.jar >> >>
IllegalArgumentException: Comparison method violates its general contract! - LTR query exception
Hi Solr Users, LTR query throwing the following exception, what could be the reasons? It is Solr 9.1.1 o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: java.lang.IllegalArgumentException: Comparison method violates its general contract! => org.apache.solr.common.SolrException: java.lang.IllegalArgumentException: Comparison method violates its general contract! at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:163) org.apache.solr.common.SolrException: java.lang.IllegalArgumentException: Comparison method violates its general contract! at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:163) ~[?:?] at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1859) ~[?:?] at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1673) ~[?:?] at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:711) ~[?:?] at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1637) ~[?:?] at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:421) ~[?:?] at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:420) ~[?:?] at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224) ~[?:?] at org.apache.solr.core.SolrCore.execute(SolrCore.java:2865) ~[?:?] at
Re: Need help with solr 8.11 DataImport
> > I've figured it out. Thank you, Mikhail >> >>
Re: Receiving 405 error messages for alias deletion
Thanks Hakan, that also sounds like a bug.
Would you be willing to file a JIRA ticket describing this issue? (I can
file one in a few days if you're not able or willing, but I would/will
mostly be quoting your email above)
Best,
Jason
On Thu, Nov 23, 2023 at 6:26 AM Hakan Özler wrote:
> Thank you for the prompt response guys!
>
> Apart from this case, I've encountered other problems that are peculiar to
> the v2 API in 9.4.
>
> 1. Creating a backup [1] request using the v2 api directives sporadically
> returns `An exception was not mapped due to exception mapper failure. The
> HTTP 500 response will be returned` with the following stack trace.
> However, the backup process is always successfully completed.
>
> java.lang.NullPointerException: Cannot invoke
> "org.apache.solr.request.SolrQueryRequest.getParams()" because
> "solrQueryRequest" is null
> at
>
> org.apache.solr.handler.api.V2ApiUtils.getMediaTypeFromWtParam(V2ApiUtils.java:92)
> at
>
> org.apache.solr.jersey.MediaTypeOverridingFilter.filter(MediaTypeOverridingFilter.java:64)
> at
>
> org.glassfish.jersey.server.ContainerFilteringStage$ResponseFilterStage.apply(ContainerFilteringStage.java:172)
> at
>
> org.glassfish.jersey.server.ContainerFilteringStage$ResponseFilterStage.apply(ContainerFilteringStage.java:139)
> at org.glassfish.jersey.process.internal.Stages.process(Stages.java:147)
> at
>
> org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:378)
>
> 2. Creating a replica [2] obviously has the same behavior. We're able to
> create a replica but we receive the message above at times.
>
> 1.
>
> https://solr.apache.org/guide/solr/latest/deployment-guide/collection-management.html#backup
> 2.
>
> https://solr.apache.org/guide/solr/latest/deployment-guide/replica-management.html#addreplica
>
>
> On Mon, 20 Nov 2023 at 17:39, Jason Gerlowski
> wrote:
>
> > Good catch! This is definitely a bug that I introduced as a part of
> > SOLR-16393 - sorry for the trouble.
> >
> > The problem, counter-intuitively, is this line. [1]. The annotations in
> > this file are overriding the ones we need in 'DeleteAliasApi' (which is
> > where the path and verb are specified).
> >
> > I'll get this fixed and backported for 9.4.1/9.5. I imagine it's cold
> > comfort, but I really appreciate you guys trying out the v2 APIs and
> > catching bugs - sorry it ended up burning you guys this time.
> >
> > Best,
> >
> > Jason
> >
> > [1]
> >
> >
> https://github.com/apache/solr/blob/9463f666f40f1d72a0ac7b3c46c7f8048ef8eaa9/solr/core/src/java/org/apache/solr/handler/admin/api/DeleteAlias.java#L55
> >
> > On Mon, Nov 20, 2023 at 6:46 AM ufuk yılmaz >
> > wrote:
> >
> > > +1 I also get the same error with a fresh solrcloud 9.4 installation.
> > > Jetty configuration doesn’t seem to have anything to block that
> request.
> > >
> > > Any idea where the implementation for this API interface is? Is it
> > > automatically generated?
> > >
> > >
> >
> https://github.com/apache/solr/blob/main/solr/api/src/java/org/apache/solr/client/api/endpoint/DeleteAliasApi.java
> > >
> > > --ufuk yilmaz
> > >
> > > Sent from Mail for Windows
> > >
> > > From: Hakan Özler
> > > Sent: Monday, November 20, 2023 1:15 PM
> > > To: users@solr.apache.org
> > > Subject: Receiving 405 error messages for alias deletion
> > >
> > > Hi there!
> > >
> > > While I was working on alias management in SOLR, I noticed that the V2
> of
> > > alias deletion [1] is not allowed in Solr 9.4, and that there seems to
> > be a
> > > missing endpoint causing 405s with every attempt using the following
> > > request:
> > >
> > > List aliases:
> > > $ curl -i -L -X GET 'http://localhost:8983/api/aliases'
> > >
> > >
> >
> {"responseHeader":{"status":0,"QTime":5},"aliases":{"my-alias":"my-cloud-collection"},"properties":{}}
> > >
> > > Delete V2 alias 'my-alias':
> > > $ curl -i -L -X DELETE 'http://localhost:8983/api/aliases/my-alias'
> > > HTTP/1.1 405 Method Not Allowed
> > >
> > > However, its V1 API works fine:
> > > $ curl -i -L -X DELETE '
> > >
> > >
> >
> http://localhost:8983/solr/admin/collections?action=DELETEALIAS&name=my-alias
> > > '
> > > {"responseHeader":{"status":0,"QTime":5}}
> > >
> > > If I'm not mistaken according to the docs, do you think this might be
> > > related to these improvements: [2]?
> > >
> > > Thanks,
> > > Hakan
> > >
> > > 1.
> > >
> > >
> >
> https://solr.apache.org/guide/solr/latest/deployment-guide/alias-management.html#deletealias
> > > 2. https://issues.apache.org/jira/browse/SOLR-16393
> > >
> > >
> >
>
Re: IllegalArgumentException: Comparison method violates its general contract! - LTR query exception
: LTR query throwing the following exception, what could be the reasons? It : is Solr 9.1.1 It means there is a really bad bug in Solr that the JVM is complaining about, and your data just happens to hit it ... but at a glance I'm not seeing any obvious problem with the code. (unless maybe the internal call to Float.compare() is returning Integer.MIN_VALUE and we're getting int rollover when it's negated?) Does this error reliably reproduce for you with a given query? with lots of queries? Are you familiar with how to build solr from source? If i gave you a patch that added a bunch of debug logging, would you be able to recompile solr and try the same query against the same data and put the (very verbose) debug logging up on a pastebin somewhere for review? -Hoss http://www.lucidworks.com/
Re: Need help with solr 8.11 DataImport
Good Job, Vince! On Mon, Nov 27, 2023 at 7:45 PM Vince McMahon wrote: > > > > I've figured it out. Thank you, Mikhail > >> > >> > -- Sincerely yours Mikhail Khludnev
