Re: term frequency in solr
Hi,
I think you are missing the main query parameter? q=*:*
By the way you may get more response in the sole-user mailing list.
Ahmet
On Wednesday, January 4, 2017 4:59 PM, huda barakat
wrote:
Please help me with this:
I have this code which return term frequency from techproducts example:
/
import java.util.List;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrRequest;
import org.apache.solr.client.solrj.impl.HttpSolrClient;
import org.apache.solr.client.solrj.request.QueryRequest;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.client.solrj.response.TermsResponse;
public class test4 {
public static void main(String[] args) throws Exception {
String urlString = "http://localhost:8983/solr/techproducts";;
SolrClient solr = new HttpSolrClient.Builder(urlString).build();
SolrQuery query = new SolrQuery();
query.setTerms(true);
query.addTermsField("name");
SolrRequest req = new QueryRequest(query);
QueryResponse rsp = req.process(solr);
System.out.println(rsp);
System.out.println("numFound: " + rsp.getResults().getNumFound());
TermsResponse termResp =rsp.getTermsResponse();
List terms = termResp.getTerms("name");
System.out.print("size="+ terms.size());
}
}
/
the result is 0 records I don't know why?? this is what I got:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
{responseHeader={status=0,QTime=0,params={terms=true,terms.fl=name,wt=javabin,version=2}},response={numFound=0,start=0,docs=[]}}
numFound: 0
Exception in thread "main" java.lang.NullPointerException
at solr_test.solr.test4.main(test4.java:29)
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: term frequency in solr
Thank you for fast reply, I add the query in the code but still not working:
import java.util.List;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrRequest;
import org.apache.solr.client.solrj.impl.HttpSolrClient;
import org.apache.solr.client.solrj.request.QueryRequest;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.client.solrj.response.TermsResponse;
public class App3 {
public static void main(String[] args) throws Exception {
String urlString = "http://localhost:8983/solr/techproducts";;
SolrClient solr = new HttpSolrClient.Builder(urlString).build();
SolrQuery query = new SolrQuery();
query.setQuery("*:*");
query.setTerms(true);
query.addTermsField("name");
SolrRequest req = new QueryRequest(query);
QueryResponse rsp = req.process(solr);
System.out.println(rsp);
System.out.println("numFound: " + rsp.getResults().getNumFound());
TermsResponse termResp =rsp.getTermsResponse();
List terms = termResp.getTerms("name");
System.out.print("size="+ terms.size());
}
}
///
I got this error:
numFound: 32
Exception in thread "main" java.lang.NullPointerException
at testPkg.App3.main(App3.java:30)
On 5 January 2017 at 18:25, Ahmet Arslan wrote:
> Hi,
>
> I think you are missing the main query parameter? q=*:*
>
> By the way you may get more response in the sole-user mailing list.
>
> Ahmet
>
>
> On Wednesday, January 4, 2017 4:59 PM, huda barakat <
> eng.huda.bara...@gmail.com> wrote:
> Please help me with this:
>
>
> I have this code which return term frequency from techproducts example:
>
>
> /
> import java.util.List;
>
> import org.apache.solr.client.solrj.SolrClient;
> import org.apache.solr.client.solrj.SolrQuery;
> import org.apache.solr.client.solrj.SolrRequest;
> import org.apache.solr.client.solrj.impl.HttpSolrClient;
> import org.apache.solr.client.solrj.request.QueryRequest;
> import org.apache.solr.client.solrj.response.QueryResponse;
> import org.apache.solr.client.solrj.response.TermsResponse;
>
> public class test4 {
> public static void main(String[] args) throws Exception {
> String urlString = "http://localhost:8983/solr/techproducts";;
> SolrClient solr = new HttpSolrClient.Builder(urlString).build();
>
> SolrQuery query = new SolrQuery();
> query.setTerms(true);
> query.addTermsField("name");
> SolrRequest req = new QueryRequest(query);
> QueryResponse rsp = req.process(solr);
>
> System.out.println(rsp);
>
> System.out.println("numFound: " + rsp.getResults().getNumFound());
>
> TermsResponse termResp =rsp.getTermsResponse();
> List terms = termResp.getTerms("name");
> System.out.print("size="+ terms.size());
> }
> }
>
> /
>
> the result is 0 records I don't know why?? this is what I got:
>
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> {responseHeader={status=0,QTime=0,params={terms=true,
> terms.fl=name,wt=javabin,version=2}},response={
> numFound=0,start=0,docs=[]}}
> numFound: 0
> Exception in thread "main" java.lang.NullPointerException
> at solr_test.solr.test4.main(test4.java:29)
>
> -
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>
Re: term frequency in solr
Hi,
I guess you are working with default techproducts.
can you try using the terms request handler:
query.setRequestHandler("terms")
Ahmet
On Friday, January 6, 2017 1:19 AM, huda barakat
wrote:
Thank you for fast reply, I add the query in the code but still not working:
import java.util.List;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrRequest;
import org.apache.solr.client.solrj.impl.HttpSolrClient;
import org.apache.solr.client.solrj.request.QueryRequest;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.client.solrj.response.TermsResponse;
public class App3 {
public static void main(String[] args) throws Exception {
String urlString = "http://localhost:8983/solr/techproducts";;
SolrClient solr = new HttpSolrClient.Builder(urlString).build();
SolrQuery query = new SolrQuery();
query.setQuery("*:*");
query.setTerms(true);
query.addTermsField("name");
SolrRequest req = new QueryRequest(query);
QueryResponse rsp = req.process(solr);
System.out.println(rsp);
System.out.println("numFound: " + rsp.getResults().getNumFound());
TermsResponse termResp =rsp.getTermsResponse();
List terms = termResp.getTerms("name");
System.out.print("size="+ terms.size());
}
}
///
I got this error:
numFound: 32
Exception in thread "main" java.lang.NullPointerException
at testPkg.App3.main(App3.java:30)
On 5 January 2017 at 18:25, Ahmet Arslan wrote:
> Hi,
>
> I think you are missing the main query parameter? q=*:*
>
> By the way you may get more response in the sole-user mailing list.
>
> Ahmet
>
>
> On Wednesday, January 4, 2017 4:59 PM, huda barakat <
> eng.huda.bara...@gmail.com> wrote:
> Please help me with this:
>
>
> I have this code which return term frequency from techproducts example:
>
>
> /
> import java.util.List;
>
> import org.apache.solr.client.solrj.SolrClient;
> import org.apache.solr.client.solrj.SolrQuery;
> import org.apache.solr.client.solrj.SolrRequest;
> import org.apache.solr.client.solrj.impl.HttpSolrClient;
> import org.apache.solr.client.solrj.request.QueryRequest;
> import org.apache.solr.client.solrj.response.QueryResponse;
> import org.apache.solr.client.solrj.response.TermsResponse;
>
> public class test4 {
> public static void main(String[] args) throws Exception {
> String urlString = "http://localhost:8983/solr/techproducts";;
> SolrClient solr = new HttpSolrClient.Builder(urlString).build();
>
> SolrQuery query = new SolrQuery();
> query.setTerms(true);
> query.addTermsField("name");
> SolrRequest req = new QueryRequest(query);
> QueryResponse rsp = req.process(solr);
>
> System.out.println(rsp);
>
> System.out.println("numFound: " + rsp.getResults().getNumFound());
>
> TermsResponse termResp =rsp.getTermsResponse();
> List terms = termResp.getTerms("name");
> System.out.print("size="+ terms.size());
> }
> }
>
> /
>
> the result is 0 records I don't know why?? this is what I got:
>
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> {responseHeader={status=0,QTime=0,params={terms=true,
> terms.fl=name,wt=javabin,version=2}},response={
> numFound=0,start=0,docs=[]}}
> numFound: 0
> Exception in thread "main" java.lang.NullPointerException
> at solr_test.solr.test4.main(test4.java:29)
>
> -
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: Lucene performance benchmark | search throughput
OK, got it One thing still I need to know (which is not clear to me) How does Lucene calculates the most restrictive clause? Correct me, if I am wrong in my understanding (in abstract): 1. During indexing, Lucene keeps information of documents count against every indexed items. 2. During search, it first checks, which condition has less number of documents count before actually iterating. 3. Then, it iterates that restricted set against other set of conditions. If the above is correct then how does Lucene calculates most restrictive clause in case of Wildcard conditions? Also, if Lucene first check for most restrictive clause, and then iterate to match documents to the other clauses, Then when will the merging of documents happen? Coming on to my main query for which I ask question in Lucene community: What is the search performance benchmark against Lucene version, so that I can benchmark my application throughput? Regards Rajnish On Tue, Jan 3, 2017 at 6:09 PM, Rajnish kamboj wrote: > OK, got it > > One thing still I need to know (which is not clear to me) > How does Lucene calculates the most restrictive clause? > > Correct me, if I am wrong in my understanding (in abstract): > 1. During indexing, Lucene keeps information of documents count against > every indexed items. > 2. During search, it first checks, which condition has less number of > documents count before actually iterating. > 3. Then, it iterates that restricted set against other set of conditions. > > If the above is correct then how does Lucene calculates most restrictive > clause in case of Wildcard conditions? > Also, if Lucene first check for most restrictive clause, and then iterate > to match documents to the other clauses, > Then when will the merging of documents happen? > > Coming on to my main query for which I ask question in Lucene community: > What is the search performance benchmark against Lucene version, so that I > can benchmark my application throughput? > > > > On Tue, Jan 3, 2017 at 5:12 PM, Michael McCandless < > luc...@mikemccandless.com> wrote: > >> When you add MUST sub-clauses to a BooleanQuery (AND to the query >> parsers) it can make the search run faster because Lucene will take >> the most restrictive clause and use that to "drive" the iteration of >> matching documents to the other clauses, allowing those other clauses >> to iterate much faster than they would otherwise require if they were >> not AND'd. >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> >> On Tue, Jan 3, 2017 at 6:33 AM, Rajnish kamboj >> wrote: >> > The answer is not clear. >> > >> > Suppose I have following query and I want 10 records. >> > Condition1 AND Condition2 AND Condition3 >> > >> > As per my understanding Lucene will first evaluate all conditions >> > separately and then merge the Documents as per AND/OR clauses. >> > At last it will return me 10 records. >> > >> > So, if I add one more condition, then it will add to search time and >> merge >> > time and hence increase latency, which results in decreased throughput. >> > >> > >> > Also, what is the search performance benchmark against Lucene version? >> > >> > >> > Regards >> > Rajnish >> > >> > >> > >> > >> > >> > >> > >> > On Tuesday 3 January 2017, Michael Wilkowski >> wrote: >> > >> >> My guess: more conditions = less documents to score and sort to return. >> >> >> >> On Mon, Jan 2, 2017 at 7:23 PM, Rajnish kamboj < >> rajnishk7.i...@gmail.com> >> >> wrote: >> >> >> >> > Hi >> >> > >> >> > Is there any Lucene performance benchmark against certain set of >> data? >> >> > [i.e Is there any stats for search throughput which Lucene can >> provide >> >> for >> >> > a certain data?] >> >> > >> >> > Search throughput Example: >> >> > Max. 200 TPS for 50K data on Lucene 5.3.1 on RHEL version x (with >> SSD) >> >> > Max. 150 TPS for 100K data on Lucene 5.3.1 on RHEL version x (with >> SSD) >> >> > Max. 300 TPS for 50K data on Lucene 6.0.0 on RHEL version x (with >> SSD) >> >> > etc. >> >> > >> >> > Also, does the index size matters for search throughput? >> >> > >> >> > Our observation: >> >> > When we increase the data size (hence index size) the search >> throughput >> >> > decreases. >> >> > When we add more AND conditions, the search throughput increases. >> Why? >> >> > Ideally if we add more conditions then the Lucene should have more >> work >> >> to >> >> > do (including merging) and the throughput should decrease but the >> >> > throughput increases? >> >> > >> >> > >> >> > Regards >> >> > Rajnish >> >> > >> >> >> > >
