I have a grid which queries a table with 100's of millions of rows. The count query is too slow so I return an estimate. I wrote a page selector that does not allow selecting the last page and the prepare only selects the first n rows. This make the grid work for queries that could return millions of rows. A user can't easily get to the last page but in my case that's OK.
On Fri, Jan 17, 2014 at 9:16 AM, Ville Virtanen < ville.virta...@orientimport.fi> wrote: > The griddatasource is natural fit for sql databases, but > lucene/solr/elasticsearch etc. may not fit it 100% > > That said, you can access the current page of the grid that is selected > (grid. getCurrentPage()) and rowsPerPage (grid. getRowsPerPage()) and do > the math yourself to be able to pull in correct content in the first place. > > Tapestry uses the total amount of rows available to only make sure, that > the > selected page is not out of the range of possible pages, and that the end > index is never more than there actually are rows. (The last page may not be > full.) > > Ville > > -----Alkuperäinen viesti----- > Lähettäjä: George Christman [mailto:gchrist...@cardaddy.com] > Lähetetty: 17. tammikuuta 2014 15:49 > Vastaanottaja: Tapestry users > Aihe: Re: How to use grid with large data sets. > > I wouldn't say it's premature optimization when I'm essentially performing > two of the exact same free text queries, it's twice the work ;-) My company > is hosting our dev database remotely on awful hardware to help us to find > performance issues. They believe if we can make something fast on dev, it > will be lighting fast in production, so everything counts lol. > > Now if I understand GridDataSource correctly, you need to return the count > in availableRows() before the prepare() method can be called, is this > correct? If that is the case, I'm not sure how to go about combining this > into a single query do to the fact I would first need to get the count > before prepare is called to pass in the start / end index. > > hibernate-search, which is based on the lucene free text search engine will > return the total filtered result count while only returning the pagination > results. When using the Tap5 loop component, I typically just return an > object containing the total count and an array list of the results, but > seeing the prepare is called secondly, I'm not sure how to do this. I guess > worse case I can just do two queries. > > > > On Thu, Jan 16, 2014 at 5:51 PM, Lance Java > <lance.j...@googlemail.com>wrote: > > > If there's a concept in lucene that allows you to bring back the total > > rowcount and a page of data in a single query then the GridDataSource > > interface can support that. > > > > > > On 16 January 2014 22:36, Lance Java <lance.j...@googlemail.com> wrote: > > > > > One query is for the rowcount, the other is for the results (for a > > > single page). > > > The only way to do it with one query is to add an extra column to > > > the result set for the row count. This value will be exactly the > > > same for > > every > > > row in the result set. > > > > > > I personally don't see an issue with 2 queries... premature > > > optimisation perhaps ;) > > > > > > > > > On 16 January 2014 20:23, George Christman <gchrist...@cardaddy.com > > >wrote: > > > > > >> I found the following documentation > > >> > > >> http://wiki.apache.org/tapestry/Tapestry5HibernateGridDatasource2 > > >> > > >> However I'm still a little confused, I'm seeing two queries needed > > >> to > > get > > >> this to work, one containing the filtered results to get the > > availableRow > > >> count > > >> and the second one to get the prepare filtered result set > > >> containing startindex / endindex for get getRowValue(); > > >> > > >> With my hibernate search service, I'm able to get everything in a > > >> single query, is there anyway to do this in a single query? > > >> > > >> > > >> On Thu, Jan 16, 2014 at 2:48 PM, George Christman > > >> <gchrist...@cardaddy.com>wrote: > > >> > > >> > So I'm playing with the GridDateSource and was wondering how you > > >> > are suppose to get the filtered result count for availableRows()? > > >> > With hibernate search you get this count after the results have > > >> > been > > >> filtered, > > >> > but the way the methods are called in GridDataSource, the > > availableRows > > >> > method is called before prepare. > > >> > > > >> > The next question is how do you get your filters back to > > >> > GridDataSouce prepare? I'm wondering if I'm going about this the > right way. > > >> > > > >> > Example I'm working from > > >> > > > >> > > > >> > > http://jumpstart.doublenegative.com.au/jumpstart/examples/tables/gridd > > atasources > > >> > > > >> > > > >> > On Thu, Jan 16, 2014 at 12:26 PM, George Christman < > > >> > gchrist...@cardaddy.com> wrote: > > >> > > > >> >> I guys, thanks for the help, I'm using hibernate-search with > lucene. > > >> >> > > >> >> > > >> >> On Thu, Jan 16, 2014 at 12:10 PM, Ville Virtanen < > > >> >> ville.virta...@orientimport.fi> wrote: > > >> >> > > >> >>> Hi, > > >> >>> > > >> >>> here is one example: > > >> >>> > > >> >>> > > >> >>> > > >> > > http://jumpstart.doublenegative.com.au/jumpstart7/examples/tables/grid > > dataso > > >> >>> urces > > >> >>> > > >> >>> Ville > > >> >>> > > >> >>> -----Alkuperäinen viesti----- > > >> >>> Lähettäjä: George Christman [mailto:gchrist...@cardaddy.com] > > >> >>> Lähetetty: 16. tammikuuta 2014 18:34 > > >> >>> Vastaanottaja: Tapestry users > > >> >>> Aihe: How to use grid with large data sets. > > >> >>> > > >> >>> Hello, I'm wondering how to use the grid component with large > > >> >>> data > > >> sets. > > >> >>> Currently the grid is grabbing the entire data set rather than > > >> >>> a sub > > >> set. > > >> >>> How do I pass back the current page and row count as well as > > >> >>> the > > >> sorts? > > >> >>> Can > > >> >>> this be done with the grid component, or would I need to builds > > >> something > > >> >>> custom with a loop? > > >> >>> > > >> >>> -- > > >> >>> George Christman > > >> >>> www.CarDaddy.com > > >> >>> P.O. Box 735 > > >> >>> Johnstown, New York > > >> >>> > > >> >>> > > >> >>> > > --------------------------------------------------------------------- > > >> >>> To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org > > >> >>> For additional commands, e-mail: users-h...@tapestry.apache.org > > >> >>> > > >> >>> > > >> >> > > >> >> > > >> >> -- > > >> >> George Christman > > >> >> www.CarDaddy.com > > >> >> P.O. Box 735 > > >> >> Johnstown, New York > > >> >> > > >> >> > > >> > > > >> > > > >> > -- > > >> > George Christman > > >> > www.CarDaddy.com > > >> > P.O. Box 735 > > >> > Johnstown, New York > > >> > > > >> > > > >> > > >> > > >> -- > > >> George Christman > > >> www.CarDaddy.com > > >> P.O. Box 735 > > >> Johnstown, New York > > >> > > > > > > > > > > > > -- > George Christman > www.CarDaddy.com > P.O. Box 735 > Johnstown, New York > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org > For additional commands, e-mail: users-h...@tapestry.apache.org > >
