Hi Daniel, I believe this happens because unlike review requests, we have no way of determining with the search indexing stuff we use if a user's information is "new" to the index. Basically, we have no "last updated" for the users.
I do see an area in the search index for users that isn't as optimal, so we can probably shave down the number of queries a fair amount, which will help. I assume you're using the default search backend of Whoosh? If so, at your scale, it might be time to install Elasticsearch. We can give you some info on how to configure that in settings_local.py (the upcoming Review Board 3.0 makes this very easy -- just select it as your search backend and fill in fields). Indexing with Elasticsearch is more efficient for large amounts of data. Christian -- Christian Hammond President/CEO of Beanbag <https://www.beanbaginc.com/> Makers of Review Board <https://www.reviewboard.org/> On Mon, May 1, 2017 at 12:59 PM, Daniel Kuecker <[email protected]> wrote: > I have a fresh 2.5.9 install with a db taken from a backup of another > 2.5.9 install (basically a backup of my production instance). > > On the production instance, I am experiencing 100% cpu usage every time > the update_index cron job runs causing my users to complain about > responsiveness of the site. > > On my production system, the cron job shows it is indexing all the records > in the user table while only the new records in the review requests table. > > For debugging, I restored the production db backup to my dev machine and > set up a 2.5.9 instance pointing to it. When update_index runs on the dev > box (given the "-v 3" arg), I get the following warning: > >> >> WARNING:py.warnings:/usr/lib/python2.7/site-packages/haystack/indexes.py:167: >> UserWarning: No updated date field found for 'User' - not restricting by >> age. >> warnings.warn(update_field_msg) >> Indexing 1971 users >> indexed 1 - 1000 of 1971 (by 16819). >> indexed 1001 - 1971 of 1971 (by 16819). > > > It then proceeds to index the entire user table and only the changed > review requests just as my production instance does. > > I have run the rebuild_index command on both systems. > > Am I missing something? I have the cron job set to run ever 1/2 hour and > am about to have to turn search indexing off to ensure the site is > responsive. > > --DK > > > > -- > Supercharge your Review Board with Power Pack: > https://www.reviewboard.org/powerpack/ > Want us to host Review Board for you? Check out RBCommons: > https://rbcommons.com/ > Happy user? Let us know! https://www.reviewboard.org/users/ > --- > You received this message because you are subscribed to the Google Groups > "reviewboard" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/ Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/ Happy user? Let us know! https://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups "reviewboard" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
