Got it. I thought that index building and index rebuilding are essentially the same, but now I see that they are different: index rebuilding cares about all indexes at once while index building cares about particular ones.
Kirill's approach sounds good. Stan > On 20 Jul 2020, at 14:54, Alexey Goncharuk <alexey.goncha...@gmail.com> wrote: > > Stan, > > Currently we never build indexes one-by-one - we always use a cache data > row visitor which either updates all indexes (see IndexRebuildFullClosure) > or updates a set of all indexes that need to catch up (see > IndexRebuildPartialClosure). GIven that, I do not see any need for > per-index rebuild status as this status will be updated for all outdated > indexes simultaneously. > > Kirill's approach for the total number of processed keys per cache seems > reasonable to me. > > --AG > > пт, 3 июл. 2020 г. в 10:12, ткаленко кирилл <tkalkir...@yandex.ru>: > >> Hi, Stan! >> >> Perhaps it is worth clarifying what exactly I wanted to say. >> Now we have 2 processes: building and rebuilding indexes. >> >> At moment, we have some metrics for rebuilding indexes: >> "IsIndexRebuildInProgress", "IndexBuildCountPartitionsLeft". >> >> I suggest adding another metric "Indexrebuildkeyprocessed", which will >> allow you to determine how many records are left to rebuild for cache. >> >> I think your comments are more about building an index that may need more >> metrics, but I think you should do it in a separate ticket. >> >> 03.07.2020, 03:09, "Stanislav Lukyanov" <stanlukya...@gmail.com>: >>> If multiple indexes are to be built "number of indexed keys" metric may >> be misleading. >>> >>> As a cluster admin, I'd like to know: >>> - Are all indexes ready on a node? >>> - How many indexes are to be built? >>> - How much resources are used by the index building (how many threads >> are used)? >>> - Which index(es?) is being built right now? >>> - How much time until the current (single) index building finishes? Here >> "time" can be a lot of things: partitions, entries, percent of the cache, >> minutes and hours >>> - How much time until all indexes are built? >>> - How much does it take to build each of my indexes / a single index of >> my cache on average? >>> >>> I think we need a set of metrics and/or log messages to solve all of >> these questions. >>> I imaging something like: >>> - numberOfIndexesToBuild >>> - a standard set of metrics on the index building thread pool (do we >> already have it?) >>> - currentlyBuiltIndexName (assuming we only build one at a time which is >> probably not true) >>> - for the "time" metrics I think percentage might be the best as it's >> the easiest to understand; we may add multiple metrics though. >>> - For "time per each index" I'd add detailed log messages stating how >> long did it take to build a particular index >>> >>> Thanks, >>> Stan >>> >>>> On 26 Jun 2020, at 12:49, ткаленко кирилл <tkalkir...@yandex.ru> >> wrote: >>>> >>>> Hi, Igniters. >>>> >>>> I would like to know if it is possible to estimate how much the index >> rebuild will take? >>>> >>>> At the moment, I have found the following metrics [1] and [2] and >> since the rebuild is based on caches, I think it would be useful to know >> how many records are processed in indexing. This way we can estimate how >> long we have to wait for the index to be rebuilt by subtracting [3] and how >> many records are indexed. >>>> >>>> I think we should add this metric [4]. >>>> >>>> Comments, suggestions? >>>> >>>> [1] - https://issues.apache.org/jira/browse/IGNITE-12184 >>>> [2] - >> org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl#idxBuildCntPartitionsLeft >>>> [3] - org.apache.ignite.cache.CacheMetrics#getCacheSize >>>> [4] - org.apache.ignite.cache.CacheMetrics#getNumberIndexedKeys >>
