[
https://issues.apache.org/jira/browse/IGNITE-22776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Steshin updated IGNITE-22776:
--------------------------------------
Description:
To build a better plan, H2 can rely on the statistics. We can enable it with
{code:java}
control.sh --property set --name 'statistics. Usage. State' --val 'ON'
{code}
and
{code:sql}
ANALYZE MY_TABLE
{code}
But table statistic tends update with CRUD only once. Despite the obsolescence
threshold, by default
{code:java}
int StatisticsObjectConfiguration#DEFAULT_OBSOLESCENCE_MAX_PERCENT = 15;
{code}
Partition statistics obsolescence is checked every 60 seconds:
{code:java}
int IgniteStatisticsManagerImpl#OBSOLESCENCE_INTERVAL = 60;
if (serverNode) {
obsolescenceSchedule = ctx.timeout().schedule(() -> {
obsolescenceBusyExecutor.execute(() -> processObsolescence());
}, OBSOLESCENCE_INTERVAL * 1000, OBSOLESCENCE_INTERVAL * 1000);
}
{code}
Regarding my research, the problem is within
{code:java}
private Set<Integer>
IgniteStatisticsManagerImpl#calculateObsolescencedPartitions(
StatisticsObjectConfiguration cfg,
IntMap<ObjectPartitionStatisticsObsolescence> parts
) {
Set<Integer> res = new HashSet<>();
parts.forEach((k, v) -> {
ObjectPartitionStatisticsImpl partStat =
statsRepos.getLocalPartitionStatistics(cfg.key(), k);
if (partStat == null || partStat.rowCount() == 0 ||
(double)v.modified() * 100 / partStat.rowCount() >
cfg.maxPartitionObsolescencePercent())
res.add(k);
});
// Will add even empty list of partitions to recollect just to force
obsolescence info to be stored.
return res;
}
{code}
Where
{code:java}
v.modified() (`ong ObjectPartitionStatisticsObsolescence#modified())
{code}
is always 0, never updates. Probably because
{code:java}
IgniteStatisticsManagerImpl#calculateObsolescencedPartitions#onRowUpdated(String
schemaName, String objName, int partId, byte[] keyBytes)
{code}
is called nowhere. But
{code:java}
partStat.rowCount()
{code}
is 0 at the beginning. This is why the statistic is calculated once.
was:
To build a better plan, H2 can rely on the statistics. We can enable it with
{code:java}
control.sh --property set --name 'statistics. Usage. State' --val 'ON'
{code}
and
{code:sql}
ANALYZE MY_TABLE
{code}
But table statistic tends update with CRUD only once. Despite the obsolescence
threshold, by default
{code:java}
int StatisticsObjectConfiguration#DEFAULT_OBSOLESCENCE_MAX_PERCENT = 15;
{code}
Partition statistics obsolescence is checked every 60 seconds:
{code:java}
int IgniteStatisticsManagerImpl#OBSOLESCENCE_INTERVAL = 60;
if (serverNode) {
obsolescenceSchedule = ctx.timeout().schedule(() -> {
obsolescenceBusyExecutor.execute(() -> processObsolescence());
}, OBSOLESCENCE_INTERVAL * 1000, OBSOLESCENCE_INTERVAL * 1000);
}
{code}
Regarding my research, the problem is within
{code:java}
private Set<Integer>
IgniteStatisticsManagerImpl#calculateObsolescencedPartitions(
StatisticsObjectConfiguration cfg,
IntMap<ObjectPartitionStatisticsObsolescence> parts
) {
Set<Integer> res = new HashSet<>();
parts.forEach((k, v) -> {
ObjectPartitionStatisticsImpl partStat =
statsRepos.getLocalPartitionStatistics(cfg.key(), k);
if (partStat == null || partStat.rowCount() == 0 ||
(double)v.modified() * 100 / partStat.rowCount() >
cfg.maxPartitionObsolescencePercent())
res.add(k);
});
// Will add even empty list of partitions to recollect just to force
obsolescence info to be stored.
return res;
}
{code}
Where
{code:java}
v.modified() (`ong ObjectPartitionStatisticsObsolescence#modified())
{code}
is always 0, never updates. Probably because
{code:java}
IgniteStatisticsManagerImpl#calculateObsolescencedPartitions#onRowUpdated(String
schemaName, String objName, int partId, byte[] keyBytes)
{code}
is called nowhere. But
{code:java}
partStat.rowCount()
{code}
is 0 at the beginning. This is why the statistic is calculated one.
> SQL table statistic is updated only once.
> -----------------------------------------
>
> Key: IGNITE-22776
> URL: https://issues.apache.org/jira/browse/IGNITE-22776
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 2.15
> Reporter: Vladimir Steshin
> Priority: Major
> Attachments: StatisticsObsolescenceTest.java
>
>
> To build a better plan, H2 can rely on the statistics. We can enable it with
> {code:java}
> control.sh --property set --name 'statistics. Usage. State' --val 'ON'
> {code}
> and
> {code:sql}
> ANALYZE MY_TABLE
> {code}
> But table statistic tends update with CRUD only once. Despite the
> obsolescence threshold, by default
> {code:java}
> int StatisticsObjectConfiguration#DEFAULT_OBSOLESCENCE_MAX_PERCENT = 15;
> {code}
> Partition statistics obsolescence is checked every 60 seconds:
> {code:java}
> int IgniteStatisticsManagerImpl#OBSOLESCENCE_INTERVAL = 60;
> if (serverNode) {
> obsolescenceSchedule = ctx.timeout().schedule(() -> {
> obsolescenceBusyExecutor.execute(() -> processObsolescence());
> }, OBSOLESCENCE_INTERVAL * 1000, OBSOLESCENCE_INTERVAL * 1000);
> }
> {code}
> Regarding my research, the problem is within
> {code:java}
> private Set<Integer>
> IgniteStatisticsManagerImpl#calculateObsolescencedPartitions(
> StatisticsObjectConfiguration cfg,
> IntMap<ObjectPartitionStatisticsObsolescence> parts
> ) {
> Set<Integer> res = new HashSet<>();
> parts.forEach((k, v) -> {
> ObjectPartitionStatisticsImpl partStat =
> statsRepos.getLocalPartitionStatistics(cfg.key(), k);
> if (partStat == null || partStat.rowCount() == 0 ||
> (double)v.modified() * 100 / partStat.rowCount() >
> cfg.maxPartitionObsolescencePercent())
> res.add(k);
> });
> // Will add even empty list of partitions to recollect just to force
> obsolescence info to be stored.
> return res;
> }
> {code}
> Where
> {code:java}
> v.modified() (`ong ObjectPartitionStatisticsObsolescence#modified())
> {code}
> is always 0, never updates. Probably because
> {code:java}
> IgniteStatisticsManagerImpl#calculateObsolescencedPartitions#onRowUpdated(String
> schemaName, String objName, int partId, byte[] keyBytes)
> {code}
> is called nowhere. But
> {code:java}
> partStat.rowCount()
> {code}
> is 0 at the beginning. This is why the statistic is calculated once.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
