[
https://issues.apache.org/jira/browse/IGNITE-21753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Puchkovskiy updated IGNITE-21753:
---------------------------------------
Description:
To enable Metastorage compaction (that is, removal of overwritten revisions
that are not needed to anyone any more), there is IGNITE-19417, but it might
take some time to define the corresponding LWM's rules.
As a crutch, we might implement a simplified procedure for some keys. For
instance, a big percentage of garbage is created by a constantly overwritten
key related to lease tracking.
If a key is never scanned (either distributively, via range()/prefix(), or
locally, via xxxLocally() returning a Cursor) and only its latest version is
always accessed (this seems to be true for the above mentioned lease-related
key), we can just do compactKey() for such a key after any update involving it.
We might also need to reimplement getAndPut() to read before doing a put as the
current implementation requires 2 latest versions to be stored (because it
makes a put first and only then reads using the 'before the put' revision).
The crutch might be implemented by whitelisting all such 'special' keys in the
KV storage backing the Metastorage to make it opaque for the API, or an API
might be temporarily changed.
was:
To enable Metastorage compaction (that is, removal of overwritten revisions
that are not needed to anyone any more), there is IGNITE-19417, but it might
take some time to define the corresponding LWM's rules.
As a crutch, we might implement a simplified procedure for some keys. For
instance, a big percentage of garbage is created by a constantly overwritten
key related to lease tracking.
If a key is never scanned (either distributively, via range()/prefix(), or
locally, via xxxLocally() returning a Cursor) and only its latest version is
always accessed (this seems to be true for the above mentioned lease-related
key), we can just do compactKey() for such a key after any update involving it.
We might also need to reimplement getAndPut() to read before doing a put as the
current implementation requires 2 latest versions to be stored (because it
makes a put first and only then reads using the 'before the put' revision).
> Implement a hacky self-compaction for selected Metastorage keys
> ---------------------------------------------------------------
>
> Key: IGNITE-21753
> URL: https://issues.apache.org/jira/browse/IGNITE-21753
> Project: Ignite
> Issue Type: Improvement
> Reporter: Roman Puchkovskiy
> Priority: Major
> Labels: ignite-3
>
> To enable Metastorage compaction (that is, removal of overwritten revisions
> that are not needed to anyone any more), there is IGNITE-19417, but it might
> take some time to define the corresponding LWM's rules.
> As a crutch, we might implement a simplified procedure for some keys. For
> instance, a big percentage of garbage is created by a constantly overwritten
> key related to lease tracking.
> If a key is never scanned (either distributively, via range()/prefix(), or
> locally, via xxxLocally() returning a Cursor) and only its latest version is
> always accessed (this seems to be true for the above mentioned lease-related
> key), we can just do compactKey() for such a key after any update involving
> it.
> We might also need to reimplement getAndPut() to read before doing a put as
> the current implementation requires 2 latest versions to be stored (because
> it makes a put first and only then reads using the 'before the put' revision).
> The crutch might be implemented by whitelisting all such 'special' keys in
> the KV storage backing the Metastorage to make it opaque for the API, or an
> API might be temporarily changed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
