Hi!

I need to clean old data and use a query for that:

DELETE FROM IgniteTransfer WHERE _KEY IN (SELECT TOP ? _KEY FROM IgniteTransfer WHERE AccountId = ? ORDER BY TimeStamp ASC)

But sometimes I get a warning

[00:05:47 WRN] Query execution is too long [time=6476 ms, sql='SELECT
_KEY,
_VAL
FROM "LocalTransfers2".IGNITETRANSFER
WHERE _KEY IN( SELECT
_KEY
FROM "LocalTransfers2".IGNITETRANSFER
WHERE ACCOUNTID = ?2
ORDER BY =TIMESTAMP LIMIT ?1 )', plan=
SELECT
    _KEY,
    _VAL
FROM "LocalTransfers2".IGNITETRANSFER
    /* "LocalTransfers2"."_key_PK": _KEY IN(SELECT
        _KEY
    FROM "LocalTransfers2".IGNITETRANSFER
        /++ "LocalTransfers2".IGNITETRANSFER_ACCOUNTID_ASC_TIMESTAMP_ASC_IDX: ACCOUNTID = ?2 ++/
    WHERE ACCOUNTID = ?2
    ORDER BY =TIMESTAMP
    LIMIT ?1)
     */
WHERE _KEY IN(
    SELECT
        _KEY
    FROM "LocalTransfers2".IGNITETRANSFER
        /* "LocalTransfers2".IGNITETRANSFER_ACCOUNTID_ASC_TIMESTAMP_ASC_IDX: ACCOUNTID = ?2 */
    WHERE ACCOUNTID = ?2
    ORDER BY =TIMESTAMP
    LIMIT ?1)
, parameters=[2000, 1001]]

6 seconds to remove 2000 elements is too much. What is the complexity of this operation?

What is a most efficient way to remove old data in FIFO order?

All inserts and deletes handled with a dedicated thread. Multithreaded reading.

Cache config

var query =new QueryEntity(typeof(long),typeof(IgniteTransfer))
{
    Fields =new[]
    {
        new QueryField {Name =nameof(IgniteTransfer.AccountId),FieldType 
=typeof(int)},
        new QueryField {Name =nameof(IgniteTransfer.TransferId),FieldType 
=typeof(long)},
        new QueryField {Name =nameof(IgniteTransfer.AssetId),FieldType 
=typeof(int)},
        new QueryField {Name =nameof(IgniteTransfer.Reason),FieldType 
=typeof(int)},
        new QueryField {Name =nameof(IgniteTransfer.TimeStamp),FieldType 
=typeof(long)},
        new QueryField {Name =nameof(IgniteTransfer.IsDeposit),FieldType 
=typeof(bool)}, },
    Indexes =new[]
    {
        new QueryIndex
        {
            Fields =new[]
            {
                new QueryIndexField {Name =nameof(IgniteTransfer.AccountId)},
                new QueryIndexField {Name =nameof(IgniteTransfer.AssetId)}
            },
            IndexType = QueryIndexType.Sorted },
        new QueryIndex
        {
            Fields =new[]
            {
                new QueryIndexField {Name =nameof(IgniteTransfer.AccountId)},
                new QueryIndexField {Name =nameof(IgniteTransfer.Reason)}
            },
            IndexType = QueryIndexType.Sorted },
        new QueryIndex
        {
            Fields =new[]
            {
                new QueryIndexField {Name =nameof(IgniteTransfer.AccountId)},
                new QueryIndexField {Name =nameof(IgniteTransfer.TimeStamp)}
            },
            IndexType = QueryIndexType.Sorted }
    }
};
var localCacheCfg =new CacheConfiguration("LocalTransfers2", query)
{
    CacheMode = CacheMode.Local,
    AtomicityMode = CacheAtomicityMode.Atomic };

var cache  = ignite.GetOrCreateCache<long, IgniteTransfer>(localCacheCfg);

Reply via email to