Sure ShaoFeng, I add a storageType field in the SnapshotDesc to support
different materialized storages.
At 2018-04-01 09:57:23, "ShaoFeng Shi" <shaofeng...@apache.org> wrote:
>Thank you Ma Gang; This is a good proposal. Externalizing the lookup
>snapshots will reduce the burden of Kylin query servers.
>
>My only comment is, does this implementation support extension? You know
>since Kylin 1.5, Kylin has the plug-in architecture, HBase is one
>implementation for the storage. Kylin core modules don't directly depend on
>HBase anymore; so please take this into consideration when you implement
>it.
>
>2018-03-30 17:57 GMT+08:00 magang <mg4w...@163.com>:
>
>> Hi all,
>>
>> There are two limitations for current lookup table design:
>>
>> 1. lookup table size is limited, because table snapshot need to be cached
>> in
>> Kylin server, too large snapshot table will break Kylin server, also the
>> snapshot building may take very long time when it is too large.
>> 2. each segment has its own lookup table snapshot, but some users may need
>> a
>> global snapshot table, which means when the global table is updated, the
>> query for all segments need to reflect the change.
>>
>> To resolve the above limitations, I have created ticket:
>> https://issues.apache.org/jira/browse/KYLIN-3221
>> <https://issues.apache.org/jira/browse/KYLIN-3221> , and put initial new
>> design doc there, any comments and suggestions are welcome.
>>
>> --
>> Sent from: http://apache-kylin.74782.x6.nabble.com/
>>
>
>
>
>--
>Best regards,
>
>Shaofeng Shi 史少锋
