Thank you, Gang! Looking forward to seeing this feature in Kylin. 2018-04-01 17:35 GMT+08:00 Ma Gang <mg4w...@163.com>:
> Sure ShaoFeng, I add a storageType field in the SnapshotDesc to support > different materialized storages. > At 2018-04-01 09:57:23, "ShaoFeng Shi" <shaofeng...@apache.org> wrote: > >Thank you Ma Gang; This is a good proposal. Externalizing the lookup > >snapshots will reduce the burden of Kylin query servers. > > > >My only comment is, does this implementation support extension? You know > >since Kylin 1.5, Kylin has the plug-in architecture, HBase is one > >implementation for the storage. Kylin core modules don't directly depend > on > >HBase anymore; so please take this into consideration when you implement > >it. > > > >2018-03-30 17:57 GMT+08:00 magang <mg4w...@163.com>: > > > >> Hi all, > >> > >> There are two limitations for current lookup table design: > >> > >> 1. lookup table size is limited, because table snapshot need to be > cached > >> in > >> Kylin server, too large snapshot table will break Kylin server, also the > >> snapshot building may take very long time when it is too large. > >> 2. each segment has its own lookup table snapshot, but some users may > need > >> a > >> global snapshot table, which means when the global table is updated, the > >> query for all segments need to reflect the change. > >> > >> To resolve the above limitations, I have created ticket: > >> https://issues.apache.org/jira/browse/KYLIN-3221 > >> <https://issues.apache.org/jira/browse/KYLIN-3221> , and put initial > new > >> design doc there, any comments and suggestions are welcome. > >> > >> -- > >> Sent from: http://apache-kylin.74782.x6.nabble.com/ > >> > > > > > > > >-- > >Best regards, > > > >Shaofeng Shi 史少锋 > -- Best regards, Shaofeng Shi 史少锋
