Hi devs, I’d like to push FLIP-221 forward a little bit. Recently we had some offline discussions and updated the FLIP. Here’s the diff compared to the previous version:
1. (Async)LookupFunctionProvider is designed as a base interface for constructing lookup functions. 2. From the LookupFunction we extend PartialCaching / FullCachingLookupProvider for partial and full caching mode. 3. Introduce CacheReloadTrigger for specifying reload stratrgy in full caching mode, and provide 2 default implementations (Periodic / TimedCacheReloadTrigger) Looking forward to your replies~ Best, Qingsheng > On Jun 2, 2022, at 17:15, Qingsheng Ren <renqs...@gmail.com> wrote: > > Hi Becket, > > Thanks for your feedback! > > 1. An alternative way is to let the implementation of cache to decide > whether to store a missing key in the cache instead of the framework. > This sounds more reasonable and makes the LookupProvider interface > cleaner. I can update the FLIP and clarify in the JavaDoc of > LookupCache#put that the cache should decide whether to store an empty > collection. > > 2. Initially the builder pattern is for the extensibility of > LookupProvider interfaces that we could need to add more > configurations in the future. We can remove the builder now as we have > resolved the issue in 1. As for the builder in DefaultLookupCache I > prefer to keep it because we have a lot of arguments in the > constructor. > > 3. I think this might overturn the overall design. I agree with > Becket's idea that the API design should be layered considering > extensibility and it'll be great to have one unified interface > supporting both partial, full and even mixed custom strategies, but we > have some issues to resolve. The original purpose of treating full > caching separately is that we'd like to reuse the ability of > ScanRuntimeProvider. Developers just need to hand over Source / > SourceFunction / InputFormat so that the framework could be able to > compose the underlying topology and control the reload (maybe in a > distributed way). Under your design we leave the reload operation > totally to the CacheStrategy and I think it will be hard for > developers to reuse the source in the initializeCache method. > > Best regards, > > Qingsheng > > On Thu, Jun 2, 2022 at 1:50 PM Becket Qin <becket....@gmail.com> wrote: >> >> Thanks for updating the FLIP, Qingsheng. A few more comments: >> >> 1. I am still not sure about what is the use case for cacheMissingKey(). >> More specifically, when would users want to have getCache() return a >> non-empty value and cacheMissingKey() returns false? >> >> 2. The builder pattern. Usually the builder pattern is used when there are >> a lot of variations of constructors. For example, if a class has three >> variables and all of them are optional, so there could potentially be many >> combinations of the variables. But in this FLIP, I don't see such case. >> What is the reason we have builders for all the classes? >> >> 3. Should the caching strategy be excluded from the top level provider API? >> Technically speaking, the Flink framework should only have two interfaces >> to deal with: >> A) LookupFunction >> B) AsyncLookupFunction >> Orthogonally, we *believe* there are two different strategies people can do >> caching. Note that the Flink framework does not care what is the caching >> strategy here. >> a) partial caching >> b) full caching >> >> Putting them together, we end up with 3 combinations that we think are >> valid: >> Aa) PartialCachingLookupFunctionProvider >> Ba) PartialCachingAsyncLookupFunctionProvider >> Ab) FullCachingLookupFunctionProvider >> >> However, the caching strategy could actually be quite flexible. E.g. an >> initial full cache load followed by some partial updates. Also, I am not >> 100% sure if the full caching will always use ScanTableSource. Including >> the caching strategy in the top level provider API would make it harder to >> extend. >> >> One possible solution is to just have *LookupFunctionProvider* and >> *AsyncLookupFunctionProvider >> *as the top level API, both with a getCacheStrategy() method returning an >> optional CacheStrategy. The CacheStrategy class would have the following >> methods: >> 1. void open(Context), the context exposes some of the resources that may >> be useful for the the caching strategy, e.g. an ExecutorService that is >> synchronized with the data processing, or a cache refresh trigger which >> blocks data processing and refresh the cache. >> 2. void initializeCache(), a blocking method allows users to pre-populate >> the cache before processing any data if they wish. >> 3. void maybeCache(RowData key, Collection<RowData> value), blocking or >> non-blocking method. >> 4. void refreshCache(), a blocking / non-blocking method that is invoked by >> the Flink framework when the cache refresh trigger is pulled. >> >> In the above design, partial caching and full caching would be >> implementations of the CachingStrategy. And it is OK for users to implement >> their own CachingStrategy if they want to. >> >> Thanks, >> >> Jiangjie (Becket) Qin >> >> >> On Thu, Jun 2, 2022 at 12:14 PM Jark Wu <imj...@gmail.com> wrote: >> >>> Thank Qingsheng for the detailed summary and updates, >>> >>> The changes look good to me in general. I just have one minor improvement >>> comment. >>> Could we add a static util method to the "FullCachingReloadTrigger" >>> interface for quick usage? >>> >>> #periodicReloadAtFixedRate(Duration) >>> #periodicReloadWithFixedDelay(Duration) >>> >>> I think we can also do this for LookupCache. Because users may not know >>> where is the default >>> implementations and how to use them. >>> >>> Best, >>> Jark >>> >>> >>> >>> >>> >>> >>> On Wed, 1 Jun 2022 at 18:32, Qingsheng Ren <renqs...@gmail.com> wrote: >>> >>>> Hi Jingsong, >>>> >>>> Thanks for your comments! >>>> >>>>> AllCache definition is not flexible, for example, PartialCache can use >>>> any custom storage, while the AllCache can not, AllCache can also be >>>> considered to store memory or disk, also need a flexible strategy. >>>> >>>> We had an offline discussion with Jark and Leonard. Basically we think >>>> exposing the interface of full cache storage to connector developers >>> might >>>> limit our future optimizations. The storage of full caching shouldn’t >>> have >>>> too many variations for different lookup tables so making it pluggable >>>> might not help a lot. Also I think it is not quite easy for connector >>>> developers to implement such an optimized storage. We can keep optimizing >>>> this storage in the future and all full caching lookup tables would >>> benefit >>>> from this. >>>> >>>>> We are more inclined to deprecate the connector `async` option when >>>> discussing FLIP-234. Can we remove this option from this FLIP? >>>> >>>> Thanks for the reminder! This option has been removed in the latest >>>> version. >>>> >>>> Best regards, >>>> >>>> Qingsheng >>>> >>>> >>>>> On Jun 1, 2022, at 15:28, Jingsong Li <jingsongl...@gmail.com> wrote: >>>>> >>>>> Thanks Alexander for your reply. We can discuss the new interface when >>> it >>>>> comes out. >>>>> >>>>> We are more inclined to deprecate the connector `async` option when >>>>> discussing FLIP-234 [1]. We should use hint to let planner decide. >>>>> Although the discussion has not yet produced a conclusion, can we >>> remove >>>>> this option from this FLIP? It doesn't seem to be related to this FLIP, >>>> but >>>>> more to FLIP-234, and we can form a conclusion over there. >>>>> >>>>> [1] https://lists.apache.org/thread/9k1sl2519kh2n3yttwqc00p07xdfns3h >>>>> >>>>> Best, >>>>> Jingsong >>>>> >>>>> On Wed, Jun 1, 2022 at 4:59 AM Jing Ge <j...@ververica.com> wrote: >>>>> >>>>>> Hi Jark, >>>>>> >>>>>> Thanks for clarifying it. It would be fine. as long as we could >>> provide >>>> the >>>>>> no-cache solution. I was just wondering if the client side cache could >>>>>> really help when HBase is used, since the data to look up should be >>>> huge. >>>>>> Depending how much data will be cached on the client side, the data >>> that >>>>>> should be lru in e.g. LruBlockCache will not be lru anymore. In the >>>> worst >>>>>> case scenario, once the cached data at client side is expired, the >>>> request >>>>>> will hit disk which will cause extra latency temporarily, if I am not >>>>>> mistaken. >>>>>> >>>>>> Best regards, >>>>>> Jing >>>>>> >>>>>> On Mon, May 30, 2022 at 9:59 AM Jark Wu <imj...@gmail.com> wrote: >>>>>> >>>>>>> Hi Jing Ge, >>>>>>> >>>>>>> What do you mean about the "impact on the block cache used by HBase"? >>>>>>> In my understanding, the connector cache and HBase cache are totally >>>> two >>>>>>> things. >>>>>>> The connector cache is a local/client cache, and the HBase cache is a >>>>>>> server cache. >>>>>>> >>>>>>>> does it make sense to have a no-cache solution as one of the >>>>>>> default solutions so that customers will have no effort for the >>>> migration >>>>>>> if they want to stick with Hbase cache >>>>>>> >>>>>>> The implementation migration should be transparent to users. Take the >>>>>> HBase >>>>>>> connector as >>>>>>> an example, it already supports lookup cache but is disabled by >>>> default. >>>>>>> After migration, the >>>>>>> connector still disables cache by default (i.e. no-cache solution). >>> No >>>>>>> migration effort for users. >>>>>>> >>>>>>> HBase cache and connector cache are two different things. HBase cache >>>>>> can't >>>>>>> simply replace >>>>>>> connector cache. Because one of the most important usages for >>> connector >>>>>>> cache is reducing >>>>>>> the I/O request/response and improving the throughput, which can >>>> achieve >>>>>>> by just using a server cache. >>>>>>> >>>>>>> Best, >>>>>>> Jark >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, 27 May 2022 at 22:42, Jing Ge <j...@ververica.com> wrote: >>>>>>> >>>>>>>> Thanks all for the valuable discussion. The new feature looks very >>>>>>>> interesting. >>>>>>>> >>>>>>>> According to the FLIP description: "*Currently we have JDBC, Hive >>> and >>>>>>> HBase >>>>>>>> connector implemented lookup table source. All existing >>>> implementations >>>>>>>> will be migrated to the current design and the migration will be >>>>>>>> transparent to end users*." I was only wondering if we should pay >>>>>>> attention >>>>>>>> to HBase and similar DBs. Since, commonly, the lookup data will be >>>> huge >>>>>>>> while using HBase, partial caching will be used in this case, if I >>> am >>>>>> not >>>>>>>> mistaken, which might have an impact on the block cache used by >>> HBase, >>>>>>> e.g. >>>>>>>> LruBlockCache. >>>>>>>> Another question is that, since HBase provides a sophisticated cache >>>>>>>> solution, does it make sense to have a no-cache solution as one of >>> the >>>>>>>> default solutions so that customers will have no effort for the >>>>>> migration >>>>>>>> if they want to stick with Hbase cache? >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Jing >>>>>>>> >>>>>>>> On Fri, May 27, 2022 at 11:19 AM Jingsong Li < >>> jingsongl...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> I think the problem now is below: >>>>>>>>> 1. AllCache and PartialCache interface on the non-uniform, one >>> needs >>>>>> to >>>>>>>>> provide LookupProvider, the other needs to provide CacheBuilder. >>>>>>>>> 2. AllCache definition is not flexible, for example, PartialCache >>> can >>>>>>> use >>>>>>>>> any custom storage, while the AllCache can not, AllCache can also >>> be >>>>>>>>> considered to store memory or disk, also need a flexible strategy. >>>>>>>>> 3. AllCache can not customize ReloadStrategy, currently only >>>>>>>>> ScheduledReloadStrategy. >>>>>>>>> >>>>>>>>> In order to solve the above problems, the following are my ideas. >>>>>>>>> >>>>>>>>> ## Top level cache interfaces: >>>>>>>>> >>>>>>>>> ``` >>>>>>>>> >>>>>>>>> public interface CacheLookupProvider extends >>>>>>>>> LookupTableSource.LookupRuntimeProvider { >>>>>>>>> >>>>>>>>> CacheBuilder createCacheBuilder(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> public interface CacheBuilder { >>>>>>>>> Cache create(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> public interface Cache { >>>>>>>>> >>>>>>>>> /** >>>>>>>>> * Returns the value associated with key in this cache, or null >>>>>> if >>>>>>>>> there is no cached value for >>>>>>>>> * key. >>>>>>>>> */ >>>>>>>>> @Nullable >>>>>>>>> Collection<RowData> getIfPresent(RowData key); >>>>>>>>> >>>>>>>>> /** Returns the number of key-value mappings in the cache. */ >>>>>>>>> long size(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> ``` >>>>>>>>> >>>>>>>>> ## Partial cache >>>>>>>>> >>>>>>>>> ``` >>>>>>>>> >>>>>>>>> public interface PartialCacheLookupFunction extends >>>>>>> CacheLookupProvider { >>>>>>>>> >>>>>>>>> @Override >>>>>>>>> PartialCacheBuilder createCacheBuilder(); >>>>>>>>> >>>>>>>>> /** Creates an {@link LookupFunction} instance. */ >>>>>>>>> LookupFunction createLookupFunction(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> public interface PartialCacheBuilder extends CacheBuilder { >>>>>>>>> >>>>>>>>> PartialCache create(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> public interface PartialCache extends Cache { >>>>>>>>> >>>>>>>>> /** >>>>>>>>> * Associates the specified value rows with the specified key >>> row >>>>>>>>> in the cache. If the cache >>>>>>>>> * previously contained value associated with the key, the old >>>>>>>>> value is replaced by the >>>>>>>>> * specified value. >>>>>>>>> * >>>>>>>>> * @return the previous value rows associated with key, or null >>>>>> if >>>>>>>>> there was no mapping for key. >>>>>>>>> * @param key - key row with which the specified value is to be >>>>>>>>> associated >>>>>>>>> * @param value – value rows to be associated with the specified >>>>>>> key >>>>>>>>> */ >>>>>>>>> Collection<RowData> put(RowData key, Collection<RowData> value); >>>>>>>>> >>>>>>>>> /** Discards any cached value for the specified key. */ >>>>>>>>> void invalidate(RowData key); >>>>>>>>> } >>>>>>>>> >>>>>>>>> ``` >>>>>>>>> >>>>>>>>> ## All cache >>>>>>>>> ``` >>>>>>>>> >>>>>>>>> public interface AllCacheLookupProvider extends >>> CacheLookupProvider { >>>>>>>>> >>>>>>>>> void registerReloadStrategy(ScheduledExecutorService >>>>>>>>> executorService, Reloader reloader); >>>>>>>>> >>>>>>>>> ScanTableSource.ScanRuntimeProvider getScanRuntimeProvider(); >>>>>>>>> >>>>>>>>> @Override >>>>>>>>> AllCacheBuilder createCacheBuilder(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> public interface AllCacheBuilder extends CacheBuilder { >>>>>>>>> >>>>>>>>> AllCache create(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> public interface AllCache extends Cache { >>>>>>>>> >>>>>>>>> void putAll(Iterator<Map<RowData, RowData>> allEntries); >>>>>>>>> >>>>>>>>> void clearAll(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> public interface Reloader { >>>>>>>>> >>>>>>>>> void reload(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> ``` >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Jingsong >>>>>>>>> >>>>>>>>> On Fri, May 27, 2022 at 11:10 AM Jingsong Li < >>> jingsongl...@gmail.com >>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Thanks Qingsheng and all for your discussion. >>>>>>>>>> >>>>>>>>>> Very sorry to jump in so late. >>>>>>>>>> >>>>>>>>>> Maybe I missed something? >>>>>>>>>> My first impression when I saw the cache interface was, why don't >>>>>> we >>>>>>>>>> provide an interface similar to guava cache [1], on top of guava >>>>>>> cache, >>>>>>>>>> caffeine also makes extensions for asynchronous calls.[2] >>>>>>>>>> There is also the bulk load in caffeine too. >>>>>>>>>> >>>>>>>>>> I am also more confused why first from LookupCacheFactory.Builder >>>>>> and >>>>>>>>> then >>>>>>>>>> to Factory to create Cache. >>>>>>>>>> >>>>>>>>>> [1] https://github.com/google/guava >>>>>>>>>> [2] https://github.com/ben-manes/caffeine/wiki/Population >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Jingsong >>>>>>>>>> >>>>>>>>>> On Thu, May 26, 2022 at 11:17 PM Jark Wu <imj...@gmail.com> >>> wrote: >>>>>>>>>> >>>>>>>>>>> After looking at the new introduced ReloadTime and Becket's >>>>>> comment, >>>>>>>>>>> I agree with Becket we should have a pluggable reloading >>> strategy. >>>>>>>>>>> We can provide some common implementations, e.g., periodic >>>>>>> reloading, >>>>>>>>> and >>>>>>>>>>> daily reloading. >>>>>>>>>>> But there definitely be some connector- or business-specific >>>>>>> reloading >>>>>>>>>>> strategies, e.g. >>>>>>>>>>> notify by a zookeeper watcher, reload once a new Hive partition >>> is >>>>>>>>>>> complete. >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Jark >>>>>>>>>>> >>>>>>>>>>> On Thu, 26 May 2022 at 11:52, Becket Qin <becket....@gmail.com> >>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Qingsheng, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for updating the FLIP. A few comments / questions below: >>>>>>>>>>>> >>>>>>>>>>>> 1. Is there a reason that we have both "XXXFactory" and >>>>>>>> "XXXProvider". >>>>>>>>>>>> What is the difference between them? If they are the same, can >>>>>> we >>>>>>>> just >>>>>>>>>>> use >>>>>>>>>>>> XXXFactory everywhere? >>>>>>>>>>>> >>>>>>>>>>>> 2. Regarding the FullCachingLookupProvider, should the reloading >>>>>>>>> policy >>>>>>>>>>>> also be pluggable? Periodical reloading could be sometimes be >>>>>>> tricky >>>>>>>>> in >>>>>>>>>>>> practice. For example, if user uses 24 hours as the cache >>>>>> refresh >>>>>>>>>>> interval >>>>>>>>>>>> and some nightly batch job delayed, the cache update may still >>>>>> see >>>>>>>> the >>>>>>>>>>>> stale data. >>>>>>>>>>>> >>>>>>>>>>>> 3. In DefaultLookupCacheFactory, it looks like InitialCapacity >>>>>>>> should >>>>>>>>> be >>>>>>>>>>>> removed. >>>>>>>>>>>> >>>>>>>>>>>> 4. The purpose of LookupFunctionProvider#cacheMissingKey() >>>>>> seems a >>>>>>>>>>> little >>>>>>>>>>>> confusing to me. If Optional<LookupCacheFactory> >>>>>> getCacheFactory() >>>>>>>>>>> returns >>>>>>>>>>>> a non-empty factory, doesn't that already indicates the >>>>>> framework >>>>>>> to >>>>>>>>>>> cache >>>>>>>>>>>> the missing keys? Also, why is this method returning an >>>>>>>>>>> Optional<Boolean> >>>>>>>>>>>> instead of boolean? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Jiangjie (Becket) Qin >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, May 25, 2022 at 5:07 PM Qingsheng Ren < >>>>>> renqs...@gmail.com >>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Lincoln and Jark, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for the comments! If the community reaches a consensus >>>>>>> that >>>>>>>> we >>>>>>>>>>> use >>>>>>>>>>>>> SQL hint instead of table options to decide whether to use sync >>>>>>> or >>>>>>>>>>> async >>>>>>>>>>>>> mode, it’s indeed not necessary to introduce the “lookup.async” >>>>>>>>> option. >>>>>>>>>>>>> >>>>>>>>>>>>> I think it’s a good idea to let the decision of async made on >>>>>>> query >>>>>>>>>>>>> level, which could make better optimization with more >>>>>> infomation >>>>>>>>>>> gathered >>>>>>>>>>>>> by planner. Is there any FLIP describing the issue in >>>>>>> FLINK-27625? >>>>>>>> I >>>>>>>>>>>>> thought FLIP-234 is only proposing adding SQL hint for retry on >>>>>>>>> missing >>>>>>>>>>>>> instead of the entire async mode to be controlled by hint. >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> >>>>>>>>>>>>> Qingsheng >>>>>>>>>>>>> >>>>>>>>>>>>>> On May 25, 2022, at 15:13, Lincoln Lee < >>>>>> lincoln.8...@gmail.com >>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Jark, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your reply! >>>>>>>>>>>>>> >>>>>>>>>>>>>> Currently 'lookup.async' just lies in HBase connector, I have >>>>>>> no >>>>>>>>> idea >>>>>>>>>>>>>> whether or when to remove it (we can discuss it in another >>>>>>> issue >>>>>>>>> for >>>>>>>>>>> the >>>>>>>>>>>>>> HBase connector after FLINK-27625 is done), just not add it >>>>>>> into >>>>>>>> a >>>>>>>>>>>>> common >>>>>>>>>>>>>> option now. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Best, >>>>>>>>>>>>>> Lincoln Lee >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Jark Wu <imj...@gmail.com> 于2022年5月24日周二 20:14写道： >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Lincoln, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I have taken a look at FLIP-234, and I agree with you that >>>>>> the >>>>>>>>>>>>> connectors >>>>>>>>>>>>>>> can >>>>>>>>>>>>>>> provide both async and sync runtime providers simultaneously >>>>>>>>> instead >>>>>>>>>>>>> of one >>>>>>>>>>>>>>> of them. >>>>>>>>>>>>>>> At that point, "lookup.async" looks redundant. If this >>>>>> option >>>>>>> is >>>>>>>>>>>>> planned to >>>>>>>>>>>>>>> be removed >>>>>>>>>>>>>>> in the long term, I think it makes sense not to introduce it >>>>>>> in >>>>>>>>> this >>>>>>>>>>>>> FLIP. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>> Jark >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, 24 May 2022 at 11:08, Lincoln Lee < >>>>>>>> lincoln.8...@gmail.com >>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Qingsheng, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Sorry for jumping into the discussion so late. It's a good >>>>>>> idea >>>>>>>>>>> that >>>>>>>>>>>>> we >>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>> have a common table option. I have a minor comments on >>>>>>>>>>> 'lookup.async' >>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>> not make it a common option: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The table layer abstracts both sync and async lookup >>>>>>>>> capabilities, >>>>>>>>>>>>>>>> connectors implementers can choose one or both, in the case >>>>>>> of >>>>>>>>>>>>>>> implementing >>>>>>>>>>>>>>>> only one capability(status of the most of existing builtin >>>>>>>>>>> connectors) >>>>>>>>>>>>>>>> 'lookup.async' will not be used. And when a connector has >>>>>>> both >>>>>>>>>>>>>>>> capabilities, I think this choice is more suitable for >>>>>> making >>>>>>>>>>>>> decisions >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> the query level, for example, table planner can choose the >>>>>>>>> physical >>>>>>>>>>>>>>>> implementation of async lookup or sync lookup based on its >>>>>>> cost >>>>>>>>>>>>> model, or >>>>>>>>>>>>>>>> users can give query hint based on their own better >>>>>>>>>>> understanding. If >>>>>>>>>>>>>>>> there is another common table option 'lookup.async', it may >>>>>>>>> confuse >>>>>>>>>>>>> the >>>>>>>>>>>>>>>> users in the long run. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> So, I prefer to leave the 'lookup.async' option in private >>>>>>>> place >>>>>>>>>>> (for >>>>>>>>>>>>> the >>>>>>>>>>>>>>>> current hbase connector) and not turn it into a common >>>>>>> option. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> WDYT? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>> Lincoln Lee >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Qingsheng Ren <renqs...@gmail.com> 于2022年5月23日周一 14:54写道： >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi Alexander, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks for the review! We recently updated the FLIP and >>>>>> you >>>>>>>> can >>>>>>>>>>> find >>>>>>>>>>>>>>>> those >>>>>>>>>>>>>>>>> changes from my latest email. Since some terminologies has >>>>>>>>>>> changed so >>>>>>>>>>>>>>>> I’ll >>>>>>>>>>>>>>>>> use the new concept for replying your comments. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1. Builder vs ‘of’ >>>>>>>>>>>>>>>>> I’m OK to use builder pattern if we have additional >>>>>> optional >>>>>>>>>>>>> parameters >>>>>>>>>>>>>>>>> for full caching mode (“rescan” previously). The >>>>>>>>>>> schedule-with-delay >>>>>>>>>>>>>>> idea >>>>>>>>>>>>>>>>> looks reasonable to me, but I think we need to redesign >>>>>> the >>>>>>>>>>> builder >>>>>>>>>>>>> API >>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>> full caching to make it more descriptive for developers. >>>>>>> Would >>>>>>>>> you >>>>>>>>>>>>> mind >>>>>>>>>>>>>>>>> sharing your ideas about the API? For accessing the FLIP >>>>>>>>> workspace >>>>>>>>>>>>> you >>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>> just provide your account ID and ping any PMC member >>>>>>> including >>>>>>>>>>> Jark. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 2. Common table options >>>>>>>>>>>>>>>>> We have some discussions these days and propose to >>>>>>> introduce 8 >>>>>>>>>>> common >>>>>>>>>>>>>>>>> table options about caching. It has been updated on the >>>>>>> FLIP. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 3. Retries >>>>>>>>>>>>>>>>> I think we are on the same page :-) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> For your additional concerns: >>>>>>>>>>>>>>>>> 1) The table option has been updated. >>>>>>>>>>>>>>>>> 2) We got “lookup.cache” back for configuring whether to >>>>>> use >>>>>>>>>>> partial >>>>>>>>>>>>> or >>>>>>>>>>>>>>>>> full caching mode. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Qingsheng >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On May 19, 2022, at 17:25, Александр Смирнов < >>>>>>>>>>> smirale...@gmail.com> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Also I have a few additions: >>>>>>>>>>>>>>>>>> 1) maybe rename 'lookup.cache.maximum-size' to >>>>>>>>>>>>>>>>>> 'lookup.cache.max-rows'? I think it will be more clear >>>>>> that >>>>>>>> we >>>>>>>>>>> talk >>>>>>>>>>>>>>>>>> not about bytes, but about the number of rows. Plus it >>>>>> fits >>>>>>>>> more, >>>>>>>>>>>>>>>>>> considering my optimization with filters. >>>>>>>>>>>>>>>>>> 2) How will users enable rescanning? Are we going to >>>>>>> separate >>>>>>>>>>>>> caching >>>>>>>>>>>>>>>>>> and rescanning from the options point of view? Like >>>>>>> initially >>>>>>>>> we >>>>>>>>>>> had >>>>>>>>>>>>>>>>>> one option 'lookup.cache' with values LRU / ALL. I think >>>>>>> now >>>>>>>> we >>>>>>>>>>> can >>>>>>>>>>>>>>>>>> make a boolean option 'lookup.rescan'. RescanInterval can >>>>>>> be >>>>>>>>>>>>>>>>>> 'lookup.rescan.interval', etc. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>> Alexander >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> чт, 19 мая 2022 г. в 14:50, Александр Смирнов < >>>>>>>>>>> smirale...@gmail.com >>>>>>>>>>>>>>>> : >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi Qingsheng and Jark, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 1. Builders vs 'of' >>>>>>>>>>>>>>>>>>> I understand that builders are used when we have >>>>>> multiple >>>>>>>>>>>>>>> parameters. >>>>>>>>>>>>>>>>>>> I suggested them because we could add parameters later. >>>>>> To >>>>>>>>>>> prevent >>>>>>>>>>>>>>>>>>> Builder for ScanRuntimeProvider from looking redundant I >>>>>>> can >>>>>>>>>>>>> suggest >>>>>>>>>>>>>>>>>>> one more config now - "rescanStartTime". >>>>>>>>>>>>>>>>>>> It's a time in UTC (LocalTime class) when the first >>>>>> reload >>>>>>>> of >>>>>>>>>>> cache >>>>>>>>>>>>>>>>>>> starts. This parameter can be thought of as >>>>>> 'initialDelay' >>>>>>>>> (diff >>>>>>>>>>>>>>>>>>> between current time and rescanStartTime) in method >>>>>>>>>>>>>>>>>>> ScheduleExecutorService#scheduleWithFixedDelay [1] . It >>>>>>> can >>>>>>>> be >>>>>>>>>>> very >>>>>>>>>>>>>>>>>>> useful when the dimension table is updated by some other >>>>>>>>>>> scheduled >>>>>>>>>>>>>>> job >>>>>>>>>>>>>>>>>>> at a certain time. Or when the user simply wants a >>>>>> second >>>>>>>> scan >>>>>>>>>>>>>>> (first >>>>>>>>>>>>>>>>>>> cache reload) be delayed. This option can be used even >>>>>>>> without >>>>>>>>>>>>>>>>>>> 'rescanInterval' - in this case 'rescanInterval' will be >>>>>>> one >>>>>>>>>>> day. >>>>>>>>>>>>>>>>>>> If you are fine with this option, I would be very glad >>>>>> if >>>>>>>> you >>>>>>>>>>> would >>>>>>>>>>>>>>>>>>> give me access to edit FLIP page, so I could add it >>>>>> myself >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 2. Common table options >>>>>>>>>>>>>>>>>>> I also think that FactoryUtil would be overloaded by all >>>>>>>> cache >>>>>>>>>>>>>>>>>>> options. But maybe unify all suggested options, not only >>>>>>> for >>>>>>>>>>>>> default >>>>>>>>>>>>>>>>>>> cache? I.e. class 'LookupOptions', that unifies default >>>>>>>> cache >>>>>>>>>>>>>>> options, >>>>>>>>>>>>>>>>>>> rescan options, 'async', 'maxRetries'. WDYT? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 3. Retries >>>>>>>>>>>>>>>>>>> I'm fine with suggestion close to >>>>>>> RetryUtils#tryTimes(times, >>>>>>>>>>> call) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ScheduledExecutorService.html#scheduleWithFixedDelay-java.lang.Runnable-long-long-java.util.concurrent.TimeUnit- >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>> Alexander >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ср, 18 мая 2022 г. в 16:04, Qingsheng Ren < >>>>>>>> renqs...@gmail.com >>>>>>>>>> : >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi Jark and Alexander, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks for your comments! I’m also OK to introduce >>>>>> common >>>>>>>>> table >>>>>>>>>>>>>>>>> options. I prefer to introduce a new >>>>>>> DefaultLookupCacheOptions >>>>>>>>>>> class >>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>> holding these option definitions because putting all >>>>>> options >>>>>>>>> into >>>>>>>>>>>>>>>>> FactoryUtil would make it a bit ”crowded” and not well >>>>>>>>>>> categorized. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> FLIP has been updated according to suggestions above: >>>>>>>>>>>>>>>>>>>> 1. Use static “of” method for constructing >>>>>>>>>>> RescanRuntimeProvider >>>>>>>>>>>>>>>>> considering both arguments are required. >>>>>>>>>>>>>>>>>>>> 2. Introduce new table options matching >>>>>>>>>>> DefaultLookupCacheFactory >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>> Qingsheng >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Wed, May 18, 2022 at 2:57 PM Jark Wu < >>>>>>> imj...@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hi Alex, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> 1) retry logic >>>>>>>>>>>>>>>>>>>>> I think we can extract some common retry logic into >>>>>>>>> utilities, >>>>>>>>>>>>>>> e.g. >>>>>>>>>>>>>>>>> RetryUtils#tryTimes(times, call). >>>>>>>>>>>>>>>>>>>>> This seems independent of this FLIP and can be reused >>>>>> by >>>>>>>>>>>>>>> DataStream >>>>>>>>>>>>>>>>> users. >>>>>>>>>>>>>>>>>>>>> Maybe we can open an issue to discuss this and where >>>>>> to >>>>>>>> put >>>>>>>>>>> it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> 2) cache ConfigOptions >>>>>>>>>>>>>>>>>>>>> I'm fine with defining cache config options in the >>>>>>>>> framework. >>>>>>>>>>>>>>>>>>>>> A candidate place to put is FactoryUtil which also >>>>>>>> includes >>>>>>>>>>>>>>>>> "sink.parallelism", "format" options. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>> Jark >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Wed, 18 May 2022 at 13:52, Александр Смирнов < >>>>>>>>>>>>>>>> smirale...@gmail.com> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hi Qingsheng, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thank you for considering my comments. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> there might be custom logic before making retry, >>>>>> such >>>>>>> as >>>>>>>>>>>>>>>>> re-establish the connection >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yes, I understand that. I meant that such logic can >>>>>> be >>>>>>>>>>> placed in >>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>> separate function, that can be implemented by >>>>>>> connectors. >>>>>>>>>>> Just >>>>>>>>>>>>>>>> moving >>>>>>>>>>>>>>>>>>>>>> the retry logic would make connector's LookupFunction >>>>>>>> more >>>>>>>>>>>>>>> concise >>>>>>>>>>>>>>>> + >>>>>>>>>>>>>>>>>>>>>> avoid duplicate code. However, it's a minor change. >>>>>> The >>>>>>>>>>> decision >>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>> up >>>>>>>>>>>>>>>>>>>>>> to you. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> We decide not to provide common DDL options and let >>>>>>>>>>> developers >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> define their own options as we do now per connector. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> What is the reason for that? One of the main goals of >>>>>>>> this >>>>>>>>>>> FLIP >>>>>>>>>>>>>>> was >>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>> unify the configs, wasn't it? I understand that >>>>>> current >>>>>>>>> cache >>>>>>>>>>>>>>>> design >>>>>>>>>>>>>>>>>>>>>> doesn't depend on ConfigOptions, like was before. But >>>>>>>> still >>>>>>>>>>> we >>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>> put >>>>>>>>>>>>>>>>>>>>>> these options into the framework, so connectors can >>>>>>> reuse >>>>>>>>>>> them >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>> avoid code duplication, and, what is more >>>>>> significant, >>>>>>>>> avoid >>>>>>>>>>>>>>>> possible >>>>>>>>>>>>>>>>>>>>>> different options naming. This moment can be pointed >>>>>>> out >>>>>>>> in >>>>>>>>>>>>>>>>>>>>>> documentation for connector developers. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>> Alexander >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> вт, 17 мая 2022 г. в 17:11, Qingsheng Ren < >>>>>>>>>>> renqs...@gmail.com>: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Hi Alexander, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks for the review and glad to see we are on the >>>>>>> same >>>>>>>>>>> page! >>>>>>>>>>>>> I >>>>>>>>>>>>>>>>> think you forgot to cc the dev mailing list so I’m also >>>>>>>> quoting >>>>>>>>>>> your >>>>>>>>>>>>>>>> reply >>>>>>>>>>>>>>>>> under this email. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> We can add 'maxRetryTimes' option into this class >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> In my opinion the retry logic should be implemented >>>>>> in >>>>>>>>>>> lookup() >>>>>>>>>>>>>>>>> instead of in LookupFunction#eval(). Retrying is only >>>>>>>> meaningful >>>>>>>>>>>>> under >>>>>>>>>>>>>>>> some >>>>>>>>>>>>>>>>> specific retriable failures, and there might be custom >>>>>> logic >>>>>>>>>>> before >>>>>>>>>>>>>>>> making >>>>>>>>>>>>>>>>> retry, such as re-establish the connection >>>>>>>>>>> (JdbcRowDataLookupFunction >>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>> example), so it's more handy to leave it to the connector. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I don't see DDL options, that were in previous >>>>>>> version >>>>>>>> of >>>>>>>>>>>>> FLIP. >>>>>>>>>>>>>>>> Do >>>>>>>>>>>>>>>>> you have any special plans for them? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> We decide not to provide common DDL options and let >>>>>>>>>>> developers >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> define their own options as we do now per connector. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> The rest of comments sound great and I’ll update the >>>>>>>> FLIP. >>>>>>>>>>> Hope >>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>> can finalize our proposal soon! >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Qingsheng >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On May 17, 2022, at 13:46, Александр Смирнов < >>>>>>>>>>>>>>>> smirale...@gmail.com> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Qingsheng and devs! >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I like the overall design of updated FLIP, however >>>>>> I >>>>>>>> have >>>>>>>>>>>>>>> several >>>>>>>>>>>>>>>>>>>>>>>> suggestions and questions. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 1) Introducing LookupFunction as a subclass of >>>>>>>>>>> TableFunction >>>>>>>>>>>>>>> is a >>>>>>>>>>>>>>>>> good >>>>>>>>>>>>>>>>>>>>>>>> idea. We can add 'maxRetryTimes' option into this >>>>>>>> class. >>>>>>>>>>>>> 'eval' >>>>>>>>>>>>>>>>> method >>>>>>>>>>>>>>>>>>>>>>>> of new LookupFunction is great for this purpose. >>>>>> The >>>>>>>> same >>>>>>>>>>> is >>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>> 'async' case. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 2) There might be other configs in future, such as >>>>>>>>>>>>>>>>> 'cacheMissingKey' >>>>>>>>>>>>>>>>>>>>>>>> in LookupFunctionProvider or 'rescanInterval' in >>>>>>>>>>>>>>>>> ScanRuntimeProvider. >>>>>>>>>>>>>>>>>>>>>>>> Maybe use Builder pattern in LookupFunctionProvider >>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>> RescanRuntimeProvider for more flexibility (use one >>>>>>>>> 'build' >>>>>>>>>>>>>>>> method >>>>>>>>>>>>>>>>>>>>>>>> instead of many 'of' methods in future)? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 3) What are the plans for existing >>>>>>>> TableFunctionProvider >>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>> AsyncTableFunctionProvider? I think they should be >>>>>>>>>>> deprecated. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 4) Am I right that the current design does not >>>>>> assume >>>>>>>>>>> usage of >>>>>>>>>>>>>>>>>>>>>>>> user-provided LookupCache in re-scanning? In this >>>>>>> case, >>>>>>>>> it >>>>>>>>>>> is >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>> very >>>>>>>>>>>>>>>>>>>>>>>> clear why do we need methods such as 'invalidate' >>>>>> or >>>>>>>>>>> 'putAll' >>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>> LookupCache. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 5) I don't see DDL options, that were in previous >>>>>>>> version >>>>>>>>>>> of >>>>>>>>>>>>>>>> FLIP. >>>>>>>>>>>>>>>>> Do >>>>>>>>>>>>>>>>>>>>>>>> you have any special plans for them? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> If you don't mind, I would be glad to be able to >>>>>> make >>>>>>>>> small >>>>>>>>>>>>>>>>>>>>>>>> adjustments to the FLIP document too. I think it's >>>>>>>> worth >>>>>>>>>>>>>>>> mentioning >>>>>>>>>>>>>>>>>>>>>>>> about what exactly optimizations are planning in >>>>>> the >>>>>>>>>>> future. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>> Smirnov Alexander >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> пт, 13 мая 2022 г. в 20:27, Qingsheng Ren < >>>>>>>>>>> renqs...@gmail.com >>>>>>>>>>>>>>>> : >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Hi Alexander and devs, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thank you very much for the in-depth discussion! >>>>>> As >>>>>>>> Jark >>>>>>>>>>>>>>>>> mentioned we were inspired by Alexander's idea and made a >>>>>>>>>>> refactor on >>>>>>>>>>>>>>> our >>>>>>>>>>>>>>>>> design. FLIP-221 [1] has been updated to reflect our >>>>>> design >>>>>>>> now >>>>>>>>>>> and >>>>>>>>>>>>> we >>>>>>>>>>>>>>>> are >>>>>>>>>>>>>>>>> happy to hear more suggestions from you! >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Compared to the previous design: >>>>>>>>>>>>>>>>>>>>>>>>> 1. The lookup cache serves at table runtime level >>>>>>> and >>>>>>>> is >>>>>>>>>>>>>>>>> integrated as a component of LookupJoinRunner as discussed >>>>>>>>>>>>> previously. >>>>>>>>>>>>>>>>>>>>>>>>> 2. Interfaces are renamed and re-designed to >>>>>> reflect >>>>>>>> the >>>>>>>>>>> new >>>>>>>>>>>>>>>>> design. >>>>>>>>>>>>>>>>>>>>>>>>> 3. We separate the all-caching case individually >>>>>> and >>>>>>>>>>>>>>> introduce a >>>>>>>>>>>>>>>>> new RescanRuntimeProvider to reuse the ability of >>>>>> scanning. >>>>>>> We >>>>>>>>> are >>>>>>>>>>>>>>>> planning >>>>>>>>>>>>>>>>> to support SourceFunction / InputFormat for now >>>>>> considering >>>>>>>> the >>>>>>>>>>>>>>>> complexity >>>>>>>>>>>>>>>>> of FLIP-27 Source API. >>>>>>>>>>>>>>>>>>>>>>>>> 4. A new interface LookupFunction is introduced to >>>>>>>> make >>>>>>>>>>> the >>>>>>>>>>>>>>>>> semantic of lookup more straightforward for developers. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> For replying to Alexander: >>>>>>>>>>>>>>>>>>>>>>>>>> However I'm a little confused whether InputFormat >>>>>>> is >>>>>>>>>>>>>>> deprecated >>>>>>>>>>>>>>>>> or not. Am I right that it will be so in the future, but >>>>>>>>> currently >>>>>>>>>>>>> it's >>>>>>>>>>>>>>>> not? >>>>>>>>>>>>>>>>>>>>>>>>> Yes you are right. InputFormat is not deprecated >>>>>> for >>>>>>>>> now. >>>>>>>>>>> I >>>>>>>>>>>>>>>> think >>>>>>>>>>>>>>>>> it will be deprecated in the future but we don't have a >>>>>>> clear >>>>>>>>> plan >>>>>>>>>>>>> for >>>>>>>>>>>>>>>> that. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks again for the discussion on this FLIP and >>>>>>>> looking >>>>>>>>>>>>>>> forward >>>>>>>>>>>>>>>>> to cooperating with you after we finalize the design and >>>>>>>>>>> interfaces! >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-221+Abstraction+for+lookup+source+cache+and+metric >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Qingsheng >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Fri, May 13, 2022 at 12:12 AM Александр >>>>>> Смирнов < >>>>>>>>>>>>>>>>> smirale...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jark, Qingsheng and Leonard! >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Glad to see that we came to a consensus on almost >>>>>>> all >>>>>>>>>>>>> points! >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> However I'm a little confused whether InputFormat >>>>>>> is >>>>>>>>>>>>>>> deprecated >>>>>>>>>>>>>>>>> or >>>>>>>>>>>>>>>>>>>>>>>>>> not. Am I right that it will be so in the future, >>>>>>> but >>>>>>>>>>>>>>> currently >>>>>>>>>>>>>>>>> it's >>>>>>>>>>>>>>>>>>>>>>>>>> not? Actually I also think that for the first >>>>>>> version >>>>>>>>>>> it's >>>>>>>>>>>>> OK >>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> use >>>>>>>>>>>>>>>>>>>>>>>>>> InputFormat in ALL cache realization, because >>>>>>>>> supporting >>>>>>>>>>>>>>> rescan >>>>>>>>>>>>>>>>>>>>>>>>>> ability seems like a very distant prospect. But >>>>>> for >>>>>>>>> this >>>>>>>>>>>>>>>>> decision we >>>>>>>>>>>>>>>>>>>>>>>>>> need a consensus among all discussion >>>>>> participants. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> In general, I don't have something to argue with >>>>>>> your >>>>>>>>>>>>>>>>> statements. All >>>>>>>>>>>>>>>>>>>>>>>>>> of them correspond my ideas. Looking ahead, it >>>>>>> would >>>>>>>> be >>>>>>>>>>> nice >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>>>>>>>>> on this FLIP cooperatively. I've already done a >>>>>> lot >>>>>>>> of >>>>>>>>>>> work >>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>>>>>>>>>>> join caching with realization very close to the >>>>>> one >>>>>>>> we >>>>>>>>>>> are >>>>>>>>>>>>>>>>> discussing, >>>>>>>>>>>>>>>>>>>>>>>>>> and want to share the results of this work. >>>>>> Anyway >>>>>>>>>>> looking >>>>>>>>>>>>>>>>> forward for >>>>>>>>>>>>>>>>>>>>>>>>>> the FLIP update! >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>> Smirnov Alexander >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> чт, 12 мая 2022 г. в 17:38, Jark Wu < >>>>>>>> imj...@gmail.com >>>>>>>>>> : >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Alex, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for summarizing your points. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> In the past week, Qingsheng, Leonard, and I have >>>>>>>>>>> discussed >>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>> several times >>>>>>>>>>>>>>>>>>>>>>>>>>> and we have totally refactored the design. >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm glad to say we have reached a consensus on >>>>>>> many >>>>>>>> of >>>>>>>>>>> your >>>>>>>>>>>>>>>>> points! >>>>>>>>>>>>>>>>>>>>>>>>>>> Qingsheng is still working on updating the >>>>>> design >>>>>>>> docs >>>>>>>>>>> and >>>>>>>>>>>>>>>>> maybe can be >>>>>>>>>>>>>>>>>>>>>>>>>>> available in the next few days. >>>>>>>>>>>>>>>>>>>>>>>>>>> I will share some conclusions from our >>>>>>> discussions: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1) we have refactored the design towards to >>>>>> "cache >>>>>>>> in >>>>>>>>>>>>>>>>> framework" way. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2) a "LookupCache" interface for users to >>>>>>> customize >>>>>>>>> and >>>>>>>>>>> a >>>>>>>>>>>>>>>>> default >>>>>>>>>>>>>>>>>>>>>>>>>>> implementation with builder for users to >>>>>> easy-use. >>>>>>>>>>>>>>>>>>>>>>>>>>> This can both make it possible to both have >>>>>>>>> flexibility >>>>>>>>>>> and >>>>>>>>>>>>>>>>> conciseness. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 3) Filter pushdown is important for ALL and LRU >>>>>>>> lookup >>>>>>>>>>>>>>> cache, >>>>>>>>>>>>>>>>> esp reducing >>>>>>>>>>>>>>>>>>>>>>>>>>> IO. >>>>>>>>>>>>>>>>>>>>>>>>>>> Filter pushdown should be the final state and >>>>>> the >>>>>>>>>>> unified >>>>>>>>>>>>>>> way >>>>>>>>>>>>>>>>> to both >>>>>>>>>>>>>>>>>>>>>>>>>>> support pruning ALL cache and LRU cache, >>>>>>>>>>>>>>>>>>>>>>>>>>> so I think we should make effort in this >>>>>>> direction. >>>>>>>> If >>>>>>>>>>> we >>>>>>>>>>>>>>> need >>>>>>>>>>>>>>>>> to support >>>>>>>>>>>>>>>>>>>>>>>>>>> filter pushdown for ALL cache anyway, why not >>>>>> use >>>>>>>>>>>>>>>>>>>>>>>>>>> it for LRU cache as well? Either way, as we >>>>>> decide >>>>>>>> to >>>>>>>>>>>>>>>> implement >>>>>>>>>>>>>>>>> the cache >>>>>>>>>>>>>>>>>>>>>>>>>>> in the framework, we have the chance to support >>>>>>>>>>>>>>>>>>>>>>>>>>> filter on cache anytime. This is an optimization >>>>>>> and >>>>>>>>> it >>>>>>>>>>>>>>>> doesn't >>>>>>>>>>>>>>>>> affect the >>>>>>>>>>>>>>>>>>>>>>>>>>> public API. I think we can create a JIRA issue >>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>> discuss it when the FLIP is accepted. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 4) The idea to support ALL cache is similar to >>>>>>> your >>>>>>>>>>>>>>> proposal. >>>>>>>>>>>>>>>>>>>>>>>>>>> In the first version, we will only support >>>>>>>>> InputFormat, >>>>>>>>>>>>>>>>> SourceFunction for >>>>>>>>>>>>>>>>>>>>>>>>>>> cache all (invoke InputFormat in join operator). >>>>>>>>>>>>>>>>>>>>>>>>>>> For FLIP-27 source, we need to join a true >>>>>> source >>>>>>>>>>> operator >>>>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>>>>>>>>>>>>>> calling it embedded in the join operator. >>>>>>>>>>>>>>>>>>>>>>>>>>> However, this needs another FLIP to support the >>>>>>>>> re-scan >>>>>>>>>>>>>>>> ability >>>>>>>>>>>>>>>>> for FLIP-27 >>>>>>>>>>>>>>>>>>>>>>>>>>> Source, and this can be a large work. >>>>>>>>>>>>>>>>>>>>>>>>>>> In order to not block this issue, we can put the >>>>>>>>> effort >>>>>>>>>>> of >>>>>>>>>>>>>>>>> FLIP-27 source >>>>>>>>>>>>>>>>>>>>>>>>>>> integration into future work and integrate >>>>>>>>>>>>>>>>>>>>>>>>>>> InputFormat&SourceFunction for now. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I think it's fine to use >>>>>>> InputFormat&SourceFunction, >>>>>>>>> as >>>>>>>>>>>>> they >>>>>>>>>>>>>>>>> are not >>>>>>>>>>>>>>>>>>>>>>>>>>> deprecated, otherwise, we have to introduce >>>>>>> another >>>>>>>>>>>>> function >>>>>>>>>>>>>>>>>>>>>>>>>>> similar to them which is meaningless. We need to >>>>>>>> plan >>>>>>>>>>>>>>> FLIP-27 >>>>>>>>>>>>>>>>> source >>>>>>>>>>>>>>>>>>>>>>>>>>> integration ASAP before InputFormat & >>>>>>> SourceFunction >>>>>>>>> are >>>>>>>>>>>>>>>>> deprecated. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>>>>>>>> Jark >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, 12 May 2022 at 15:46, Александр Смирнов >>>>>> < >>>>>>>>>>>>>>>>> smirale...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Martijn! >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Got it. Therefore, the realization with >>>>>>> InputFormat >>>>>>>>> is >>>>>>>>>>> not >>>>>>>>>>>>>>>>> considered. >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for clearing that up! >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>> Smirnov Alexander >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> чт, 12 мая 2022 г. в 14:23, Martijn Visser < >>>>>>>>>>>>>>>>> mart...@ververica.com>: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> With regards to: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if there are plans to refactor all >>>>>>> connectors >>>>>>>>> to >>>>>>>>>>>>>>>> FLIP-27 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, FLIP-27 is the target for all connectors. >>>>>>> The >>>>>>>>> old >>>>>>>>>>>>>>>>> interfaces will be >>>>>>>>>>>>>>>>>>>>>>>>>>>>> deprecated and connectors will either be >>>>>>>> refactored >>>>>>>>> to >>>>>>>>>>>>> use >>>>>>>>>>>>>>>>> the new ones >>>>>>>>>>>>>>>>>>>>>>>>>>>> or >>>>>>>>>>>>>>>>>>>>>>>>>>>>> dropped. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> The caching should work for connectors that >>>>>> are >>>>>>>>> using >>>>>>>>>>>>>>>> FLIP-27 >>>>>>>>>>>>>>>>> interfaces, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> we should not introduce new features for old >>>>>>>>>>> interfaces. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Martijn >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, 12 May 2022 at 06:19, Александр >>>>>> Смирнов >>>>>>> < >>>>>>>>>>>>>>>>> smirale...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jark! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry for the late response. I would like to >>>>>>> make >>>>>>>>>>> some >>>>>>>>>>>>>>>>> comments and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> clarify my points. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) I agree with your first statement. I think >>>>>>> we >>>>>>>>> can >>>>>>>>>>>>>>>> achieve >>>>>>>>>>>>>>>>> both >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> advantages this way: put the Cache interface >>>>>> in >>>>>>>>>>>>>>>>> flink-table-common, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but have implementations of it in >>>>>>>>>>> flink-table-runtime. >>>>>>>>>>>>>>>>> Therefore if a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> connector developer wants to use existing >>>>>> cache >>>>>>>>>>>>>>> strategies >>>>>>>>>>>>>>>>> and their >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations, he can just pass >>>>>> lookupConfig >>>>>>> to >>>>>>>>> the >>>>>>>>>>>>>>>>> planner, but if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> he wants to have its own cache implementation >>>>>>> in >>>>>>>>> his >>>>>>>>>>>>>>>>> TableFunction, it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be possible for him to use the existing >>>>>>>>>>> interface >>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> purpose (we can explicitly point this out in >>>>>>> the >>>>>>>>>>>>>>>>> documentation). In >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this way all configs and metrics will be >>>>>>> unified. >>>>>>>>>>> WDYT? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a filter can prune 90% of data in the >>>>>>> cache, >>>>>>>> we >>>>>>>>>>> will >>>>>>>>>>>>>>>>> have 90% of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lookup requests that can never be cached >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Let me clarify the logic filters >>>>>>> optimization >>>>>>>> in >>>>>>>>>>> case >>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>> LRU cache. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It looks like Cache<RowData, >>>>>>>> Collection<RowData>>. >>>>>>>>>>> Here >>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>> always >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> store the response of the dimension table in >>>>>>>> cache, >>>>>>>>>>> even >>>>>>>>>>>>>>>>> after >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applying calc function. I.e. if there are no >>>>>>> rows >>>>>>>>>>> after >>>>>>>>>>>>>>>>> applying >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filters to the result of the 'eval' method of >>>>>>>>>>>>>>>> TableFunction, >>>>>>>>>>>>>>>>> we store >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the empty list by lookup keys. Therefore the >>>>>>>> cache >>>>>>>>>>> line >>>>>>>>>>>>>>>> will >>>>>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filled, but will require much less memory (in >>>>>>>>> bytes). >>>>>>>>>>>>>>> I.e. >>>>>>>>>>>>>>>>> we don't >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completely filter keys, by which result was >>>>>>>> pruned, >>>>>>>>>>> but >>>>>>>>>>>>>>>>> significantly >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reduce required memory to store this result. >>>>>> If >>>>>>>> the >>>>>>>>>>> user >>>>>>>>>>>>>>>>> knows about >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this behavior, he can increase the 'max-rows' >>>>>>>>> option >>>>>>>>>>>>>>> before >>>>>>>>>>>>>>>>> the start >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the job. But actually I came up with the >>>>>>> idea >>>>>>>>>>> that we >>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>> do this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> automatically by using the 'maximumWeight' >>>>>> and >>>>>>>>>>> 'weigher' >>>>>>>>>>>>>>>>> methods of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GuavaCache [1]. Weight can be the size of the >>>>>>>>>>> collection >>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>> rows >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (value of cache). Therefore cache can >>>>>>>> automatically >>>>>>>>>>> fit >>>>>>>>>>>>>>>> much >>>>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> records than before. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Flink SQL has provided a standard way to do >>>>>>>>> filters >>>>>>>>>>> and >>>>>>>>>>>>>>>>> projects >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pushdown, i.e., SupportsFilterPushDown and >>>>>>>>>>>>>>>>> SupportsProjectionPushDown. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jdbc/hive/HBase haven't implemented the >>>>>>>>> interfaces, >>>>>>>>>>>>>>> don't >>>>>>>>>>>>>>>>> mean it's >>>>>>>>>>>>>>>>>>>>>>>>>>>> hard >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to implement. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's debatable how difficult it will be to >>>>>>>>> implement >>>>>>>>>>>>>>> filter >>>>>>>>>>>>>>>>> pushdown. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But I think the fact that currently there is >>>>>> no >>>>>>>>>>> database >>>>>>>>>>>>>>>>> connector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with filter pushdown at least means that this >>>>>>>>> feature >>>>>>>>>>>>>>> won't >>>>>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supported soon in connectors. Moreover, if we >>>>>>>> talk >>>>>>>>>>> about >>>>>>>>>>>>>>>>> other >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> connectors (not in Flink repo), their >>>>>> databases >>>>>>>>> might >>>>>>>>>>>>> not >>>>>>>>>>>>>>>>> support all >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Flink filters (or not support filters at >>>>>> all). >>>>>>> I >>>>>>>>>>> think >>>>>>>>>>>>>>>> users >>>>>>>>>>>>>>>>> are >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interested in supporting cache filters >>>>>>>> optimization >>>>>>>>>>>>>>>>> independently of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supporting other features and solving more >>>>>>>> complex >>>>>>>>>>>>>>> problems >>>>>>>>>>>>>>>>> (or >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unsolvable at all). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) I agree with your third statement. >>>>>> Actually >>>>>>> in >>>>>>>>> our >>>>>>>>>>>>>>>>> internal version >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also tried to unify the logic of scanning >>>>>> and >>>>>>>>>>>>> reloading >>>>>>>>>>>>>>>>> data from >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> connectors. But unfortunately, I didn't find >>>>>> a >>>>>>>> way >>>>>>>>> to >>>>>>>>>>>>>>> unify >>>>>>>>>>>>>>>>> the logic >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of all ScanRuntimeProviders (InputFormat, >>>>>>>>>>>>> SourceFunction, >>>>>>>>>>>>>>>>> Source,...) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and reuse it in reloading ALL cache. As a >>>>>>> result >>>>>>>> I >>>>>>>>>>>>>>> settled >>>>>>>>>>>>>>>>> on using >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InputFormat, because it was used for scanning >>>>>>> in >>>>>>>>> all >>>>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> connectors. (I didn't know that there are >>>>>> plans >>>>>>>> to >>>>>>>>>>>>>>>> deprecate >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InputFormat in favor of FLIP-27 Source). IMO >>>>>>>> usage >>>>>>>>> of >>>>>>>>>>>>>>>>> FLIP-27 source >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in ALL caching is not good idea, because this >>>>>>>>> source >>>>>>>>>>> was >>>>>>>>>>>>>>>>> designed to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work in distributed environment >>>>>>> (SplitEnumerator >>>>>>>> on >>>>>>>>>>>>>>>>> JobManager and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SourceReaders on TaskManagers), not in one >>>>>>>> operator >>>>>>>>>>>>>>> (lookup >>>>>>>>>>>>>>>>> join >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator in our case). There is even no >>>>>> direct >>>>>>>> way >>>>>>>>> to >>>>>>>>>>>>>>> pass >>>>>>>>>>>>>>>>> splits from >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SplitEnumerator to SourceReader (this logic >>>>>>> works >>>>>>>>>>>>> through >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SplitEnumeratorContext, which requires >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OperatorCoordinator.SubtaskGateway to send >>>>>>>>>>>>>>> AddSplitEvents). >>>>>>>>>>>>>>>>> Usage of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InputFormat for ALL cache seems much more >>>>>>> clearer >>>>>>>>> and >>>>>>>>>>>>>>>>> easier. But if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there are plans to refactor all connectors to >>>>>>>>>>> FLIP-27, I >>>>>>>>>>>>>>>>> have the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following ideas: maybe we can refuse from >>>>>>> lookup >>>>>>>>> join >>>>>>>>>>>>> ALL >>>>>>>>>>>>>>>>> cache in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> favor of simple join with multiple scanning >>>>>> of >>>>>>>>> batch >>>>>>>>>>>>>>>> source? >>>>>>>>>>>>>>>>> The point >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is that the only difference between lookup >>>>>> join >>>>>>>> ALL >>>>>>>>>>>>> cache >>>>>>>>>>>>>>>>> and simple >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> join with batch source is that in the first >>>>>>> case >>>>>>>>>>>>> scanning >>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>> performed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple times, in between which state >>>>>> (cache) >>>>>>> is >>>>>>>>>>>>> cleared >>>>>>>>>>>>>>>>> (correct me >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if I'm wrong). So what if we extend the >>>>>>>>>>> functionality of >>>>>>>>>>>>>>>>> simple join >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to support state reloading + extend the >>>>>>>>>>> functionality of >>>>>>>>>>>>>>>>> scanning >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> batch source multiple times (this one should >>>>>> be >>>>>>>>> easy >>>>>>>>>>>>> with >>>>>>>>>>>>>>>>> new FLIP-27 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> source, that unifies streaming/batch reading >>>>>> - >>>>>>> we >>>>>>>>>>> will >>>>>>>>>>>>>>> need >>>>>>>>>>>>>>>>> to change >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only SplitEnumerator, which will pass splits >>>>>>>> again >>>>>>>>>>> after >>>>>>>>>>>>>>>>> some TTL). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> WDYT? I must say that this looks like a >>>>>>> long-term >>>>>>>>>>> goal >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> will make >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the scope of this FLIP even larger than you >>>>>>> said. >>>>>>>>>>> Maybe >>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>> can limit >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ourselves to a simpler solution now >>>>>>>> (InputFormats). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So to sum up, my points is like this: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There is a way to make both concise and >>>>>>>> flexible >>>>>>>>>>>>>>>>> interfaces for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> caching in lookup join. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Cache filters optimization is important >>>>>> both >>>>>>>> in >>>>>>>>>>> LRU >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> ALL caches. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) It is unclear when filter pushdown will be >>>>>>>>>>> supported >>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>> Flink >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> connectors, some of the connectors might not >>>>>>> have >>>>>>>>> the >>>>>>>>>>>>>>>>> opportunity to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support filter pushdown + as I know, >>>>>> currently >>>>>>>>> filter >>>>>>>>>>>>>>>>> pushdown works >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only for scanning (not lookup). So cache >>>>>>> filters >>>>>>>> + >>>>>>>>>>>>>>>>> projections >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> optimization should be independent from other >>>>>>>>>>> features. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4) ALL cache realization is a complex topic >>>>>>> that >>>>>>>>>>>>> involves >>>>>>>>>>>>>>>>> multiple >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> aspects of how Flink is developing. Refusing >>>>>>> from >>>>>>>>>>>>>>>>> InputFormat in favor >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of FLIP-27 Source will make ALL cache >>>>>>> realization >>>>>>>>>>> really >>>>>>>>>>>>>>>>> complex and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not clear, so maybe instead of that we can >>>>>>> extend >>>>>>>>> the >>>>>>>>>>>>>>>>> functionality of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simple join or not refuse from InputFormat in >>>>>>>> case >>>>>>>>> of >>>>>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>> join ALL >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cache? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Smirnov Alexander >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>> https://guava.dev/releases/18.0/api/docs/com/google/common/cache/CacheBuilder.html#weigher(com.google.common.cache.Weigher) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> чт, 5 мая 2022 г. в 20:34, Jark Wu < >>>>>>>>> imj...@gmail.com >>>>>>>>>>>> : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's great to see the active discussion! I >>>>>>> want >>>>>>>> to >>>>>>>>>>>>> share >>>>>>>>>>>>>>>> my >>>>>>>>>>>>>>>>> ideas: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) implement the cache in framework vs. >>>>>>>> connectors >>>>>>>>>>> base >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't have a strong opinion on this. Both >>>>>>> ways >>>>>>>>>>> should >>>>>>>>>>>>>>>>> work (e.g., >>>>>>>>>>>>>>>>>>>>>>>>>>>> cache >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pruning, compatibility). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The framework way can provide more concise >>>>>>>>>>> interfaces. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The connector base way can define more >>>>>>> flexible >>>>>>>>>>> cache >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strategies/implementations. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We are still investigating a way to see if >>>>>> we >>>>>>>> can >>>>>>>>>>> have >>>>>>>>>>>>>>>> both >>>>>>>>>>>>>>>>>>>>>>>>>>>> advantages. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We should reach a consensus that the way >>>>>>> should >>>>>>>>> be a >>>>>>>>>>>>>>> final >>>>>>>>>>>>>>>>> state, >>>>>>>>>>>>>>>>>>>>>>>>>>>> and we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are on the path to it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) filters and projections pushdown: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree with Alex that the filter pushdown >>>>>>> into >>>>>>>>>>> cache >>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>> benefit a >>>>>>>>>>>>>>>>>>>>>>>>>>>> lot >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ALL cache. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However, this is not true for LRU cache. >>>>>>>>> Connectors >>>>>>>>>>> use >>>>>>>>>>>>>>>>> cache to >>>>>>>>>>>>>>>>>>>>>>>>>>>> reduce >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IO >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> requests to databases for better throughput. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a filter can prune 90% of data in the >>>>>>> cache, >>>>>>>> we >>>>>>>>>>> will >>>>>>>>>>>>>>>>> have 90% of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> requests that can never be cached >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and hit directly to the databases. That >>>>>> means >>>>>>>> the >>>>>>>>>>> cache >>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>>>>>>>>>> meaningless in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this case. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IMO, Flink SQL has provided a standard way >>>>>> to >>>>>>> do >>>>>>>>>>>>> filters >>>>>>>>>>>>>>>>> and projects >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pushdown, i.e., SupportsFilterPushDown and >>>>>>>>>>>>>>>>>>>>>>>>>>>> SupportsProjectionPushDown. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jdbc/hive/HBase haven't implemented the >>>>>>>>> interfaces, >>>>>>>>>>>>>>> don't >>>>>>>>>>>>>>>>> mean it's >>>>>>>>>>>>>>>>>>>>>>>>>>>> hard >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implement. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> They should implement the pushdown >>>>>> interfaces >>>>>>> to >>>>>>>>>>> reduce >>>>>>>>>>>>>>> IO >>>>>>>>>>>>>>>>> and the >>>>>>>>>>>>>>>>>>>>>>>>>>>> cache >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> size. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That should be a final state that the scan >>>>>>>> source >>>>>>>>>>> and >>>>>>>>>>>>>>>>> lookup source >>>>>>>>>>>>>>>>>>>>>>>>>>>> share >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the exact pushdown implementation. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't see why we need to duplicate the >>>>>>>> pushdown >>>>>>>>>>> logic >>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>> caches, >>>>>>>>>>>>>>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will complex the lookup join design. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) ALL cache abstraction >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> All cache might be the most challenging part >>>>>>> of >>>>>>>>> this >>>>>>>>>>>>>>> FLIP. >>>>>>>>>>>>>>>>> We have >>>>>>>>>>>>>>>>>>>>>>>>>>>> never >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provided a reload-lookup public interface. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we put the reload logic in the >>>>>>> "eval" >>>>>>>>>>> method >>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>> TableFunction. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That's hard for some sources (e.g., Hive). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Ideally, connector implementation should >>>>>> share >>>>>>>> the >>>>>>>>>>>>> logic >>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>> reload >>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scan, i.e. ScanTableSource with >>>>>>>>>>>>>>>>> InputFormat/SourceFunction/FLIP-27 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Source. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However, InputFormat/SourceFunction are >>>>>>>>> deprecated, >>>>>>>>>>> and >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> FLIP-27 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> source >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is deeply coupled with SourceOperator. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If we want to invoke the FLIP-27 source in >>>>>>>>>>> LookupJoin, >>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>> may make >>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope of this FLIP much larger. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We are still investigating how to abstract >>>>>> the >>>>>>>> ALL >>>>>>>>>>>>> cache >>>>>>>>>>>>>>>>> logic and >>>>>>>>>>>>>>>>>>>>>>>>>>>> reuse >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the existing source interfaces. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jark >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, 5 May 2022 at 20:22, Roman Boyko < >>>>>>>>>>>>>>>>> ro.v.bo...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's a much more complicated activity and >>>>>>> lies >>>>>>>>> out >>>>>>>>>>> of >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> scope of >>>>>>>>>>>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> improvement. Because such pushdowns should >>>>>> be >>>>>>>>> done >>>>>>>>>>> for >>>>>>>>>>>>>>>> all >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ScanTableSource >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementations (not only for Lookup ones). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, 5 May 2022 at 19:02, Martijn >>>>>> Visser < >>>>>>>>>>>>>>>>>>>>>>>>>>>> martijnvis...@apache.org> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> One question regarding "And Alexander >>>>>>>> correctly >>>>>>>>>>>>>>>> mentioned >>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>>>> filter >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pushdown still is not implemented for >>>>>>>>>>>>>>> jdbc/hive/hbase." >>>>>>>>>>>>>>>>> -> Would >>>>>>>>>>>>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> alternative solution be to actually >>>>>>> implement >>>>>>>>>>> these >>>>>>>>>>>>>>>> filter >>>>>>>>>>>>>>>>>>>>>>>>>>>> pushdowns? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> imagine that there are many more benefits >>>>>> to >>>>>>>>> doing >>>>>>>>>>>>>>> that, >>>>>>>>>>>>>>>>> outside >>>>>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> caching and metrics. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Martijn Visser >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://twitter.com/MartijnVisser82 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/MartijnVisser >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, 5 May 2022 at 13:58, Roman Boyko < >>>>>>>>>>>>>>>>> ro.v.bo...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi everyone! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving such a valuable >>>>>>>> improvement! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I do think that single cache >>>>>> implementation >>>>>>>>>>> would be >>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>> nice >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> opportunity >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users. And it will break the "FOR >>>>>>> SYSTEM_TIME >>>>>>>>> AS >>>>>>>>>>> OF >>>>>>>>>>>>>>>>> proc_time" >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> semantics >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> anyway - doesn't matter how it will be >>>>>>>>>>> implemented. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Putting myself in the user's shoes, I can >>>>>>> say >>>>>>>>>>> that: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) I would prefer to have the opportunity >>>>>>> to >>>>>>>>> cut >>>>>>>>>>> off >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> cache >>>>>>>>>>>>>>>>>>>>>>>>>>>> size >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simply filtering unnecessary data. And >>>>>> the >>>>>>>> most >>>>>>>>>>>>> handy >>>>>>>>>>>>>>>>> way to do >>>>>>>>>>>>>>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apply >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it inside LookupRunners. It would be a >>>>>> bit >>>>>>>>>>> harder to >>>>>>>>>>>>>>>>> pass it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> through the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LookupJoin node to TableFunction. And >>>>>>>> Alexander >>>>>>>>>>>>>>>> correctly >>>>>>>>>>>>>>>>>>>>>>>>>>>> mentioned >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filter pushdown still is not implemented >>>>>>> for >>>>>>>>>>>>>>>>> jdbc/hive/hbase. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) The ability to set the different >>>>>> caching >>>>>>>>>>>>>>> parameters >>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>>>>> different >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tables >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is quite important. So I would prefer to >>>>>>> set >>>>>>>> it >>>>>>>>>>>>>>> through >>>>>>>>>>>>>>>>> DDL >>>>>>>>>>>>>>>>>>>>>>>>>>>> rather >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have similar ttla, strategy and other >>>>>>> options >>>>>>>>> for >>>>>>>>>>>>> all >>>>>>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>>>>>>>>>>>>> tables. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) Providing the cache into the framework >>>>>>>>> really >>>>>>>>>>>>>>>>> deprives us of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> extensibility (users won't be able to >>>>>>>> implement >>>>>>>>>>>>> their >>>>>>>>>>>>>>>> own >>>>>>>>>>>>>>>>>>>>>>>>>>>> cache). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> most >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> probably it might be solved by creating >>>>>>> more >>>>>>>>>>>>>>> different >>>>>>>>>>>>>>>>> cache >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strategies >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a wider set of configurations. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> All these points are much closer to the >>>>>>>> schema >>>>>>>>>>>>>>> proposed >>>>>>>>>>>>>>>>> by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Alexander. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qingshen Ren, please correct me if I'm >>>>>> not >>>>>>>>> right >>>>>>>>>>> and >>>>>>>>>>>>>>>> all >>>>>>>>>>>>>>>>> these >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> facilities >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be simply implemented in your >>>>>>>>> architecture? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Roman Boyko >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e.: ro.v.bo...@gmail.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, 4 May 2022 at 21:01, Martijn >>>>>>> Visser < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> martijnvis...@apache.org> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't have much to chip in, but just >>>>>>>> wanted >>>>>>>>> to >>>>>>>>>>>>>>>>> express that >>>>>>>>>>>>>>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> really >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appreciate the in-depth discussion on >>>>>> this >>>>>>>>> topic >>>>>>>>>>>>>>> and I >>>>>>>>>>>>>>>>> hope >>>>>>>>>>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> others >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will join the conversation. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Martijn >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 3 May 2022 at 10:15, Александр >>>>>>>>> Смирнов < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> smirale...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Qingsheng, Leonard and Jark, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for your detailed feedback! >>>>>>>> However, I >>>>>>>>>>> have >>>>>>>>>>>>>>>>> questions >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> some of your statements (maybe I didn't >>>>>>> get >>>>>>>>>>>>>>>>> something?). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caching actually breaks the semantic >>>>>> of >>>>>>>> "FOR >>>>>>>>>>>>>>>>> SYSTEM_TIME >>>>>>>>>>>>>>>>>>>>>>>>>>>> AS OF >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> proc_time” >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree that the semantics of "FOR >>>>>>>>> SYSTEM_TIME >>>>>>>>>>> AS >>>>>>>>>>>>>>> OF >>>>>>>>>>>>>>>>>>>>>>>>>>>> proc_time" >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fully implemented with caching, but as >>>>>>> you >>>>>>>>>>> said, >>>>>>>>>>>>>>>> users >>>>>>>>>>>>>>>>> go >>>>>>>>>>>>>>>>>>>>>>>>>>>> on it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consciously to achieve better >>>>>> performance >>>>>>>> (no >>>>>>>>>>> one >>>>>>>>>>>>>>>>> proposed >>>>>>>>>>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> enable >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> caching by default, etc.). Or by users >>>>>> do >>>>>>>> you >>>>>>>>>>> mean >>>>>>>>>>>>>>>>> other >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> developers >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> connectors? In this case developers >>>>>>>>> explicitly >>>>>>>>>>>>>>>> specify >>>>>>>>>>>>>>>>>>>>>>>>>>>> whether >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> connector supports caching or not (in >>>>>> the >>>>>>>>> list >>>>>>>>>>> of >>>>>>>>>>>>>>>>> supported >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> options), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> no one makes them do that if they don't >>>>>>>> want >>>>>>>>>>> to. >>>>>>>>>>>>> So >>>>>>>>>>>>>>>>> what >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exactly is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the difference between implementing >>>>>>> caching >>>>>>>>> in >>>>>>>>>>>>>>>> modules >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-table-runtime and in >>>>>>>> flink-table-common >>>>>>>>>>> from >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> considered >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> point of view? How does it affect on >>>>>>>>>>>>>>>>> breaking/non-breaking >>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> semantics of "FOR SYSTEM_TIME AS OF >>>>>>>>> proc_time"? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> confront a situation that allows table >>>>>>>>>>> options in >>>>>>>>>>>>>>>> DDL >>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> control >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior of the framework, which has >>>>>>> never >>>>>>>>>>>>> happened >>>>>>>>>>>>>>>>>>>>>>>>>>>> previously >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be cautious >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If we talk about main differences of >>>>>>>>> semantics >>>>>>>>>>> of >>>>>>>>>>>>>>> DDL >>>>>>>>>>>>>>>>>>>>>>>>>>>> options >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> config options("table.exec.xxx"), isn't >>>>>>> it >>>>>>>>>>> about >>>>>>>>>>>>>>>>> limiting >>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the options + importance for the user >>>>>>>>> business >>>>>>>>>>>>>>> logic >>>>>>>>>>>>>>>>> rather >>>>>>>>>>>>>>>>>>>>>>>>>>>> than >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specific location of corresponding >>>>>> logic >>>>>>> in >>>>>>>>> the >>>>>>>>>>>>>>>>> framework? I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mean >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in my design, for example, putting an >>>>>>>> option >>>>>>>>>>> with >>>>>>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>>>>>>>>>>>>> cache >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> strategy in configurations would be >>>>>> the >>>>>>>>> wrong >>>>>>>>>>>>>>>>> decision, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> directly affects the user's business >>>>>>> logic >>>>>>>>> (not >>>>>>>>>>>>>>> just >>>>>>>>>>>>>>>>>>>>>>>>>>>> performance >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> optimization) + touches just several >>>>>>>>> functions >>>>>>>>>>> of >>>>>>>>>>>>>>> ONE >>>>>>>>>>>>>>>>> table >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (there >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be multiple tables with different >>>>>>> caches). >>>>>>>>>>> Does it >>>>>>>>>>>>>>>>> really >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> matter for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the user (or someone else) where the >>>>>>> logic >>>>>>>> is >>>>>>>>>>>>>>>> located, >>>>>>>>>>>>>>>>>>>>>>>>>>>> which is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> affected by the applied option? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I can remember DDL option >>>>>>>>>>> 'sink.parallelism', >>>>>>>>>>>>>>>>> which in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> some way >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "controls the behavior of the >>>>>> framework" >>>>>>>> and >>>>>>>>> I >>>>>>>>>>>>>>> don't >>>>>>>>>>>>>>>>> see any >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduce a new interface for this >>>>>>>>> all-caching >>>>>>>>>>>>>>>>> scenario >>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> design >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would become more complex >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is a subject for a separate >>>>>>>> discussion, >>>>>>>>>>> but >>>>>>>>>>>>>>>>> actually >>>>>>>>>>>>>>>>>>>>>>>>>>>> in our >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> internal version we solved this problem >>>>>>>> quite >>>>>>>>>>>>>>> easily >>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>>>>>>>>>>> reused >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InputFormat class (so there is no need >>>>>>> for >>>>>>>> a >>>>>>>>>>> new >>>>>>>>>>>>>>>> API). >>>>>>>>>>>>>>>>> The >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> point is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that currently all lookup connectors >>>>>> use >>>>>>>>>>>>>>> InputFormat >>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scanning >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data in batch mode: HBase, JDBC and >>>>>> even >>>>>>>> Hive >>>>>>>>>>> - it >>>>>>>>>>>>>>>> uses >>>>>>>>>>>>>>>>>>>>>>>>>>>> class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PartitionReader, that is actually just >>>>>> a >>>>>>>>>>> wrapper >>>>>>>>>>>>>>>> around >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InputFormat. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The advantage of this solution is the >>>>>>>> ability >>>>>>>>>>> to >>>>>>>>>>>>>>>> reload >>>>>>>>>>>>>>>>>>>>>>>>>>>> cache >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parallel (number of threads depends on >>>>>>>> number >>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>> InputSplits, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an upper limit). As a result cache >>>>>> reload >>>>>>>>> time >>>>>>>>>>>>>>>>> significantly >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reduces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (as well as time of input stream >>>>>>>> blocking). I >>>>>>>>>>> know >>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>>>> usually >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to avoid usage of concurrency in Flink >>>>>>>> code, >>>>>>>>>>> but >>>>>>>>>>>>>>>> maybe >>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>>>>>>> one >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an exception. BTW I don't say that it's >>>>>>> an >>>>>>>>>>> ideal >>>>>>>>>>>>>>>>> solution, >>>>>>>>>>>>>>>>>>>>>>>>>>>> maybe >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are better ones. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Providing the cache in the framework >>>>>>> might >>>>>>>>>>>>>>> introduce >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's possible only in cases when the >>>>>>>>> developer >>>>>>>>>>> of >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>> connector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> won't >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> properly refactor his code and will use >>>>>>> new >>>>>>>>>>> cache >>>>>>>>>>>>>>>>> options >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incorrectly >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (i.e. explicitly provide the same >>>>>> options >>>>>>>>> into >>>>>>>>>>> 2 >>>>>>>>>>>>>>>>> different >>>>>>>>>>>>>>>>>>>>>>>>>>>> code >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> places). For correct behavior all he >>>>>> will >>>>>>>>> need >>>>>>>>>>> to >>>>>>>>>>>>>>> do >>>>>>>>>>>>>>>>> is to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> redirect >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing options to the framework's >>>>>>>>>>> LookupConfig >>>>>>>>>>>>> (+ >>>>>>>>>>>>>>>>> maybe >>>>>>>>>>>>>>>>>>>>>>>>>>>> add an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> alias >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for options, if there was different >>>>>>>> naming), >>>>>>>>>>>>>>>> everything >>>>>>>>>>>>>>>>>>>>>>>>>>>> will be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> transparent for users. If the developer >>>>>>>> won't >>>>>>>>>>> do >>>>>>>>>>>>>>>>>>>>>>>>>>>> refactoring at >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nothing will be changed for the >>>>>> connector >>>>>>>>>>> because >>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>> backward >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility. Also if a developer >>>>>> wants >>>>>>> to >>>>>>>>> use >>>>>>>>>>>>> his >>>>>>>>>>>>>>>> own >>>>>>>>>>>>>>>>>>>>>>>>>>>> cache >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> he just can refuse to pass some of the >>>>>>>>> configs >>>>>>>>>>>>> into >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instead make his own implementation >>>>>> with >>>>>>>>>>> already >>>>>>>>>>>>>>>>> existing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configs >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metrics (but actually I think that >>>>>> it's a >>>>>>>>> rare >>>>>>>>>>>>>>> case). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filters and projections should be >>>>>> pushed >>>>>>>> all >>>>>>>>>>> the >>>>>>>>>>>>>>> way >>>>>>>>>>>>>>>>> down >>>>>>>>>>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> table >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> function, like what we do in the scan >>>>>>>> source >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's the great purpose. But the truth >>>>>> is >>>>>>>> that >>>>>>>>>>> the >>>>>>>>>>>>>>>> ONLY >>>>>>>>>>>>>>>>>>>>>>>>>>>> connector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supports filter pushdown is >>>>>>>>>>> FileSystemTableSource >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (no database connector supports it >>>>>>>>> currently). >>>>>>>>>>>>> Also >>>>>>>>>>>>>>>>> for some >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> databases >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's simply impossible to pushdown such >>>>>>>>> complex >>>>>>>>>>>>>>>> filters >>>>>>>>>>>>>>>>>>>>>>>>>>>> that we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in Flink. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only applying these optimizations to >>>>>> the >>>>>>>>> cache >>>>>>>>>>>>>>> seems >>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> quite >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> useful >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Filters can cut off an arbitrarily >>>>>> large >>>>>>>>>>> amount of >>>>>>>>>>>>>>>> data >>>>>>>>>>>>>>>>>>>>>>>>>>>> from the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dimension table. For a simple example, >>>>>>>>> suppose >>>>>>>>>>> in >>>>>>>>>>>>>>>>> dimension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> table >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'users' >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we have column 'age' with values from >>>>>> 20 >>>>>>> to >>>>>>>>> 40, >>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> input >>>>>>>>>>>>>>>>>>>>>>>>>>>> stream >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'clicks' that is ~uniformly distributed >>>>>>> by >>>>>>>>> age >>>>>>>>>>> of >>>>>>>>>>>>>>>>> users. If >>>>>>>>>>>>>>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filter 'age > 30', >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there will be twice less data in cache. >>>>>>>> This >>>>>>>>>>> means >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> user >>>>>>>>>>>>>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> increase 'lookup.cache.max-rows' by >>>>>>> almost >>>>>>>> 2 >>>>>>>>>>>>> times. >>>>>>>>>>>>>>>> It >>>>>>>>>>>>>>>>> will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> gain a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> huge >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performance boost. Moreover, this >>>>>>>>> optimization >>>>>>>>>>>>>>> starts >>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>>> really >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> shine >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in 'ALL' cache, where tables without >>>>>>>> filters >>>>>>>>>>> and >>>>>>>>>>>>>>>>> projections >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can't >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fit >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in memory, but with them - can. This >>>>>>> opens >>>>>>>> up >>>>>>>>>>>>>>>>> additional >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> possibilities >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for users. And this doesn't sound as >>>>>> 'not >>>>>>>>> quite >>>>>>>>>>>>>>>>> useful'. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It would be great to hear other voices >>>>>>>>>>> regarding >>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>> topic! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Because >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we have quite a lot of controversial >>>>>>>> points, >>>>>>>>>>> and I >>>>>>>>>>>>>>>>> think >>>>>>>>>>>>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> help >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of others it will be easier for us to >>>>>>> come >>>>>>>>> to a >>>>>>>>>>>>>>>>> consensus. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Smirnov Alexander >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> пт, 29 апр. 2022 г. в 22:33, Qingsheng >>>>>>> Ren >>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>> renqs...@gmail.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Alexander and Arvid, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the discussion and sorry >>>>>> for >>>>>>> my >>>>>>>>>>> late >>>>>>>>>>>>>>>>> response! >>>>>>>>>>>>>>>>>>>>>>>>>>>> We >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> had >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> internal discussion together with Jark >>>>>>> and >>>>>>>>>>> Leonard >>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> I’d >>>>>>>>>>>>>>>>>>>>>>>>>>>> like >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> summarize our ideas. Instead of >>>>>>>> implementing >>>>>>>>>>> the >>>>>>>>>>>>>>>> cache >>>>>>>>>>>>>>>>>>>>>>>>>>>> logic in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> table >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> runtime layer or wrapping around the >>>>>>>>>>> user-provided >>>>>>>>>>>>>>>>> table >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> function, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> prefer to introduce some new APIs >>>>>>> extending >>>>>>>>>>>>>>>>> TableFunction >>>>>>>>>>>>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> these >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> concerns: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. Caching actually breaks the >>>>>> semantic >>>>>>> of >>>>>>>>>>> "FOR >>>>>>>>>>>>>>>>>>>>>>>>>>>> SYSTEM_TIME >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AS OF >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> proc_time”, because it couldn’t truly >>>>>>>> reflect >>>>>>>>>>> the >>>>>>>>>>>>>>>>> content >>>>>>>>>>>>>>>>>>>>>>>>>>>> of the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> table at the moment of querying. If >>>>>> users >>>>>>>>>>> choose >>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> enable >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> caching >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lookup table, they implicitly indicate >>>>>>> that >>>>>>>>>>> this >>>>>>>>>>>>>>>>> breakage is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> acceptable >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exchange for the performance. So we >>>>>>> prefer >>>>>>>>> not >>>>>>>>>>> to >>>>>>>>>>>>>>>>> provide >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> caching on >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> table runtime level. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. If we make the cache implementation >>>>>>> in >>>>>>>>> the >>>>>>>>>>>>>>>>> framework >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (whether >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> runner or a wrapper around >>>>>>> TableFunction), >>>>>>>> we >>>>>>>>>>> have >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>>> confront a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> situation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that allows table options in DDL to >>>>>>> control >>>>>>>>> the >>>>>>>>>>>>>>>>> behavior of >>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> framework, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which has never happened previously and >>>>>>>>> should >>>>>>>>>>> be >>>>>>>>>>>>>>>>> cautious. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Under >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> current design the behavior of the >>>>>>>> framework >>>>>>>>>>>>> should >>>>>>>>>>>>>>>>> only be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> specified >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations (“table.exec.xxx”), and >>>>>>> it’s >>>>>>>>>>> hard >>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> apply >>>>>>>>>>>>>>>>>>>>>>>>>>>> these >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> general >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configs to a specific table. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3. We have use cases that lookup >>>>>> source >>>>>>>>> loads >>>>>>>>>>> and >>>>>>>>>>>>>>>>> refresh >>>>>>>>>>>>>>>>>>>>>>>>>>>> all >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> records >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> periodically into the memory to achieve >>>>>>>> high >>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>>>>>>>>>>>>> performance >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (like >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hive >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> connector in the community, and also >>>>>>> widely >>>>>>>>>>> used >>>>>>>>>>>>> by >>>>>>>>>>>>>>>> our >>>>>>>>>>>>>>>>>>>>>>>>>>>> internal >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> connectors). Wrapping the cache around >>>>>>> the >>>>>>>>>>> user’s >>>>>>>>>>>>>>>>>>>>>>>>>>>> TableFunction >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> works >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fine >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for LRU caches, but I think we have to >>>>>>>>>>> introduce a >>>>>>>>>>>>>>>> new >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interface for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all-caching scenario and the design >>>>>> would >>>>>>>>>>> become >>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>>>>>>>>>>>>> complex. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4. Providing the cache in the >>>>>> framework >>>>>>>>> might >>>>>>>>>>>>>>>>> introduce >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues to existing lookup sources like >>>>>>>> there >>>>>>>>>>> might >>>>>>>>>>>>>>>>> exist two >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> caches >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> totally different strategies if the >>>>>> user >>>>>>>>>>>>>>> incorrectly >>>>>>>>>>>>>>>>>>>>>>>>>>>> configures >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> table >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (one in the framework and another >>>>>>>> implemented >>>>>>>>>>> by >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> source). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As for the optimization mentioned by >>>>>>>>>>> Alexander, I >>>>>>>>>>>>>>>>> think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filters >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> projections should be pushed all the >>>>>> way >>>>>>>> down >>>>>>>>>>> to >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> table >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> function, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> what we do in the scan source, instead >>>>>> of >>>>>>>> the >>>>>>>>>>>>>>> runner >>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cache. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goal of using cache is to reduce the >>>>>>>> network >>>>>>>>>>> I/O >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>> pressure >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> external system, and only applying >>>>>> these >>>>>>>>>>>>>>>> optimizations >>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cache >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not quite useful. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I made some updates to the FLIP[1] to >>>>>>>>> reflect >>>>>>>>>>> our >>>>>>>>>>>>>>>>> ideas. >>>>>>>>>>>>>>>>>>>>>>>>>>>> We >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> prefer to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> keep the cache implementation as a part >>>>>>> of >>>>>>>>>>>>>>>>> TableFunction, >>>>>>>>>>>>>>>>>>>>>>>>>>>> and we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provide some helper classes >>>>>>>>>>> (CachingTableFunction, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AllCachingTableFunction, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CachingAsyncTableFunction) to >>>>>> developers >>>>>>>> and >>>>>>>>>>>>>>> regulate >>>>>>>>>>>>>>>>>>>>>>>>>>>> metrics >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cache. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, I made a POC[2] for your >>>>>> reference. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking forward to your ideas! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-221+Abstraction+for+lookup+source+cache+and+metric >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>> https://github.com/PatrickRen/flink/tree/FLIP-221 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qingsheng >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Apr 26, 2022 at 4:45 PM >>>>>>> Александр >>>>>>>>>>> Смирнов >>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> smirale...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the response, Arvid! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have few comments on your message. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but could also live with an easier >>>>>>>>> solution >>>>>>>>>>> as >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>> first >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> step: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think that these 2 ways are >>>>>> mutually >>>>>>>>>>> exclusive >>>>>>>>>>>>>>>>>>>>>>>>>>>> (originally >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> proposed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by Qingsheng and mine), because >>>>>>>>> conceptually >>>>>>>>>>>>> they >>>>>>>>>>>>>>>>> follow >>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> goal, but implementation details are >>>>>>>>>>> different. >>>>>>>>>>>>>>> If >>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>>>>>>>>>>> will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go one >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moving to another way in the future >>>>>>> will >>>>>>>>> mean >>>>>>>>>>>>>>>>> deleting >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and once again changing the API for >>>>>>>>>>> connectors. >>>>>>>>>>>>>>> So >>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>>>>>>>>>>> think we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reach a consensus with the community >>>>>>>> about >>>>>>>>>>> that >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> together >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on this FLIP, i.e. divide the work on >>>>>>>> tasks >>>>>>>>>>> for >>>>>>>>>>>>>>>>> different >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parts >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flip (for example, LRU cache >>>>>>> unification >>>>>>>> / >>>>>>>>>>>>>>>>> introducing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> proposed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metrics / further work…). WDYT, >>>>>>>> Qingsheng? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as the source will only receive the >>>>>>>>> requests >>>>>>>>>>>>>>> after >>>>>>>>>>>>>>>>>>>>>>>>>>>> filter >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Actually if filters are applied to >>>>>>> fields >>>>>>>>> of >>>>>>>>>>> the >>>>>>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> table, we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> firstly must do requests, and only >>>>>>> after >>>>>>>>>>> that we >>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>>>>>>>>>> filter >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> responses, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because lookup connectors don't have >>>>>>>> filter >>>>>>>>>>>>>>>>> pushdown. So >>>>>>>>>>>>>>>>>>>>>>>>>>>> if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filtering >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is done before caching, there will be >>>>>>>> much >>>>>>>>>>> less >>>>>>>>>>>>>>>> rows >>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cache. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Alexander unfortunately, your >>>>>>>>> architecture >>>>>>>>>>> is >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>>>>>>>> shared. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solution to share images to be >>>>>> honest. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry for that, I’m a bit new to such >>>>>>>> kinds >>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>> conversations >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> :) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have no write access to the >>>>>>> confluence, >>>>>>>>> so >>>>>>>>>>> I >>>>>>>>>>>>>>>> made a >>>>>>>>>>>>>>>>>>>>>>>>>>>> Jira >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where described the proposed changes >>>>>> in >>>>>>>>> more >>>>>>>>>>>>>>>> details >>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-27411. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Will happy to get more feedback! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Smirnov Alexander >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> пн, 25 апр. 2022 г. в 19:49, Arvid >>>>>>> Heise >>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>> ar...@apache.org>: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Qingsheng, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this; the >>>>>>>> inconsistency >>>>>>>>>>> was >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> satisfying >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> me. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I second Alexander's idea though but >>>>>>>> could >>>>>>>>>>> also >>>>>>>>>>>>>>>> live >>>>>>>>>>>>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> easier >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solution as the first step: Instead >>>>>> of >>>>>>>>>>> making >>>>>>>>>>>>>>>>> caching >>>>>>>>>>>>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> detail of TableFunction X, rather >>>>>>>> devise a >>>>>>>>>>>>>>> caching >>>>>>>>>>>>>>>>>>>>>>>>>>>> layer >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> around X. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> proposal would be a >>>>>>> CachingTableFunction >>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>>>> delegates to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> X in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> case >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> misses and else manages the cache. >>>>>>>> Lifting >>>>>>>>>>> it >>>>>>>>>>>>>>> into >>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operator >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> model >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> proposed would be even better but is >>>>>>>>>>> probably >>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessary >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> first step >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a lookup source (as the source >>>>>>> will >>>>>>>>> only >>>>>>>>>>>>>>>> receive >>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> requests >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filter; applying projection may be >>>>>>> more >>>>>>>>>>>>>>>> interesting >>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>>> save >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> memory). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Another advantage is that all the >>>>>>>> changes >>>>>>>>> of >>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>> FLIP >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limited to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> options, no need for new public >>>>>>>>> interfaces. >>>>>>>>>>>>>>>>> Everything >>>>>>>>>>>>>>>>>>>>>>>>>>>> else >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> remains >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementation of Table runtime. >>>>>> That >>>>>>>>> means >>>>>>>>>>> we >>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>>>>>>>>>> easily >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incorporate >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> optimization potential that >>>>>> Alexander >>>>>>>>>>> pointed >>>>>>>>>>>>>>> out >>>>>>>>>>>>>>>>>>>>>>>>>>>> later. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Alexander unfortunately, your >>>>>>>>> architecture >>>>>>>>>>> is >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>>>>>>>> shared. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solution to share images to be >>>>>> honest. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Apr 22, 2022 at 5:04 PM >>>>>>>> Александр >>>>>>>>>>>>>>> Смирнов >>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> smirale...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Qingsheng! My name is Alexander, >>>>>>> I'm >>>>>>>>>>> not a >>>>>>>>>>>>>>>>>>>>>>>>>>>> committer >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> yet, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'd >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> really like to become one. And this >>>>>>>> FLIP >>>>>>>>>>>>> really >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> interested >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> me. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Actually I have worked on a similar >>>>>>>>>>> feature in >>>>>>>>>>>>>>> my >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> company’s >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Flink >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fork, and we would like to share >>>>>> our >>>>>>>>>>> thoughts >>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>>>>>>>>>>>>> this and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> open source. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think there is a better >>>>>> alternative >>>>>>>>> than >>>>>>>>>>>>>>>>>>>>>>>>>>>> introducing an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> abstract >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class for TableFunction >>>>>>>>>>>>> (CachingTableFunction). >>>>>>>>>>>>>>>> As >>>>>>>>>>>>>>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TableFunction exists in the >>>>>>>>>>> flink-table-common >>>>>>>>>>>>>>>>>>>>>>>>>>>> module, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provides >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only an API for working with >>>>>> tables – >>>>>>>>> it’s >>>>>>>>>>>>> very >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> convenient >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> importing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in connectors. In turn, >>>>>>>>>>> CachingTableFunction >>>>>>>>>>>>>>>>> contains >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> runtime execution, so this class >>>>>> and >>>>>>>>>>>>>>> everything >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> connected >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be located in another >>>>>> module, >>>>>>>>>>> probably >>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-table-runtime. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But this will require connectors to >>>>>>>>> depend >>>>>>>>>>> on >>>>>>>>>>>>>>>>> another >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> module, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> contains a lot of runtime logic, >>>>>>> which >>>>>>>>>>> doesn’t >>>>>>>>>>>>>>>>> sound >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> good. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I suggest adding a new method >>>>>>>>>>>>> ‘getLookupConfig’ >>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LookupTableSource >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or LookupRuntimeProvider to allow >>>>>>>>>>> connectors >>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> only >>>>>>>>>>>>>>>>>>>>>>>>>>>> pass >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configurations to the planner, >>>>>>>> therefore >>>>>>>>>>> they >>>>>>>>>>>>>>>> won’t >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> depend on >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> runtime >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> realization. Based on these configs >>>>>>>>> planner >>>>>>>>>>>>>>> will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> construct a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lookup >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> join operator with corresponding >>>>>>>> runtime >>>>>>>>>>> logic >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (ProcessFunctions >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> module flink-table-runtime). >>>>>>>> Architecture >>>>>>>>>>>>> looks >>>>>>>>>>>>>>>>> like >>>>>>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pinned >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> image (LookupConfig class there is >>>>>>>>> actually >>>>>>>>>>>>>>> yours >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CacheConfig). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Classes in flink-table-planner, >>>>>> that >>>>>>>> will >>>>>>>>>>> be >>>>>>>>>>>>>>>>>>>>>>>>>>>> responsible >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> – >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CommonPhysicalLookupJoin and his >>>>>>>>>>> inheritors. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Current classes for lookup join in >>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-table-runtime >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LookupJoinRunner, >>>>>>>> AsyncLookupJoinRunner, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LookupJoinRunnerWithCalc, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AsyncLookupJoinRunnerWithCalc. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I suggest adding classes >>>>>>>>>>>>>>> LookupJoinCachingRunner, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LookupJoinCachingRunnerWithCalc, >>>>>> etc. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> And here comes another more >>>>>> powerful >>>>>>>>>>> advantage >>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>> such a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solution. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we have caching logic on a lower >>>>>>> level, >>>>>>>>> we >>>>>>>>>>> can >>>>>>>>>>>>>>>>> apply >>>>>>>>>>>>>>>>>>>>>>>>>>>> some >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> optimizations to it. >>>>>>>>>>> LookupJoinRunnerWithCalc >>>>>>>>>>>>>>> was >>>>>>>>>>>>>>>>>>>>>>>>>>>> named >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because it uses the ‘calc’ >>>>>> function, >>>>>>>>> which >>>>>>>>>>>>>>>> actually >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mostly >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consists >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filters and projections. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For example, in join table A with >>>>>>>> lookup >>>>>>>>>>> table >>>>>>>>>>>>>>> B >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> condition >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ‘JOIN … >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ON >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> A.id = B.id AND A.age = B.age + 10 >>>>>>>> WHERE >>>>>>>>>>>>>>>> B.salary > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1000’ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ‘calc’ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> function will contain filters >>>>>> A.age = >>>>>>>>>>> B.age + >>>>>>>>>>>>>>> 10 >>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> B.salary > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1000. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If we apply this function before >>>>>>>> storing >>>>>>>>>>>>>>> records >>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cache, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> size >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cache will be significantly >>>>>> reduced: >>>>>>>>>>> filters = >>>>>>>>>>>>>>>>> avoid >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> useless >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> records in cache, projections = >>>>>>> reduce >>>>>>>>>>>>> records’ >>>>>>>>>>>>>>>>>>>>>>>>>>>> size. So >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initial >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max number of records in cache can >>>>>> be >>>>>>>>>>>>> increased >>>>>>>>>>>>>>>> by >>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> user. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What do you think about it? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2022/04/19 02:47:11 Qingsheng >>>>>> Ren >>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi devs, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yuan and I would like to start a >>>>>>>>>>> discussion >>>>>>>>>>>>>>>> about >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FLIP-221[1], >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> introduces an abstraction of lookup >>>>>>>> table >>>>>>>>>>>>> cache >>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>> its >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> standard >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metrics. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently each lookup table source >>>>>>>>> should >>>>>>>>>>>>>>>>> implement >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> own >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cache to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> store lookup results, and there >>>>>>> isn’t a >>>>>>>>>>>>>>> standard >>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metrics >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> developers to tuning their jobs >>>>>> with >>>>>>>>> lookup >>>>>>>>>>>>>>>> joins, >>>>>>>>>>>>>>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> quite >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> common >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use case in Flink table / SQL. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore we propose some new APIs >>>>>>>>>>> including >>>>>>>>>>>>>>>>> cache, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metrics, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrapper >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> classes of TableFunction and new >>>>>>> table >>>>>>>>>>>>> options. >>>>>>>>>>>>>>>>>>>>>>>>>>>> Please >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> take a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> look >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FLIP page [1] to get more details. >>>>>>> Any >>>>>>>>>>>>>>>> suggestions >>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> comments >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> appreciated! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-221+Abstraction+for+lookup+source+cache+and+metric >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qingsheng >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qingsheng Ren >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Real-time Computing Team >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Alibaba Cloud >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Email: renqs...@gmail.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Roman Boyko >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e.: ro.v.bo...@gmail.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Qingsheng Ren >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Real-time Computing Team >>>>>>>>>>>>>>>>>>>>>>>>> Alibaba Cloud >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Email: renqs...@gmail.com >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>>> >>>
