Hi all,
Here is async in-memory cache that I've implemented for one of my projects:
https://gist.github.com/IlyaSkriblovsky/5aba53b661acd49b65efeb4ce41a8b52
It properly handles problem #2 described by Maarten. But it doesn't
bother with eviction because it wasn't needed at the time of writing
(because key space was limited).
Usage example:
@defer.inlineCallbacks
def load_content(url: str):
# does some long-running async task such as loading url content
@defer.inlineCallbacks
def main():
# DeferredCache receives loader function as an argument
# Loader function must return Deferred
cache = DeferredCache(load_content)
# this will actually download content
page1 = yield cache.get('http://example.com')
# this will use cached one (note that this get() also returns
Deferred, but that
# Deferred will be already-succeeded, so `yield` will return
the content immediately)
page2 = yield cache.get('http://example.com')
# illustration of problem #2 described by Maarten
# note there is no yield here, we are running these to get()s
simultaneously
deferred_page1 = cache.get('https://www.nasa.gov/')
deferred_page2 = cache.get('https://www.nasa.gov/')
# actually waiting for results
page1, page2 = yield deferred.gatherResults([deferred_page1,
deferred_page2])
# load_content() will be called only once with
'https://www.nasa.gov/' by this point
-- Ilya
пт, 27 сент. 2019 г. в 06:59, Maarten ter Huurne <maar...@treewalker.org>:
>
> On Friday, 27 September 2019 05:48:35 CEST Waqar Khan wrote:
> > Hi Maarten,
> > I think you have hit the problem in the head. I do think this is
> > feasible as I have observed that as size of cache increases, things do
> > get better which might support your theory.
> >
> > Is there a simple example you can add on "put a Deferred for the fetch
> > operation ". I am really just getting started with twisted.
> > Thanks for all the help.
>
> Unfortunately I don't think I have any code lying around that implements
> this kind of cache.
>
> Bye,
> Maarten
>
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
