To avoid doing something like that, and for a web scrapping tool I wrote, I implemented a basic subclass of ZnClient (called ZnCachingClient) that used a disk cache where each key/file was the hash of the requested URL.
I haven't used in a while but it should continue to work, in any case, I published it in a Github repository: https://github.com/eMaringolo/zinc-caching-client Best regards, Esteban A. Maringolo On Wed, Jan 6, 2021 at 3:37 PM Richard Sargent <richard.sarg...@gemtalksystems.com> wrote: > > > > On Wed, Jan 6, 2021 at 10:34 AM Roelof Wobben via Pharo-users > <pharo-users@lists.pharo.org> wrote: >> >> >> Thanks, >> >> Right now im downloading/fetching the images every time again. >> >> As I see it, the biggest bottleneck is that I have 10 images. >> And for all 10 I fetching the image and the data I could display when a >> user wants it. >> So that will be 20 calls to the api. >> >> So maybe some cache could be handy. >> Any hints how to make a cache in smalltalk. > > > Dictionary coupled with #at:ifAbsentPut: > >> >> Roelof >> >> >> >> Op 6-1-2021 om 19:19 schreef Sven Van Caekenberghe: >> > Roelof, >> > >> > Working with multiple high resolution images, as I believe you are >> > doing, is always going to be a real challenge, performance wise. It >> > just takes time to transfer lots of data. >> > >> > First you have to make sure that you are not doing too much work >> > (double downloads, using too high resolutions for previews or >> > browsing). Also, make sure your ultimate client (the browser) can >> > cache as well if applicable (set modification dates on the response). >> > >> > Next you could cache images locally (on your app server) so that next >> > time you need the same image, you do not need to download it again. Of >> > course, this only helps if your hit rate is higher than zero (if you >> > actually ask for the same image multiple times). >> > >> > It is also possible to do multiple download requests concurrently: if >> > the other end is fast enough, that can certainly help. >> > >> > HTH, >> > >> > Sven >> > >> >> On 6 Jan 2021, at 18:11, Roelof Wobben via Pharo-users >> >> <pharo-users@lists.pharo.org> wrote: >> >> >> >> >> >> I did it on the root document and see this : >> >> >> >> <nkdkdknfekaflcfc.png> >> >> >> >> So as far as I see it , The most time it taken by getting all the >> >> data from all the 10 images. >> >> >> >> I hope someone can look at me if im on the right track and will help >> >> me to figure out faster ways to achieve the same >> >> >> >> Roelof >> >> >> >> >> >> >> >> Op 5-1-2021 om 05:16 schreef Richard O'Keefe: >> >>> Before you take another step, explore the root document. >> >>> >> >>> Profiling is easy. >> >>> Open a Playground. >> >>> Type an expression such as >> >>> 3 tinyBenchmarks >> >>> Right click and select 'Profile it'. >> >>> >> >>> More generally, in a browser, look at the "Tool - Profilers" >> >>> class category. The classic approach was >> >>> MessageTally spyOn: [3 tinyBenchmarks] >> >>> If I understand correctly, 'Profile it' uses TimeProfiler, >> >>> which has a nicer interface. (This is in Pharo 8.) >> >>> >> >>> >> >>> On Sun, 3 Jan 2021 at 23:03, Roelof Wobben <r.wob...@home.nl> wrote: >> >>> I want that the code fetches a url and some data from the >> >>> Rijksmuseaum api. >> >>> And as far as I see it the second it not pointless because it >> >>> getting more detailed info about the painting as in the first get. >> >>> >> >>> I did not profiled it because I never learned how to do that in Pharo. >> >>> >> >>> Roelof >> >>> >> >>> >> >>> >> >>> Op 3-1-2021 om 01:09 schreef Richard O'Keefe: >> >>>> What do you want the code to do? >> >>>> Have you profiled the code to see where the time is going? >> >>>> >> >>>> A quick look at the code shows >> >>>> - Paintings does one web get >> >>>> - each Painting does two more web gets >> >>>> ! and the first of those seems to be pretty pointless, >> >>>> as it refetches an object that Paintings already fetched >> >>>> and just looked at. >> >>>> >> >>>> >> >>>> >> >>>> On Sun, 3 Jan 2021 at 01:16, Roelof Wobben via Pharo-users >> >>>> <pharo-users@lists.pharo.org> wrote: >> >>>> Hello, >> >>>> >> >>>> I have now this code : https://github.com/RoelofWobben/Rijksmuseam >> >>>> >> >>>> but it seems to be slow. >> >>>> >> >>>> Can anyone help me with a way I can use a sort of cache so the page >> >>>> looks first at the cache if a image is there . >> >>>> If so, take the image from there , if not , ask the api for the url of >> >>>> the image. >> >>>> >> >>>> Roelof
