[sr-dev] Re: topos module with storage "shm"

2024-10-29 Thread Henning Westerholt via sr-dev
Cc: Stefan Mititelu Subject: [sr-dev] Re: topos module with storage "shm" I am going to look into the following dev idea: 1. from htable module try export a bind() API functions (e.g. similar to usrloc) and check that basic htable module ops can be done from another module 2. next

[sr-dev] Re: topos module with storage "shm"

2024-10-28 Thread Stefan Mititelu via sr-dev
I am going to look into the following dev idea: 1. from htable module try export a bind() API functions (e.g. similar to usrloc) and check that basic htable module ops can be done from another module 2. next, create a topos_htable module that will work with those htable module API functions 3. las

[sr-dev] Re: topos module with storage "shm"

2024-10-22 Thread Henning Westerholt via sr-dev
Hi Stefan, thanks for the reply. DMQ is usually fast, but it can have data loss or data synchronization issues if you are using in concurrent access scenarios. Regarding using the shared memory as a cache, this is of course frequently used in usrloc module or others. It would certainly help for

[sr-dev] Re: topos module with storage "shm"

2024-10-21 Thread Stefan Mititelu via sr-dev
So far I've been thinking of this topos SHM htable stuff as a replacement for existing storages (redis/mysql). Now, I am thinking of this topos SHM htable as a cache between kama procs and existing storages. So this should happen: 1. on store, always keep in shm htable and write in storage db 2. o

[sr-dev] Re: topos module with storage "shm"

2024-10-21 Thread Stefan Mititelu via sr-dev
Hi, Thanks for the feedback for this. My motivation is related to an active-active scenario and supposing that redis db becomes unavailable => topology info will then become unavailable. Now that I am double thinking about DMQ sync, I think you are right. If DMQ does not sync fast enough, there w

[sr-dev] Re: topos module with storage "shm"

2024-10-21 Thread Henning Westerholt via sr-dev
Hi Stefan, certainly, a new storage mode “shared memory” could be added, to avoid any additional dependencies for topos. This sounds useful. I personally don’t see a large case for adding also DMQ synchronisation capabilities for it. The DMQ synchronisation is not 100% reliable in all corner c