Re: Global question

On 5/19/07, Will Fould <[EMAIL PROTECTED]> wrote: I'm afraid that: 1. hashes get really big (greater than a few MB's each) 2. re-caching entire hash just b/c 1 key updated (waste). 3. latency for pulling cache data from remote DB. 4. doing this for all children. The most common way

Re: Global question

my .02¢ • ldap would be silly unless you're clustering -- most implementations use bdb as their backend • bdb and cache::fastmmap would make more sense if you're on 1 machine also i think your hash system may be better off rethought you have: $CACHE_1{id}='foo'

Re: Global question

Thanks a lot Perrin - I really like the current method (if it were to stay on 1 machine and not grow). Caching per child has not really been a problem once I got beyond the emotional hangup of what seemed to be duplicative, waste of memory. I am totally amazed how fast and efficient using modper

Re: Global question

On 5/19/07, Will Fould <[EMAIL PROTECTED]> wrote: Here's the situation: We have a fully normalized relational database (mysql) now being accessed by a web application and to save a lot of complex joins each time we grab rows from the database, I currently load and cache a few simple hashes (1-10

Re: Global question

Maybe I should restate this question -- I'm wondering if BerkleyDB, LDAP, or something like IPC::MM will help me with this but I have little experience with these, in heavy practice. Here's the situation: We have a fully normalized relational database (mysql) now being accessed by a web applicat

Re: Global question

On 5/7/07, Carl Johnstone <[EMAIL PROTECTED]> wrote: You can use shared memory between apache processes. Check: *Apache::SharedMem* *Tie::ShareLite*

Re: Global question

You can use shared memory between apache processes. Check: *Apache::SharedMem* *Tie::ShareLite* *Cache::SharedMemoryCache*

Re: Global question

On 5/7/07, Will Fould <[EMAIL PROTECTED]> wrote: C/Would anyone recommend any of the IPC::*** shared memory packages for what I'm doing? No, they have terrible performance for any significant amount of data. Much worse than a simple shared file approach. If you can break up your data into a ha

Re: Global question

simpler by checking last mod time on a shared file Good idea. C/Would anyone recommend any of the IPC::*** shared memory packages for what I'm doing? On 5/7/07, Perrin Harkins <[EMAIL PROTECTED]> wrote: On 5/7/07, Will Fould <[EMAIL PROTECTED]> wrote: > Can apache processes meaningfully acce

Re: Global question

On 5/7/07, Jonathan Vanasco <[EMAIL PROTECTED]> wrote: I thought the same as you , until that lighttpd posting ( March 2006 ).But according to Jan , who was at MySQL at the time , MySQL only had a synchronous API. An async was on a todo list, but selects were blocking on the whole server.

Re: Global question

On 5/7/07, Will Fould <[EMAIL PROTECTED]> wrote: Can apache processes meaningfully access any external ( i.e. shell, other) structures? It's the same as any other process, i.e. the usual IPC methods are available. If you want to update the whole data structure at once, I think what you're doin

Re: Global question

On May 7, 2007, at 2:01 PM, Perrin Harkins wrote: It does when you shut down the BDB "environment", but there's no reason to do that unless your processes are exiting. ah, that makes sense. so long as one process has bdb running, its there's a shared bdb memory section. Blocking? You me

Re: Global question

Thanks guys. (I'm sure Perrin is tired of answering these same old question in all of it's forms.) The lists are functionally similar to Unix security lists (group id=name, etc). With thousands of users, these key lists are getting larger and the time to re-build them will continue to grow but

Re: Global question

On 5/7/07, Jonathan Vanasco <[EMAIL PROTECTED]> wrote: that's interesting. since its not persistant, i figured it just released everything once you closed the file. It does when you shut down the BDB "environment", but there's no reason to do that unless your processes are exiting. if you've

Re: Global question

On May 7, 2007, at 1:12 PM, Perrin Harkins wrote: I didn't know that BDB does shared memory caching. And no socket overhead too. All the calls are in-process. that's interesting. since its not persistant, i figured it just released everything once you closed the file. > Primary key lo

Re: Global question

On 5/7/07, Jonathan Vanasco <[EMAIL PROTECTED]> wrote: Ah, I reread the post. I saw "large lists" and thought "complex data structure", not simple text. I think we were talking about different things, actually. For reading and writing a large and complex data structure in its entirety, a Stor

Re: Global question

On May 7, 2007, at 11:59 AM, Perrin Harkins wrote: Storable is fast, but not using it is considerably faster. There's no need to use it for storing simple strings. BerkeleyDB does shared memory caching, so commonly accessed data doesn't need to go to disk. Ah, I reread the post. I saw "lar

Re: Global question

On 5/7/07, Jonathan Vanasco <[EMAIL PROTECTED]> wrote: IIRC, Wouldn't the fastest method possible be using Storable and a simple local file on the server ? Building a perl var from a storable has almost no overhead. Storable is fast, but not using it is considerably faster. There's no

Re: Global question

On May 7, 2007, at 11:26 AM, Perrin Harkins wrote: Of course, the problem with using a database to get the lists (besides the lists being the result of a munge), is that they are rather large. Ideally you would load only the part you need, rather than the whole thing. A local shared stora

Re: Global question

On 5/5/07, Will Fould <[EMAIL PROTECTED]> wrote: But, I'd like to do something similar; have a separate process that can alter parent data receive signals and re-cache accordingly. There's no way to alter data in the parent process without restarting the server. Of course, the problem with us

Re: Global question

On 5/7/07, Adam Prime x443 <[EMAIL PROTECTED]> wrote: If they change really rarely couldln't you just have the children automatically die off when the stuff needs to change and reload it? You'd have to create the datastructure using a ChildInit handler i assume, but couldn't a setup like that pot

RE: Global question

riginal Message- From: Will Fould [mailto:[EMAIL PROTECTED] Sent: Saturday, May 05, 2007 11:44 AM To: Jonathan Vanasco Cc: modperl Subject: Re: Global question Yes. I currently use a semaphore scheme to cache large lists within child processes that rarely change. It works quite well. If the semaph

Re: Global question

Yes. I currently use a semaphore scheme to cache large lists within child processes that rarely change. It works quite well. If the semaphore is set, the child knows to re-cache; children set the semaphore when they do something that would require other children to re-cache. But, I'd like to do

Re: Global question

On 5/4/07, Will Fould <[EMAIL PROTECTED]> wrote: Can lists and other global objects created at apache startup be altered as an *indirect* result of child processes (i.e. some type of semaphore/listener scheme?). Are you asking if changing a perl data structure in one process will affect it in a

Global question

Can lists and other global objects created at apache startup be altered as an *indirect* result of child processes (i.e. some type of semaphore/listener scheme?). Thanks