Harrison, That's useful information. Yes, I'll only be doing lookups, which simplifies things quite a bit. Given what you said about the Movable Type software, I assume DB_FILE would be a good way to keep track of website user names, passwords, cookies, and the like?
- Chris -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Harrison Dekker Sent: Friday, June 20, 2008 2:19 PM To: Christopher Morgan Subject: Re: Practicality of using DB_File on a Perl-based book site? Chris, I'm no expert, but it seems to me, that there should be less overhead using Berkeley DB compared to a relational DB, assuming that all you're doing is lookups. If you've got a bunch of post processing going on involving multiple large retrieval sets then you'll probably lose that edge, but that's only because your perl code would be doing the work that a more optimized SQL engine could be doing. SQL doesn't give you any improvement, however, when all you're doing is a key/value type lookup. Movable Type blog software uses BDB, at least it did in the past, and as far as I know it's quite reliable/scalable. I use BDB for one web servicey type application and I do have to throttle my requests if I'm sending them in batch, but the db isn't the bottleneck, it's apache or the php xml functions I use. -Harrison On Fri, Jun 20, 2008 at 10:42 AM, Christopher Morgan <[EMAIL PROTECTED]> wrote: > > I'm designing a web site that will display MARC authority files > onscreen. I use a Perl hash that's tied to a (read-only) Berkeley > DB_file, and it works nicely. How practical is this approach if > there's going to be moderate traffic on a site? > > My DB_FILE is about 200MB, but of course Perl brings only small pieces > of the database into memory at any one time. Would the site bog down > if people were accessing records at the rate of, say, every few > seconds? Should I consider mySQL instead? I'd prefer to stick to > DB_FILE, since it's so easy and elegant -- and I can easily create complex data structures. > > What if one of my data files was significantly bigger (say, a GB or > two of MARC book records)? I don't have a feel for the pros and cons > of the various approaches to accessing large databases using Perl, but > tied hashes are pretty fast! In any case, I know I'll have to lock the > file during each read, via "flock" or the like. I haven't tried implementing the latter yet. > > Does anyone have any ideas about this? Are there other Perl forums I > should investigate regarding this topic? > > Many thanks! > > - Chris Morgan > > -- Harrison Dekker -- Coordinator of Data Services -- UC Berkeley Libraries 510-642-8095 :: GTalk:vagrantscholar :: AIM:hdekker :: Meebo:ucbdekker http://sunsite.berkeley.edu/wikis/datalab/ ------------------------ Q: Why is this email 5 sentences or less? A: http://five.sentenc.es
