Totally agree with everything brien says below.
I also run a web site with a large number of media objects. I've been
involved in filing system and media projects for many years both in
research and production capacity.
The slight increase in complexity of the solution to maintain the media
objects outside the database is completely offset by the huge
improvement in performance of the database.
Also it is particularly worth stressing the backup advantages. If you
have 10,000 users (not a large number these days) each with 5 images
and/or one podcast you are going to have very, very large monolithic
database backups which are either going to be in a binary form and thus
very opaque or simply huge as sql dumps unless you separate the binary data.
Another advantage is that it allows you to have a very scalable media
server. The media server does not have to participate in the session.
You can have a number of independent media servers who simply download
files identified by an ID. The media server(s) don't have to be on the
same box, platform or even in the same building. The issues of
clustering simply go away (a media download is a single operation and
thus does not benefit from clustering.) If you are running a large
number of concurrent downloads, you only need a trivial load balancing
front end server to direct requests at the media servers. If you require
authentication to restrict access to the media then you may have a bit
more work to do but it is not too difficult.
So if you have a site with a lot of media, you can tune the application
component to be small and responsive and the media delivery components
to be simple and fast and optimizied for continuous download.
brien colwell wrote:
On the topic of DB versus filesystem for media, I prefer storing media
in a filesystem and meta data in a DB. The advantages of storing large
binary files outside of the DB are
* Reduce contention in the DB -- it's doing so much
* You have more control with a filesystem where your data goes -- e.g.
you can have a disk for one type of data, and another disk for a
different type
* Filesystems are optimized to stream out data ... which I think makes sense
That first point is really key. If you're serving data to a UI from
the same database you're storing large files into, the whole
experience is going to suffer. The best article I've read on this
topic is
http://mysqldatabaseadministration.blogspot.com/2008/01/i-will-not-blob.html
The main scale issue I'm careful of is how many files I put per
directory. For large amounts of files, I sometimes link directories
into trees. I might be a little paranoid about this, but it seems to
scale well.
In terms of backup and security, I think a FS is easier to manage --
e.g. chmod and rsync on a cron job -- though you'll have to write more
code and scripts. You also have to be careful of things like shell
injection if you're running shell commands from your webserver.
Anyway, hope that gives some help. I was also confused with this when
I started writing DB backed apps.
On Feb 9, 2008 4:58 PM, <[EMAIL PROTECTED]> wrote:
Thanks for the suggestions!
I like using the database for storage for it's easy maintenance and security,
but marshalling a lot of binary data (like a large image library) adds a bit of
overhead to the application.
I'll look into pursuing a storage directory external to the webapp.
-------------- Original message ----------------------
From: "Johnny Kewl" <[EMAIL PROTECTED]>
---------------------------------------------------------------------------
HARBOR: http://coolharbor.100free.com/index.htm
The most powerful application server on earth.
The only real POJO Application Server.
Making the Java dream come true.
---------------------------------------------------------------------------
----- Original Message -----
From: <[EMAIL PROTECTED]>
To: <users@tomcat.apache.org>
Sent: Friday, February 08, 2008 11:13 PM
Subject: managing user uploads best practices
Yes... outside.
Its been a long long time now, vaguely remember struggling with Apache
uploader then eventually getting all to work...
Anyway... what I did is store the files in an Apache httpD folder, so I
could spy on the uploads, and they available for viewing again.
And whats cool is because Apache is also the load balancer in my case... can
have lots of TC's doing their thing.
I was making a kind of wiki thing for an estate agency... thats how I did
it.... way back when...
What's the current wisdom on managing user uploaded files to a web app
that's
deployed via a WAR?
In other words, when the WAR is updated, the directory containing uploaded
files
would be wiped out.
Do people save uploaded files outside of the web app root directory?
Security
issues with this?
Do people not use auto-expanding WAR files and manage the deployment by
hand?
Do you not include the directory for uploaded files in the WAR (but create
it at
runtime) and then trust that the expanded WAR won't overwrite it on
deployment?
Any pointers greatly appreciated!
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
!DSPAM:47aed67d224182136417547!
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
