Good question :)
Originally I was about to use only the first letter of uuid_key :) and
a parameter true or false :)
It's the gnome way to doing things. Keep it simple to the
endusers... :-D
But after that I considered to use the first two characters than came
the idea to have an integer parameter to specify the length of
characters to use.
It's totally sufficient to me to have only two letters. It's up to
you :-o
I'm thinking about that the use of only that the first or the first
two letter could be useful... and in the perspective of web2py you
could hardcode the two...
Has somebody an opinion about it?
On jún. 9, 21:07, mdipierro <mdipie...@cs.depaul.edu> wrote:
> Does it make sense to have the value of uploadseparate be any
> different than 2? It is already 3844 subfolders.
>
> On Jun 9, 11:15 am, szimszon <szims...@gmail.com> wrote:
>
> > Already done :)
>
> > On jún. 9, 15:45, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > please email me this as an attachment.
>
> > > On Jun 9, 6:50 am, szimszon <szims...@gmail.com> wrote:
>
> > > > Hmm... there are a lot of possibility to separate files into
> > > > folders...
>
> > > > I typically would have lots of files with the same type...
>
> > > > In the mean time I have a path to separate the uploaded files into
> > > > subdirectories based on uuid_key's first X character:
>
> > > > ---- cut ----
> > > > --- sql.py.orig 2010-06-08 23:45:04.000000000 +0200
> > > > +++ sql.py 2010-06-09 12:50:46.000000000 +0200
> > > > @@ -49,7 +49,7 @@
>
> > > > table_field = re.compile('[\w_]+\.[\w_]+')
> > > > oracle_fix = re.compile("[^']*('[^']*'[^']*)*\:(?P<clob>CLOB\('([^']
> > > > +|'')*'\))")
> > > > -regex_content = re.compile('([\w\-]+\.){3}(?P<name>\w+)\.\w+$')
> > > > +regex_content = re.compile('([\w\-]+\.){2}(?P<uuidkey>[\w\-]+)\.(?
> > > > P<name>\w+)\.\w+$')
> > > > regex_cleanup_fn = re.compile('[\'"\s;]+')
>
> > > > # list of drivers will be built on the fly
> > > > @@ -2600,7 +2600,9 @@
> > > > # 'a_field_name' means store in this
> > > > field in db
> > > > # False means file content will be
> > > > discarded.
> > > > writable=True, readable=True, update=None,
> > > > authorize=None,
> > > > - autodelete=False, represent=None, uploadfolder=None)
> > > > + autodelete=False, represent=None, uploadfolder=None,
> > > > + uploadseparate=None # upload to separate directories by
> > > > uuid_keys
> > > > + # first X character)
>
> > > > to be used as argument of SQLDB.define_table
>
> > > > @@ -2636,6 +2638,7 @@
> > > > autodelete=False,
> > > > represent=None,
> > > > uploadfolder=None,
> > > > + uploadseparate=None,
> > > > compute=None,
> > > > ):
>
> > > > @@ -2655,6 +2658,7 @@
> > > > self.unique = unique
> > > > self.uploadfield = uploadfield
> > > > self.uploadfolder = uploadfolder
> > > > + self.uploadseparate = uploadseparate
> > > > self.widget = widget
> > > > self.label = label
> > > > self.comment = comment
> > > > @@ -2695,6 +2699,10 @@
> > > > path = self.uploadfolder
> > > > else:
> > > > path = os.path.join(self._db._folder, '..',
> > > > 'uploads')
> > > > + if self.uploadseparate:
> > > > + path =
> > > > os.path.join(path,uuid_key[:self.uploadseparate])
> > > > + if not os.path.exists(path):
> > > > + os.mkdir(path)
> > > > pathfilename = os.path.join(path, newfilename)
> > > > dest_file = open(pathfilename, 'wb')
> > > > shutil.copyfileobj(file, dest_file)
> > > > @@ -2728,6 +2736,9 @@
> > > > path = self.uploadfolder
> > > > else:
> > > > path = os.path.join(self._db._folder, '..',
> > > > 'uploads')
> > > > + if self.uploadseparate:
> > > > + u = m.group('uuidkey')
> > > > + path = os.path.join(path,u[:self.uploadseparate])
> > > > return (filename, open(os.path.join(path, name), 'rb'))
>
> > > > def formatter(self, value):
> > > > --- cut ---
>
> > > > I introduced the "uploadseparate" parameter which is either None --
> > > > default (backward compatible) or the number of characters of
> > > > uuid_key...
> > > > The directory is created automatically and this works with
> > > > uploadfolder too (uploadfolder/uploadseparate)...
>
> > > > I don't know the name "uploadseparate" is acceptable... or the
> > > > patch :) There could be a better way...
>
> > > > On jún. 9, 12:40, Jason Brower <encomp...@gmail.com> wrote:
>
> > > > > I wonder if it would be better to sort by type.
> > > > > /uploads/table_name/field_name/
> > > > > Working with those images/files should be done from a database don't
> > > > > you
> > > > > think? When I deal with large amounts of files I use the console.
> > > > > BR,
> > > > > Jason Brower
>
> > > > > On Tue, 2010-06-08 at 23:45 -0700, szimszon wrote:
> > > > > > I wasn't able to continue the thread in
> > > > > > http://groups.google.com/group/web2py/browse_frm/thread/a81248fec1dce...
>
> > > > > > So...
>
> > > > > > I imagine that I would have lots of files say some 10 000 or more.
> > > > > > :)
> > > > > > I think with ext3/ext2... filesystems so many files in one directory
> > > > > > is a mess.
>
> > > > > > Is there absoute out of question to have upload/download to handle
> > > > > > this issue in trunk?
>
> > > > > > I think of some kind of directory structure like one directory (say
> > > > > > upload/0) has X number of files then the new one (upload/1) is
> > > > > > created
> > > > > > and the new files are stored in it...
> > > > > > ...and download could handle it out of the box.
>
> > > > > > Or the generated filenames first or first two character is the
> > > > > > directory name under upload/ and the file is stored under that
> > > > > > directory... it could be a Field switch which defaults to the old
> > > > > > behavior...
>
>
