Does it make sense to have the value of uploadseparate be any
different than 2? It is already 3844 subfolders.
On Jun 9, 11:15 am, szimszon <szims...@gmail.com> wrote:
> Already done :)
>
> On jún. 9, 15:45, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > please email me this as an attachment.
>
> > On Jun 9, 6:50 am, szimszon <szims...@gmail.com> wrote:
>
> > > Hmm... there are a lot of possibility to separate files into
> > > folders...
>
> > > I typically would have lots of files with the same type...
>
> > > In the mean time I have a path to separate the uploaded files into
> > > subdirectories based on uuid_key's first X character:
>
> > > ---- cut ----
> > > --- sql.py.orig 2010-06-08 23:45:04.000000000 +0200
> > > +++ sql.py 2010-06-09 12:50:46.000000000 +0200
> > > @@ -49,7 +49,7 @@
>
> > > table_field = re.compile('[\w_]+\.[\w_]+')
> > > oracle_fix = re.compile("[^']*('[^']*'[^']*)*\:(?P<clob>CLOB\('([^']
> > > +|'')*'\))")
> > > -regex_content = re.compile('([\w\-]+\.){3}(?P<name>\w+)\.\w+$')
> > > +regex_content = re.compile('([\w\-]+\.){2}(?P<uuidkey>[\w\-]+)\.(?
> > > P<name>\w+)\.\w+$')
> > > regex_cleanup_fn = re.compile('[\'"\s;]+')
>
> > > # list of drivers will be built on the fly
> > > @@ -2600,7 +2600,9 @@
> > > # 'a_field_name' means store in this
> > > field in db
> > > # False means file content will be
> > > discarded.
> > > writable=True, readable=True, update=None,
> > > authorize=None,
> > > - autodelete=False, represent=None, uploadfolder=None)
> > > + autodelete=False, represent=None, uploadfolder=None,
> > > + uploadseparate=None # upload to separate directories by
> > > uuid_keys
> > > + # first X character)
>
> > > to be used as argument of SQLDB.define_table
>
> > > @@ -2636,6 +2638,7 @@
> > > autodelete=False,
> > > represent=None,
> > > uploadfolder=None,
> > > + uploadseparate=None,
> > > compute=None,
> > > ):
>
> > > @@ -2655,6 +2658,7 @@
> > > self.unique = unique
> > > self.uploadfield = uploadfield
> > > self.uploadfolder = uploadfolder
> > > + self.uploadseparate = uploadseparate
> > > self.widget = widget
> > > self.label = label
> > > self.comment = comment
> > > @@ -2695,6 +2699,10 @@
> > > path = self.uploadfolder
> > > else:
> > > path = os.path.join(self._db._folder, '..',
> > > 'uploads')
> > > + if self.uploadseparate:
> > > + path =
> > > os.path.join(path,uuid_key[:self.uploadseparate])
> > > + if not os.path.exists(path):
> > > + os.mkdir(path)
> > > pathfilename = os.path.join(path, newfilename)
> > > dest_file = open(pathfilename, 'wb')
> > > shutil.copyfileobj(file, dest_file)
> > > @@ -2728,6 +2736,9 @@
> > > path = self.uploadfolder
> > > else:
> > > path = os.path.join(self._db._folder, '..',
> > > 'uploads')
> > > + if self.uploadseparate:
> > > + u = m.group('uuidkey')
> > > + path = os.path.join(path,u[:self.uploadseparate])
> > > return (filename, open(os.path.join(path, name), 'rb'))
>
> > > def formatter(self, value):
> > > --- cut ---
>
> > > I introduced the "uploadseparate" parameter which is either None --
> > > default (backward compatible) or the number of characters of
> > > uuid_key...
> > > The directory is created automatically and this works with
> > > uploadfolder too (uploadfolder/uploadseparate)...
>
> > > I don't know the name "uploadseparate" is acceptable... or the
> > > patch :) There could be a better way...
>
> > > On jún. 9, 12:40, Jason Brower <encomp...@gmail.com> wrote:
>
> > > > I wonder if it would be better to sort by type.
> > > > /uploads/table_name/field_name/
> > > > Working with those images/files should be done from a database don't you
> > > > think? When I deal with large amounts of files I use the console.
> > > > BR,
> > > > Jason Brower
>
> > > > On Tue, 2010-06-08 at 23:45 -0700, szimszon wrote:
> > > > > I wasn't able to continue the thread in
> > > > > http://groups.google.com/group/web2py/browse_frm/thread/a81248fec1dce...
>
> > > > > So...
>
> > > > > I imagine that I would have lots of files say some 10 000 or more. :)
> > > > > I think with ext3/ext2... filesystems so many files in one directory
> > > > > is a mess.
>
> > > > > Is there absoute out of question to have upload/download to handle
> > > > > this issue in trunk?
>
> > > > > I think of some kind of directory structure like one directory (say
> > > > > upload/0) has X number of files then the new one (upload/1) is created
> > > > > and the new files are stored in it...
> > > > > ...and download could handle it out of the box.
>
> > > > > Or the generated filenames first or first two character is the
> > > > > directory name under upload/ and the file is stored under that
> > > > > directory... it could be a Field switch which defaults to the old
> > > > > behavior...
