Thanks... On Saturday, October 6, 2012 12:02:58 AM UTC+3, Niphlod wrote: > > depends on the underlying filesystem. Just try it out before hitting your > platform's limits. > Someone's got to help me with math. Suppose that for ~10 millions of files > in a single table, uuid are "evenly distributed". > Folders are "2 letter prefixes" of an uuid, that is composed of > hexadecimal characters. > This means 16^2 folders in total, 256. > 10 millions images evenly distributed in such 256 folders make ~39062 > images per single folder. > FAT32 holds 65634 files per folder, ok. > NTFS 4 millions, ok. > XFS, etx3, etx4, jfs, etc are ok (depends on available inodes, but for > normal disk-sizes noaday it's a no-issue) > > Depending on the filesystem and on the number of actual files, you may > encounter some delays with high number of files in a single folder because > every filesystem implements it's own indexing system (not of the file > contents, just "pointers to where the file is"). > > BTW: lately there have been much improvements on supporting in web2py > whatever you need to use as a "filesystem", even if it's virtual. > pyfilesystem is supported and you can also write your own function to > store/retrieve the file (e.g. if you want to store it in a similar way, but > with a "4 character prefix" (10 millions files would end up being in > folders of 152 files each), or for using GridFS, MogileFS, etc). > > There is no docs ready yet on it, but see dal source (from line 8360), > it's quite straightforward. > > > On Friday, October 5, 2012 9:02:26 PM UTC+2, apps in tables wrote: >> >> Thank you for the immediate reply... >> >> I did it manually. >> is uploadseparate=True the best configuration for having millions of >> images with reasonable performance? >> >> On Friday, October 5, 2012 9:56:37 PM UTC+3, Niphlod wrote: >>> >>> it's stated very clearly on the book that it's not possible to "migrate" >>> between uploadseparate or not. Web2py should have to move within one >>> request millions of files. >>> """ >>> ATTENTION: You cannot change the value of uploadseparate from True to >>> False without breaking the system. web2py either uses the separate >>> subfolders or it does not. Changing the behavior after files have been >>> uploaded will prevent web2py from being able to retrieve those files. If >>> this happens it is possible to move files and fix the problem but this is >>> not described here. >>> """ >>> you can do that by hand if you need to, just move every file in a >>> subfolder that is composed as the first 2 letters of the file itself. >>> >>> On Friday, October 5, 2012 8:44:06 PM UTC+2, apps in tables wrote: >>>> >>>> Hi >>>> >>>> Web2py is great... >>>> >>>> I always get a smooth migration after changing the structure of the >>>> database. >>>> >>>> unfortunately, when i changed 'upload' data to uploadseparate = True , >>>> I lost them. >>>> >>>> is this needs to be included in the auto migration? >>>> >>>> regards, >>>> >>>> Ashraf >>>> >>>
--
