Hello, everyone. I'm trying to develop a web app to rid colleagues of mine of the "SAP -> Wordpad -> Excel -> Access -> Excel" workflow by offering them django instead.
I'd like to allow them to upload csv-like text files to an app in our intranet that would load the data onto a temporary DB table, apply *limited* custom SQL logic (such as filter, group by, aggregate) and save the results to a permanent database. The problem with our current workflow is that these files are really big (1.5gb) and they take quite a while to move around from excel to access to whatnot and keeping that much data in spreadsheets is error prone at best, and just plain bad IMHO. I'd like to offer users the possibility of browsing their data, filtering, etc. without having to open excel at all. Ideally, I users would be able to specify "import templates" (which would most likely be a Model itself), so that they can repeat processes every month for similar data. I wrote a model that would take some options such as "ignore bad lines", "delimiter", "first column as headers", "skip the first N lines in the uploaded file", but I need to allow users to specify column data types and choose which columns they'd like to have uploaded. So design question #1 is: How should I let users choose the column types in "ImportTemplate"? Should I use a FK to a "Field" model? Are there caveats I should know of? I'd rather not reinvent the wheel with this. Finally, I can't really wrap my head around how to construct the whole process. The biggest problem is: How can I create a temporary table based on the column types specified in ImportTemplate and run a query to generate the data for the permanent table? I've looked at some dynamic model tips on djangoproject.com[1] and blogs[2], but I haven't gotten far at all. Assuming I can create a dynamic table, I can easily populate it (I've written a wxPython that does exactly that, but used SQLAlchemy instead) and save the results of a query to the permanent table. I'd really appreciate any help you can give me at all. Thank you in advance Sincerely, André Terra (airstrike) [1] https://code.djangoproject.com/wiki/DynamicModels [2] http://family.ubuntu-fl.org/mhall/fun-with-django-meta-classes-and-dynamic-models/#comment-733 -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
