On Jul 21, 3:18 pm, Wayne Koorts <wkoo...@gmail.com> wrote: > Hi Steve, > > > We have an application where we periodically import data from an > > external vendor, and the process is mostly automated, but we need to > > review the data before having it go live. > > > We were considering an option where we would run processes on another > > database, do the manual validation, and then replicate the DB changes > > to our production database (which would have the same schema). > > I do exactly this with one of my sites. This is my scenario: > > 1. Main database table gets updated automatically every night (this > is a kind of data that can be automatically refreshed, but that should > be beside the point for this discussion). The data first goes into a > pending table on my dev server, which is exactly the same as the main > table, with "_pending" appended to the name. > 2. The data is then dumped to a file using "psql" and sent via FTP up > to an equivalent pending table on the production server (via Python > script in my dev environment). There is a Python script set up on the > server to monitor a specific folder for a specific dump file name at > regular intervals, and if it finds it, imports it into the table also > with "psql", then deletes the dump file. > 3. The data sits in the pending table on the production server until > I approve it via an admin interface which I have set up Django. > Basically there is a "DB Admin" area on the site which I have set up > which shows me four chunks of the data at various points in the table, > 100 at the start, 100 at 1/4 through, 100 at 3/4 through and then 100 > at the end. It also shows me the total number of records so that I > can do a quick visual integrity check. > 4. There are "approve" and "reject" buttons at the bottom of the > approval page. If I click "approve" it moves the current live table > data into a backup table (the db admin area also has a restore > function) and moves the pending data into the live table using a raw > nested SQL SELECT statement via Django and then empties the pending > table. If I click "reject" it just empties the pending table. > > These same general steps can be adjusted to accomodate more tables > etc. as per your requirements. >
Wayne, Thanks for your response. The way that you are doing it was definitely an option we were considering, but we have just enough tables that I think scaling up the "pending_"<table> methodology that you describe below would not really be practical. The overall process will be very similar, though. We'll do the import on some sort of staging database and have an application with approve/ reject buttons to validate the new data is quality for production. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
