On Fri, Feb 19, 2010 at 11:23 AM, Michael Lissner < mliss...@michaeljaylissner.com> wrote:
> Thanks for the reply. > > My teammate that writing the scrapers is a java guy, and he's planning to > use that rather than django/python. I haven't looked at the kinds of > databases that django creates, but do you think that will cause any > problems? > > If not, I think we'll proceed with plan B, below. > I don't think it should be a problem, as long as you guys can come to a consensus about how your tables will be defined and named (I made the table naming mistake early on - django expects certain table names names for models you have defined in your models.py files - of-course you can override that by setting db_table in the Meta class for said model) . Also, if he plans on serializing any data, make sure he does so in something like JSON or XML so that you can have access to it via Python. Also, have you taken a look at the inspectdb management command - it might go a long way in helping you get your models.py files setup - see http://docs.djangoproject.com/en/dev/ref/django-admin/#inspectdb > > Michael lissnermliss...@michaeljaylissner.com > 909-576-4123 > > > Rishabh Manocha wrote on 02/18/2010 07:16 PM: > > > > On Fri, Feb 19, 2010 at 10:04 AM, mjlissner <mjliss...@gmail.com> wrote: > >> I'm using django for a final project in my masters program at UC >> Berkeley, and I'm trying to sort out exactly how the database works. I >> would spend a bunch of time figuring this out myself, but I am working >> in a team, and my teammates need this info asap. >> >> My teammates are making scrapers that pull information from >> various .gov websites, and then dump that information into a mysql >> database. >> >> It's my job to use django to make queries on that database, and to >> build a web front end to it. >> >> We've developed a data model for the database, but my question is >> what's the best way to interface between the data they're dumping in, >> and django? I have two thoughts about this...both of which are >> probably wrong: >> 1. They can build a separate database, and I can pull from it using >> custom sql queries within my django code; or >> 2. I can use django models to flesh out the database, and then they >> can dump into the tables that were created by the syncdb command. >> >> Can people opine as to what the best way to do this might be, or what >> kinds of problems we can anticipate if we do the above? >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To post to this group, send email to django-us...@googlegroups.com. >> To unsubscribe from this group, send email to >> django-users+unsubscr...@googlegroups.com<django-users%2bunsubscr...@googlegroups.com> >> . >> For more options, visit this group at >> http://groups.google.com/group/django-users?hl=en. >> >> > I've done something similar with my project. I setup up my database tabels > such that they work with both my scraping code as well as my django code. To > insert data from my scraping code, I used sqlalchemy (though, you can just > as easily use django's ORM for this - this method would probably be better > in the longer run 'cause you can define your models in one place and just > include them in both your scraping code as well as your django site) and > then just accessed that data from my django site accessing the same database > (no replication, parsing etc.). This method has worked out quiet well for > me, so far. > > -- > > Best, > > R > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-us...@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-users?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-us...@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com<django-users%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/django-users?hl=en. > -- Best, R -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
