Héllo (again),
2014-05-14 22:41 GMT+02:00 Sayth Renshaw <flebber.c...@gmail.com>: > Thanks Tom, I have set up a way to extract the elements I want with > xmltodict and played with lxml.objectify last night and that would work as > well. > > But what exactly in django models am I using to bring in the data? > Django knows how to load data from json, xml, whatnot only if it knows about the format. I don't remember we considered Django native serialization/unserialization support (see my mail on Python ML). The thing is that, in your project, it seems like import is a "one time thing". So maybe using a framework thingy like django-swallow or a custom django's fixture thing is too much. Anyway, here is the "getting started" document of django swallow<https://github.com/liberation/django-swallow/blob/master/doc/source/walkthrough.rst>, summed up below: - Builder class: entry point for an import, this used among other thing to trigger the import from command line. - Mapper class: create "mapping object" for the input xml. Basically it breaks down a xml into sub-documents (if it's rss/atom like xml) or a single object if it's actually only one document, pythonize the xml data and make it easily accessible as object properties... no heavy processing of data or mapping to data already in the database. For instance, if you have an identifier, that maps to a row in the database, you retrieve the populator (see below) will retrieve the identifier as an integer or string and convert it to a Django model... - Populator class: it helps populating the target django model object. It implements common operations, like «model.foo = mapper.foo» and other stuff. The idea behind django swallow, is that an import is breaked down into several steps: - parse xml, and turn in into simple python object(s) with python types (Mapper class) - populate one django model instance (or several) with based on the data from the first step (Populator class) They are extra optional steps ;) I'm not sure about what is the repository, but for instance you can create a base mapper, which will populate a plain python objects from xpath and then the populator can be as simple as: class Populator(BasePopulator): _fields_one_to_one = None # XXX: in this case None means map every properties of the mapper object to the model fields with the same name... _fields_if_instance_already_exists = None _fields_if_instance_modified_from_last_import = None It's may seem kind of overkill. But when you have 5+ imports running everyday, business critical, processing several dozen of files, for different kind of documents it's helpful to know that the implementation is built in a canvas, a framework, you know where to look for the code implementing a behaviour depending on the behaviour... whether it is for debugging or building new imports... There is a full but simple example, importing items from an atom file: https://github.com/liberation/django-swallow/blob/master/example/config.py HTH, Amirouche > Sayth > On 15/05/2014 1:13 am, "Tom Evans" <tevans...@googlemail.com> wrote: > >> On Tue, May 13, 2014 at 8:40 PM, Sayth Renshaw <flebber.c...@gmail.com> >> wrote: >> > Can I ask for some assistance please. >> > >> > For my project I will need to upate the database from an external xml >> file >> > consistently. The xml is like this one >> > http://old.racingnsw.com.au/Site/_content/racebooks/20140515GOSF0.xml >> > If i have create my model objects, most of which will have a direct >> > correlation to the xml file, but there will be extra model objects and >> > fields relating to user input. How can I do updates to my database via >> xml. >> > >> > There are 2 docs I have been reviewing >> > >> https://docs.djangoproject.com/en/1.6/howto/initial-data/#providing-initial-data-with-fixtures >> > and https://docs.djangoproject.com/en/1.6/topics/serialization/. >> > >> > The guides talk about creating a fixture but I am not sure it is >> appropriate >> > exactly and I am unsure from the guide how I would setup a mapping so >> that >> > the xml data went to the correct model object. >> > How do I do this? >> > >> >> I don't think you want fixtures, even though there are XML fixtures. >> This is basically a data feed which periodically you wish to import in >> to your own database. Fixtures are normally exports of your own >> database. I think it would be very difficult to ensure that your >> database always matches the contents of an XML file produced by a 3rd >> party. >> >> You could use xslt to transform the 3rd party xml in to the right >> format for an xml serialisation, but that would involve using xslt. >> >> What I would do is write a management command that parses the XML (I >> like lxml), and then iterates through it, using xpath to pull out >> relevant parts of the doc, and then use standard django models to >> create this data in the database, taking care not to insert duplicate >> data when it exists in both DB and XML. >> >> Cheers >> >> Tom >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Django users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/django-users/TZ2GbmlVk1o/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> django-users+unsubscr...@googlegroups.com. >> >> To post to this group, send email to django-users@googlegroups.com. >> Visit this group at http://groups.google.com/group/django-users. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/CAFHbX1KCPYxxL%3Duj2CQkX1EiC5Y%3D-DsBSfOqm5YykbxXWT%2B_dA%40mail.gmail.com >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-users+unsubscr...@googlegroups.com. > To post to this group, send email to django-users@googlegroups.com. > Visit this group at http://groups.google.com/group/django-users. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/CAN2of00-Lcv6VAeL0Lv%3Dbi7HniBB_YuzitWwy%3DsWiA0g3u0TnQ%40mail.gmail.com<https://groups.google.com/d/msgid/django-users/CAN2of00-Lcv6VAeL0Lv%3Dbi7HniBB_YuzitWwy%3DsWiA0g3u0TnQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAL7_Mo9_TQttwaMHzhhtZfiT0GV12HDzzniGMrQVVJhLy9XSLw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
