Hi,

That's why I suggested ArangoDB since it's really transactional like normal relational databases. It's harder to get it out of sync than for example MongoDB.

Also surprisingly ArangoDB query language is really close to SQL.

You should give it a spin. That was the reason we picked ArangoDB for out projects.

On 6.2.2018 15.16, Marc wrote:
This is an interesting article against document databases, I am sure though that there are equally valid arguments for
http://www.sarahmei.com/blog/2013/11/11/why-you-should-never-use-mongodb/

On Tuesday, February 6, 2018 at 1:14:41 PM UTC, Marc wrote:

    Thank you Jani, for taking the time to respond. We are currently
    at the design stage so are considering all options. We have used
    document databases (MongoDB) for other projects and in those cases
    it turned out to not be good fit. I agree that at first glance a
    document database does feels like a good fit, but I want to
    explore all options. The organisation is asking for quite
    ambitious requirements (moon on a stick), they are going to have
    to make compromises I am sure. A good compromise does seem to be
    the JOSN column data type in both PostgreSQL and MySQL, I believe
    that both Ruby on Rails and Django have support

    On Tuesday, February 6, 2018 at 12:14:59 PM UTC, Jani Tiainen wrote:

        Hi,

        Since you're uploading more like documents than similary
        structured data, why not to use more proper tool, like
        document database (ArangoDB would get my vote) instead of
        trying to do what Django is not designed for and doesn't
        support well with concepts like "dynamic models".

        We do similar thing with certain uploadable reports and it
        works pretty fine.

        On 6.2.2018 13.35, Marc wrote:

        Hello

        Currently working on a project to allow researchers to upload
        data files into a repository. The system will only handle the
        import, validation and transformation of the data, so that it
        meets the required standards. When a file is upload the for
        the first time the user has to be able to define the
        structure of the destination based on the csv/xls. This
        includes key field, data types, foreign keys etc. There is no
        requirement for the data to be displayed or edited once in
        the repository by this system, currently. We are currently
        planning on using PostgreSQL as the database.

        Our organisation already has another system, with a similar
        requirement, that is partially written by an external
        contractor in Ruby on Rails. The dynamic creation of the
        tables is no problem but it is the dynamic creation of the
        models and controllers that is proving difficult and is the
        reason why this other project is not finished.

        Reading through the forums it looks like Django may offer a
        better solution through meta-programming.
        https://code.djangoproject.com/wiki/DynamicModels
        <https://code.djangoproject.com/wiki/DynamicModels> This does
        look like quite an old article though.

        The other solution would be to have one table that holds all
        the records for all the imported files and have a JSONB (in
        PostgreSQL) to hold the columns that differ between files.
        This means that we would not be able to represent the
        relationships between files though. Ideally the organisation
        would like the data in a relational form to make subsequent
        analysis with other tools easier.

        Above all it is important that the system is robust and
        maintainable. Any solution will of course be tested by
        creating a proof of concept first.

        Is it still possible to dynamically create the models in the
        latest versions of Django (1.11 and 2.02)?

        Thank you

        Marc

-- You received this message because you are subscribed to the
        Google Groups "Django users" group.
        To unsubscribe from this group and stop receiving emails from
        it, send an email to django-users...@googlegroups.com.
        To post to this group, send email to django...@googlegroups.com.
        Visit this group at
        https://groups.google.com/group/django-users
        <https://groups.google.com/group/django-users>.
        To view this discussion on the web visit
        
https://groups.google.com/d/msgid/django-users/d707e803-3dbf-43e6-95a8-13869043c280%40googlegroups.com
        
<https://groups.google.com/d/msgid/django-users/d707e803-3dbf-43e6-95a8-13869043c280%40googlegroups.com?utm_medium=email&utm_source=footer>.
        For more options, visit https://groups.google.com/d/optout
        <https://groups.google.com/d/optout>.

-- Jani Tiainen

--
You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com <mailto:django-users+unsubscr...@googlegroups.com>. To post to this group, send email to django-users@googlegroups.com <mailto:django-users@googlegroups.com>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/117a9163-ad2e-4d18-b599-63f766bb5587%40googlegroups.com <https://groups.google.com/d/msgid/django-users/117a9163-ad2e-4d18-b599-63f766bb5587%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

--
Jani Tiainen

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/066808f3-5340-0ad0-735d-87cc28fbaf4d%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to