On 8/12/15 10:29 PM, Clint Byrum wrote:
Excerpts from Dan Smith's message of 2015-08-12 23:12:23 +0800:
If OTOH we are referring to the width of the columns and the join is
such that you're going to get the same A identity over and over again,
if you join A and B you get a "wide" row with all of A and B with a very
large amount of redundant data sent over the wire again and again (note
that the database drivers available to us in Python always send all rows
and columns over the wire unconditionally, whether or not we fetch them
in application code).
Yep, it was this. N instances times M rows of metadata each. If you pull
100 instances and they each have 30 rows of system metadata, that's a
lot of data, and most of it is the instance being repeated 30 times for
each metadata row. When we first released code doing this, a prominent
host immediately raised the red flag because their DB traffic shot
through the roof.
In the past I've taken a different approach to problematic one to
many relationships and have made the metadata a binary JSON blob.
Is there some reason that won't work? Of course, this type of thing
can run into concurrency issues on update, but these can be handled by
SELECT..FOR UPDATE + intelligent retry on deadlock. Since the metadata
is nearly always queried as a whole, this seems like a valid approach
that would keep DB traffic low but also ease the burden of reassembling
the collection in nova-api.
JSON blobs have the disadvantages that you are piggybacking an entirely
different storage model on top of the relational one, losing all the
features you might like about the relational model like rich datatypes
(I understand our JSON decoders trip up on plain datetimes?), insert
defaults, nullability constraints, a fixed, predefined schema that can
be altered in a controlled, all-or-nothing way, efficient storage
characteristics, and of course reasonable querying capabilities. They
are useful IMO only for small sections of data that are amenable to
ad-hoc changes in schema like simple bags of key-value pairs containing
miscellaneous features.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
