Thanks! I stress it would be optional, we could add it in a release after the main couchdb-on-fdb in response to pressure from users finding the 10mb (etc) limits too restrictive, or we can do it as a neat enhancement in its own right (the validation aspect) that just happens to allow us to optimise the lengths of keys.
B. -- Robert Samuel Newson rnew...@apache.org On Thu, 31 Jan 2019, at 17:36, Adam Kocoloski wrote: > I like the idea, both for the efficiencies it enables in the > FoundationDB data model and for the ability to cover a lot of validation > functionality without shelling out to JS. > > It’s pretty obviously a big, meaty topic unto itself, one that needs > some careful thought and design. Also an awful lot of opportunity for > scope creep. But a good topic nonetheless. > > Adam > > > On Jan 31, 2019, at 12:05 PM, Robert Newson <b...@rsn.io> wrote: > > > > Hi, > > > > An enhancement over the first idea (where we flatten JSON documents into > > keys and values where the key is the full path to every terminal value, > > regardless of depth in the JSON) is to allow users to register schemas. > > > > For documents that match a registered schema (suggestion, a top level field > > called "_schema" is required to mark the documents) we can convert to > > key/value pairs much more compactly. The key name, barring whatever prefix > > identifies the database itself, is just the name of the schema and the > > ordinal of the schema item relative to the declaration. > > > > These schema documents (living under /dbname/_schema/$scheme_name akin to > > design documents) would list all required and optional fields, their types > > and perhaps other constraints (like valid ranges or relationships with > > other fields). We could get arbitrarily complex in schema definitions over > > time. Effectively, these are validate_doc_update functions without the > > Javascript evaluation pain. > > > > We don't necessarily need this in the first version, but it feels like a > > better response to the worries over the restrictions that the flattening > > idea is causing than switching to an opaque series of 100k chunks. > > > > thoughts? > > B > > > > -- > > Robert Newson > > b...@rsn.io > > > > On Thu, 31 Jan 2019, at 16:26, Adam Kocoloski wrote: > >> > >>> On Jan 31, 2019, at 1:47 AM, ermouth <ermo...@gmail.com> wrote: > >>> > >>>> As I don't see the 10k limitation as having significant merit > >>> > >>> Not sure it’s relevant here, but Mango indexes put selected doc values > >>> into > >>> keys. > >>> > >>> ermouth > >> > >> Totally relevant. Not just because of the possibility of putting a large > >> scalar value into the index, but because someone could create an index > >> on a field that is itself a container, and Mango could just dump the > >> entire container into the index as the key. > >> > >> Presumably there’s a followup discussion dedicated to indexing where we > >> can suss out what to do in that scenario. > >> > >> Adam >
