On 2/28/2013 10:59 AM, Benoit Jacob wrote:
Because the raw crash files do not include new metadata fields, this has
led to weird engineering practices like shoving interesting metadata into
the freeform app notes field, and then parsing that data back out later.
I'm worried about perpetuating this kind of behavior, which is hard on the
database and leads to very arcane queries in many cases.
I don't agree with the notion that freeform fields are bad. freeform plain
text is an amazing file format. It allows to add any kind of data without
administrative overhead and is still easy to parse (if the data was that
was added was formatted with easy parsing in mind).
The obvious disadvantage is that it is much more difficult to
machine-process. For example elasticsearch can't index on it (at least
not without lots of custom parsing), and in general you can't ask tools
like hbase or elasticsearch to filter on that without a user defined
function. (Regexes might work for some kinds of text processing.)
But if one considers it a bad thing that people use it, then one should
address the issues that are causing people to use it. As you mention, raw
crash files may not include newer metadata fields. So maybe that can be
fixed by making it easier or even automatable to include new fields in raw
crash files?
Yes, that is all filed. We can't automatically include the field,
because we don't know whether they are supposed to be public or private,
but we should soon be able to have a dynamically updateable list.
Note that if mcmanus is correct, we're going to be dealing with 1M
fields per day here. That's a lot more than the 250k from crash-stats,
especially because the payload is bigger. I believe that the flat files
from crash-stats are a really useful kludge because we couldn't figure
out a better way to expose the raw data. But that kludge will start to
fall over pretty quickly, and perhaps we should just expose a better way
to do queries using the databases, which are surprisingly good at doing
these kinds of queries efficiently.
--BDS
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
